logo

Run Buildkite Jobs on Namespace

Namespace provides ephemeral, self-hosted Buildkite agents without requiring you to operate agent infrastructure.

Prefer to manage the integration through Buildkite? Buildkite Hosted Agents use Namespace infrastructure and are configured directly in Buildkite.

Learn about Buildkite Hosted Agents

How it works

Connecting a Buildkite organization creates a Namespace Cluster with a default queue and an agent token. A Buildkite webhook notifies Namespace when a job is scheduled on that cluster. Namespace then provisions an isolated instance, starts an agent for the queue, runs the job, and tears the instance down when the job completes.

You can select the operating system, architecture, instance shape, base image, and persistent cache for each step using agent tags.

Reach out to get enrolled into Namespace-managed agents.

Contact support

Getting Started

Create a Buildkite API access token

Create an API access token for the Buildkite organization you want to connect. Namespace uses this token for initial setup, ongoing build reconciliation, and build annotations, so its expiry must be set to Never. The token can be revoked at any time through the Buildkite UI. Enable these REST API scopes:

  • read_pipelines
  • read_builds and write_builds
  • read_clusters and write_clusters
  • read_organizations

Connect your Buildkite organization

Open Buildkite → Jobs in the Namespace dashboard, then enter the API token and your Buildkite organization slug, and then click Associate. Copy the generated webhook URL for the next step.

Namespace creates the Namespace Cluster, its default queue, and an agent token in your Buildkite organization.

Configure the Buildkite webhook

On the next page, click on Create new Webhook and copy and paste the provided webhook URL and token.

Buildkite webhook configuration for Namespace

Select the following events in the Buildkite webhook interface:

  • ping
  • job.scheduled
  • job.started
  • job.finished
Buildkite webhook events configuration for Namespace

Make sure the webhook is enabled for All Pipelines.

Buildkite pipelines configuration for Namespace

Add a pipeline to the Namespace Cluster

In your Buildkite pipeline's settings, select Namespace Cluster under General > Cluster to ensure that jobs for that pipeline are eligible to run on Namespace agents.

Add and remove queues

When you connect a Buildkite organization, Namespace creates a Namespace Cluster and registers its default queue to run on Namespace-managed agents.

To add an additional Namespace-managed queue:

  1. In Buildkite, navigate to Agents and select the cluster for which you want to create a queue.
  2. Create a new self hosted queue.
Buildkite queue creation
  1. Open Buildkite → Queues in the Namespace dashboard.
  2. Find the queue under its Buildkite cluster and click Register.
Make sure that your webhook is configured to send events from the pipelines using this queue.

Webhook Settings

Selecting a queue

Set the queue agent tag on a pipeline or step to send jobs to the registered queue:

agents:
  queue: "my-queue"

Removing queues

To remove a queue, navigate to Buildkite → Queues and click Unregister next to the queue. A queue cannot be removed while it has scheduled or running jobs in Namespace.

Unregistering a queue stops Namespace from provisioning agents for it and revokes the agent token created during registration. It does not delete the queue or its cluster from Buildkite.

Checking out code

Cloning repositories over HTTPS works by default and requires no additional configuration.

To clone over SSH, follow Buildkite's code access setup for self-hosted agents. Store your private key as a Buildkite secret, then set checkout.ssh_secret on each of your pipeline's steps that require SSH access:

steps:
  - label: "Build"
    command: "make build"
    checkout:
      ssh_secret: "MY_SSH_KEY"

Next Steps

Configure Buildkite Agents. Choose compute resources, base images, and persistent caches with agent tags.

Last updated