Bazel Remote Execution

How it works

A Bazel Remote Execution cluster has three components:

• Scheduler: accepts Bazel Remote Execution API requests and assigns actions to workers.
• Storage: stores content-addressable storage (CAS) entries durably across the cluster.
• Workers: start on demand and execute Bazel actions.

Each worker advertises a fixed number of execution slots. A slot represents one unit of concurrent work, so a worker with four slots can run up to four actions at the same time. Namespace starts workers as your build needs capacity and routes actions to workers that match the requested execution platform.

Bazel's remote execution protocol supports multi-platform builds. For example, you can run Bazel from a Linux host while selected actions execute on macOS workers. This is useful for repositories that need to produce or test artifacts on multiple operating systems without moving the entire CI job to each platform.

Getting started

First, make sure your workspace has early-access RBE enabled. Then use the CLI to create a Bazel configuration file for the execution cluster:

$
nsc bazel execution setup --bazelrc=~/.namespace.bazelrc

$
bazel --bazelrc=~/.namespace.bazelrc build //...

For a quick early-access smoke test, generate a configuration file:

$ nsc bazel execution setup --bazelrc=~/.namespace.bazelrc --key=docs-smoke-test

If the command succeeds, Namespace provisioned the execution cluster and wrote a Bazel configuration file that points at it.

Example: Linux host to Linux workers

This example runs Bazel from a Linux host and lets Namespace execute actions on Linux workers. It uses a public Bazel repository so you can try the flow without changing your own project first.

$ git clone https://github.com/bazelbuild/examples.git
$ cd examples/cpp-tutorial/stage3
 
$ nsc bazel execution setup --bazelrc=~/.namespace.bazelrc --key=examples-linux
$ bazel --bazelrc=~/.namespace.bazelrc build //...

By default, actions without explicit platform properties are routed to Linux workers.

Example: Linux host to macOS workers

Bazel can also route selected actions to macOS workers from a Linux client. The linux-to-mac-rbe-demo repository demonstrates that setup.

This example requires macos/arm64 workers to be enabled for your workspace. If you see an error such as no on-demand worker capacity configured for platform "macos/arm64", reach out to the Namespace team to enable macOS worker capacity.

$ git clone https://github.com/namespacelabs/linux-to-mac-rbe-demo.git
$ cd linux-to-mac-rbe-demo
 
$ nsc bazel execution setup --bazelrc=~/.namespace.bazelrc --key=linux-to-mac-demo
$ bazelisk --bazelrc=~/.namespace.bazelrc build //DemoApp:DemoApp

For your own targets, route an action to macOS by setting Bazel execution properties on that target or platform, for example:

exec_properties = {
    "OSFamily": "macos",
    "Arch": "arm64",
}

Actions without macOS execution properties continue to run on the default Linux workers, allowing one Bazel invocation to use both platforms.

Custom worker base images

By default, actions run inside Namespace's standard Linux worker image. If your actions need extra tools, libraries, or a specific base environment, you can run them inside your own container image instead.

Custom images are selected per action through the standard Bazel container-image execution property. The scheduler routes actions to workers started with the requested image, and actions without a container-image property continue to run on the default worker.

A few requirements apply:

• The image reference must be pinned to an immutable digest (repo@sha256:...). Mutable tags are rejected, because a tag could move to a different image while a warm worker for the old digest is still pooled.
• The image must be optimized before it can be used. Namespace converts the image into a fast-booting disk variant; the scheduler rejects any custom image that has not been optimized yet (see below).
• The worker must be able to pull the OCI image and therefore it has to be available publicly. Pushing it to your tenant's nscr.io registry is the simplest option.
• Custom images are supported on Linux workers only and they won't work on macOS workers.

Build, push, and optimize a custom image

A custom worker image starts as an ordinary OCI image. After you push it, you pin it to a digest and optimize it once

$ nsc docker login
$ nsc build . -t nscr.io/<tenant>/my-worker:latest --platform linux/amd64 --push

$ nsc base-image upload <source-image> my-worker:latest

$ docker inspect --format '{{index .RepoDigests 0}}' nscr.io/<tenant>/my-worker:latest
nscr.io/<tenant>/my-worker@sha256:<digest>

$
nsc base-image optimize --image_ref nscr.io/<tenant>/my-worker@sha256:<digest>

jobs:
  rbe:
    runs-on:
      - namespace-profile-foobar;permissions.additional_grant=baseimage:*:*

jobs:
  rbe:
    runs-on:
      - nscloud-ubuntu-22.04-amd64-4x8-with-features
      - namespace-features:permissions.additional_grant=baseimage:*:*

Set a custom image for the whole build

Use --remote_default_exec_properties to apply one image to every action that does not override it:

$ bazel --bazelrc=~/.namespace.bazelrc build //... \
  --remote_default_exec_properties=container-image=docker://<repo>@sha256:<digest>

Set a custom image per target or platform

Set the container-image execution property on a specific target or platform so only those actions run inside the custom image:

exec_properties = {
    "container-image": "docker://<repo>@sha256:<digest>",
}

The docker:// scheme prefix is optional.

Run preparation scripts at worker startup

A custom worker image can ship preparation scripts that run once before the worker starts handling any actions. These hooks are for setup related actions that must happen on every worker boot, such as mounting an external /nix store or seeding caches.

Place executable files in /etc/namespace/prepare.d inside your image. On startup, the worker runs every executable file in that directory in lexical order (e.g. 10-mount.sh before 20-warmup.sh). Non-executable files and subdirectories are skipped. Scripts inherit the worker agent's environment and stream their output to the worker logs.

Using revocable tokens

By default, nsc bazel execution setup uses your interactive login and writes a bazelrc that authenticates to the cluster with a short-lived mTLS client certificate. For CI/CD pipelines, automation, or any environment without an interactive login, you can instead authenticate with a revocable token.

Revocable tokens are long-lived but can be revoked at any time from cloud.namespace.so/user/sessions or with nsc token revoke.

$ nsc token create \
  --name "rbe-ci-token" \
  --grant '{"resource_type":"bazel/execution","resource_id":"*","actions":["ensure"]}' \
  --grant '{"resource_type":"ingress","resource_id":"*","actions":["access"]}' \
  --token_file rbe-token.json

$ export NSC_TOKEN_FILE="$(pwd)/rbe-token.json"
$ nsc bazel execution setup --static --bazelrc=~/.namespace.bazelrc --key=rbe-revocable

$ bazel --bazelrc=~/.namespace.bazelrc build //...

Current caveats

Remote Execution is still early and the experience is actively changing:

• Worker sizing is not yet self-service. Reach out to the Namespace team to change worker shapes or slot counts.
• Action observability is still missing.
• Actions are not sandboxed from each other when running in the same worker, but workers are never shared across workspaces.
• Action input hydration and result upload performance still needs further work.

If any of these caveats block your use case, contact support@namespace.so so we can help choose the right rollout path.