Bazel Build System

Lumio uses a dual build system: Cargo for local development, Bazel for CI and container images. Developers write and test code with standard Cargo commands locally. Bazel takes over in CI where its remote caching and incremental target detection provide significant speed improvements.

Why Bazel

• Remote caching via BuildBuddy -- unchanged targets are not rebuilt across CI runs
• Incremental target detection via bazel-diff -- only affected targets are tested on PRs
• Hermetic builds -- reproducible container images from pinned toolchains and vendored dependencies
• Multi-app image builds -- release-images.yml builds and pushes images for the nine Rust apps (api, automation-worker, bot-module-worker, twitch-bot, youtube-bot, kick-bot, trovo-bot, discord-bot, innertube-proxy). The three Next.js apps (web, admin, id) are built in the same workflow with docker build -f docker/{app}.Dockerfile, not with Bazel.

Architecture

lumio/
├── MODULE.bazel                         # Root module definition
├── .bazelrc                             # Local Bazel settings
├── .github/actions/setup-bazel/         # Composite action: generates the CI bazelrc
├── misc/toolchains/
│   ├── rust.MODULE.bazel                # Rust toolchain (edition 2024, pinned version)
│   ├── proto.MODULE.bazel               # Protobuf toolchain
│   ├── v8.MODULE.bazel                  # V8 prebuilt static library
│   ├── docker.MODULE.bazel              # Base image pull (runtime-base, by `latest` tag)
│   └── BUILD.bazel                      # cc_library for V8 native linking
├── vendor/cargo/                        # Vendored crate BUILD files (~814)
│   ├── BUILD.actix-web-4.13.0.bazel
│   ├── BUILD.tokio-1.*.bazel
│   └── ...
├── crates/*/BUILD.bazel                 # One BUILD per internal crate
└── apps/*/BUILD.bazel                   # One BUILD per app (binary + image targets)

The root MODULE.bazel includes four toolchain modules and pins the rules_rust fork via git_override.

There is no checked-in .bazelrc.ci. .bazelrc deliberately carries no try-import for it -- CI settings are generated at runtime by .github/actions/setup-bazel/action.yaml and handed to bazel-contrib/setup-bazel via its bazelrc input.

zaflun/rules_rust Fork

Lumio uses a fork of rules_rust at github.com/zaflun/rules_rust. The fork fixes two issues in the upstream repository:

1. Sandbox path rotation for cargo_build_script -- the upstream OUT_DIR handling breaks when Bazel rotates sandbox paths between actions. The fork stabilizes OUT_DIR so build scripts (e.g., utoipa-swagger-ui asset embedding) produce deterministic outputs.
2. cargo_runfiles data file mapping -- the upstream mapping does not resolve data dependencies (e.g., V8 binding files) correctly in sandboxed builds. The fork patches the runfiles lookup.

The fork publishes prebuilt cargo-bazel binaries via GitHub Releases for Linux (x86_64, aarch64), macOS (aarch64), and Windows (x86_64, aarch64). These are fetched as http_archive dependencies in MODULE.bazel so that the vendor step does not require building cargo-bazel from source.

Vendored Crates

All external Rust dependencies are vendored into vendor/cargo/ as auto-generated BUILD.{crate}-{version}.bazel files (~814 of them). This avoids network fetches during builds and ensures reproducibility. The generated repository is exposed as @crate_index (wired up in misc/toolchains/rust.MODULE.bazel via the //vendor:cargo_ext.bzl extension).

Update workflow after changing Cargo.lock:

bazel run //vendor:cargo_vendor

This regenerates all BUILD.*.bazel files in vendor/cargo/ using crate_universe. The vendored files are committed to the repository.

BUILD.bazel Patterns

Crate (library + tests)

Every internal crate follows this pattern:

exports_files(["Cargo.toml"])

load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")

rust_library(
    name = "lo-common",
    srcs = glob(["src/**/*.rs"]),
    edition = "2024",
    deps = [
        "@crate_index//:actix-web",
        "@crate_index//:serde",
        "@crate_index//:serde_json",
    ],
    visibility = ["//visibility:public"],
)

rust_test(
    name = "lo-common_test",
    crate = ":lo-common",
)

The exports_files(["Cargo.toml"]) line is required -- the vendor toolchain reads Cargo.toml metadata from every workspace member. Omitting it causes vendor failures.

App (binary + container image)

App BUILD files define a rust_binary, then layer it into a container image:

load("@rules_img//img:image.bzl", "image_manifest")
load("@rules_img//img:layer.bzl", "image_layer")
load("@rules_img//img:push.bzl", "image_push")
load("@rules_rust//rust:defs.bzl", "rust_binary")

rust_binary(
    name = "api",
    srcs = glob(["src/**/*.rs"]),
    edition = "2024",
    deps = ["//crates/lo-common", ...],
)

image_layer(
    name = "api_layer",
    tags = ["manual"],
    srcs = {
        "/usr/local/bin/lumio-api": ":api",
        "/app/config/default.toml": "config/default.toml",
        "/app/config/production.toml": "config/production.toml",
        "/app/config/staging.toml": "config/staging.toml",
    },
)

image_manifest(
    name = "api_image",
    tags = ["manual"],
    base = "@runtime_base",
    layers = [":api_layer"],
    entrypoint = ["/usr/bin/tini", "--", "/usr/local/bin/lumio-api"],
    env = {
        "TZ": "Europe/Berlin",
        "RUST_LOG": "info",
        "LUMIO__SERVER__HOST": "0.0.0.0",
        "LUMIO__SERVER__PORT": "3000",
    },
    user = "1001",
    visibility = ["//visibility:public"],
)

image_push(
    name = "api_push",
    tags = ["manual"],
    image = ":api_image",
    registry = "ghcr.io",
    repository = "zaflun/lumio/api",
    tag = "{{.TAG}}",
    build_settings = {"TAG": "//misc:push_tag"},
    visibility = ["//visibility:public"],
)

Push tag

push_tag is a repo-local string_flag (//misc:push_tag, declared in misc/BUILD.bazel), not a rules_img setting — there is no @rules_img//img/settings:push_tag. Every image_push target wires it in via build_settings and expands it with tag = "{{.TAG}}"; release-images.yml overrides it per tag with --//misc:push_tag=<tag>. Without the tag attribute rules_img pushes by digest only and no tag is created.

Base image

The base image is ghcr.io/zaflun/lumio/runtime-base (Debian trixie-slim), pulled by the latest tag in misc/toolchains/docker.MODULE.bazel. Because it is pulled by tag rather than by digest, the pull rule needs unsafe_allow_tag_without_digest = True — rules_img refuses a tag-only pull otherwise and fails with missing valid digest, please specify the digest explicitly. The tag is resolved to a digest at fetch time, so the image is not reproducible across time; build-runtime-base.yml rebuilds it daily.

The base image index must not contain attestation manifests. build-runtime-base.yml therefore builds with --provenance=false --sbom=false. Without those flags docker buildx adds one unknown/unknown attestation manifest per platform to the OCI index. rules_img's image_import iterates every child manifest of an index and requires a matching config.rootfs.diff_ids entry for each layer; an attestation manifest has one in-toto layer but an empty ({}) config, so the import aborts with:

Error in fail: layer index out of range for config: 0

and every //apps/*:*_push target becomes unanalysable — the failure surfaces at @@+pull+runtime_base//:image, not in the app being pushed. A workflow fix alone does not help: the already-published image must be rebuilt, and latest is only re-tagged on main, so build-runtime-base.yml has to run from main before release-images.yml can succeed.

Manual tags

Every image_manifest / image_push target carries tags = ["manual"], so wildcards like bazel build //apps/... never expand to them, and ci.yml's Bazel jobs additionally pass --build_tag_filters=-manual. Image targets are therefore built only when named explicitly — by just bazel-images locally, or by release-images.yml on a release. No PR check analyses them, so a broken base image surfaces at release time, not in CI.

Adding a New Crate

1. Create the crate directory under crates/ with Cargo.toml and src/.
2. Add it to the workspace [members] in the root Cargo.toml.
3. Create crates/{name}/BUILD.bazel following the library + test pattern above. Include exports_files(["Cargo.toml"]) at the top.
4. Run bazel run //vendor:cargo_vendor to regenerate vendored BUILD files if the crate introduces new external dependencies.
5. Verify the build: bazel build //crates/{name}:...
6. If the crate is a dependency of a deployable app, add it to the app's deps in the app's BUILD.bazel.

Bazel deps are explicit -- adding a dependency in Cargo.toml does not add it to the corresponding BUILD.bazel. A green cargo check therefore does not imply a green Bazel build; hand-add every new dep to the deps list of the affected rust_library / rust_binary target.

V8 Integration

Crates and apps that depend on deno_core (the V8 JavaScript engine) require special linking. The V8 engine is distributed as a prebuilt static library for x86_64-unknown-linux-gnu only (matching the deployment fleet), fetched from the denoland/rusty_v8 releases.

Toolchain setup (misc/toolchains/v8.MODULE.bazel):

http_archive(
    name = "v8_prebuilt",
    build_file_content = 'exports_files(["librusty_v8.a"], visibility = ["//visibility:public"])',
    sha256 = "f48762ca10d1f1fc605a441c5ae430ec8ce1e9e80f14d78fbc42cb878c30b476",
    urls = ["https://github.com/denoland/rusty_v8/releases/download/v150.4.0/librusty_v8_simdutf_release_x86_64-unknown-linux-gnu.a.gz"],
)

Linking -- the cc_library that wraps the archive is declared once, publicly, in misc/toolchains/BUILD.bazel. App BUILD files do not redeclare it; they reference the shared target from the binary's deps:

# misc/toolchains/BUILD.bazel
load("@rules_cc//cc:cc_library.bzl", "cc_library")

cc_library(
    name = "v8_native",
    srcs = ["@v8_prebuilt//:librusty_v8.a"],
    linkstatic = True,
    linkopts = ["-lstdc++", "-ldl", "-lpthread"],
    visibility = ["//visibility:public"],
)

# apps/bot-module-worker/BUILD.bazel
rust_binary(
    name = "bot-module-worker",
    deps = [
        "//crates/lo-bot-module-worker",
        "//misc/toolchains:v8_native",
        ...
    ],
)

//misc/toolchains:v8_native is in the deps of every V8-linking binary: api, automation-worker, bot-module-worker, and the five platform bots.

Vendor BUILD for the v8 crate -- the vendored BUILD.v8-*.bazel uses $(execpath) to resolve the binding file path at build time:

rustc_env = {
    "RUSTY_V8_SRC_BINDING_PATH": "$(execpath gen/src_binding_simdutf_release_x86_64-unknown-linux-gnu.rs)",
},

Key Commands

Command	Description
just bazel-build	bazel build //crates/... //apps/...
just bazel-test	bazel test //crates/...
just bazel-images	Build the container images for api, bot-module-worker and the five platform bots
just bazel-clippy	Run clippy over //crates/... via the rust_clippy_aspect
just bazel-vendor	bazel run //vendor:cargo_vendor

just bazel-images does not cover automation-worker or innertube-proxy; build those explicitly (bazel build //apps/automation-worker:automation-worker_image) when you need them locally.

Direct Bazel invocations:

# Build a single crate
bazel build //crates/lo-common

# Test a single crate
bazel test //crates/lo-common:lo-common_test

# Build a specific app image
bazel build //apps/api:api_image

# Push an image (requires GHCR authentication)
bazel run //apps/api:api_push --//misc:push_tag="2026.5.3"

BuildBuddy

Remote caching is provided by BuildBuddy. The .github/actions/setup-bazel composite action generates the CI bazelrc at runtime and passes it to bazel-contrib/setup-bazel. The remote-cache block is emitted only when the BUILDBUDDY_API_KEY secret is present -- without it the job logs a notice and builds without a remote cache:

common --remote_cache=grpcs://remote.buildbuddy.io
common --bes_backend=grpcs://remote.buildbuddy.io
common --bes_results_url=https://app.buildbuddy.io/invocation/
common --remote_timeout=3600
common --remote_cache_compression
common --remote_header=x-buildbuddy-api-key=$BUILDBUDDY_API_KEY

Cache write policy:

Context	Writes
Base default (every job)	Read-only -- the generated bazelrc always sets common --noremote_upload_local_results
In-org pull_request	Read-write -- the Rust Tests / Rust Build jobs append --remote_upload_local_results when github.event.pull_request.head.repo.full_name == github.repository
push to main	Read-write -- same flag; main is the promotion pointer

There is no push-to-next run any more (ZAF-1070): every one was cancelled by hand after each merge, so it seeded nothing and cost ~220-360 job-min/day for zero signal. In-org PR runs are the cache seeders now. Fork PRs cannot poison the cache -- secrets.BUILDBUDDY_API_KEY is not exposed to fork pull_request runs (and pull_request_target is unused), so setup-bazel emits no remote-cache block for a fork and the upload flag is a no-op there; the head.repo == repository guard is defence-in-depth on top of that.

The same action also installs the system deps Bazel needs (libssl-dev, clang, lld), provisions a 4 GB swapfile to survive large crate compilations, and logs in to GHCR so the @runtime_base pull succeeds.

Troubleshooting

OpenSSL in sandbox

Some crates (e.g., openssl-sys) need environment variables to find system libraries inside the Bazel sandbox. If builds fail with OpenSSL linker errors, check that action_env settings in .bazelrc expose the necessary paths.

Missing exports_files for Cargo.toml

Every crate's BUILD.bazel must include exports_files(["Cargo.toml"]). Without it, the vendor toolchain cannot read crate metadata and bazel run //vendor:cargo_vendor fails with a missing-file error.

layer index out of range for config: 0

An image target fails during analysis with this rules_img error:

ERROR: in image_import rule @@+pull+runtime_base//:image:
Error in fail: layer index out of range for config: 0

The index that latest resolves to contains a buildx attestation manifest. image_import walks every child manifest of an index and needs one config.rootfs.diff_ids entry per layer; an attestation manifest has an in-toto layer but an empty config, so it never matches. The fix is to republish latest without attestations -- build-runtime-base.yml already passes --provenance=false --sbom=false, but it only tags latest from main, so that workflow has to run from main before the pull resolves to a clean index. See Base image.

UNAUTHORIZED: authentication required pulling @runtime_base

ghcr.io/zaflun/lumio/runtime-base is a private package, so analysing any image target locally requires docker login ghcr.io with a token that has read:packages. A GitHub App installation token is not accepted by GHCR -- use a classic PAT. Without it, bazel build on //apps/*:*_image or //apps/*:*_push cannot fetch the base image; every other target builds fine, so this only blocks image work.

V8 linking failures

V8 linking requires:

1. The prebuilt librusty_v8.a archive matching the deno_core version in Cargo.lock.
2. A cc_library target in the app's BUILD.bazel that links the archive with -lstdc++ -ldl -lpthread.
3. The v8 vendor BUILD must have the correct $(execpath) for the binding source file.

If the deno_core or v8 crate version changes in Cargo.lock, update the SHA and URL in misc/toolchains/v8.MODULE.bazel to match the corresponding rusty_v8 release.