Installation
Set up a local Lumio development environment.
Hosted Environments
Lumio runs three environments per app. Production lives on lumio.vision; non-production on zaflun.dev. Use these strings when configuring OAuth callbacks, CORS, env-var defaults and CI deploy targets.
| App | Production | Production Preview | Staging |
|---|---|---|---|
| Webapp | lumio.vision | lumio.web.prod.zaflun.dev | lumio.web.staging.zaflun.dev |
| ID App | id.lumio.vision | lumio.id.prod.zaflun.dev | lumio.id.staging.zaflun.dev |
| Admin App | admin.lumio.vision | lumio.admin.prod.zaflun.dev | lumio.admin.staging.zaflun.dev |
| Stats App | stats.lumio.vision | lumio.stats.prod.zaflun.dev | lumio.stats.staging.zaflun.dev |
| API | api.lumio.vision | lumio.api.prod.zaflun.dev | lumio.api.staging.zaflun.dev |
| Docs | docs.lumio.vision | lumio.docs.prod.zaflun.dev | lumio.docs.staging.zaflun.dev |
| Developer Docs | developers.lumio.vision | lumio.dev.docs.prod.zaflun.dev | lumio.dev.docs.staging.zaflun.dev |
| Overlay external URL | overlay.lumio.vision | (single per-deployment external host; configurable via NEXT_PUBLIC_OVERLAY_URL on the webapp) | (same) |
| Widget external URL | widget.lumio.vision | (single per-deployment external host; configurable via LUMIO__WIDGET__PUBLIC_URL on the API) | (same) |
Branch mapping: next → staging · main → production preview · tag 20* → production. This mapping is enforced in CI for the Cloudflare-hosted docs family (docs, developer-docs, extension-apps). The core product containers (API, web, admin, id, workers, bots) are container images promoted by release channel instead - beta for a pre-release, latest for a full release.
The rest of this guide covers a local development setup.
Prerequisites
| Tool | Version / notes |
|---|---|
| Docker + Docker Compose | Runs the local database stack |
Rust toolchain (rustup) | Edition 2024. CI builds on stable; the Bazel toolchain pins 1.97.1 |
protoc | Required - lo-youtube-api's build script compiles the YouTube gRPC protos |
| Node.js | >= 18 |
| pnpm | 10.28.0 (see packageManager in the root package.json) |
just | Task runner used by every command in these docs |
Bazel is optional locally - it is the CI build system (see Bazel below).
Dev Stack
just stack-up starts the local database + broker stack, merging dev-stack/db.docker-compose.yml and dev-stack/broker.docker-compose.yml:
just stack-up
| Service | Image | Host port | Database |
|---|---|---|---|
lumio-postgres | postgres:18-alpine | 127.0.0.1:5432 | lumio (user/password lumio) |
lumio-tsdb | timescale/timescaledb-ha:pg18 | 127.0.0.1:5433 | lumio_tsdb (user/password lumio) |
lumio-extensions-postgres | postgres:18-alpine | 127.0.0.1:5434 | lumio_extensions (owner lumio_ext, DML logins lumio_ext_api / lumio_ext_worker; password = user name) |
lumio-redis | redis:8-alpine | 127.0.0.1:6379 | password lumio |
lumio-rabbitmq | rabbitmq:4-management-alpine | 127.0.0.1:5672 (AMQP), 127.0.0.1:15672 (UI) | user/password lumio |
lumio-kafka | apache/kafka:3.9.0 | 127.0.0.1:9092 | KRaft mode (no ZooKeeper) |
The brokers back the ingest pipeline: RabbitMQ is the job/command bus and Kafka is the durable ingest firehose (Redis stays cache-only, TimescaleDB is the system of record). Dev and staging run single-node; only production is the multi-node cluster.
The extensions PostgreSQL's superuser is postgres (password postgres), and no app uses it: superusers ignore row-level security. On an empty volume dev-stack/initdb/extensions/10-roles.sh creates the DDL owner lumio_ext, the lumio_ext_dml group and the DML logins (see developer-guide/data-layer-staging.md, "Extensions PostgreSQL: roles"). A volume created before this change still has lumio_ext as its superuser. Delete that volume once with the stack stopped (find it with docker volume ls | grep lumio-extensions-pg-data; compose prefixes the project name) so the script runs, or switch it in place as described there.
The databases preload pg_stat_statements and log every statement that takes 1 s or longer. Redis has maxmemory 512mb with maxmemory-policy noeviction: a full Redis rejects writes instead of dropping keys, as on staging (see developer-guide/data-layer-staging.md). TimescaleDB keeps its data in the lumio-tsdb-pgdata volume. The older lumio-tsdb-data volume was mounted at a path the timescaledb-ha image does not use, so it holds no data and you can delete it.
The stack runs in the foreground, so use a dedicated terminal. Related recipes:
| Recipe | What it does |
|---|---|
just stack-down | Stop the stack (volumes kept) |
just stack-reset | Stop the stack, drop the volumes, start again |
just stack-logs | Tail the stack logs |
just mail-up | MailDev on http://localhost:1080 |
just minio-up | MinIO console on http://localhost:9001 |
just stack-all | stack-up + mail-up + minio-up |
just psql / just tsdb-psql / just redis-cli | Open a CLI against the running containers |
just rabbitmq-ui | Print the RabbitMQ management UI URL (http://127.0.0.1:15672, lumio/lumio) |
just kafka-topics | List Kafka topics on the local KRaft broker |
Database Migrations
Migrations live in apps/api/migrations/ (main database) and apps/api/tsdb_migrations/ (TimescaleDB) and are run by the migrate binary:
just migrate # run all pending migrations on both databases
just migrate-status # show what has been applied
just migrate-main # main database only
just migrate-tsdb # TimescaleDB only
just db-fresh # drop, create, migrate
The API also runs pending migrations on startup.
sqlx::migrate!() embeds the .sql files into the binary when it is compiled. Adding a new migration file therefore requires a rebuild - cargo build -p lumio-api (or just build-api) - not just a restart. A restart alone will silently run the old migration set.
New migrations use the reversible .up.sql / .down.sql pair format. Legacy migrations from before 20260410 use the simple single-file format and are never renamed.
Running the Apps
Each app has its own recipe and its own port:
| Recipe | App | URL |
|---|---|---|
just run-api (or just dev-api for auto-reload, which needs cargo install cargo-watch) | Rust API | http://localhost:3000 |
just dev-web | Webapp / dashboard | http://localhost:4000 |
just dev-admin | Admin panel | http://localhost:4001 |
just dev-id | ID / auth app | http://localhost:4002 |
just dev-stats | Public stats app | http://localhost:4003 |
pnpm --filter @lumio/shopify dev | Embedded Shopify admin page (apps/shopify); renders only inside the Shopify admin, see Shopify → Embedded page | http://localhost:4004 |
just dev-docs | This documentation site | Docusaurus default port - pass --port to avoid clashing with the API on 3000 |
just dev-developer-docs | Developer documentation | http://localhost:3004 |
just dev-supervisor | Extension supervisor (static) | http://localhost:3200 |
just dev-runtime | Extension runtime (static) | http://localhost:3201 |
Bazel (CI Only)
Bazel handles CI builds and container image creation. It is not required for local development -- Cargo covers that. Install Bazelisk, which automatically downloads the correct Bazel version:
# Install Bazelisk (Bazel version manager)
npm install -g @bazel/bazelisk
# Or on macOS:
brew install bazelisk
# Verify
bazel --version
See the Bazel Build System guide for details on the build architecture, vendored crates, and image targets.
Environment Configuration
Sign-in protection and support code
The API's [turnstile] section controls server-side Cloudflare Turnstile verification. enabled, site_key, secret_key, id_hostnames, web_hostnames, verify_url, and timeout_ms can be overridden as LUMIO__TURNSTILE__ENABLED, LUMIO__TURNSTILE__SITE_KEY, LUMIO__TURNSTILE__SECRET_KEY, LUMIO__TURNSTILE__ID_HOSTNAMES, LUMIO__TURNSTILE__WEB_HOSTNAMES, LUMIO__TURNSTILE__VERIFY_URL, and LUMIO__TURNSTILE__TIMEOUT_MS. Use exact ID-app hosts for registration, login, reset and resend; exact web-app hosts for email setup and resend and for the contact form (action contact). The public site key is supplied to the apps at runtime by GET /v1/auth/turnstile-config. Keep the secret in the environment. The default pair is Cloudflare's always-pass test pair, so every protected action passes on a local stack. Cloudflare answers the test secret with hostname example.com, no action and metadata.result_with_testing_key: true; the API accepts that answer without the action and hostname checks, and only when the environment is neither staging nor production. In staging and production the test secret fails closed, and a test-key answer with a real secret is rejected, so hosted environments use real keys. To run with no check at all, set LUMIO__TURNSTILE__ENABLED=false.
The API's [contact] section sets where the public contact form (POST /v1/contact, GraphQL submitContactMessage) sends its messages: recipient (default hey@lumio.vision, ENV LUMIO__CONTACT__RECIPIENT). Each message goes out through the configured [email] provider with Reply-To set to the sender and is not stored. With email.enabled = false the message is only logged as a skipped send.
Set [support].code_secret through LUMIO__SUPPORT__CODE_SECRET. It is the HMAC secret for each profile's hourly support code; rotating it invalidates existing codes. Never expose it to a browser or put it in a repository TOML file.
Local development needs no setup: when the secret is empty and environment is development (or dev), the API derives codes from a fixed, public development secret and logs this once at info on startup. Every other environment takes no default. Staging or production without the secret still starts, but logs warn support code disabled: LUMIO__SUPPORT__CODE_SECRET is empty; the support code then stays off (SUPPORT_CODE_UNAVAILABLE), so the profile header shows no code and the admin Support Identity card cannot reveal or check one.
File storage for profile pictures
Own profile pictures (POST /v1/users/me/avatar) are stored through the API's [storage] section, the same lo-storage client that channel uploads and GDPR exports use. With storage.enabled = false (LUMIO__STORAGE__ENABLED) the upload returns 503 AVATAR_UNAVAILABLE. The files live under the key prefix users/<user_id>/avatar/ (<uuid>-512.webp and <uuid>-128.webp), and users.avatar_url is storage.public_url (LUMIO__STORAGE__PUBLIC_URL) plus that key. Point storage.public_url at the API itself, <API origin>/v1/uploads/file (e.g. http://localhost:3000/v1/uploads/file, or your own port): the API serves the current picture under that path (GET /v1/uploads/file/users/<user_id>/avatar/<uuid>-512.webp) from storage, so neither the local directory nor an S3 bucket has to be publicly readable. The API re-derives every stored picture URL from the key at start-up, so changing storage.public_url heals existing pictures by itself. The dashboard's img-src allows the API origin; a CSP set in front of the web app (such as the production edge) has to allow it too. GDPR export archives use the separate private/data-exports/ prefix, which must stay private. The 5 MB picture cap is fixed in code and independent of storage.max_upload_size_bytes. Uploaded channel pictures (POST /v1/accounts/<id>/avatar) use the same storage under accounts/<id>/avatar/ and are served at GET /v1/uploads/file/accounts/<id>/avatar/<uuid>-512.webp, with the same 503 AVATAR_UNAVAILABLE when storage is off.
Billing without Stripe
Stripe billing is controlled by the API's [stripe] section (enabled, secret_key, webhook_secret, price_id_pro, price_id_enterprise; ENV LUMIO__STRIPE__ENABLED, LUMIO__STRIPE__SECRET_KEY, …). It is off in default.toml, so a local stack runs without Stripe. Then billingStatus.billingEnabled (REST GET /v1/billing/status → billing_enabled) is false, the dashboard's Subscription tab still lists the plans (from the database) and the channel's current plan, with checkout, switching to Free and cancelling disabled; the Invoices tab shows that billing is not set up instead of loading invoices; and every other billing call answers 409 BILLING_DISABLED. New channels are created on Free either way.
Configuration is split by runtime. There is no repository-root .env - each app is configured on its own.
Developer payouts and credit notes
Every developer payout is documented by a self-billed credit note (Gutschrift). The issuer printed on it comes from the API's [billing.issuer] section: name, address (one line per address line) and vat_id (ENV LUMIO__BILLING__ISSUER__NAME, LUMIO__BILLING__ISSUER__ADDRESS, LUMIO__BILLING__ISSUER__VAT_ID). They are empty in default.toml: the API then logs a warning at start-up and refuses every payout, so a local stack books none. [billing.credit_notes] sets the number prefix (number_prefix, default GS) and the VAT rate for German standard-rated developers (domestic_vat_rate_bp, default 1900). Storing and mailing the PDFs needs [storage] and [email]. Details: Developer credit notes.
The payout cycle ([payments] payout_cycle_enabled, ENV LUMIO__PAYMENTS__PAYOUT_CYCLE_ENABLED, default true) pays out developer balances once per month and sends pending Stripe and PayPal payouts every 15 minutes. A provider without credentials (stripe_secret_key; paypal_client_id + paypal_client_secret) is skipped, so a local stack sends nothing. Details: Developer payouts.
Rust services - layered TOML
The API and the bots read layered TOML from their own config/ directory:
config/default.toml → config/{env}.toml → config/local.toml → ENV vars
Later layers win, so an environment variable always overrides a file. config/default.toml is committed and holds every key with a documented default; config/local.toml is git-ignored and is where you put local secrets. The {env} layer is selected by APP_ENV for the API (development - the default - staging, or production), and by {SERVICE}_RUN_MODE for the bots (e.g. TWITCH_BOT_RUN_MODE).
List values from the environment are comma-separated. A list key takes one ENV value whose entries are separated by ,, for example LUMIO__TURNSTILE__ID_HOSTNAMES=lumio.id.staging.zaflun.dev or LUMIO__TURNSTILE__WEB_HOSTNAMES=a.example,b.example. A single entry needs no comma. Each entry is trimmed and empty entries are dropped, so " a.example , b.example ," loads as a.example and b.example. Numeric lists work the same way: CRAWLER_TWITCH__CRAWLER__OWNED_SHARDS=3 loads as [3], and =1,2 as [1, 2]. The list keys are cors.allowed_origins, youtube.innertube_proxy_urls, history.command_prefixes, webauthn.origins, turnstile.id_hostnames, turnstile.web_hostnames (API) and crawler.owned_shards (Twitch crawler). Every other key is read as one value, so a comma inside a URL, token or DSN is kept as it is.
Create apps/api/config/local.toml with at least your OAuth login credentials and a token encryption key (see OAuth Provider Setup). The database and Redis defaults in default.toml already match the dev stack:
| Config key | Default |
|---|---|
server.port | 3000 |
database.url | postgres://lumio:lumio@localhost:5432/lumio |
timescale.url | postgres://lumio:lumio@localhost:5433/lumio_tsdb |
extensions.database.url | postgres://lumio_ext_api:lumio_ext_api@localhost:5434/lumio_extensions |
extensions.database.ddl_url | postgres://lumio_ext:lumio_ext@localhost:5434/lumio_extensions |
redis.url | redis://:lumio@127.0.0.1:6379 |
web.public_url | http://localhost:4000 |
admin.public_url | http://localhost:4001 |
overlay.public_url | http://localhost:4000/overlay |
widget.public_url | http://localhost:4000/widget |
The extensions database uses two roles. extensions.database.url (LUMIO__EXTENSIONS__DATABASE__URL) is the DML pool that server functions run on: a login in the lumio_ext_dml group that may only read and write rows. extensions.database.ddl_url (LUMIO__EXTENSIONS__DATABASE__DDL_URL) is the DDL pool, at most 2 connections: the schema owner, which runs the extensions migrations, provisioning, the data purges and the storage-quota pass. At startup the API refuses to run if either role is a superuser or has BYPASSRLS, or if the DML login has CREATE or TEMPORARY on the database. Outside development it also refuses an empty ddl_url or one equal to url. In development an empty ddl_url falls back to url. The dev stack's init script creates both logins; a volume created before it still has lumio_ext as its superuser, and the API then refuses to start until the volume is recreated or switched (see the note under the dev-stack table above).
Required public URLs for hosted API deployments
The API uses [web] public_url and [admin] public_url to mint OAuth redirect URIs and to validate the admin return target for global bot OAuth. It also uses the web URL for developer-team invite emails (/invite/dev/\{code\}) and the admin URL for developer-application review notifications and emails (/developers/applications). apps/api/config/staging.toml and apps/api/config/production.toml do not pin those sections, so hosted deployments must supply them through environment variables:
| Environment | LUMIO__WEB__PUBLIC_URL | LUMIO__ADMIN__PUBLIC_URL |
|---|---|---|
| Production | https://lumio.vision | https://admin.lumio.vision |
| Production preview | https://lumio.web.prod.zaflun.dev | https://lumio.admin.prod.zaflun.dev |
| Staging | https://lumio.web.staging.zaflun.dev | https://lumio.admin.staging.zaflun.dev |
If LUMIO__WEB__PUBLIC_URL is missing outside local development, system-sourced channel and bot OAuth flows build provider redirect URIs with the default http://localhost:4000 host. Twitch, Kick and YouTube reject that redirect URI in hosted environments. If LUMIO__ADMIN__PUBLIC_URL is missing, the admin global-bot flow stores http://localhost:4001/providers as its return target; the unified callback allowlist then cannot return the browser to the hosted admin app.
DATABASE_URL is not an API settingThe API reads its connection strings from the TOML layers (or LUMIO__DATABASE__URL). The apps/api integration-test harness uses DATABASE_URL to create an isolated database for each test. Start the Dev Stack, set DATABASE_URL to its PostgreSQL server using a user that can CREATE DATABASE, then run cargo test -p lumio-api --test <name>. For a TimescaleDB-gated suite, set DATABASE_URL to the Dev Stack's TimescaleDB server and RUN_TSDB_MIGRATION_TEST=1 before running the same command. Tests that need real Redis can opt in with TEST_REDIS_URL; other database tests use the offline Redis fixture.
Agent runs use a separate host test stack with an injected DATABASE_URL. That stack runs on demand: if a test reports connection refused on ports 15432, 15433, or 16379, agents start it with docker compose -f ~/test-stack/docker-compose.yml up -d, wait for the services to be healthy, and rerun the test. Agents leave it running; only Raphael stops it. They do not replace the injected URL or switch to the developer stack.
Mandatory auth secrets
Two [auth] keys are required on any non-development deployment. default.toml ships them unset/placeholder so the local inner loop just works, but the API refuses to boot (environment = staging or production) if either is empty, the shipped placeholder (CHANGE_ME / change-me-in-production), or shorter than 16 characters:
| Key | ENV var | Why it must be a real secret |
|---|---|---|
auth.token_encryption_key | LUMIO__AUTH__TOKEN_ENCRYPTION_KEY | AES-256-GCM key for every at-rest secret (channel + BYO app_credentials OAuth tokens, StreamElements JWTs, OBS passwords, Stripe Connect state). An empty value derives SHA-256("") - a constant any clone of this repo can reproduce - so all ciphertext becomes readable by anyone who has the code. |
auth.jwt_secret | LUMIO__AUTH__JWT_SECRET | HMAC key for session tokens. A placeholder value lets anyone forge a valid session JWT (auth bypass, no DB access needed). |
Generate each with openssl rand -base64 32.
In development the boot only logs an error! and continues, so just run-api still works against the empty defaults - but data encrypted under the empty key is readable by anyone, so never expose a dev instance.
token_encryption_key is destructiveThe ciphertext carries no key-version tag, so changing this key makes every existing row permanently undecryptable (aead::Error) - rows are re-encrypted only on their next write. Set it once per deployment and never change it on a live instance. A mid-flight flip between "set" and "unset" is the exact root cause of the ZAF-1196 credential-loss incident. If you must rotate, plan a coordinated re-encrypt of every affected column.
Next.js apps - .env.local
apps/web, apps/admin and apps/id each ship a .env.example. Copy it next to the app and fill it in:
cp apps/web/.env.example apps/web/.env.local
cp apps/admin/.env.example apps/admin/.env.local
cp apps/id/.env.example apps/id/.env.local
The variables you will actually touch first:
| Variable | App | Description |
|---|---|---|
LUMIO_API_URL | web, admin, id | Server-side API base - http://localhost:3000/v1 |
LUMIO_INTERNAL_TOKEN | web, admin, id, stats | Shared secret for SSR → API calls; must match rate_limiting.internal_token in the API config (dev-internal-token locally) |
LUMIO_AUDIT_INGEST_KEY | id | System key (lm_sys_* carrying the narrow audit:ingest grant) that apps/id uses to post user-scope audit events to POST /v1/internal/audit-ingest. Leave unset locally - audit emission is then simply skipped and login still works. In staging/production, mint a dedicated key carrying only the audit:ingest chip from the admin app (System → System Keys) and set it here - see Provisioning the apps/id audit-ingest key. While it is unset the whole apps/id audit trail is dropped and GET /api/health reports audit_ingest_configured: false (see Audit Events → Events that originate outside the API) |
AUTH_SECRET | id | NextAuth session secret |
AUTH_URL | id | Public base URL of the ID app - http://localhost:4002. NextAuth builds its OAuth callbacks from it, and the ID app's own login redirect, callbackUrl and absolute share-image URLs use its origin; behind a reverse proxy the request URL carries the container's bind address (0.0.0.0:4002), so this must be the public host |
AUTH_{PROVIDER}_ID / AUTH_{PROVIDER}_SECRET | id | OAuth login credentials per provider (see OAuth Provider Setup) |
LUMIO__AUTH__DISCORD_CLIENT_SECRET | api | Discord login app secret matching AUTH_DISCORD_SECRET in the ID app; needed for rotating login-token refresh |
LUMIO__AUTH__DISCORD_BOT_CLIENT_ID / LUMIO__AUTH__DISCORD_BOT_CLIENT_SECRET | api | Global Discord bot application used for verified server invites; separate from the login pair and the static Bot Token |
DISCORD_BOT__API__SYSTEM_KEY | discord-bot | System key created under Admin → System Keys in the same environment with bots:internal; the API does not provision System keys from config |
LUMIO_APP_URL / NEXT_PUBLIC_APP_URL | web, admin | Public URL of the app itself; LUMIO_APP_URL is read on the server per request and wins. It is the origin of the login callbackUrl and of the OAuth connection-callback redirects. On web it also supplies canonical, language-alternate, share-image and sitemap URLs; only https://lumio.vision allows indexing. An unset or invalid value yields no canonical or language alternates and blocks indexing. Behind a reverse proxy the request URL carries the container's bind address (0.0.0.0:<port>), and the Host / X-Forwarded-Host headers are never trusted for it |
LUMIO_STATS_URL | stats | Public origin of the Stats app, read on the server per request. It supplies the canonical, language-alternate and sitemap URLs. Only https://stats.lumio.vision allows indexing: every other value, including the prod origin host lumio.stats.prod.zaflun.dev, and an unset or invalid one make every response noindex, nofollow (meta tag and X-Robots-Tag), robots.txt Disallow: / and the sitemap routes 404. There is no NEXT_PUBLIC_* variant and no localhost fallback |
LUMIO__PUBLIC_STATS__SITEMAP_PLATFORMS | api | Comma-separated server-side [public_stats].sitemap_platforms allowlist for channel sitemap files. The default is twitch,kick; the client cannot expand it. The sitemap includes only allowlisted channels with a login and a stream in the last 12 months. Keep it aligned with the Stats app's indexable channel-platform list. |
NEXT_PUBLIC_ID_URL | web | Public URL of the ID app |
NEXT_PUBLIC_OVERLAY_URL | web | Public overlay host - http://localhost:4000/overlay locally |
LUMIO_SUPERVISOR_URL / LUMIO_EXTENSION_RUNTIME_URL | web | Extension sandbox hosts - http://localhost:3200 / http://localhost:3201 locally. Read on the server per request, so a deployed container takes them from its runtime env. NEXT_PUBLIC_SUPERVISOR_URL / NEXT_PUBLIC_EXTENSION_RUNTIME_URL are the fallback; they only apply where they were present at next build (local checkouts) |
LUMIO_EXTENSION_RUNTIME_ORIGIN_TEMPLATE | web | Per-extension runtime origin, e.g. https://{label}.lumio.ext.staging.zaflun.dev; {label} becomes the extension id as 32 hex characters and must match the template the Supervisor and Runtime were built with. Unset = every extension shares the runtime URL above, so removing the variable and restarting the container is the rollback. Falls back to NEXT_PUBLIC_EXTENSION_RUNTIME_ORIGIN_TEMPLATE. Empty values count as unset |
COOKIE_DOMAIN | web, admin, id | Only needed for cross-subdomain deployments; leave unset locally. Must be identical across the three apps |
LUMIO_WEB_URL / NEXT_PUBLIC_WEB_URL
Absolute URL of the apps/web app, read by apps/id. It is what the ID app redirects to after a successful login (/dashboard, /dashboard/settings/channel/connections, /dashboard/settings/channel/subscription). LUMIO_WEB_URL takes precedence; NEXT_PUBLIC_WEB_URL is the client-side mirror.
- Default in code:
http://localhost:4000 - Staging:
https://lumio.web.staging.zaflun.dev - Production-preview:
https://lumio.web.prod.zaflun.dev - Production:
https://lumio.vision
apps/web itself does NOT need this variable - it uses relative paths because it hosts those pages.
Environment Variable Naming
Every TOML key in apps/api/config/*.toml is overridable via an environment variable using the format LUMIO__<SECTION>__<KEY> - double underscore between the prefix and the first segment, and double underscore between every nested segment. Examples:
| TOML path | ENV var |
|---|---|
database.url | LUMIO__DATABASE__URL |
auth.token_encryption_key | LUMIO__AUTH__TOKEN_ENCRYPTION_KEY |
webhooks.youtube_secret | LUMIO__WEBHOOKS__YOUTUBE_SECRET |
webhooks.kick_public_key (optional pin) | LUMIO__WEBHOOKS__KICK_PUBLIC_KEY |
webhooks.kick_public_key_observer.refresh_interval_secs | LUMIO__WEBHOOKS__KICK_PUBLIC_KEY_OBSERVER__REFRESH_INTERVAL_SECS |
youtube.innertube_observer.api_key_override | LUMIO__YOUTUBE__INNERTUBE_OBSERVER__API_KEY_OVERRIDE |
kick.realtime.enabled | LUMIO__KICK__REALTIME__ENABLED |
The override priority is default.toml < {env}.toml < local.toml < ENV vars, so an env-var always wins.
Kick event delivery needs no signing-key setup: the API fetches Kick's public RSA key from https://api.kick.com/public/v1/public-key itself and refreshes it ([webhooks.kick_public_key_observer], fail-closed). webhooks.kick_public_key is only an optional pin. See Kick webhook setup and Kick Webhook Public Key.
The Kick chat-moderation listener ([kick.realtime]) needs no setup either: it picks up messages deleted and users unbanned on kick.com from Kick's unofficial public chat socket, with its public key fetched and refreshed at runtime. LUMIO__KICK__REALTIME__ENABLED=false turns it off, LUMIO__KICK__REALTIME__ALERT_WEBHOOK_URL routes its two alarms (cold boot, Pusher no longer offered) to Discord, and LUMIO__KICK__REALTIME__APP_KEY_OVERRIDE + LUMIO__KICK__REALTIME__CLUSTER_OVERRIDE are an emergency pin only. All keys: Kick Chat-Moderation Listener → Configuration.
7TV event listener
The API listens to the public 7TV EventAPI for changes to the active emote
sets of connected Twitch, Kick and YouTube channels. No 7TV key or inbound
webhook is needed. [seventv] in apps/api/config/default.toml accepts these
overrides:
| Key | Environment variable | Default |
|---|---|---|
events_enabled | LUMIO__SEVENTV__EVENTS_ENABLED | true |
ws_url | LUMIO__SEVENTV__WS_URL | wss://events.7tv.io/v3 |
max_subscriptions_per_connection | LUMIO__SEVENTV__MAX_SUBSCRIPTIONS_PER_CONNECTION | 450 |
roster_refresh_secs | LUMIO__SEVENTV__ROSTER_REFRESH_SECS | 600 |
events_enabled = false stops 7TV event capture and event-driven emote-cache
refresh; the normal cache TTL still applies. See the 7TV Event Listener
for the roster, privacy boundary and cache behaviour.
Platform Credential Sources
Channel and bot OAuth flows choose their OAuth app source from the API config. The flat [platform_credentials] table applies to channel connections. The nested [platform_credentials.bot] table applies to bot connections.
# Channel-credential source. The flat form is channel.
[platform_credentials]
twitch = "system"
kick = "system"
youtube = "account"
spotify = "account"
# Bot-credential source. Discord invites use the global bot app; its gateway token is static.
[platform_credentials.bot]
twitch = "system"
kick = "system"
youtube = "system"
discord = "system"
Allowed values are "system" and "account" for OAuth channel and bot platforms. system means the OAuth app pair comes from the global config keys in [auth]; account means the pair comes from the encrypted per-account app_credentials row. Discord bot accepts only "system"; an "account" override is ignored, and a missing Discord bot app pair makes server invites fail closed.
ENV overrides use the same nested naming convention:
LUMIO__PLATFORM_CREDENTIALS__TWITCH=system
LUMIO__PLATFORM_CREDENTIALS__YOUTUBE=account
LUMIO__PLATFORM_CREDENTIALS__BOT__TWITCH=system
LUMIO__PLATFORM_CREDENTIALS__BOT__YOUTUBE=system
The global system OAuth app pairs live in [auth]:
| TOML path | ENV var | Notes |
|---|---|---|
auth.twitch_channel_client_id | LUMIO__AUTH__TWITCH_CHANNEL_CLIENT_ID | Twitch channel/bot system app; falls back to auth.twitch_client_id when empty |
auth.twitch_channel_client_secret | LUMIO__AUTH__TWITCH_CHANNEL_CLIENT_SECRET | Twitch channel/bot system app secret |
auth.kick_channel_client_id | LUMIO__AUTH__KICK_CHANNEL_CLIENT_ID | Kick channel/bot system app; no login fallback |
auth.kick_channel_client_secret | LUMIO__AUTH__KICK_CHANNEL_CLIENT_SECRET | Kick channel/bot system app secret |
YouTube Chat Transport
YouTube chat uses InnerTube as the primary transport (zero quota cost). gRPC and REST fallbacks are available but disabled by default:
| ENV var | Purpose | Default |
|---|---|---|
LUMIO__YOUTUBE__GRPC_FALLBACK_ENABLED | Enable gRPC streamList fallback when InnerTube fails | false |
LUMIO__YOUTUBE__REST_FALLBACK_ENABLED | Enable REST polling fallback when InnerTube and gRPC both fail | false |
YouTube InnerTube Settings
The InnerTube chat poller ships with sensible defaults - no configuration is required. For operational overrides, these ENV vars map onto the [youtube.innertube_observer] block in apps/api/config/default.toml:
The InnerTube API key and client version are resolved at runtime in this order: override (if set) → Redis cache → fresh youtube.com scrape → cold-boot constant. The scraped values are cached in Redis (keys lumio:yt:innertube_key / lumio:yt:innertube_version, 24 h TTL) and a shared leader-elected refresher re-scrapes every 6 h by default (REFRESH_INTERVAL_SECS), so YouTube's periodic rotations are picked up without a restart. The two *_OVERRIDE vars below are an emergency pin only - leave them empty so auto-rotation stays in charge; a non-empty value wins over every other source, so it must never carry the ordinary current value. If an override is set, the API logs a single WARN the first time the credential is resolved - innertube: rotation disabled by pin (client_version_override=…) - unset to resume auto-rotation - so a forgotten pin never silently disables rotation (the API key value is redacted in that log line).
The *_COLD_BOOT vars are different: they configure the last-resort value used only when override, cache, and scrape have all come up empty (Stage 4). Because they are consulted last they can never pin the credential or disable auto-rotation - they are a fallback seed, not an override. default.toml ships them set to the current cold-boot values so a fresh deployment starts from a known-good credential; the daily drift-check keeps them in lockstep with the compiled constants (DEFAULT_INNERTUBE_API_KEY / DEFAULT_CLIENT_VERSION), which remain the last-ditch fallback if a field is cleared. Clear one (or set an empty ENV var) to fall back to the compiled constant, or set one to a newer value to give a fresh deployment a better cold-start credential without recompiling.
Reaching Stage 4 (cold boot) means Redis was empty and the youtube.com scrape failed - a fault, not a normal state. Point COLD_BOOT_ALERT_WEBHOOK_URL at the same Discord channel the InnerTube health CI (.github/workflows/innertube-health.yml) already posts to (do not create a second channel) and the refresher fires a single alert per incident. COLD_BOOT_ALERT_AFTER_FAILURES (default 1 = immediate) debounces it: raise it to require that many consecutive 6 h cycles at cold boot before alerting. Alerting is fail-open - an unset or unreachable webhook only produces a WARN log and never touches the chat path.
| ENV var | Purpose | Default |
|---|---|---|
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__API_KEY_OVERRIDE | Emergency pin for the InnerTube API key. Empty by default; set only if auto-detection fails after a Google rotation | (empty) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__CLIENT_VERSION_OVERRIDE | Emergency pin for the InnerTube client version. Empty by default; set only if auto-detection fails | (empty) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__API_KEY_COLD_BOOT | Stage-4 cold-boot API key; used only after override/cache/scrape all miss. Absent/empty → compiled DEFAULT_INNERTUBE_API_KEY. Never a pin | (current key, shipped in default.toml) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__CLIENT_VERSION_COLD_BOOT | Stage-4 cold-boot client version; used only after override/cache/scrape all miss. Absent/empty → compiled DEFAULT_CLIENT_VERSION. Never a pin | 2.20260731.00.00 |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__CACHE_TTL_SECONDS | Member + tier-badge entry TTL in Redis | 1209600 (14 d) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__REFRESH_INTERVAL_SECS | How often the shared refresher re-scrapes youtube.com and warms Redis | 21600 (6 h) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__COLD_BOOT_ALERT_WEBHOOK_URL | Discord webhook for the InnerTube health channel; alert on a Stage-4 cold-boot fall-through. Empty → alerting disabled (still WARN-logged). Reuse the CI channel, not a new one | (empty) |
LUMIO__YOUTUBE__INNERTUBE_OBSERVER__COLD_BOOT_ALERT_AFTER_FAILURES | Consecutive cold-boot refresh cycles before alerting; 1 = immediate. 0 treated as 1 | 1 |
api_key_override, api_key_cold_boot, and cold_boot_alert_webhook_url are all wrapped in a SensitiveString and never appear in logs or tracing output.
WebSocket
[websocket] configures the /ws endpoint (see WebSocket API).
| Variable | Purpose | Default |
|---|---|---|
LUMIO__WEBSOCKET__HEARTBEAT_INTERVAL | Seconds between internal heartbeat ticks (credential re-validation and token-expiry checks; the server sends no Ping) | 5 |
LUMIO__WEBSOCKET__CLIENT_TIMEOUT | Seconds without an inbound frame before the session is reaped; also bounds every outbound send | 30 |
LUMIO__WEBSOCKET__MAX_MESSAGE_SIZE | Largest accepted inbound text frame, in bytes | 65536 |
LUMIO__WEBSOCKET__MAX_CHANNELS_PER_SESSION | Channels one session may hold at once | 100 |
LUMIO__WEBSOCKET__SUBSCRIBE_BURST | subscribe/unsubscribe frames a session may send back-to-back before RATE_LIMITED. A value below max_channels_per_session is raised to it at startup, with a warning | 120 |
LUMIO__WEBSOCKET__SUBSCRIBE_REFILL_PER_SECOND | Sustained subscribe/unsubscribe frames per second once the burst is spent. 0 is treated as 1 | 5 |
ECB exchange rates
[fx] controls the API's ECB reference-rate loader for Stream History. It fetches the daily file every six hours and imports the historical ZIP when the stored rate history is short. Disabling it leaves amounts in their source currencies and marks conversion unavailable; it does not block history reads. The base source_url has eurofxref-daily.xml and eurofxref-hist.zip appended. Downloads are size-capped (2 MiB for the daily file, 8 MiB for the history ZIP, 32 MiB for the CSV inside it); a larger response is rejected and retried on the next tick.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__FX__ENABLED | Load ECB reference rates | true |
LUMIO__FX__SOURCE_URL | Base URL of the ECB rate files | https://www.ecb.europa.eu/stats/eurofxref |
LUMIO__FX__REFRESH_INTERVAL_SECS | Daily-file refresh interval in seconds | 21600 |
Stream history command prefixes
[history] command_prefixes sets the prefixes a chat message must start with to count as a command in the Stream History report's command list. The default is ["!", "#"]: ! is the convention of third-party bots (Nightbot, StreamElements, Fossabot), # counts hashtag-style commands. After the prefix, the name must start with a letter ([A-Za-z][A-Za-z0-9_-]{0,31}). The list must include the bot.prefix of Lumio's chat bots: the bots read their prefix from their own service config, which the API does not see. A multi-character prefix is allowed; the longest match wins. Changing the list affects sessions finalized afterwards and reports built from the chat log; frozen reports keep their list. See Stream History → Links and commands.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__HISTORY__COMMAND_PREFIXES | Comma-separated command prefixes | !,# |
Chat Retention
chat_retention_days comes from the account's account_limits override when set, otherwise its plan (Free 365 days; Pro 1095 days). A background sweep hard-deletes older chat messages and events in TimescaleDB; a separate sweep deletes chatter profiles, notes and moderation records in PostgreSQL after the same effective window, capped at 1095 days. A downgrade shortens the window on the next daily sweep. The sweeps are gated so dev/test/staging environments keep their history (see Chat → Retention and Chatter retention).
| Variable | Purpose | Default |
|---|---|---|
LUMIO__CHAT__RETENTION_ENFORCEMENT_ENABLED | Master on/off switch for the retention sweep. Off in default.toml/staging; on in production.toml | false (dev) · true (production) |
LUMIO__CHAT__RETENTION_SWEEP_INTERVAL_SECS | How often the sweep runs. Ignored when enforcement is off; a sweep also runs once on startup | 86400 (24 h) |
LUMIO__CHAT__ORPHAN_SWEEP_ENABLED | Delete the chatter PII (chat, events, platform_users, notes, moderation log) of accounts that no longer exist (Chat → Erasure). Only correct when the TimescaleDB belongs to this database alone | false (dev) · true (production) |
LUMIO__CHAT__ORPHAN_SWEEP_INTERVAL_SECS | How often the orphan sweep runs; a sweep also runs once on startup | 86400 (24 h) |
LUMIO__CHAT__CHATTER_RETENTION_ENABLED | Delete chatters and moderation records (moderation-state chatters, notes, moderation log) after the account's plan window, at most 1095 days (Chat → Chatter retention) | false (dev) · true (production) |
LUMIO__CHAT__CHATTER_RETENTION_INTERVAL_SECS | How often the chatter retention sweep runs; a sweep also runs once on startup. 86400 is the maximum, because a row outlives its window by up to one interval: the API refuses to start with a larger value or 0 | 86400 (24 h) |
Shopify app
The public Lumio Shopify app ([shopify]). Setup in the Shopify Dev Dashboard: OAuth setup → Shopify.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__SHOPIFY__CLIENT_ID | Client ID of the app | empty (Shopify not configured) |
LUMIO__SHOPIFY__CLIENT_SECRET | Client secret; signs the embedded app's ID tokens, the launch query and every webhook, and authenticates the token exchange. ENV only | empty (Shopify not configured) |
LUMIO__SHOPIFY__CLIENT_SECRET_PREVIOUS | The previous client secret while a rotation is in progress; verifiers (ID token, launch query, webhooks) try it after the current one (the metric shopify_client_secret_matches_total{secret="previous"} shows when it is no longer used; steps in Shopify → Client-secret rotation). Must differ from the client secret, or the API refuses to start. ENV only | empty (no rotation) |
LUMIO__SHOPIFY__API_VERSION | GraphQL Admin API version every call pins | 2026-10 |
LUMIO__SHOPIFY__SCOPES | Comma-separated scopes the app needs; the token exchange checks Shopify's scope readback against it (the scopes Shopify grants come from apps/shopify/shopify.app.<env>.toml) | read_orders,read_products,read_legal_policies |
LUMIO__SHOPIFY__APP_STORE_URL | The App Store listing the connect page links to | empty |
The embedded admin page apps/shopify (a Next.js app on its own origin) reads these at request time:
| Variable | Purpose | Default |
|---|---|---|
LUMIO_API_URL | API base including /v1; the page's route handlers forward to /v1/shopify/embedded/* | http://localhost:3000/v1 |
LUMIO_WEB_URL | Lumio web app origin for "In Lumio öffnen" (NEXT_PUBLIC_WEB_URL as fallback) | http://localhost:4000 |
LUMIO__SHOPIFY__CLIENT_ID | Same value as the API: the shopify-api-key meta tag for App Bridge and the theme editor deep link | empty |
LUMIO__SHOPIFY__CLIENT_SECRET / LUMIO__SHOPIFY__CLIENT_SECRET_PREVIOUS | Same values as the API: verify the hmac of Shopify's launch query, which decides frame-ancestors. Without them every page answers frame-ancestors 'none' and cannot be embedded. ENV only | empty |
The web app (apps/web) reads one Shopify value at request time:
| Variable | Purpose | Default |
|---|---|---|
LUMIO_SHOPIFY_APP_HANDLE | The app's handle from apps/shopify/shopify.app.<env>.toml (lumio-staging on staging). The Shopify card's "Open in Shopify" opens https://admin.shopify.com/store/<store>/apps/<handle>. Lowercase letters, digits and -; anything else reads as the default | lumio |
Abuse report retention
Closed copyright abuse reports (status resolved or rejected) hold the reporter's name, e-mail address and reason. A daily worker deletes them once the case has been closed for longer than the configured period; the privacy policy promises three years (§ 195 BGB). Open (pending) reports are never deleted. The worker is always on (a legal maximum has no off switch) and also runs once on startup.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__ABUSE_REPORTS__RETENTION_YEARS | Years a closed report is kept after it was closed (abuse_reports.retention_years). Values below 1 count as 1 | 3 |
Chatter List
The chatter list polls a platform while someone has the panel open (a chatters:\{account_id\} WebSocket subscriber) and — with live_polling — while the stream is live for accounts that opened the list in the last 7 days. Cost of live polling: one Twitch Helix list call per 30 s and one Kick web list call per 60 s per live account that used the list in the last 7 days; nothing for accounts that never use it. [chatters] holds the global settings; provider tables are [chatters.twitch] and [chatters.kick]. Kick uses kick.com’s undocumented active-chatters endpoint without an OAuth token or new scope; it lists recent writers, not lurkers. The ordinary chatter group is capped at 100, with the complete role groups and a full platform count.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__CHATTERS__ENABLED | Global kill-switch for every provider. Off: every platform reports available = false, reason = "disabled", and no poller starts | true |
LUMIO__CHATTERS__INTEREST_TTL_SECS | Lifetime of the per-account interest key. Renewed every 5 s while a subscriber is connected; once it lapses the poller stops | 90 |
LUMIO__CHATTERS__RECENTLY_LEFT_RETENTION_SECS | How long a chatter who left stays listed as present = false (recently left), counted from their last_seen_at; then they are dropped | 1800 |
LUMIO__CHATTERS__WANTED_MARKER_TTL_SECS | Lifetime of the per-account lumio:chatters:wanted:\{account_id\} marker, renewed on every subscription | 604800 (7 days) |
LUMIO__CHATTERS__LIVE_POLLING | Poll marked accounts while their stream is live, even with no panel open. false = panel-driven only (kill switch) | true |
LUMIO__CHATTERS__TWITCH__ENABLED | Twitch provider switch (reason = "disabled" when off) | true |
LUMIO__CHATTERS__TWITCH__POLL_INTERVAL_SECS | Seconds between two Helix snapshots while the list is polled | 30 |
LUMIO__CHATTERS__TWITCH__MAX_CHATTERS | Chatters fetched per snapshot (1000 per Helix page); larger chats report truncated with Twitch's full total | 5000 |
LUMIO__CHATTERS__KICK__ENABLED | Kick provider switch; with a Kick channel connection, off reports reason = "disabled" | true |
LUMIO__CHATTERS__KICK__POLL_INTERVAL_SECS | Seconds between kick.com active-chatters reads; the source caches for 60 s | 60 |
LUMIO__CHATTERS__KICK__MAX_CHATTERS | Lumio’s upper bound per snapshot. Kick itself lists at most 100 ordinary chatters plus its complete role groups, so the platform’s own cap can set truncated first | 5000 |
Automation Worker
The API dispatches extension automation-node handlers to the Automation Worker over HTTP. Configured under [automation_worker] in apps/api/config/default.toml.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__AUTOMATION_WORKER__URL | Base URL of the Automation Worker (apps/automation-worker, no trailing slash) | http://automation-worker:8091 |
LUMIO__AUTOMATION_WORKER__SYSTEM_KEY | System key presented as Authorization: SystemKey <key>; must match the worker's worker.system_key. Empty (the default) disables extension-node dispatch and fails closed | (empty) |
Bot Module Worker (server functions)
The API runs extension handler server functions (query() / mutation() / action(), REST /execute and GraphQL extensionExecuteHandler) on the Bot Module Worker's V8 executor (POST {url}/v1/server-functions/execute). Configured under [bot_module_worker] in apps/api/config/default.toml.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__BOT_MODULE_WORKER__URL | Base URL of the Bot Module Worker (apps/bot-module-worker, no trailing slash) | http://bot-module-worker:8090 |
LUMIO__BOT_MODULE_WORKER__SYSTEM_KEY | System key presented as Authorization: SystemKey <key>; must match the worker's worker.system_key. Empty (the default) turns handler execution off: /execute answers 503 EXECUTOR_UNAVAILABLE | (empty) |
Give the worker a key of its own. Do not reuse an API System key that carries broader grants (for example *:*): the worker runs untrusted extension code, and a key it holds should open nothing on the API beyond what the worker needs.
Automation triggers
Chat, keyword, command, event and timer triggers of automations run only when this switch is on. It is off in every shipped config (default.toml, staging.toml, production.toml). With it off, automations run only when executed manually. Turning it on for staging or production is a founder decision. Configured under [automation] in apps/api/config/default.toml.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__AUTOMATION__TRIGGERS_ENABLED | Starts the leader-elected trigger and timer workers (Redis lease lumio:automation:triggers:leader; one replica fires) | false |
LUMIO__AUTOMATION__MAX_CONCURRENT_EXECUTIONS | Executions of one account that may run at once. A trigger at the cap is dropped and logged | 8 |
LUMIO__AUTOMATION__EXECUTION_TIMEOUT_SECS | Wall-clock limit for one triggered execution. Keep it above the 1-hour Delay cap | 3900 |
Bot outbound throttle
Every tenant shares the one Twitch bot connection, so the Twitch bot throttles its
outgoing messages with a token bucket and gives each built-in command a per-channel
cooldown. Configured under [bot] in apps/twitch-bot/config/default.toml. A send
that would have to wait more than 2 s for a token is dropped and logged as
Outbound rate limit reached.
| Variable | Purpose | Default |
|---|---|---|
TWITCH_BOT__BOT__OUTBOUND_MESSAGES_PER_30S | Sustained outbound budget in messages per 30 s, across all channels. The default is Twitch's per-30 s limit for an account that is moderator in the channel; raise it only for a Twitch-verified bot account | 100 |
TWITCH_BOT__BOT__OUTBOUND_BURST | Messages that may go out back-to-back before the sustained rate applies | 20 |
TWITCH_BOT__BOT__BUILTIN_COOLDOWN_SECS | Per-channel cooldown of each built-in command (!song, !np, !uptime, !commands); 0 turns it off. The Kick and YouTube bots use a fixed 5 s | 5 |
Handler-execution telemetry (workers)
Both handler-execution workers (apps/bot-module-worker, apps/automation-worker) optionally record per-execution telemetry into TimescaleDB - one extension_errors row on failure and one extension_install_logs row on every run (see Developer Extension Endpoints). The writer is fail-open: it never fails or delays a handler request, and is simply disabled when no TimescaleDB pool is configured. Env vars use each worker's own prefix - BOT_MODULE_WORKER__… and AUTOMATION_WORKER__….
| Variable | Purpose | Default |
|---|---|---|
…__TIMESCALE__URL | TimescaleDB connection URL. Omit the whole [timescale] section to disable the telemetry writer | (unset → writer off) |
…__WORKER__TELEMETRY_SALT | Per-deployment secret salt used to HMAC-anonymize account/install IDs in extension_errors. Set a stable value in production; an empty salt still hashes but is weaker | (empty) |
Worker registry sync
Each worker (apps/bot-module-worker, apps/automation-worker) populates its in-memory InstallRegistry from the API's internal handler-bundle feed on startup and re-syncs on a poll - see Bot Modules → Worker sync-loop. Configured under [sync], with the env prefix BOT_MODULE_WORKER__SYNC__* and AUTOMATION_WORKER__SYNC__* respectively. When the section is not fully configured the worker starts with an empty registry (every request short-circuits) rather than failing.
| Variable (per worker prefix) | Purpose | Default |
|---|---|---|
…__SYNC__ENABLED | Run the registry sync loop | true |
…__SYNC__API_BASE_URL | API origin the feed is served from (no trailing slash), e.g. http://api:8080. Empty disables the loop | (empty) |
…__SYNC__SYSTEM_KEY | System key presented to the feed as Authorization: Bearer <key>. Empty disables the loop | (empty) |
…__SYNC__POLL_INTERVAL_SECS | Full-snapshot re-sync interval. The Bot Module Worker applies install changes and the kill switch live over [redis], so its poll is a safety net; without [redis] it polls at most every 30 s | Bot Module Worker 300, Automation Worker 30 |
Bot-module secrets key
Bot-module handlers read their extension's secrets with ctx.secrets.get (see Bot Modules → V8 Execution). The API re-encrypts each value for the Bot Module Worker under a key that only these two services share, so the worker never holds auth.token_encryption_key. Set the same independent random value (openssl rand -base64 32) on both sides:
| Variable | Service | Purpose | Default |
|---|---|---|---|
LUMIO__EXTENSIONS__BOT_MODULE_SECRETS_KEY | API | Wraps secrets for the worker's internal secrets feed. Empty turns ctx.secrets off (the feed answers 503). A placeholder or a value under 16 characters aborts the boot outside development | (empty) |
BOT_MODULE_WORKER__SYNC__SECRETS_KEY | Bot Module Worker | Opens the wrapped values. Also needs …__SYNC__API_BASE_URL and …__SYNC__SYSTEM_KEY. Empty turns ctx.secrets off (calls throw ctx.secrets is unavailable); a placeholder or a value under 16 characters aborts the start | (empty) |
Never reuse auth.token_encryption_key here. To rotate, restart the API and every worker with the new value together; expect up to 60 seconds of ctx.secrets is unavailable. Nothing stored is re-encrypted.
Bot Module Worker ctx.db
Bot-module handlers read and write their extension's tables through the worker's own pool on the extensions database (the API's [extensions.database], lumio_extensions). Each op runs through the same executor as the API's server functions, under the same row-level security; see Bot Modules. The API applies the extensions-database migrations (apps/api/migrations/extensions/) itself at startup, on its DDL pool. The worker only ever gets a DML login (dev stack: lumio_ext_worker), never the schema owner. At startup it refuses to run if that login is a superuser, has BYPASSRLS, or has CREATE or TEMPORARY on the database; if the database is unreachable at startup, ctx.db stays off until the next restart.
# dev stack
BOT_MODULE_WORKER__EXTENSIONS__DATABASE__URL=postgres://lumio_ext_worker:lumio_ext_worker@localhost:5434/lumio_extensions
| Variable | Purpose | Default |
|---|---|---|
BOT_MODULE_WORKER__EXTENSIONS__DATABASE__URL | Extensions database URL as a DML login of the lumio_ext_dml group (its own login, not the API's). Empty turns ctx.db off (calls throw ctx.db is unavailable) | (empty) |
BOT_MODULE_WORKER__EXTENSIONS__DATABASE__MAX_CONNECTIONS | Connections per worker replica. An op waits at most 100 ms (sync) / 1 s (async) for one, then throws ctx.db: db busy. Keep 20 × API replicas + 5 × worker replicas plus headroom under the extensions Postgres max_connections | 5 |
Bot Module Worker chat triggers
The Bot Module Worker runs extension keyword and pattern triggers that the bots publish on Redis (lumio:botmod:msg:*) and posts each handler's reply back through the bots - see Bot Modules → Bot Module Worker. It needs the same Redis instance as the API and the bots. Without a [redis] section the listener does not start and keyword/pattern triggers do nothing; command triggers (HTTP) are unaffected.
| Variable | Purpose | Default |
|---|---|---|
BOT_MODULE_WORKER__REDIS__URL | Redis URL (same instance as the API and the bots). Remove the [redis] section to run without chat triggers | redis://:lumio@127.0.0.1:6379 |
BOT_MODULE_WORKER__CHAT_TRIGGERS__ENABLED | Run the keyword/pattern trigger listener | true |
BOT_MODULE_WORKER__CHAT_TRIGGERS__MAX_IN_FLIGHT | Handlers that may run at the same time; a trigger that arrives while all are busy is dropped with a warning | 32 |
Bot Module Worker event triggers and timers
The Bot Module Worker also runs bot-module event handlers for the stream events the API publishes on lumio:events:*, and bot-module timers - see Bot Modules → Bot Module Worker. Both need [redis]; timers also need a configured [sync] section, because the timer list comes from the handler feed. Each event and each timer slot is claimed in Redis, so running several worker replicas runs each one once.
| Variable | Purpose | Default |
|---|---|---|
BOT_MODULE_WORKER__EVENT_TRIGGERS__ENABLED | Deliver stream events to bot modules that list them in triggers.events | true |
BOT_MODULE_WORKER__EVENT_TRIGGERS__MAX_IN_FLIGHT | Event handlers that may run at the same time; a delivery that arrives while all are busy is dropped with a warning | 32 |
BOT_MODULE_WORKER__SERVER_FUNCTIONS__MAX_IN_FLIGHT | Server-function handlers (posted by the API) that may run at the same time on this replica; a call that finds every slot busy is refused with RATE_LIMITED | 16 |
BOT_MODULE_WORKER__SERVER_FUNCTIONS__MAX_IN_FLIGHT_PER_ACCOUNT | Of those, the handlers one account may run at the same time | 4 |
BOT_MODULE_WORKER__TIMERS__ENABLED | Run bot-module timers (triggers.timers / triggers.timer) | true |
BOT_MODULE_WORKER__TIMERS__MAX_IN_FLIGHT | Timer handlers that may run at the same time; a firing while all are busy is skipped | 32 |
Bot-module handler actions
The actions a bot-module handler returns are run by the API, not by the worker or the bots: they are posted to the internal POST /v1/internal/bot-modules/{install_id}/actions endpoint. The Bot Module Worker posts them with its [sync] api_base_url and system_key (without both, its actions are dropped with a warning); the bots use their [api] settings. The key needs the bots:internal grant.
| Variable | Service | Purpose | Default |
|---|---|---|---|
LUMIO__EXTENSIONS__BOT_MODULE_ACTION_RATE_PER_MINUTE | API | Most actions one bot-module install may run per minute. Over the cap an action is refused and logged once per minute; if Redis cannot count, actions are refused. 0 means the default | 120 |
GeoIP
The optional GeoIP service resolves client IP addresses to a country/city using the free MaxMind GeoLite2 City database. It powers the Location column of the Audit Log. It is disabled by default - no MaxMind account is needed to run Lumio; leave enabled = false and the location fields simply stay empty.
When enabled with a license key, the API downloads the .mmdb database on startup if it is missing and, when auto_update is on, refreshes it every update_interval_days. All GeoIP state lives under [geoip] in apps/api/config/default.toml; every key is overridable via the LUMIO__GEOIP__* env vars below.
| Variable | Purpose | Default |
|---|---|---|
LUMIO__GEOIP__ENABLED | Master on/off switch. When false, no database is loaded and location fields stay empty | false |
LUMIO__GEOIP__DATABASE_PATH | Path to the GeoLite2 City .mmdb file. The default lives under the container WORKDIR (/app); the runtime-base image pre-creates /app/data/geoip owned by uid 1001, so the file is downloadable and refreshable by the non-root user. Mount a persistent volume there to survive restarts | /app/data/geoip/GeoLite2-City.mmdb |
LUMIO__GEOIP__LICENSE_KEY | MaxMind license key used to download/update the database. Get a free key. Without it, an existing database is still read but cannot be auto-downloaded | (empty) |
LUMIO__GEOIP__ACCOUNT_ID | MaxMind numeric account ID. Required (with the license key) for the Privacy Exclusions API | (empty) |
LUMIO__GEOIP__AUTO_UPDATE | Re-download the database on a schedule when it is older than update_interval_days | false |
LUMIO__GEOIP__UPDATE_INTERVAL_DAYS | Minimum database age before a re-download. MaxMind publishes GeoLite2 weekly | 7 |
LUMIO__GEOIP__PRIVACY_EXCLUSIONS_ENABLED | Honour MaxMind Privacy Exclusions (user geolocation opt-outs). Requires license_key + account_id | false |
LUMIO__GEOIP__PRIVACY_EXCLUSIONS_REFRESH_HOURS | How often the cached exclusions list is refreshed | 24 |
LUMIO__GEOIP__PRIVACY_EXCLUSIONS_CACHE_PATH | Where the exclusions list is cached for persistence across restarts. Lives on the same writable /app/data/geoip directory as the database | /app/data/geoip/geoip-privacy-exclusions.json |
For a one-off database refresh outside the server, run the geoip-update maintenance binary: cargo run -p lumio-api --bin geoip-update -- --help.
Privacy. GeoIP derives only a coarse country/city from the IP - no precise coordinates are stored. Enabling
privacy_exclusions_enabledmakes Lumio respect MaxMind's Privacy Exclusions opt-out list so IPs of users who opted out are not resolved.
Diagnostics file (Rust services)
Every Rust service can write a selected subset of its logs to a local rotating file, in addition to stdout. By default the subset is every warning and error of the service (all_warnings), plus the ingest_diag::<platform> capture lines for event types ingest does not understand yet. Payment-webhook logs are always excluded. The file's selection is independent of [logging] level and RUST_LOG. Details on the selection, redaction and read-back are in Logging → Rust services: diagnostics file.
Each key below is set under [logging.diagnostics] in the service's config/ or through ENV. Replace <PREFIX> with LUMIO (api), TWITCH_BOT, KICK_BOT, YOUTUBE_BOT, DISCORD_BOT, CRAWLER_TWITCH, BOT_MODULE_WORKER or AUTOMATION_WORKER.
| Variable | Purpose | Default |
|---|---|---|
<PREFIX>__LOGGING__DIAGNOSTICS__ENABLED | Master switch. While off, no file and no directory is created. On in staging.toml for api, twitch-bot, kick-bot and youtube-bot, and in the api's production.toml | false |
<PREFIX>__LOGGING__DIAGNOSTICS__DIR | Target directory, relative to the working directory. In containers this is /app/data/diagnostics, which the runtime-base image creates (owned by uid 1001). Mount a host volume there | data/diagnostics |
<PREFIX>__LOGGING__DIAGNOSTICS__ALL_WARNINGS | Select every warn and error line of every component (auth, database, billing, routes, workers, ingest). Off: only the platform target sets below | true |
<PREFIX>__LOGGING__DIAGNOSTICS__TWITCH | Add the Twitch target set (IRC, Helix, EventSub, IRC tags, twitch-bot) | true |
<PREFIX>__LOGGING__DIAGNOSTICS__KICK | Add the Kick target set (API client, kick* workers, webhook route and event service, kick-bot) | true |
<PREFIX>__LOGGING__DIAGNOSTICS__YOUTUBE | Add the YouTube target set (InnerTube/gRPC client, YouTube and InnerTube-credential workers, webhook route, youtube-bot) | true |
<PREFIX>__LOGGING__DIAGNOSTICS__FILTER | Extra EnvFilter directives, merged after the platform sets. A directive for a target already in a set replaces it, and a bare level (error) replaces the all_warnings default. One comma-separated string, e.g. twitch_irc=off,lo_kick_api=debug | (empty) |
<PREFIX>__LOGGING__DIAGNOSTICS__ROTATION | daily or hourly. Boundaries are UTC | daily |
<PREFIX>__LOGGING__DIAGNOSTICS__MAX_FILES | Rotated files kept per service. This is the retention bound: 14 means 14 days with daily rotation. Must be at least 1 | 14 |
<PREFIX>__LOGGING__DIAGNOSTICS__FORMAT | json (one object per line, .jsonl) or compact (one text line, .log) | json |
<PREFIX>__LOGGING__DIAGNOSTICS__REDACT_USER_FIELDS | Replace viewer names and message text with [redacted] in the file. Stdout is unaffected | false |
Files are named <dir>/<service>.<YYYY-MM-DD>.jsonl (api, twitch-bot, …), so several services can share one directory. A misconfiguration never stops the service: it logs one warning and keeps running on stdout only.
Hosted deployments. Mount a persistent host directory at /app/data/diagnostics; otherwise the file disappears with the container. Create that directory owned by uid 1001 before the first start (sudo install -d -o 1001 -g 1001 <host-dir>). The staging compose files mount ${DIAGNOSTICS_DIR:-./data/diagnostics} (see dev-stack/staging/README.md). On the production host, add the same volumes: line to each service that enables the file:
volumes:
- ${DIAGNOSTICS_DIR:-./data/diagnostics}:/app/data/diagnostics
Error tracking (Rust services)
Every Rust service sends errors and warnings to Sentry through lo-telemetry when [sentry] is enabled and has a DSN. Each event carries release = <service>@<version>, the service and target tags, and a fingerprint of target plus message template. Viewer names and message text are removed before sending. Details: Error Tracking → Rust backend.
Each key below is set under [sentry] in the service's config/ or through ENV, with the same <PREFIX> as above. Only these keys enable Sentry: a bare SENTRY_DSN is not read by these services.
| Variable | Purpose | Default |
|---|---|---|
<PREFIX>__SENTRY__ENABLED | Master switch. Needs a DSN as well | false |
<PREFIX>__SENTRY__DSN | DSN from the Sentry project settings. An unparseable DSN leaves Sentry off | (empty) |
<PREFIX>__SENTRY__ENVIRONMENT | environment of every event | development |
<PREFIX>__SENTRY__TRACES_SAMPLE_RATE | Performance trace sample rate, clamped to 0.0–1.0 | 1.0 (0.1 for crawler-twitch) |
<PREFIX>__SENTRY__SAMPLE_RATE | Event sample rate, clamped to 0.0–1.0 | 1.0 |
<PREFIX>__SENTRY__DEBUG | Sentry SDK debug output | false |
<PREFIX>__SENTRY__EVENT_LEVEL | Lowest log level that becomes an event: warn or error. Lower levels down to info become breadcrumbs. Any other value logs a warning and uses warn | warn |
<PREFIX>__SENTRY__MAX_EVENTS_PER_FINGERPRINT | Volume guard: events with the same fingerprint sent per window. The rest is counted and reported as suppressed_count on the next event. 0 switches the guard off | 10 |
<PREFIX>__SENTRY__FINGERPRINT_WINDOW_SECS | Volume guard window in seconds | 60 |
innertube-proxy has no TOML config. It enables Sentry from SENTRY_DSN and SENTRY_ENVIRONMENT (default production) and installs the same lo-telemetry tracing layers as the other services, so its warn/error logs become Sentry events. See InnerTube Proxy → Logging and error tracking.
Frontend Environment & Logging
apps/web, apps/admin, and apps/id share two env-vars that drive the logger and the Sentry integration. Set both the server-side and the NEXT_PUBLIC_* mirror so the browser bundle picks them up too.
| Variable | Allowed values | Default | Purpose |
|---|---|---|---|
LUMIO_ENV / NEXT_PUBLIC_LUMIO_ENV | development | staging | production | development | Logger threshold default + Sentry environment tag |
LUMIO_LOG_LEVEL / NEXT_PUBLIC_LUMIO_LOG_LEVEL | debug | info | warn | error | per env (debug/info/warn) | Override the auto-threshold |
Sentry env-vars per app (apps/web, apps/admin, apps/id, and apps/shopify on the server only) - leave blank locally to disable, fill them in the CI build / deployment environment for staging + production:
| Variable | Where | Purpose |
|---|---|---|
LUMIO_PUBLIC_SENTRY_DSN | Run time (browser + server) | Public DSN. Set this on deployed containers: the root layout hands it to the browser SDK per request (web, ID; after Functional consent) |
NEXT_PUBLIC_SENTRY_DSN | Build time | Local-dev fallback only - frozen at next build, no effect on a prebuilt image |
SENTRY_DSN | Server only | Optional override; falls back to LUMIO_PUBLIC_SENTRY_DSN, then NEXT_PUBLIC_SENTRY_DSN. apps/admin and apps/shopify run no browser SDK, so only this (or its fallbacks) applies there |
SENTRY_ORG | Build (CI) | Sentry org slug for source-map upload |
SENTRY_PROJECT | Build (CI) | Sentry project slug |
SENTRY_AUTH_TOKEN | Build (CI) | Auth token for source-map upload |
Each app also has a proxy.ts (Next.js 16's renamed middleware.ts) that prints one structured line per HTTP request to the SSR terminal - including the real client IP (x-real-ip / x-forwarded-for / cf-connecting-ip) with IPv4/IPv6 family classification. See Logging → Per-request access logging for details.
Verify Installation
just stack-up- all six containers report healthy.just migrate-status- every migration shows as applied.just run-api- the API answers onhttp://localhost:3000/v1.GET /v1/graphiqlserves the GraphiQL IDE andGET /v1/schemathe GraphQL SDL (both are on by default indevelopmentand gated by[graphql].playground/[graphql].schema_sdl). The REST surface is documented at/v1/swagger-ui/with the raw spec at/v1/openapi.json(both gated by[docs].swagger_ui/[docs].openapi_json; production turns the Swagger UI off).just dev-web- visithttp://localhost:4000and log in through the ID app onhttp://localhost:4002.
Before you push
| Recipe | What it runs |
|---|---|
just verify-all | cargo fmt --check + clippy (-D warnings) + cargo check + ESLint + tsc - the fast inner loop |
just lint-all | clippy + ESLint only |
just test-all | cargo test (workspace) + the V8 extension tests + Vitest |
just verify-release | verify-all plus regenerating the OpenAPI/GraphQL schemas and the Bazel vendor tree, failing on drift. Run this after a version bump or a Cargo.lock change - verify-all does not detect that drift |
Passkey relying party configuration
The API's [webauthn] settings bind passkeys to the web and ID apps. rp_id is the shared relying-party domain, rp_name is the name shown by an authenticator, and origins is the exact list of browser origins allowed to register or use a passkey. The default local values are:
[webauthn]
rp_id = "localhost"
rp_name = "Lumio"
origins = ["http://localhost:4000", "http://localhost:4002"]
Staging uses rp_id = "staging.zaflun.dev" with https://lumio.web.staging.zaflun.dev and https://lumio.id.staging.zaflun.dev. Production uses rp_id = "lumio.vision" with https://lumio.vision and https://id.lumio.vision. The API checks origins exactly: do not include overlay, widget, admin or extension hosts. Environment overrides are LUMIO__WEBAUTHN__RP_ID and LUMIO__WEBAUTHN__ORIGINS (comma-separated, e.g. https://lumio.vision,https://id.lumio.vision); rp_name is also a TOML setting. Never change rp_id after the first real passkey exists. Existing passkeys are bound to that ID and would stop working.