Logging

Lumio's frontend apps share a single env-aware structured logger from the @lumio/logger workspace package. It wraps console.* so log lines land in the SSR terminal and the browser DevTools console alike, and forwards error / warn events to Sentry when an integration is wired up. The same package also ships a ready-made Next.js request-logging helper.

The Rust API and bots stay on tracing + lo-telemetry; this logger is frontend-only. For the Rust side's selected ingest-error file, see Rust services: diagnostics file.

Package layout

File	Purpose
`shared/logger/src/index.ts`	Default logger (`logger`), factory (`createLogger`), Sentry binding (`bindSentry`).
`shared/logger/src/middleware.ts`	`createRequestLoggerMiddleware` - a self-contained request logger for a `proxy.ts` that has no other work to do.

The package exports both entry points:

import { logger, createLogger, bindSentry } from "@lumio/logger";
import { createRequestLoggerMiddleware } from "@lumio/logger/middleware";

Environment

The logger reads two env vars to decide what to print:

Variable	Server	Browser	Allowed values	Default
`LUMIO_ENV` / `NEXT_PUBLIC_LUMIO_ENV`	yes	yes	`development` \| `staging` \| `production`	`development`
`LUMIO_LOG_LEVEL` / `NEXT_PUBLIC_LUMIO_LOG_LEVEL`	yes	yes	`debug` \| `info` \| `warn` \| `error`	per env (see below)

NEXT_PUBLIC_* wins over the bare name in both cases (it is the only form the browser bundle can see, and Next.js inlines it at build time). If neither env var is set, resolveEnv() falls back to NODE_ENV, and to development if that is not one of the three recognised values. An unrecognised *_LOG_LEVEL value is ignored rather than erroring.

Default thresholds when *_LOG_LEVEL is absent:

development → debug (everything)
staging → info (no debug)
production → warn (only warnings + errors)

LUMIO_ENV also tags Sentry events so staging and production stay on separate dashboards.

Levels

logger.debug("...", context);
logger.info("...", context);
logger.warn("...", context);            // → Sentry capture-message
logger.error("...", error, context);    // → Sentry capture-exception

error always logs regardless of the configured threshold - errors signal real user-visible problems and must not be silenced. The other three respect the threshold.

context is an arbitrary Record<string, unknown> that gets JSON-serialised and appended to the line. It also flows through to Sentry as the extra payload.

Named loggers

Use createLogger({ name }) or logger.child(scope) to prefix every line with a [scope] tag. Useful for tracing which component produced which log.

const log = logger.child("chat-page");
log.info("Loading chat", { userId });
// → [chat-page] Loading chat {"userId":"…"}

const sender = log.child("sender");
sender.error("Failed to send", err, { platform: "twitch" });
// → [chat-page:sender] Failed to send {"platform":"twitch"} <err>

Per-request access logging (`proxy.ts`)

apps/{web,admin,id}/src/proxy.ts (Next 16's file rename of middleware.ts) each own auth, token refresh, and request logging in one proxy() function, so none of them mounts createRequestLoggerMiddleware - they build a named logger with createLogger({ name: "web" }) and emit the access line themselves at the top of proxy():

const log = createLogger({ name: "web" });

export async function proxy(request: NextRequest) {
  const { pathname } = request.nextUrl;
  const ip = resolveClientIp(request);

  log.info(`${request.method} ${pathname}`, {
    ip,
    user_agent: request.headers.get("user-agent") ?? "-",
    elapsed_ms: 0,
  });
  // … auth / refresh / popout-token handling
}

Output:

[web] GET /dashboard/chat {"ip":"91.99.42.7","user_agent":"Mozilla/5.0…","elapsed_ms":0}

The line is emitted before the request is handled, so elapsed_ms is always 0 - treat it as an access log, not a latency metric.

Resolution order for the client IP (identical in the apps and in createRequestLoggerMiddleware):

cf-connecting-ip (Cloudflare proxy)
x-real-ip (Caddy / nginx / Hetzner LB)
First entry of x-forwarded-for
request.ip (only some runtimes expose it)
"unknown"

createRequestLoggerMiddleware additionally classifies the address as ip_family: ipv4, ipv6, or unknown, with IPv4-mapped IPv6 (::ffff:1.2.3.4) counted as ipv4. The apps' own inline logging does not emit that field.

All three apps share the same matcher, which skips Next internals and static image assets so hot-reload chatter doesn't drown the log:

export const config = {
  matcher: [
    "/((?!_next/static|_next/image|favicon.ico|.*\\.(?:svg|png|jpg|jpeg|gif|webp|ico)$).*)",
  ],
};

Note - Next.js 16 expects the export to be named proxy, not middleware, and refuses to boot if a middleware.ts sits alongside it. Fold request-interception logic into proxy.ts rather than adding a second file.

Sentry binding

bindSentry() wires the logger to a Sentry-compatible binding. Each Next.js app does this once at boot - server-side via src/instrumentation.ts, browser-side via sentry.client.config.ts. After binding, logger.error and logger.warn automatically forward to Sentry.captureException / Sentry.captureMessage (translating warn → Sentry's warning severity along the way).

In the browser the binding is consent-gated: Sentry only initializes once the user has granted the Functional cookie category, and bindSentry(null) unbinds it when consent is withdrawn. Server-side binding is unconditional.

The binding is optional - without a DSN, the logger still works as a plain console logger. See Error Tracking for the full Sentry setup.

Best practices

Never log secrets. Tokens, passwords, OAuth secrets, session cookies - none of them should land in context. Sentry's beforeSend strips Authorization / Cookie headers, but the context object is up to the caller.
One scope per concern. Pick a stable name per component / route so log filtering downstream (Loki, Grafana, Sentry) groups cleanly. Avoid stamp-of-the-day dynamic prefixes.
Errors as Error objects. Prefer throw new Error("msg") and logger.error("Failed", err, ctx) over stringifying - Sentry's stack-frame symbolication relies on the Error instance.
debug is debug-only. Production threshold defaults to warn; anything you put in debug will silently disappear in prod, which is the point. Don't put critical state-tracking there.

Verifying it works

Local dev:

just dev-web
# in another terminal, hit a page:
curl http://localhost:4000/
# → terminal shows: [web] GET / {"ip":"127.0.0.1","user_agent":"curl/…","elapsed_ms":0}

Throw a test error inside a Server Action / route handler:

import { logger } from "@lumio/logger";

export async function POST() {
  try {
    throw new Error("test");
  } catch (err) {
    logger.error("test handler failed", err, { hint: "expected" });
    return new Response("ok", { status: 200 });
  }
}

Terminal shows the error + stack; if a Sentry DSN is configured (SENTRY_DSN or LUMIO_PUBLIC_SENTRY_DSN), the event also lands in Sentry under the matching environment tag.

Rust services: diagnostics file

Every Rust service sets up logging through lo_telemetry::init_tracing(&config.logging, "<service>"): api (plus api-migrate / api-geoip-update), twitch-bot, kick-bot, youtube-bot, discord-bot, crawler-twitch, bot-module-worker and automation-worker. The function installs these tracing layers, and each one has its own filter:

Layer	Filter	Output
stdout (`[logging] format`: `json` \| `pretty`)	`RUST_LOG`, else `[logging] level`	container stdout
Sentry (`BreadcrumbLayer`, `ErrorLayer`, `SpanLayer`)	the same filter as stdout; then `[sentry] event_level` decides event vs. breadcrumb, and spans from `info` up become transactions	Sentry, when `[sentry]` is configured (see Error Tracking → Rust backend)
diagnostics file (only when `[logging.diagnostics] enabled`)	the diagnostics selection below	`<dir>/<service>.<date>.jsonl`

Because each layer filters independently, the file can capture a line that stdout drops. For example, production runs level = "warn", but the info capture lines below still reach the file. RUST_LOG does not affect the file.

Selection

With all_warnings = true (the default) the directive starts with a bare warn default, so every warn and error line of every component lands in the file: auth, database, cache, billing, routes, WebSocket, workers and ingest. Set all_warnings = false to go back to the platform sets alone.

On top of that, the directive is built from three per-platform target sets. Each target is selected at warn, and each platform's capture target at info:

Toggle	Targets (`warn`)	Capture target (`info`)
`twitch`	`twitch_irc`, `lo_twitch_irc`, `lo_twitch_api`, `api::workers::twitch_eventsub`, `api::workers::twitch_irc_tags`, `twitch_bot`	`ingest_diag::twitch`
`kick`	`lo_kick_api`, `api::workers::kick` (prefix, so it covers every `kick_*` worker), `api::services::kick_webhook_events`, `api::routes::webhooks`, `kick_bot`	`ingest_diag::kick`
`youtube`	`lo_youtube_api`, `api::workers::youtube`, `api::workers::innertube_credentials`, `api::routes::webhooks`, `youtube_bot`	`ingest_diag::youtube`

Target matching is prefix-based (starts_with). api::routes::webhooks would therefore also match api::routes::webhooks_payments, and all_warnings selects every target anyway. Whenever all_warnings, Kick or YouTube is on, the selection adds api::routes::webhooks_payments=off. Payment-webhook logs carry billing data and never reach the file.

filter is a comma-separated EnvFilter directive string that is merged after the platform sets. A directive whose target a platform set already names replaces that entry, a bare level (error, info, …) replaces the all_warnings default, and any other directive is appended. The merge happens by target in code, not by string concatenation, so precedence is deterministic. The toggles switch a whole platform on or off; filter fine-tunes. Examples:

twitch_irc=off: drops the twitch-irc library warnings and keeps the rest of the Twitch set.
lo_kick_api=debug: raises one target's level.
error: only errors from every component (plus the platform sets and capture lines).
ingest_diag=info with all_warnings and all three toggles off: only the capture lines.

The `ingest_diag::<platform>` convention

A log line that exists to capture the payload of something ingest does not understand yet uses the stable target ingest_diag::<platform> instead of its module path. The filter then selects it by intent, not by file location. Each line fires on the first sighting per process of a given type. The set is capped at 64 types, and lines are never lost to the cap. Repeats log at debug on the module target, or not at all.

Platform	Line	Fields
Twitch	Unrecognized `channel.chat.notification` `notice_type`: one that is neither mapped nor in the worker's list of types Twitch documents (`DOCUMENTED_UNMAPPED_NOTICE_TYPES`). Twitch's catch-all `unknown` is logged (`api::workers::twitch_eventsub`)	`notice_type`, `event` (raw EventSub event)
Kick	Unhandled webhook event type (`api::services::kick_webhook_events`). The module-level `warn!` stays the stdout signal	`event_type`, `data` (raw event data)
YouTube	InnerTube chat item whose renderer no typed field matched (`lo_youtube_api::innertube::parser`). `ChatItem::other` (`#[serde(flatten)]`) keeps the unknown renderer key	`renderer`, `item` (renderer body)

When you add a new "unknown type, dropping" path, give it the same treatment: a bounded first-sighting set plus info!(target: "ingest_diag::<platform>", …).

Keys

All keys live under [logging.diagnostics], in every service's config/default.toml. Each key can be set from TOML or ENV (<PREFIX>__LOGGING__DIAGNOSTICS__<KEY>, prefixes listed in Installation).

Key	Default	Meaning
`enabled`	`false`	Master switch. While off, no file and no directory is created.
`dir`	`data/diagnostics`	Target directory, relative to the working directory (`/app` in containers).
`all_warnings`	`true`	Select every `warn`/`error` line of every target (bare `warn` default).
`twitch` / `kick` / `youtube`	`true`	Add that platform's target set.
`filter`	`""`	Extra `EnvFilter` directives (see above). A single scalar string, never a list key.
`rotation`	`daily`	`daily` (`<service>.YYYY-MM-DD.jsonl`) or `hourly` (`<service>.YYYY-MM-DD-HH.jsonl`). Rotation boundaries are UTC.
`max_files`	`14`	Rotated files kept per service, which is the retention bound (14 days with daily rotation). Must be ≥ 1. Old files are pruned whenever a new file is opened, so a service that is switched off leaves its last files on disk until they are removed by hand.
`format`	`json`	`json`: one `fmt::layer().json()` object per line, `.jsonl`. `compact`: one text line, `.log`.
`redact_user_fields`	`false`	Replaces viewer names and message text with `"[redacted]"` in the file only.

The writer is tracing-appender behind a non-blocking, lossy channel. A stalled disk drops diagnostics lines instead of blocking a Tokio worker. The TelemetryGuard returned by init_tracing flushes the channel when it is dropped, so hold it for the whole process (let _telemetry = …).

Redaction

With redact_user_fields = true, the file layer redacts:

Top-level fields whose key names a person or what they wrote: keys ending in user_name, user_login, display_name or author_name, plus username, login, slug, sender, author, text, content, chat_message, message_text, system_message. Matching ignores case, _ and -. tracing's own message field (the log text) is exempt.
JSON documents logged as a field (event = %json), redacted recursively. Under a matching key (here message counts too), every string leaf becomes "[redacted]", except id, *_id and type.

IDs, event types, numbers, unknown keys and the document structure stay, so "what does this new event look like" still works. Stdout is never redacted.

Fail-open

The service never refuses to start because of the diagnostics file. Each of these cases logs one warn!, Diagnostics file disabled, continuing on stdout only: <reason>, naming the key, and the service runs with stdout and Sentry only:

dir is not writable
filter does not parse
rotation or format has an unknown value
max_files = 0
the selection is empty (all_warnings and all three platforms off, filter empty)

Where it runs and how to read it

On staging, api, twitch-bot, kick-bot and youtube-bot enable the file in their staging.toml, and api also in its production.toml. The bots have no production.toml, so production enables them through <PREFIX>__LOGGING__DIAGNOSTICS__ENABLED=true. The image creates /app/data/diagnostics, owned by uid 1001. Every staging compose file bind-mounts ${DIAGNOSTICS_DIR:-./data/diagnostics} there; without the mount the file would disappear with the container. The host directory must belong to uid 1001 (sudo install -d -o 1001 -g 1001 data/diagnostics; see dev-stack/staging/README.md).

jq -c 'select(.target | startswith("ingest_diag"))' data/diagnostics/*.jsonl
jq -c 'select(.level == "ERROR")' data/diagnostics/kick-bot.*.jsonl

The diagnostics file is telemetry. It emits no audit event and needs no permission.

Package layout​

Environment​

Levels​

Named loggers​

Per-request access logging (proxy.ts)​

Sentry binding​

Best practices​

Verifying it works​

Rust services: diagnostics file​

Selection​

The ingest_diag::<platform> convention​

Keys​

Redaction​

Fail-open​

Where it runs and how to read it​