Running mechabedo as a server

Beam has the four shapes a server-side installation takes and how to start each one. This page is what happens after that: the volume, capacity, what to watch, and getting it back when the machine does not.

Everything here assumes vendor mode, which is what a server-shaped installation runs: authentication is mandatory and one config document is the deployment's interface.

The volume is the installation

An image is disposable. /data is not, and everything mutable is under it:

Drawing the diagram…

Path	Contents	Losing it costs
/data/config	mechabedo.config.jsonc, the deployment's interface	Nothing; you wrote it
/data/settings	server.jsonc, the signing secret, the installation id, the enhance catalog	Every session gets a new login, and the beam slot reads this as a different machine
/data/home/.mechabedo/mainy/data/pglite	The database: conversations, transcripts, memory, config	The archive
/data/home	$HOME: the above, plus mise, the harnesses, their own config and the workspaces	Everything not in /data/settings

The image sets MECHABEDO_SETTINGS_DIR=/data/settings and HOME=/data/home, and leaves the data directory to the default, which is why the database ends up under $HOME rather than beside the settings. A container holds one profile, always mainy: profile switching is a desktop feature, and a deployment's state is its volume.

The database and the settings directory are one unit. Restoring half of them produces a state that never existed: a database whose sessions reference a signing secret that is gone, or settings pointing at conversations that are not there. mise run appdata and Settings, App data both take the two together for exactly that reason.

The installation id is in /data/settings/server.jsonc. It is what tells the beam host "this machine restarting" from "a second machine arriving", so a settings directory restored onto a different machine will reclaim that machine's slot, and a lost one will be refused it. That is correct in both cases; it is worth knowing before it surprises you.

It prepares itself

A deployment installs the full recommended set on its own, in the background, the first time it starts: node, ripgrep, fd, rtk and hashline, then every harness this server can drive and the ACP adapters two of them need. So the first person to log in finds an installation that can open a session, rather than a bootstrap page with a button on it.

It is the same job the card on /settings/bootstrap runs, and the same page is where you watch it. A member already installed contributes no step, so a restart on a volume that has them costs one detection pass and nothing else.

"bootstrap": { "autoInstall": "minimal" }   // or "off"

minimal is Claude Code and Codex with their adapters; off leaves every row to the page. Two things worth knowing before leaving it on full:

• hashline is one release pinned by version and checksum, so a replaced download fails the install rather than landing. The linux/arm64 image has no hashline, because that release has no ARM build; the agent edits by text replacement there instead.
• Everything lands on /data/home, so it survives a restart and an image upgrade, and a wiped volume installs it all again.

This is the one place mechabedo installs without being asked. On a machine you sit at, every row stays a button.

Capacity

The server itself is small. What costs is what it spawns: every harness session is a child process with its own model client, its own toolchain, and sometimes a build.

Sessions at once	Sensible shape
1-2	2 vCPU, 4 GB
3-5	4 vCPU, 8 GB
More	Split across slots rather than growing one box

Two limits are worth setting explicitly, because a harness can spawn a compiler:

--pids-limit 512 --memory 8g

Without them one runaway build takes the whole container with it, including the tunnel, and the failure looks like the beam rather than like a build.

Disk grows with the archive. Transcripts are the bulk of it, and logbook sync adds whatever the other installations push. Settings, App data reports the database size, and the status bar can carry it.

The deployment's interface is the config document

In vendor mode, what the config document owns is written back on every start. A setting changed in the UI that the document also names is overwritten the next time the container restarts.

So: durable deployment settings go in the file, and only there. The gateway, the loadouts, the auth mode, the harness environment. Everything else can live in the UI. Kits seed like loadouts: kits and enhance.defaultKit land on the volume once, on the first start, and the Enhance page owns them afterwards.

docker/mechabedo.config.example.jsonc is the annotated reference. MECHABEDO_VENDOR_CONFIG carries the same document as strict JSON in one environment variable, ahead of the file, for a platform that has secrets and environment but no way to put a file on a volume before the first start.

A missing or invalid auth section is the one thing that exits rather than degrading. The container stops and the log lists every problem, which is what you want: a deployment that came up unauthenticated would be worse.

With OIDC, the groups decide the role. auth.oidc.adminGroups signs its members in as admins and auth.oidc.allowedGroups as members; with either list set, an account in neither is refused. auth.oidc.groupsClaim names the ID token claim the groups are in, groups by default, and the provider has to put it in the ID token. With neither list set, every account the provider issues for the client is an admin, and the log warns about it on the first sign-in. What a member may do is on Settings > Access.

"auth": {
  "mode": "oidc",
  "oidc": {
    "issuer": "https://idp.example.com",
    "clientId": "mechabedo",
    "adminGroups": ["platform"],
    "allowedGroups": ["engineering"]
  }
}

Health

The image carries a HEALTHCHECK that curls /api/health every thirty seconds, with a forty-five second start period because PGlite opening a cold volume for the first time is slower than every start after it.

docker inspect --format '{{.State.Health.Status}}' mechabedo
curl -s -H 'x-beam-slot: kdlab' https://beam.example.com/api/health | jq

/api/health needs no credential and answers what the process is: its mode, its profile, whether the request arrived through a beam, and the build. Everything past it does need one.

Three things separate "up" from "working", and only the third needs a person:

1. The container is healthy.
2. The log carries tunnel open to <host> on slot <name>, and GET /__beam/slots on the host lists that slot as connected.
3. A chat session opens and stays open while idle. The remote leg has no heartbeat of its own, so an idle session staying connected is the one thing only a real session proves.

Logs

MECHABEDO_LOG_LEVEL takes trace, debug, info, warning, error or fatal; info is the default and is state changes only. A repeating reading - a poll answering the same thing every tick - is deliberately not one, so an info log that is quiet means nothing is changing rather than that nothing is happening.

mise run docker logs --follow          # a workstation container
docker logs -f mechabedo                # anywhere else
journalctl -u mechabedo -f              # the systemd shape

Turn debug on for a specific problem and turn it back off. The tunnel logs every dial at debug, which is a line every few seconds while a host is away.

For anything beyond reading lines, the server speaks OpenTelemetry, and on a server-shaped installation that is the difference between a log you grep and a log you query. The next section is how.

Telemetry

kubectl logs shows the console sink's text. The record behind each of those lines has a level, a category and typed properties, and the OTLP export is where they survive. A hosted deployment is also the one shape that cannot answer "what are the harnesses costing" any other way: the process gauges register only when telemetry is on.

Everything goes in the config document, and every field is overridden by the matching OTEL_ variable, so a platform that already injects them needs none of it:

"telemetry": {
  "enabled": true,
  "endpoint": "http://otel-collector-opentelemetry-collector.monitoring.svc:4318",
  "serviceName": "mechabedo-kdlab",
  "resourceAttributes": "deployment.environment.name=production",
  "runtime": true,
}

One name per deployment, not per pod. Which pod, which namespace and which workload are resource attributes a collector stamps on; what only the process knows is its build, its install id, its channel and its beam slot, and those are added for you.

The three signals

Signal	Default	What you get
Logs	on	Every log() call as a record: level, category, typed properties, the exception fields on an error, and the trace it happened in
Metrics	on	http.server.* and the v8 heap from the runtime, plus mechabedo.process.*: what this server and its harnesses are using
Traces	on with metrics	Every request and every call out to the LLM gateway, instrumented by the runtime with no code

runtime is the switch for the second and third together. That is the runtime's granularity and not a choice made here: OTEL_DENO turns on traces and metrics at once.

It has to be set on the launch. The runtime reads OTEL_DENO before the config document is loaded, so a document that says "runtime": true on a container that does not set it warns at startup and changes nothing. Saying it anyway is worth doing: it is what turns a variable dropped from a manifest into a line in the log rather than signals quietly stopping.

OTEL_DENO_CONSOLE=ignore is the other launch-time one, and the container's entrypoint sets it for you. Without it the console sink's line is exported as a second, unstructured copy of every record.

Traces are worth turning on and not worth storing

Leave them on. Then decide, at your collector and not here, what to keep:

• A streaming route is one span that lasts for hours. Event streams, the database socket, terminals and the beam tunnel all close their span when the connection does. Counted into a latency histogram they make it meaningless.
• A turn is not a span. An agent turn runs for minutes over a child process, and it already logs twice with the model, the stop reason, the timings and the tokens on the end line. That line is the record, not a span.

So what a trace is actually read for here is "was that slow request us or the gateway", and per-route latency and error rate. A collector can derive the second from spans it then throws away. The k0s-bootstrap platform does exactly that: spans over five minutes are dropped and the rest feed a spanmetrics connector, so there is no trace store and the numbers are still there.

Looking

Settings, Telemetry reports what resolved: the collector, the protocol, the header names, the resource, and which fields the environment pinned. It is also where the reader for this run's log file is, which needs no collector at all.

Backup and restore

The unit is the database and the settings directory together.

From inside the app is the one to prefer. Settings, App data writes a timestamped directory holding manifest.json, database.tar.gz taken through PGlite's own dumpDataDir, and a file-for-file copy of the settings directory. mise run appdata is the same operation on the command line. It is consistent and it does not stop anything, because the database exports itself rather than being copied out from under its own write-ahead log. A restore or a wipe asked for while a server holds the profile is parked and applied at the next start.

From outside, with the container stopped, the whole volume is the safe unit:

docker stop mechabedo
docker run --rm -v mechabedo-data:/data -v "$PWD":/out alpine \
  tar -C /data -czf "/out/mechabedo-$(date -u +%Y%m%dT%H%M%SZ).tgz" .
docker start mechabedo

Stopping it is not a formality. A tar of a running PGlite is a tar of a database mid-write. Do not tar settings alone either: the database is not in it.

Restoring is putting both halves back, together, into a stopped container, and starting it. The rest of /data/home is worth keeping but is rebuildable: the next start installs mise and the harnesses again by itself.

Do not restore one installation's settings directory onto a second machine that is also running: they will share an installation id and fight over one beam slot. Restore it onto the machine it came from, or onto its replacement.

The workstation shape

mise run docker is this whole thing in one command, for a container on a machine you sit at:

export MECHABEDO_BEAM_TOKEN=<the slot's token>
mise run docker up --beam-url https://beam.example.com --beam-slot kdlab

mise run docker status          # container, image, mounts, beam slot, health
mise run docker logs --follow
mise run docker restart         # the same container, from what it was started with
mise run docker down            # the container goes, the data directory stays

Every flag is remembered in <data>/deploy.json at mode 0600, the tunnel token included, so restart and update reproduce the container without repeating any of them. --forget-token opts out of keeping the token, for a deployment whose secret lives in a store; then the environment has to carry it on every run.

--mount /srv/projects gives the agents a host directory, at the same path inside and out, so a path in a transcript means the same thing on both sides.

mise run docker config prints the resolved config document with the secrets redacted, and needs no engine running.

Next

• Upgrading - including mise run docker update
• Running a beam host - the other end of the slot