cocoonbox

cocoonbox — design and execution plan

Status: draft v1, 2026-09-30.

cocoonbox gives each user a persistent cloud computer with an agent living in it, driven by any model through our gateway, with a native macOS app to watch, talk to, take over, and connect accounts and the local Mac. Meta’s Muse is a design reference only: no code is copied.

1. Stack selection

Part Choice Why Rejected
macOS app Swift, SwiftUI + AppKit; live view = swift-rfb (loomhand/swift-rfb, MIT): event-driven RFB client with a Metal renderer, fed directly by Session.Screen native idle ~30 MB vs Electron’s ~130 MB / 400 MB bundle; framebuffer path stays off a webview canvas Electron, Tauri (WKWebView startup + canvas path)
Mac local helper cocoonboxd Swift, LaunchAgent, dials out to the box survives the app window; runs “on my Mac” requests; approvals re-checked locally helper inside the app process
In-box agent cocoonbox-agent Rust, single static binary (tokio, tonic, rusqlite) per-VM RSS is density: ~20 MB vs a Node host’s hundreds; same language as gw and cua-driver Node/TS AI-SDK host
Control plane + edge cocoonbox-api Go, a systemd service on the bench host sandboxd claim lifecycle behind the machines interface; routes app ↔ box Rust
Wire gRPC over HTTP/2 everywhere (tonic / grpc-go / grpc-swift 2); bidi streams carry events, RFB bytes and local-exec one connection per app, streaming both ways, protobuf contracts shared by three languages HTTP+SSE JSON, WebSocket per channel
Screen executor trycua cua-driver v0.30.4 (MIT, Rust), serve on its 0600 unix socket AT-SPI element tree + element-token clicks + pixel input, one daemon for Linux/Windows/macOS cua computer-server (a11y stub), OSWorld server (benchmark-shaped, no auth)
Desktop in guest real Xorg with xserver-xorg-video-dummy at 1280×800 + /dev/uinput, XFCE, AT-SPI on, x11vnc on loopback cua-driver’s focus-free pixel input needs real Xorg + uinput; TigerVNC Xvnc cannot hotplug the MPX devices (linux-mpx-recovery.md:41-42) Xvnc (foreground-only input)
Browser Google Chrome (.deb), CDP on loopback, --force-renderer-accessibility structured channel ahead of pixels snap chromium
Models gw v0.2.6 on the bench host, SQLite ledger, bridge-only bind tenant caps, one endpoint for every model provider  
Machines the cbsbx sandboxd instance on the bench host (§14; K3s + vk-cocoon + CocoonSet until 2026-10-01) a claim = persistent box with hibernate/wake, warm pools, egress proxy and per-claim secrets eru (host-local snapshots, no migration)

Hot-path cost: none. No change to cocoon, sandboxd or gw code; all of it is configuration plus new cocoonbox repos.

2. Topology

cocoonbox.app ─┐                              bench host
cocoonboxd  ───┼─ gRPC/h2+TLS ─▶ cocoonbox-api (edge + control) ──▶ sandboxd claim ─▶ VM      
               │                  │ routes by box_id, never sees payload secrets        │
               │                  └─ gRPC ─────────────────────────────────────────────▶│ cocoonbox-agent :7443
               │                                                                        │  ├─ cua-driver (unix socket)
               │                                                                        │  ├─ Chrome CDP (loopback)
               │                                                                        │  ├─ x11vnc :5900 (loopback, via agent)
               │                                                                        │  ├─ stdio MCP servers (plugins)
               │                                                                        │  └─ proxy → host egress doors (§14.6, §16.3)
               │                                                                        └──▶ gw 10.203.0.1:18090 ─▶ OpenRouter / vendors

The app and cocoonboxd always dial out (no inbound port on the Mac). The edge authenticates the user, resolves box_id → sandboxd claim, and proxies through the sandboxd port relay the gRPC stream; account credentials never enter the box (§16).

3. In-box agent (cocoonbox-agent)

Tool tiers, cheapest and most reliable first:

  1. Connectors / MCP tools from installed plugins.
  2. Shell + files: persistent PTY sessions, read / write / edit / ls / grep under /workspace.
  3. Browser: CDP; page snapshot as a ref-tagged element list (- role "name" [ref=eN], capped), act by ref.
  4. Computer: cua-driver get_window_state (screenshot + AT-SPI tree) and element-token actions first, pixel click/type/press_key/hotkey/scroll/drag last. Only the computer subagent holds this tool; the main agent gets a read-only screenshot when its model has vision.

Models (resolved per role in agent config, all via gw):

Role Default Wire
main a text model /v1/chat/completions
computer subagent a vision model /v1/messages when the upstream is the model provider’s own API, else chat
summarize / memory small cheap model chat

Rule: the agent speaks the upstream’s native wire; gw translation is not relied on (silent-distortion risk: cache_control, stop reasons, parallel tools). Images from tool results go into a following user image part for vision models and are replaced by text for text-only models.

Privilege split: the loop runs as cbagent; tools (shell, files, browser, cua-driver, MCP servers) run as the desktop user user in its session. The model’s shell cannot read the loop’s database or config. The loop holds no real gw key either: gw calls go through the egress proxy, which injects the per-box gw key like any other surrogate, so a key leaked by a tool is worthless and the per-box tenant cap bounds any abuse.

State: SQLite (/var/lib/cocoonbox/agent.db) for agents, transcript, approvals, installed plugins; /workspace for user files. Both survive hibernate (disk is in the snapshot).

4. Interfaces

Proto packages live in one repo (cocoonbox/proto), generated for Go, Rust and Swift. Sketch below; field lists are the contract to refine in M2.

4.1 Control plane — cocoonbox.control.v1.Boxes (edge, per user)

service Boxes {
  rpc Ensure(EnsureRequest) returns (Box);                 // create or wake; idempotent per (user, flavor)
  rpc Get(GetBoxRequest) returns (Box);
  rpc List(ListBoxesRequest) returns (ListBoxesResponse);
  rpc Watch(WatchBoxesRequest) returns (stream ListBoxesResponse); // the list at once and after every change; never wakes a box
  rpc Hibernate(BoxRef) returns (Operation);                // sandboxd hibernate
  rpc Recreate(RecreateRequest) returns (Operation);        // preserve_data: snapshot /workspace+agent.db, fresh image
  rpc WatchOperation(WatchRequest) returns (stream OperationEvent); // resumable by offset_key
}
message Box { string id = 1; State state = 2; string image = 3; Flavor flavor = 4; bool image_update_available = 5; }
enum State { ABSENT = 0; STARTING = 1; RUNNING = 2; HIBERNATING = 3; HIBERNATED = 4; FAILED = 5; }
message OperationEvent { string operation_id = 1; Phase phase = 2; string detail = 3; string offset_key = 4; int64 at_ms = 5; }
enum Phase { BACKING_UP = 0; CREATING = 1; MOVING = 2; CLEANING_UP = 3; DONE = 4; FAILED_PHASE = 5; }

All box RPCs address box_id in metadata; the edge forwards cocoonbox.agent.v1.* calls to that box.

4.2 Connect

Three different things, three permission models (Muse’s split):

(a) App ↔ box — cocoonbox.agent.v1.Session

service Session {
  rpc Hello(HelloRequest) returns (HelloResponse);          // box_public_key (X25519), agent version, capabilities
  rpc Events(EventsRequest) returns (stream Event);         // channels + from_offset; transcript, agents, approvals, computer-action, box-disk
  rpc Screen(stream ScreenClientFrame) returns (stream ScreenServerFrame); // RFB bytes to/from x11vnc; take-over = client input frames
}

(b) Accounts: superseded by §16. The in-VM vault sketched here kept credentials at rest in the VM.

(c) The user’s Mac — cocoonbox.agent.v1.LocalExec (cocoonboxd dials out)

service LocalExec {
  rpc Attach(stream LocalFrame) returns (stream LocalRequest);  // hello{computer_id, root, label}; requests: exec, read, ls, upload, download, cancel
}

4.3 Marketplace

Superseded by §17.

4.4 Agents — cocoonbox.agent.v1.Agents

service Agents {
  rpc Create(CreateAgentRequest) returns (Agent);           // client_nonce dedupe
  rpc List(ListAgentsRequest) returns (ListAgentsResponse);
  rpc Send(SendRequest) returns (SendAccepted);             // prompt + attachments + selected plugin capabilities; client_nonce
  rpc Transcript(TranscriptRequest) returns (TranscriptPage);
  rpc Respond(RespondRequest) returns (Empty);              // approvals, widgets, "take over done, continue"
  rpc Cancel(AgentRef) returns (Empty);
}

5. Guest image cocoonbox-desktop:24.04

Cocoon OCI VM image (vmlinuz + initrd.img + rootfs) built from cocoon’s os-image/ubuntu/24.04 template:

6. Bench host layout (isolated env cb)

The layout: the cbsbx sandboxd instance under /srv/cb/sbx (its own cocoon root, net none pools, egress proxy), gw and the MinIO state store on the bridge address 10.203.0.1 (cbc0, which holds no VMs), cocoonbox-api behind Caddy, and the guards in cb-host-net. The K3s, vk-cocoon, cocoon-operator, registry and cocoon-cb stack that ran the boxes until M4.8 was removed on 2026-10-01.

7. Milestones

# Deliverable Acceptance evidence Size
M0 isolated cb env, K3s, registry, vk-cocoon, operator, gw kubectl get node cocoon-cb Ready; curl 10.203.0.1:18090/health; a real completion through gw with the new key 0.5 d
M1 guest image v1 + cua-driver spike VM from CocoonSet boots; cua-driver doctor clean; get_window_state on Mousepad and Chrome returns a non-empty AT-SPI tree; an element-token click and a background pixel click both land (visible in x11vnc); hibernate → registry cb/<vm>:hibernate → wake keeps a marker file 1–1.5 d
M2 cocoonbox-agent v0 (Rust): gw client (chat + messages), shell/files, browser (CDP refs), computer subagent via cua-driver, Agents/Session gRPC, SQLite from grpcurl: the text-model main agent runs Shell; delegates to the vision computer subagent which opens Mousepad and types text; browser task reads example.com H1 via refs 4–6 d
M3 cocoonbox-api (Go): Boxes on CocoonSet, WatchOperation, edge routing by box_id, user tokens Ensure from zero to RUNNING; Hibernate/Ensure round trip with a resumable operation stream; agent RPCs reach the box through the edge only 3 d
M4 macOS app v0: sign-in, box state, agent list, streaming transcript, live view + take-over RoyalVNCKit spike first (encodings against x11vnc: ZRLE/Tight); app shows the desktop at 1280×800 and the transcript live; measured: app RSS, time to first frame, input RTT 4–5 d
M4.5 sandbox line evaluation docs/m45-sandbox-eval.md: measured lifecycle, relay, egress, density, failure model; recommendation done
M4.6 state layer (§11): user data to object storage, SaveState, Rebase SaveState twice on a box (second incremental); Rebase v6 → next image keeps /workspace, a logged-in Chrome cookie and agent.db; sizes and timings recorded in progress
M4.7 native agents: a plugin for the local agent CLIs (skill + commands + MCP config), a cocoonbox CLI (login, mcp-headers, box lifecycle), a control-plane MCP in the api (status, ensure, hibernate, save, rebase) from each CLI, with the app closed: wake the box, do a GUI task, save, hibernate; no token in any config file next
M4.8 sandboxd as the machine backend (§14), after the upstream PRs (per-claim env, dead-VMM restart) land §14.9: M3 acceptance plus live view, MCP, Rebase, H3, identity, crash, dead-api and lost-machine runs on the sandboxd backend; default flipped; cocoonset kept as fallback after PRs
M4.9 proactive agents (§15): routines in the box, api alarms with wake-then-Tick, precheck, notices in the app, limits and kill switches, dual-meter budgets (§15.9, phases a–f) §15.9 per phase; end to end: the HN digest routine wakes the hibernated box at 08:00, writes the digest, re-hibernates, and its notice shows in the app without a wake; per-run box-seconds and cost match sandboxd’s and gw’s own records 8–10 d
M5 connect (§16): host-side broker with OAuth through the Mac, connector tools for both modes, per-tool policy and approvals; M5.2 git through egress injection. cocoonboxd local-exec (§4.2(c)) is tracked separately §16.8: GitHub and Calendar connected; mode B with the box hibernated; mode A write approved in the app; no token in the guest, its hibernate image or the Mac’s configs 5–6 d, M5.2 2 d
M6 market (§17): pinned git catalog in the M4.7 plugin format, skills and secret-free stdio MCP servers in the box, visible in both modes; the catalog doubles as the local agents’ marketplace §17.8: a public agent-CLI plugin installs unchanged and works in a box turn and through the bridge; tampered, symlinked and secret-needing plugins refused; survives Rebase 3–4 d, M6.2 3 d

M0–M2 are the proof that “any model + our machines” works; M3–M4.8 make it a usable product; M5–M6 (connect, marketplace) come after the current stack runs end to end.

8. Decisions

Settled 2026-09-30:

  1. Repo: monorepo loomhand/cocoonbox, AGPL-3.0. Layout: proto/, agent/ (Rust), api/ (Go), macos/ (Swift), image/. AGPL allows reusing gw’s protocol crates.
  2. Endpoint: the bench host’s public origin. Caddy path matcher @cocoonbox path /cocoonbox.* → reverse_proxy h2c://127.0.0.1:18200 placed before the existing default handlers; gRPC method paths (/cocoonbox.control.v1.Boxes/Ensure) never collide with the host’s other routes. caddy validate before a graceful reload.

Open:

  1. The new gw key(s) and daily cap.
  2. Rust for the in-box agent (recommended).

9. Risks

Risk Mitigation
cua-driver is pre-release, ~1 release/day; open Linux bugs (#3774 background type_text silent drop, #3871 desktop freeze on Cinnamon) pin 0.30.4; M1 spike is the gate; fall back to delivery_mode: foreground for typing; keep the executor behind our own computer tool so the backend can change
Real Xorg + dummy + uinput in a cocoon guest unproven M1 checks /dev/uinput, MPX device creation, x11vnc capture of the dummy screen before anything else
RFB client cost on the hot path (resolved: RoyalVNCKit polled at 100 Hz; replaced by swift-rfb, idle 0.03% CPU) spike encodings early in M4; our own Metal layer; H.264 (RFB encoding 50, done before in cocoonstack/scrcpy-rfb) later if bandwidth demands
The main text model’s tool-calling quality under a long tool list per-role model map; acceptance uses one Shell call first

11. State layer (user data in object storage)

Decided 2026-09-30. The box is disposable across images and nodes; the user’s data is not. Hibernate and same-node checkpoints stay the fast path (memory and running programs kept); the state layer is the durable path that survives node loss and image rebase.

12. Box identity without secrets (replaces EDGE_TOKEN, audit H1(a); gw relay, H3)

Decided 2026-09-30, edge side reviewed 2026-10-01. Constraints:

The adversary is the box’s own user (user, unprivileged), or root inside one box after a guest compromise.

12.1 Edge → agent

  A. Edge-signed box-bound credential B. Per-box token in tmpfs C. vsock transport
Mechanism The edge signs key_id · box_id · wake_id · exp with ed25519 (header x-cocoonbox-credential). The agent verifies it with the public key from its identity file, and checks key_id, box_id and wake_id against that file and exp against its clock. The api delivers a random per-box token at every boot and wake into /run/cocoonbox/edge-token (tmpfs, root:cbagent 0640). The agent compares it like today. The agent accepts calls only on vsock from CID 2 (the host). A node-local bridge carries edge traffic onto each VM’s vsock socket.
Secret in the guest none yes, in RAM, so it lands in every hibernate image until the next wake rotates it none
Misrouted call (stale pod IP now held by another box) rejected: box_id mismatch rejected: token mismatch impossible: the channel is the VM
Guest root compromise can use only its own box’s credentials; forging needs the edge’s private key leaks its own token until the next wake none reusable
Host plumbing none new; api writes one file per wake same file write, plus per-box token storage in the api per-node TCP↔vsock bridge with its own auth to the edge
Clock dependence yes (exp); bounded by wake_id (below) no no

Decided: A (ruling 2026-09-30).

Credential format. x-cocoonbox-credential: v1.<b64url(key_id|box_id|wake_id|exp_unix)>.<b64url(ed25519 signature over the first part)>: base64url without padding, |-separated fields.

Where the agent learns its identity and the edge key. On every boot and wake, the api writes /run/cocoonbox-identity/identity, {"box_id":…,"wake_id":<random>,"key_id":…,"public_key":<ed25519>}, through sandboxd exec (silkd over vsock, running as root in the guest).

Clock. A woken guest’s clock may lag until NTP syncs. wake_id changes on every wake, so it carries the real freshness: a credential from before a wake is dead whatever the clock says. exp is a coarse bound (10 min, with a small skew allowance of 2 min) against replay within one wake. Replay needs a copy of the edge’s traffic, which runs host-local through the sandboxd port relay.

Rotation. The public key arrives in the identity file, not in the image. Rotation is therefore: the api starts signing with the new key, then rewrites the identity files on the next wake. No image cut is needed.

Rebase and hibernate. A rebased machine gets its own wake_id, so the old machine’s credentials die with it. A hibernate image carries the identity file; that is harmless because it is not secret, and the wake rewrites it with a new wake_id.

MCP uses the same header, so the edge treats /mcp exactly like gRPC.

12.2 Box → host (gw relay, H3)

A signature can’t prove “request from box X” without a secret in X. The proof must come from the channel.

12.3 Changes

13. Native agents without the app (M4.7)

Proposed 2026-10-01, for review. The user’s own local coding agent drives their box with the app closed: it wakes the box, works on it through the box tools, saves and hibernates. The model subscription stays on the Mac (mode B); the box sees only MCP calls.

Constraints:

13.1 Pieces

Piece Where What
cocoonbox CLI macos/ (Swift, a new product next to the app, on CocoonBoxKit) login and logout (Keychain), box lifecycle commands, the stdio MCP bridge, a headersHelper command, per-client setup
Control MCP api service layer, https://<endpoint>/cocoonbox.mcp/control box lifecycle tools for the caller’s own box, over the existing Boxes semantics
Plugin plugins/cocoonbox/ (no code) one directory with a manifest per client, a shared .mcp.json that runs cocoonbox mcp, a skill and slash commands

The CLI is Swift on CocoonBoxKit because it needs the Keychain and the app already solved endpoint parsing, the loopback-only plaintext rule, the HTTP/2 connection and the Boxes calls. That is vertical reuse inside one component (macos/), not coupling to another. It ships inside the app bundle (CocoonBox.app/Contents/MacOS/cocoonbox), and cocoonbox install links it into ~/.local/bin; a standalone build works the same way for users who never open the app.

13.2 How a client reaches the box: a local stdio bridge

Every client runs the same MCP server entry, with no URL and no token in it:

{ "mcpServers": { "cocoonbox": { "command": "cocoonbox", "args": ["mcp"] } } }

cocoonbox mcp is a stdio MCP server. It reads the token from the Keychain at start and keeps it only in memory. It forwards each JSON-RPC message to the edge over MCP Streamable HTTP with Authorization: Bearer: box tools go to /cocoonbox.mcp/<box id>, and lifecycle tools go to the control MCP. It resolves the box id itself (Boxes.List), so a Rebase or a new box never breaks a client’s config.

Why a bridge rather than pointing clients at the HTTPS endpoint directly (what the app’s Connect sheet does today):

The cost is one local stdio hop per message, next to a 230 ms WAN round trip; the bridge holds one HTTP/2 connection for both endpoints.

Tool list while the box is down. The box tool names and input schemas are the contract, so the bridge can answer tools/list without the box. It caches the box’s own tools/list answer (not secret) in ~/Library/Caches/cocoonbox/tools.json whenever the box is running, falls back to a copy compiled into the CLI, and sends notifications/tools/list_changed when the live list differs. Lifecycle tools come from the control MCP, or from a compiled-in copy when the edge is unreachable.

Connector tools (§16). At initialize the bridge also opens a session to /cocoonbox.mcp/connect with the client’s own initialize, and holds that door’s GET stream. Its tools/list answer appends the door’s tools, after waiting at most 3 s for the first list. A connector tool whose name a control or box tool already has is left out, with a line on stderr. On the door’s list_changed the bridge lists again, and sends notifications/tools/list_changed when the connector tools differ from those the client last listed. A call to a listed connector tool goes to the door and wakes no box. When the door fails, the bridge serves the other tools and retries with backoff, from 1 s up to 60 s. A 401, 403, 404 or 405 from an api without the door ends the retries.

Waking on demand. When a box tool is called and the box is not RUNNING, the bridge calls the control tool box_ensure. It reports the wake as MCP progress notifications (“waking: restoring memory snapshot”), then forwards the call. A wake takes 5–17 s today. A client whose tool timeout is shorter than that, or a user who does not want implicit wakes, uses cocoonbox mcp --no-wake: the call then fails with an error that tells the model to call box_ensure.

Direct fallback. The Connect sheet recipes stay for users who don’t install the CLI. The claude recipe may use "headersHelper": "cocoonbox mcp-headers", which prints {"Authorization":"Bearer …"} to the CLI’s pipe, instead of ${COCOONBOX_TOKEN}. There is no cocoonbox env command: an exported token lives in every child of that shell.

13.3 Control MCP (api)

The api serves it itself; the edge does not proxy it to a box. Auth is the user token, which the api checks the way it checks Boxes, and every tool acts only on the caller’s own box. Tools (names are the contract; add them to contracts.md):

Tool Arguments Result
box_status – state, image, latest_image, last_saved_at_ms, box id
box_ensure wait (default true) final state; progress notifications from WatchOperation while waiting
box_hibernate – operation result; progress notifications
box_save – operation result with bytes; says the browser closes during the save
box_upgrade confirm: true (required) operation result; refused without confirm, with the text “running programs restart; files, logins and agent history are kept”

These are thin wrappers over Ensure, Hibernate, SaveState, Rebase and WatchOperation, so the api keeps one implementation of the lifecycle. The CLI’s own cocoonbox status|ensure|hibernate|save|upgrade commands call Boxes over gRPC directly, as the app does.

13.4 CLI

Command Does
cocoonbox login [--endpoint URL] [--browser \| --token - \| --from-app] [--invite CODE] on a terminal (or with --browser), signs in through the browser and gets a device token of its own (contracts “Browser sign-in and device tokens”); with --token -, over SSH, or with stdin not a terminal, reads a token from stdin or a hidden prompt. Stores it in the Keychain and checks it with Boxes.List
cocoonbox invite create [--days N] / invite list with an admin’s invite-scoped token from COCOONBOX_INVITE_TOKEN or a file, makes a one-use invite link or lists the token’s invites (contracts “Invite-scoped tokens”)
cocoonbox logout deletes the Keychain item, also when the api is unreachable; first revokes the token at /cocoonbox.oauth/revoke when the CLI got it through the browser. A token copied from the app (--from-app or the implicit copy) is the app’s device and is not revoked. cli.json records where each endpoint’s token came from
cocoonbox status / ensure / hibernate / save / upgrade [--yes] lifecycle, with progress on stderr and --json for scripts
cocoonbox mcp [--no-wake] the stdio bridge (13.2)
cocoonbox mcp-headers the headers helper for the direct claude recipe
cocoonbox rsh [--box B] [HOST] COMMAND… runs a command in the box with its stdin, stdout and stderr; rsync’s and git’s remote shell (13.7)
cocoonbox setup claude\|codex\|cursor [--print] installs the plugin, or prints the manual config for that client
cocoonbox install links the CLI into ~/.local/bin

Keychain sharing with the app. Both use service io.github.loomhand.cocoonbox with the endpoint URL as the account. With a Developer ID signature, app and CLI share one keychain access group, so neither prompts. Until then, builds are ad hoc signed and every rebuild has a new code hash. So cocoonbox login writes its own item, and cocoonbox login --from-app copies the app’s token once, after a single “Always Allow”.

13.5 Plugin packaging

plugins/cocoonbox/:

None of these files contains a URL or a token: the endpoint comes from cocoonbox login and the box from Boxes.List. Installation, per the installed CLIs’ own help (claude 2.1.285, codex 0.159.2, cursor-agent 2026.09.28):

CI checks the directory with claude plugin validate.

13.6 Acceptance and work split

Acceptance, from each of the three agent CLIs with the app closed and the box hibernated:

  1. Ask for a GUI task. The bridge wakes the box and the task completes; the app, when opened later, shows the session under External with the client’s name.
  2. Ask it to save, then to hibernate; both finish and cocoonbox status agrees.
  3. Search the three clients’ config and state directories and the shell history for the token: no match. The token is also absent from argv and the environment of every client and bridge process (ps -E).

Split:

Order: the api contract first (contracts.md on main), then api and macos in parallel.

Decided 2026-10-01:

  1. The bridge wakes the box on the first box-tool call by default; --no-wake turns that off.
  2. The CLI ships inside the app bundle and as a standalone build.
  3. No box_operations tool.

13.7 Remote shell: cocoonbox rsh

Decided 2026-10-03: offload transport is rsync/git over cocoonbox rsh; sshd deferred. The box has no sshd since v12, and a per-box sshd would need host keys, user keys and a port path through sandboxd. rsh reuses what every client already has: the user token, the HTTPS edge and the agent’s gRPC.

Hot path: none on existing calls. The edge adds no code. The agent adds one owner check per Exec stream and nothing to the other methods. The window change only grants more credit. A -t stream also runs one infocmp in the box (and tic once per new terminal type) and one local infocmp in the CLI.

Acceptance (2026-10-03, dev’s box on v19 with test agent 4228d59, from a Mac over the public endpoint, RTT 250 ms):

Check Result
rsh --box main uname -a; false; exit 42 Linux …; exit 1; exit 42; runs as user in /workspace, stderr kept apart
50 MiB random through cat sha256 equal both ways; 15.2 s round trip
Throughput, 50 MiB upload 3.3 MiB/s, download 8.0 MiB/s with the call’s startup included. ssh to the bench host on the same path moved 5.9 and 6.5 MiB/s
Startup of rsh true 1.22–1.26 s (10 runs): TCP and TLS to the endpoint, Boxes.List, the Exec stream; the TLS handshake alone took 0.50 s
rsync of the cocoonbox checkout, .git included (1025 entries, 775 files, 8.0 MB) first run 4.8 s, 4.6 MB sent; no change 2.8 s, 64 KB sent; one file changed (docs/design.md) 3.0 s, 65 KB sent and 260 B of unmatched data; trees equal by sha256
GIT_SSH_COMMAND="cocoonbox rsh --box main" git clone box:/workspace/rsh-test/main 3.1 s; git probed -G, got 255 and used its simple variant; a push back worked
Ctrl-C of a running sleep, idle and with a full stdin the remote process ended both times; the CLI exited 130
A second call while an Exec held a full stdin 1.27 s with 4228d59; it waited until the stalled command ended with 7e14603

Round 2 (agent de97cf9, the revised plan, same setup):

Check Result
Login shell login_shell on; with ~/.local/bin present it leads PATH (/home/user/.local/bin:/usr/local/sbin:…)
-t from a pty of 33×111 /dev/pts/3, TERM=xterm-256color, 33 111; after a local resize and SIGWINCH 44 132; typed input echoed and read; exit 5 passed through. tmux runs in it
SIGINT to the CLI, remote trap … INT got-int, exit 9, 0.28 s after the signal; SIGTERM to a remote sleep gives 143 and the process is gone
GIT_SSH_VARIANT=simple no -G probe; clone 3.2 s; a commit made in the box arrived by git fetch in 1.9 s
A second call while an Exec held a full stdin 1.29 s
50 MiB through cat, upload, download; startup equal; 15.2, 14.8 and 6.7 s; 1.22–1.27 s

macOS rsync is openrsync (protocol 29); the box runs rsync 3.2.7. Pushing small/ both ways through cocoonbox rsh:

Upload stays under ssh’s rate. One 1 MiB window in flight over a 250 ms RTT allows about 4 MiB/s, and Go’s HTTP/2 server, which Caddy uses, buffers 1 MiB per stream by default. That fits the numbers but is not proven.

14. M4.8: sandboxd as the machine backend

Design, 2026-10-01. It builds on the M4.5 measurements (m45-sandbox-eval.md) and on three upstream PRs: cocoonstack/sandbox#268 (per-claim env), cocoonstack/sandbox#269 (dead-VMM restart) and cocoonstack/cocoon#274 (vm list verifies the VMM PID). Only api/ and image/ change. The agent and the app do not, except for one optional proto field (§14.8).

14.1 Box ↔ claim, and where it persists

machines.Machine from the sandboxd row
ID, Owner metadata.cb_machine, metadata.cb_owner
Image key.template
Flavor key.size through the tier table (4 vCPU / 8 GiB = xlarge); a flavor that is not a tier is refused at config load
CreatedAt claimed_at
State see 14.3

14.2 Image → template and pool

14.3 Lifecycle mapping

Machines sandboxd Latency (M4.5, nested)
Ensure, no claim POST /v1/claim {template, size, ttl_seconds: 3600, on_expire: archive, metadata, env} warm 1 ms + env delivery
Ensure, hibernated or archived POST /v1/sandboxes/{id}/wake; the new incarnation is then provisioned (14.5) 0.26 s / 1.7 s + about 0.3 s to ready
Ensure, failed POST …/wake (#269: a cold boot of the claim’s disk) boot time
Hibernate POST …/hibernate 23–50 s (writes the 8 GiB memory file)
Delete POST …/release —
Watch poll the row every 500 ms while the watch is open; the index refresh between watches —

States:

Condition State
a claim or wake call in flight; restarts moved and the agent has not answered on 7443 since Starting
a hibernate call in flight Hibernating
hibernated or archived Hibernated
failed Failed, with the row’s failed as Reason
otherwise Running
row gone ErrNotFound; StateGone on a watch that saw it

Leases and tiers:

14.4 Edge → agent: Address returns a dialer

The agent is reachable only through GET /v1/sandboxes/{id}/ports/7443 with Upgrade: tcp. There is no IP. The edge serves Session.Screen itself through GET /v1/sandboxes/{id}/ports/5900, the box’s RFB server; no other port is relayed.

The interface change, backend-neutral:

// Endpoint is how to reach a machine's agent; Key changes when the machine moves.
type Endpoint struct {
	Key  string
	Dial func(ctx context.Context) (net.Conn, error)
}
Address(ctx context.Context, id string) (Endpoint, error)

Hot-path cost against today’s direct h2c to the pod IP:

14.5 Provisioning on sandboxd: Deliver, gw key, identity, meta

The service layer computes each incarnation’s identity and hands it to the backend with Deliver(ctx, id, machines.Identity, keys), where keys are the gw key and the door key as named host-only secrets. It never gets a root exec into the guest and never knows guest paths or scripts. The sandboxd backend implements the same interface:

machines sandboxd
Deliver: gw key a guest: false env entry GW_KEY (below)
Deliver: identity written by the backend to the §12 path, either through POST /v1/sandboxes/{id}/exec (runs as root in the guest through silkd; exec takes only the claim’s own token, which the backend reads with the root by-id GET /v1/sandboxes/{id} at each call and never stores; the buffered exec has no stdin, so the file travels in the request env as CB_STDIN) or as guest env. How it is written is the backend’s choice
Meta / SetMeta guest: true env entries CB_META_<key> (#268): readable with GET …/env and durable in claims.json. Claim metadata is immutable, and the values (wake id, flags) are not secret
Incarnation a backend concern. It changes on claim, wake, unarchive and cold restart, and needs a per-claim boot counter in the sandbox row (upstream, 14.7). Until then, the backend derives it from claimed_at, restarts and its own wake count, kept in meta

14.6 Network lane, crash story, node loss

14.7 Upstream mechanism vs cocoonbox

Upstream (cocoonstack, mechanism only) State
per-claim env, guest: false secrets sandbox#268, in review
dead-VMM cold restart, failed claims, vmm_exit / vmm_restart events sandbox#269, in review
vm list verifies the VMM PID cocoon#274, in review
re-deliver the claim env after a cold restart; VMM exits from cocoon daemon’s event stream in #269, stacked on #268
golden disk size per pool (storage_gib → cocoon vm run --storage) branch ready, not yet a PR
a per-claim boot counter in the sandbox row (claim, wake, unarchive, cold restart) to open: one counter field; Incarnation derives from it
CONFIG_INPUT_UINPUT in the sandbox kernel optional; the image kernel works meanwhile
binary port_forward framing only if many full-rate viewers per node
sparse or compressed memory snapshots, sparse s3 export backlog (§18)

Stays in cocoonbox:

sandboxd learns nothing about users, tiers or cocoonbox.

14.8 Changes and order

  1. Upstream: #268 and #269 merged, the env re-delivery hook in, and a cocoon release carrying #274. Then the disk-size pool field.
  2. contracts.md:
    • the static agent.env (placeholder gw key, no EDGE_TOKEN);
    • “Agent listen” on loopback via the relay;
    • the sandboxd section replacing “Boxes ↔ cluster” for this backend.
  3. api:
    • Endpoint in machines, the edge, MCP and dialer, landing first with cocoonset unchanged in behaviour;
    • Deliver, SetMeta and Incarnation in machines/sandboxd;
    • then machines/sandboxd;
    • then config backend: sandboxd | cocoonset;
    • then restore-on-missing.
  4. image: the rootfs build for cocoon image import, and the additions in 14.2 and 14.5.
  5. infra:
    • the cocoonbox sandboxd instance on the bench host (its own port, data_dir, cocoon root with run_dir on the NVMe, fenced cores);
    • the pool and egress config;
    • per-user gw keys.
  6. Optional proto: Box.restarted_at_ms, so the app can say “your box restarted, memory was lost”. Without it, the app only sees a reconnect.
  7. Migration: per user, SaveState on cocoonset, then Ensure on sandboxd, which restores through restore-on-missing. cocoonset stays configured as the fallback until the acceptance run passes. Then the default flips.

14.9 Acceptance run on the bench host

Every step runs through the public edge, with the app or grpcurl, on the sandboxd backend. Each step records its timings.

  1. M3 acceptance as it stands:
    • Ensure from zero to RUNNING;
    • a Hibernate / Ensure round trip on a resumable operation stream;
    • agent RPCs reach the box only through the edge.
  2. Live view: the app renders 1280×800 through Session.Screen over the relay. Record fps, time to first frame, input RTT, and sandboxd CPU per viewer.
  3. MCP: from a local agent CLI with the app closed, wake the box, do a GUI task, save, hibernate (M4.7’s run).
  4. State layer:
    • SaveState, then Rebase vN → vN+1 on sandboxd;
    • /workspace, a logged-in Chrome cookie and agent.db are kept.
  5. H3:
    • a gw call from the box appears in gw’s log under that user’s own key;
    • the key string is absent from the guest’s /proc/*/environ, /etc, /run and disk, and from the box’s hibernate memory file on the host;
    • GET …/env blanks it.
  6. Identity:
    • a credential for another box_id or an old wake_id is rejected;
    • after a wake, the identity file carries the new wake_id before RUNNING.
  7. Crash:
    • kill -9 of the box’s CH brings the box back to RUNNING with the marker kept, and the app reconnects by itself;
    • four kills leave it Failed, and Ensure recovers it.
  8. Dead api:
    • with the api stopped past the lease, the box is archived;
    • with the api started, Ensure wakes it from the archive, state kept.
  9. Lost machine: release the claim behind the service’s back. Ensure restores from the state layer, and the loss window is reported.
  10. Flip: backend: sandboxd is the default, and backend: cocoonset still serves an existing CocoonSet box.

The flip ran on 2026-09-30. On 2026-10-01 the cocoonset backend, the backend key and the K3s stack were removed; sandboxd is the only backend.

15. Proactive agents

Proposed 2026-10-01, for review. Status 2026-10-10: only the alarm half is built, as scheduled wake (15.13); routines, Tick, the lease, notices and budgets are not. Agents act without a prompt from the user:

The result reaches the user in the app. A risky action waits for the user, as take-over does today.

Constraints:

15.1 How others do it

  A hosted assistant’s scheduled tasks (public docs, not verified here)
Definition a one-off or recurring prompt, about 10 active per user
Where it lives the vendor’s cloud
Machine off not applicable (no user machine)
Run a new chat turn
Result push and email
Safety the vendor’s policy
Self-scheduling yes, from chat

15.2 Where a trigger lives and who wakes the box

The box keeps the routines; the api keeps an alarm clock.

Event triggers:

Trigger Source While hibernated
time (at, cron + IANA zone) agent scheduler, api alarm wakes the box
file under /workspace toolhost inotify, reported to serve none: files arrive only through a running box
page change a cron routine with a precheck (15.4); no separate watcher as time
connector event (mail, GitHub, Slack; M5) a webhook to the api, which wakes the box and calls Routines.Deliver{routine_id, event} wakes the box
box wake the Tick after a wake carries reason: resume; routines with on: resume run —

Hot path: none. The alarm index is a min-heap in the api’s service layer, off the edge path. The heartbeat costs one RPC per running box per 5 min.

15.3 In the box: a headless run

15.4 Budget and limits

The api enforces every limit through the policy lease. The per-box gw key cap (§12.2) remains the hard backstop for model spend. Every default below is api config, not code; the wake floor and runs per day get tuned from the cost of a run (15.8).

Limit First-cut default Why
routines per user 20 active bounds the alarm index and the prompt
wake floor 1 h between alarms that wake a hibernated box a wake plus a re-hibernate costs about 1 s up and 24 s writing 8 GiB of memory (15.8), and the write uses the node’s NVMe bandwidth; minute-level routines run only on a box that is already up
runs per day 48 bounds machine time
run wall time 10 min, then the turn is cancelled a stuck GUI step must not hold the box up
proactive budget per day box-seconds and model cost, per user (15.9f) spend control separate from interactive use; the dual meter of the cost model
failures backoff 1, 2, 4 … 24 h; auto-pause after 5 in a row; pause at once on an auth failure stops loops that burn tokens
away guard no app visit for 3 days and 15 unread notices → one notice “keep or pause”; no answer in 3 more days → pause all spend while the user is gone
kill switch per routine, per user (“pause all”), global (api config) operations

Precheck: a routine may carry a shell command that runs as user without a model, with a 30 s limit. The model turn starts only if the command’s output differs from the last run’s, or on exit code 0 with non-empty output. A page watch is then curl -s … | sha256sum or a small script, and it costs no tokens while nothing changes.

15.5 Safety

15.6 Results to the user

15.7 Rebase, hibernate, node loss

15.8 Cost of one proactive run

Measured on the bench host on 2026-09-30 and 2026-10-01 unless marked. Sources:

Pricing rule: a box holds its slot (4 vCPU, 8 GiB) from the wake call until its hibernate finishes. Machine cost = box-seconds × the slot price. A freed slot saves money only if another box uses it. The slot price is the planning line, as in the free-tier cost model.

Step Time Source
wake: hibernated → RUNNING (memory restore, identity written) about 1 s wake op DONE in 1.0 s; 2026-10-01 sandboxd wake at 10:03:39.95, api “provisioned … as wake” at 10:03:40
run, precheck with unchanged output about 2 s (limit 30 s) estimate: curl plus sha256sum
run, text turn (text-model main agent) model calls span a median of 4.5 s and at most 197 s; with tool time, planned at 45 s gw ledger, 12 text turns
run, GUI turn (vision computer subagent) model calls span a median of 27.5 s and at most 92 s; planned at 150 s gw ledger, 10 turns
hold: final Tick, then an incremental live save about 1 s plus 1–3 s api journal: Rebase saves “saved 1s”, “2s”, “3s”
re-hibernate: write 8 GiB of memory to the local NVMe 24 s (23–50 s) api “hibernating box” at 10:03:09, sandboxd hibernate at 10:03:32.98

vCPU-seconds. These are not measured. The guest is mostly idle while it waits on the model, and Chrome page loads are the busy part. M4.9f records them per run from the guest’s /proc/stat. A host-side count needs per-claim CPU-seconds in sandboxd. sandboxd records wake and hibernate events in usage.jsonl but no CPU counter, so this goes upstream as mechanism (15.9f).

Per run:

Run Box-seconds Machine, list Machine, Spot Tokens through gw Total, list Total, Spot
precheck, output unchanged (no save) 28 $0.0013 $0.0003–0.0005 $0 $0.0013 $0.0003–0.0005
text 73 $0.0034 $0.0009–0.0013 $0.0035 mean (median 0.0014, max 0.012) $0.0069 $0.0044–0.0048
GUI 179 $0.0085 $0.0022–0.0032 $0.051 mean (median 0.020, max 0.28) $0.059 $0.053–0.054
any run held to the 10 min wall 628 $0.030 $0.0076–0.011 bounded by the budget (15.9f)    

Per user-month (30 days; “text” and “GUI” as above):

Usage Alarm wakes a day Box-hours a month Machine, list Machine, Spot Tokens Total, list Total, Spot
3 text runs a day, each its own wake 3 1.8 $0.31 $0.08–0.12 $0.32 $0.63 $0.40–0.43
3 GUI runs a day 3 4.5 $0.76 $0.19–0.29 $4.58 $5.34 $4.77–4.87
§15.4 caps: 48 text runs a day, 1 h floor, so at most 24 wakes 24 23.6 $4.01 $1.02–1.51 $5.07 $9.08 $6.09–6.58
the same caps, GUI runs 24 65.6 $11.16 $2.84–4.20 $73.30 $84.46 $76.14–77.50
the same caps, every run held to the 10 min wall 24 245.6 $41.78 $10.63–15.74 up to the gw key cap ($10 a day)    
box up all day, for comparison — 730 $124.17 $31.61–46.79 the same per run    

15.9 Build plan (M4.9a–f)

The contract lands on main first (boundaries.md); then the phases follow in order. Every phase is gated by build, tests and lint. Bench host acceptance runs at the end of b, c, d, e and f. Routines stay off for every user not in routines.users (the same pattern as state.scheduled_users) until 9e and 9f are done.

Contract. proto/cocoonbox/agent/v1/routines.proto, sketched here; the fields are the contract to refine:

service Routines {        // clients, through the edge
  rpc List(ListRoutinesRequest) returns (ListRoutinesResponse);
  rpc Update(UpdateRoutineRequest) returns (Routine);         // pause, resume, grant, delete
  rpc RunNow(RoutineRef) returns (RunAccepted);
  rpc Runs(RunsRequest) returns (RunsPage);                   // history, newest first
}
service RoutineControl {  // api only; the edge refuses it like State
  rpc Tick(TickRequest) returns (stream TickEvent);
}
message Policy { int64 lease_until_ms = 1; string wake_id = 2; bool paused = 3; bool cancel_running = 4;
                 int32 runs_left_today = 5; int32 min_interval_s = 6; int32 max_run_s = 7; }
message TickRequest { int64 now_ms = 1; Policy policy = 2; TickReason reason = 3; uint64 ack_through = 4; }
enum TickReason { ALARM = 0; HEARTBEAT = 1; PRE_HIBERNATE = 2; RESUME = 3; CONTROL = 4; }
message TickEvent { uint64 seq = 1; oneof kind { RunStarted started = 10; RunEnded ended = 11; Notice notice = 12; TickEnd end = 13; } }
message RunEnded { string run_id = 1; string routine_id = 2; string agent_id = 3; RunStatus status = 4;
                   int64 started_ms = 5; int64 ended_ms = 6; repeated ModelUsage usage = 7; int64 guest_cpu_ms = 8; }
message TickEnd { int64 next_due_ms = 1; int32 active_runs = 2; }

The rest of the contract:

An alarm Tick streams until every run it started has ended. A heartbeat Tick returns once it has started what is due, and those runs report on a later Tick.

M4.9a Routines in the box (agent)

Scope:

Acceptance:

Left out: the api side, the precheck, notices.

M4.9b Alarms: wake, Tick, re-hibernate (api)

Scope:

Acceptance, on the bench host with the dev box hibernated and the app closed:

Left out: notices (the result is only in the transcript), budgets beyond the lease.

M4.9c Precheck (agent)

Scope:

Acceptance: a routine with the precheck curl -s https://example.com | sha256sum fires on three alarms. All three runs are skipped_precheck, and the gw ledger has no row for the machine in those windows. Changing the command’s output gives one model turn.

Left out: any precheck isolation beyond user’s shell.

M4.9d Notices and the app

Scope:

Acceptance, on the bench host:

Left out: APNs and email.

M4.9e Limits, kill switches, away guard

Scope:

Acceptance:

Left out: tiers.

M4.9f Cost metering and budgets

Scope:

Acceptance, on the bench host, five alarm runs:

Left out: billing, tiers.

15.10 Failure handling

Failure What happens
api down at an alarm No wake. At start, the api rebuilds the heap from proactive.json, and an alarm in the past wakes its box at once, within the node’s wake limits. The agent runs each missed routine once inside its catch-up window and records the rest as missed. It never replays a backlog.
api down while a box is running In-box fires stop when the lease ends, within 15 min.
box fails to wake (Ensure error, Failed) Retries at 1, 2 and 4 min. Then an api notice failed (“your box did not wake for a scheduled run”), and the alarm moves one wake floor ahead. A machine that is gone goes through restore-on-missing (§14.6), which brings back agent.db and its routines from the last save.
run past the wall The agent cancels the turn at max_run_s (status timeout, counted as a failure for backoff). The api has its own guard: if an alarm Tick stream is still open at max_run_s + 2 min, it sends cancel_running: true, waits 30 s, and then hibernates anyway. A turn caught in a hibernate image probably fails on the next wake, since its model stream is gone; this is unverified, and 9b measures it.
approval nobody answers After 10 min the turn ends as waiting_for_user, with a needs_user notice, and the box may hibernate. The user replies in the conversation; a continuation turn comes later (15.5). An M5 connector approval takes the same path (§16.5).
Rebase mid-run A Rebase saves QUIESCED, which waits 30 s for running turns and then fails (state/mod.rs:539-542). So runRebase (rebase.go:42) first sends a CONTROL Tick with paused: true, cancel_running: true. Cancelled runs end as interrupted, which is not a failure, and fire once on the new machine if they are still inside their catch-up window.
restore-on-missing mid-run The run is lost with the machine. The restored agent.db may predate the run’s start, so the run can repeat inside its catch-up window. Under observe, a repeat only rewrites files in /workspace. act will need a per-run idempotency key (open decision 6).
VMM crash (cold restart, #269) Memory is lost. recover ends open turns as failed and marks the run interrupted, and the next Tick fires it again inside its catch-up window.
clock The api sees only UTC next_wake_at_ms and never parses a trigger. The agent judges due times by the api’s now_ms plus the offset measured at each Tick. After a wake, nothing fires before the first Tick (the lease’s wake_id).
time zones and DST Cron is evaluated in the routine’s IANA zone with the image’s tzdata (/usr/share/zoneinfo). A local time that does not exist (spring forward) fires once, at the first instant after the gap. A local time that occurs twice (fall back) fires once, at its first occurrence. @every is an absolute interval and ignores DST. The app shows the next due time in the routine’s zone and in the Mac’s. tzdata changes arrive with image cuts, so a zone whose rules changed keeps the old rule until the next cut.
duplicate reports Run events are rows with a sequence number, acknowledged through ack_through. The api dedupes by run_id.

15.11 Observability

Per run (agent, routine_runs; it travels with agent.db; the newest 100 per routine are kept):

Per alarm wake (api journal, structured, never content):

Per policy change (api journal): pause all, resume, the kill switch, an away-guard pause.

Per day and user (api, proactive.json): alarm wakes, box-seconds, runs, cost as reported by the agent and from gw, and the budget left.

Second signals:

How the operator sees it:

15.12 Decisions

Decided 2026-10-01: all eight items follow the recommendation below, with one addition to item 5. M4.9 as a whole is in the backlog (§18) and is not scheduled.

  1. Day budgets per user. Recommended for the first cut: 3600 box-seconds and $0.50 of model cost a day.
    • In the worst month that is 30 box-hours ($5.10 list, $1.30–1.92 Spot) plus $15 of tokens.
    • It covers about 140 text runs, or 10 GUI runs, a day.
    • A free tier would need about 600 s and $0.05 a day, which is 3 text runs with headroom.
  2. Keep 48 runs a day and the 1 h floor. Recommended. They bound frequency and NVMe writes, and the budgets bound money.
  3. Exact time or a flex window for cron. Recommended: cron runs within 5 min of their time, spread by a hash of the box id, so that wakes and hibernates do not pile up at :00. at one-shots keep the exact time. This costs nothing.
  4. Save before re-hibernate. Recommended: a live save (1–3 s) after any run that changed /workspace or agent.db, and none after a precheck-only run. A plain hibernate does not save today, so without this a node loss loses the run’s output.
  5. Linger after a run. Recommended: none; hibernate as soon as the Tick ends. A 60 s linger, in case the user opens the app, would almost double a text run’s machine cost. Addition: never hibernate while a client is attached (the edge Presence counter, 15.9b); the user’s own session keeps the box up.
  6. Idempotency for act. Recommended: settle it before act ships. Each run gets an idempotency key that write tools and M5 connectors receive, so a run repeated after a restore cannot file the same receipt twice. observe does not need it.
  7. What counts against the budget on a running box. Recommended: only tokens, not box-seconds, since the user’s session pays for the box being up.
  8. Zero-page skip (§18). It now has a second reason: the 24 s re-hibernate is most of a short run’s machine cost and all of its NVMe write. Recommended: revisit with the 9f numbers from real wakes.

15.13 As built: scheduled wake (option B, 2026-10-10)

A user asked for a routine at 09:00 while the Mac is closed. This is the alarm of 15.2 cut down to “wake, stay up for a while, sleep again”: the api owns the time and the box owns the content, a systemd user timer that runs the routine (contracts.md, Scheduled wake). No Routines service, no Tick, no lease and no notice exist.

16. M5: Connect

Proposed 2026-10-01, for review. It replaces §4.2(b), whose in-VM vault kept credentials at rest in the VM. The user links an external account (GitHub, Google, Notion, …) once in the app. After that, both kinds of agent can use it:

Local exec on the Mac (§4.2(c)) is not part of this section.

Constraints:

References:

16.1 Where credentials live: a host-side broker

The boundaries.md order is Mac Keychain first, then host side, then VM memory for one request. How the options compare:

  Mac Keychain Host-side broker
Mode B (local agents) works works
Mode A while the Mac sleeps (proactive runs, §15) fails: every refresh and every call needs the Mac online works
Implementations a second set of connector clients in Swift, next to the host’s one
Blast radius one Mac the api host; its master key opens every grant

Decided: host custody, stored in PostgreSQL (16.1.1).

The broker is a service-layer package of the api (api/connect), not a new component. It holds:

16.1.1 Storage: PostgreSQL

Connect uses the api’s PostgreSQL database, next to the users tables, with its own migrations (api/connect/migrations, tracked in connect_schema_migrations).

Why PostgreSQL:

Where it runs:

Encryption (envelope):

Tables (migration 0001; connect_door_keys came with the box door; since migration 0004 every user_id references users ON DELETE CASCADE):

Table Columns Notes
connect_grants id PK, user_id, connector, kind, mcp_url, account_label, scopes text[], status (active / needs_reconnect), policy jsonb, sealed bytea (access token, refresh token, an API key and its header, a registered client), token_expires_at, created_at, updated_at unique (user_id, connector); policy and the expiry stay plain, so listing and the refresh check need no decrypt
connect_flows id PK, user_id, connector, kind, mcp_url, state_sha256, session_sha256, sealed bytea (PKCE verifier, the registered client), expires_at 10 min TTL; deleted when the callback takes it
connect_approvals id PK, user_id, grant_id, tool, args_sha256, summary, decision (pending, once, always, deny), expires_at, used_at used once, 10 min (16.5)
connect_door_keys key_sha256 PK, machine_id, user_id, incarnation, created_at the box door key (16.3, migration 0003); the key itself is never stored
connect_audit id bigserial PK, at, user_id, connector, tool, door, caller, decision, outcome, upstream_status, latency_ms, req_bytes, resp_bytes, args_sha256 index (user_id, connector, at); a daily job deletes rows older than 90 days

How the broker uses it:

Hot-path cost: none on the claim, screen or edge path. A connector call adds one primary-key read and one audit insert, about 1 ms against a local PostgreSQL, next to an upstream call of 100 ms or more. A refresh adds one row lock, held only for the vendor round trip.

gw’s MCP gateway does not fit as the broker:

Adding those to gw would put cocoonbox’s user policy into infra.

16.2 The OAuth flow

The flow runs in the browser, from the console’s Connect page (/cb/connect), and the code comes back to the api, which does the exchange. Decided 2026-10-05: the console page replaces the app’s loopback listener, and the app’s Connections… item opens that page.

  1. The signed-in page calls POST /cb/api/connect/{connector}/begin. The api creates a flow:
    • a random state, bound to the user, to the hash of the console session secret and to the hash of a random 32-byte nonce, with a 10 min TTL, used once;
    • a PKCE S256 verifier, kept in the api;
    • the catalog entry’s scopes.

    It sets the cookie cb_connect to the nonce, for /cocoonbox.connect/ only, and returns the vendor’s authorization URL. Its redirect_uri is the api’s https://<endpoint>/cocoonbox.connect/callback.

  2. The page opens that URL in the same browser, where the user’s accounts are already signed in.
  3. The vendor redirects to the callback. The api completes the flow only when the hash of the request’s cb_connect cookie equals the flow’s nonce hash, the state is unused and unexpired, and the console session that began the flow is still live for the flow’s user.
  4. The api exchanges the code, using its client credentials and the verifier, and stores the grant. It redirects to /cb/connect?connected=<connector>, which shows the account as connected.

The cookies:

Why the flow is bound to the console session:

OAuth clients:

16.3 How a call reaches the account

Connector tools are MCP tools, served by the broker.

Doors:

Door Caller Auth Serves
Connectors gRPC (proto/cocoonbox/control/v1/connectors.proto) the app user token catalog, connect, policy, approvals, activity, disconnect
https://<endpoint>/cocoonbox.mcp/connect the bridge (mode B) user token the user’s connector tools
http://10.203.0.1:18091/mcp, host-internal the box (mode A) a per-machine key that the box never holds the box owner’s connector tools

The box door follows §12.2 (the proof comes from the channel) and the gw pattern of §14.5:

Egress injection, for programs that don’t speak MCP (M5.2, in the first cut). Some work needs the credential on the wire of an ordinary program, above all git and gh against GitHub. For that case only:

How it works:

Limits, from sandboxd’s code:

Every other connector stays broker-only.

Per-claim credential rules (available since sandboxd e0e27fc, cocoonstack/sandbox#302). Today every injected host and secret is node config, so each new vendor would be a config change on every node, which puts cocoonbox’s catalog into infra. The upstream primitive is a per-claim credential list ({hosts, header, value}), set through the API, validated as a subset of the pool’s intercepted hosts, persisted with the claim, redacted on read and never delivered to the guest (connect-research.md §4). With it, a vendor without MCP becomes a catalog entry, and its logic runs in the box as a CLI, SDK or script. Such grants get vendor-scope approval only, and the app says so at connect time. Filed as cocoonstack/sandbox#301 (2026-10-03).

With #302, M5.2 uses claim env entries with inject instead of pool secret rules: the node config keeps only egress_ca and the intercept: true rules for the GitHub hosts, and the api sets and rotates the values per claim (rotation works on a hibernated claim). egress_ca loads only at a sandboxd start and only when a pool intercepts. cbsbx-conf writes it whenever /etc/cocoonbox/egress-ca/root.crt exists, so an image cut keeps it. Placeholder gating (sandbox #307, 858a80e) keeps the injection off the box Chrome’s github.com traffic. The recipe:

Accepted on cbsbx 2026-10-03: #302 (9 steps) and the placeholder gate with real git 2.43.0. The NSS import and the git switch were accepted the same day on a throwaway claim of a test pool that intercepts postman-echo.com and github.com, with the branch’s image files installed by hand: Chrome loaded the intercepted host with the egress intermediate as issuer, and failed with ERR_CERT_AUTHORITY_INVALID before the import; git sent no Authorization to github.com without the flag, and the placeholder with it.

16.4 Refresh, rotation, revocation

16.6 Audit and blast radius

16.7 Connectors in the first cut

Connector Auth Why Gate
GitHub our GitHub App the most asked-for by coding work; no vendor review none
Google Calendar our Google client Calendar scopes are sensitive, not restricted (to confirm at registration) Google brand verification
Notion, Linear the vendor’s MCP server, with client metadata or dynamic registration no per-vendor code none
Custom MCP server (URL, plus OAuth or a key) MCP authorization covers the long tail –
Gmail, Drive (read) our Google client high value see below
Slack – not in the first cut: no adapter waits for a directory listing (Slack’s MCP server serves only listed apps) or for per-claim credential rules (16.3)

Gmail and Drive:

Google’s Workspace MCP servers are in developer preview, and our account is not enrolled (2026-10-05). Google Calendar is therefore a grant-only connector: Connect, refresh, revoke and Activity work, it serves no tools, and programs in the box will use it through per-claim credential rules (16.3, an inject entry for www.googleapis.com with Authorization: Bearer <token>). The broker gets no adapter.

Connector events for §15.2’s triggers (GitHub webhooks, Gmail push) arrive at the broker. They are not in the first cut.

16.8 Work split, order and acceptance

Split:

Order:

  1. proto and contracts.
  2. api and macos. This gives mode B end to end on the current backend.
  3. agent: mode A, on sandboxd.
  4. M5.2, the last step of the first cut.

Hot-path cost: none on the claim, screen or edge path. A connector call adds one host hop, and the bridge opens one more MCP session at start. M5.2 adds one host-local sandboxd env call before the wake of a box with a GitHub grant, plus a vendor refresh only when the token is about to expire (16.4).

Acceptance, on the bench host through the public edge:

  1. Connect GitHub and Google Calendar from the console’s Connect page.
    • The consent page opens in the same browser, and the page shows the account as connected.
    • Connectors.List output has no token field.
    • Negative: a callback without the console session that started the flow (no cookie, another user’s session, a replayed or expired state) is refused.
  2. Mode B, box hibernated. A local agent CLI, with the app closed, lists this week’s Calendar events and the user’s GitHub issues. cocoonbox status still says hibernated.
  3. Mode A, on sandboxd.
    • The box agent lists issues. This is a read, and nothing asks.
    • Then it creates an issue. The app asks, with the broker’s summary. Approve → the issue is created.
    • A retry of the same approval with a changed title is refused. Deny → refused.
    • Activity shows all four calls.
  4. No credential in the VM. The acceptance script decrypts the grants on the host (root, with the master key) into a pipe. It then searches for the access and refresh tokens in:
    • the guest’s /proc/*/environ, /etc, /run, /home, /workspace and agent.db;
    • the box’s hibernate memory file on the host;
    • the Mac’s client configs and shell history;
    • a pg_dump of the database.

    No match. A positive control matches.

  5. Refresh. With the cached expiry set in the past, two concurrent GitHub calls, sent to two api processes on the same database, cause one refresh, and the grant still works afterwards.
  6. Disconnect.
    • GitHub reports the old token revoked (401).
    • The tools disappear in both modes (list_changed).
    • Activity records the disconnect.
  7. M5.2.
    • In the box, git clone and git push of a private repository in the selection work, with no credential helper and no token in the guest (searched as in 4).
    • GitHub refuses a repository outside the selection.
    • After the box was hibernated for longer than the token’s 8 h, the first git fetch after the wake works.

Decided: host custody of every grant, with no Mac-only option, stored in PostgreSQL (16.1, 16.1.1).

Decided:

  1. Gmail and Drive read ship in the first cut as an unverified Google app (at most 100 users, with the warning screen). Verification and CASA come before general use.
  2. In mode B, ask is the client’s own permission prompt (16.5).

Decided 2026-10-03, from connect-research.md §5:

  1. The broker is a generic MCP client of vendor-hosted MCP servers, with no per-vendor adapters (16.3).
  2. GitHub egress injection (M5.2) is in the first cut, on today’s sandboxd (16.3).
  3. Per-claim credential rules are a later upstream sandboxd primitive, an open item to file as a cocoonstack/sandbox issue (16.3).

17. M6: Market

Proposed 2026-10-01; in the backlog since 2026-10-01 until its two open questions are decided (§18). It replaces §4.3. Users install capabilities (skills, MCP servers, agent presets, desktop apps) for the box and for their local agents, from one catalog.

What goes where:

Capability Box agent (mode A) Local agents (mode B)
skill (skills/*/SKILL.md), command, agent preset (agents/*.md) loaded by the box agent from the box’s user state installed by the client’s own plugin manager from the same catalog
stdio MCP server runs in the box as user. Its tools join the dispatcher, so the box’s /mcp serves them too uses the box-hosted tools through the bridge, or installs the server locally through its own plugin manager
remote MCP server that needs OAuth or a key a connector (§16): the broker is its client, and nothing lands in the box the same connector, through /cocoonbox.mcp/connect
desktop app installed per user in the box, with Flatpak –

17.1 Package: the M4.7 plugin format

A plugin uses the agent-plugins layout that the three clients share (§13.5):

Existing public plugins install unchanged. What only the box needs goes into one extra manifest, .cocoonbox-plugin/plugin.json, which the clients ignore:

{ "requires": { "connectors": ["github"], "egress": [{"host": "api.example.com", "ports": [443]}], "apps": [{"flatpak": "org.gimp.GIMP"}] } }

A plugin without that manifest counts as requires: {}.

Variables:

17.2 Catalog, integrity, trust

17.3 Where installs land

17.4 Runtime in the box

17.5 Update, disable, uninstall

17.6 Interface

cocoonbox.agent.v1.Plugins is served by the agent. The edge routes it like Agents.

service Plugins {
  rpc Install(InstallRequest) returns (stream InstallProgress);   // catalog URL, plugin, SHA, non-secret variables
  rpc Update(InstallRequest) returns (stream InstallProgress);
  rpc SetEnabled(SetEnabledRequest) returns (InstalledPlugin);
  rpc Uninstall(PluginRef) returns (Empty);
  rpc ListInstalled(Empty) returns (ListInstalledResponse);
}

17.7 First cut and later

M6.1, the first cut:

M6.2:

17.8 Work split and acceptance

Split:

Hot-path cost: none on the claim or screen path.

Acceptance, on the bench host:

  1. Install in the box. An existing public agent-CLI plugin, made of a skill and a secret-free stdio MCP server, installs unchanged from the catalog.
    • The permission sheet shows the server’s command line.
    • In a box-agent turn, the skill loads and the MCP tool runs.
  2. Mode B. With the plugin installed, a local agent CLI (app closed, through the bridge) sees the plugin’s tool after list_changed, and runs it in the box.
  3. Local install. The same plugin installs locally in claude and in codex, from the catalog used as a marketplace.
  4. Refusals. Each of these is refused with a message that says why:
    • a tampered tarball (sha256 mismatch);
    • an archive with a symlink;
    • a plugin with a secret variable for a stdio server in the box.
  5. Disable and uninstall.
    • Disable → the tool is gone in both modes.
    • Uninstall → the files, the record and the process are gone.
    • An idle server stops after 10 min, as the process list shows.
  6. Rebase. After a Rebase, the plugin is still installed and works.
  7. Memory. Box memory with the plugin installed and idle is back at the baseline, because the server stopped. Measure how long the first call takes, including the server start.

Open (deferred to the backlog):

  1. Whether the catalog repository is public: loomhand/cocoonbox-plugins. It must be public for cursor-agent.
  2. No hooks in the box in the first cut.

18. Backlog

19. Several boxes per user

Proposed 2026-10-01. Direction: an agent in a box drives only that box, and a local coding agent may drive any of the user’s boxes, told apart on the MCP side.

19.1 Which agent drives which box

19.2 Where the split lives: the bridge

The local bridge (cocoonbox mcp, §13) already merges the control MCP and one box’s MCP into one stdio server. It becomes the multiplexer:

Not chosen:

The app’s + → Local agent starts the CLI through the bridge (stdio) instead of the direct HTTP config, with --box set to the box selected in the app.

19.3 Control plane

19.4 App

Decided 2026-10-03: an agent is an agent, a box is a box.

19.5 Order and acceptance

Order:

  1. proto and contracts.
  2. api: Create, Delete, name and quota.
  3. Bridge multiplexing and the control tools.
  4. App switcher and + through the bridge.

Hot path: none on the screen or edge path; the bridge adds one map lookup per call.

Acceptance, on the bench host:

  1. Create test next to the existing box. Running a second box at once is refused, and works after hibernating the first.
  2. One local agent CLI session, with no restart and no config change:
    • lists the boxes;
    • creates and wakes test;
    • runs uname -n in both boxes with box set;
    • each box’s transcript shows only its own call.
  3. A third box is refused by the total quota.
  4. Delete test: its machine and gw key are gone, and its last snapshot is still in the store.
  5. Another user’s box name or id in box is refused by the edge.

19.6 As built (2026-10-03)

Reconciled with §22–§26 before building:

Not built: per-box agents in MockBox, a CLI list command, per-box cursor-agent workspaces (two cursor-agent agents for different boxes share one .cursor/mcp.json, written just before each starts). A rename made elsewhere reaches a running bridge only on its next name miss, create or delete, so until then a box tool may still find the box under its old name.

Acceptance: deploy/accept/multibox.sh on the bench host with a scratch user (19.5 steps 1, 3, 4, 5 scripted, and step 2’s routing checked over the edge by each box’s identity file, since clones share a hostname; the agent CLI session itself stays manual and the script prints it. Step 2’s per-box transcripts no longer apply, since the owner’s sessions are not tracked).

20. Sign in to a website with a password saved on the Mac

Proposed 2026-10-01. The user has a site’s password in the Mac’s Passwords (iCloud Keychain). The box’s Chrome should sign in with it, and the password should never be written in the VM or seen by any model.

boundaries.md invariant 2 allows exactly this shape: “inside the VM only in memory for one request”. The session cookie that results is the existing exception, browser sessions, encrypted in the state layer.

20.1 Flow

  1. Pick on the Mac. A third-party app cannot read Passwords entries directly, and an ad hoc signed app gets no AutoFill on an arbitrary site (20.4). The app opens a small native sheet with a username field and a password field (NSTextField and NSSecureTextField). The user pastes the values from Passwords or 1Password, or types them, and the app holds them in memory only. The fields keep contentType .username and .password, which cost nothing and may get AutoFill once the app has Developer ID signing.
  2. Send. The app calls a new agent RPC, Session.FillCredential{origin, username, password, submit}, through the edge with the user token. It is app-only:
    • it is not an MCP tool;
    • neither the box’s loop nor a local agent can call it or see its arguments;
    • it never appears in a transcript, an event or a log.
  3. Fill. The toolhost picks the Chrome tab whose top-level origin equals origin (scheme, host and port) and fills the page’s sign-in form through CDP (the focused field, else the first username/password pair), then submits if asked.
    • A tab on any other origin is refused, so a phishing page or a redirect gets nothing.
    • The values live in the agent and toolhost memory for this call only, and are zeroed after.
  4. Keep it off disk. The image sets Chrome’s PasswordManagerEnabled=false (managed policy), so Chrome never offers to save the password. The resulting cookies follow the browser-session rule.

20.2 Entry points

20.3 Not covered, and second factors

20.4 AutoFill in the sheet

20.5 Work split and acceptance

Work split:

Hot path: none.

Acceptance, on the dev box:

  1. Sign in to a test site (a local page served in the box with a known account) from a Passwords entry. The page shows signed in.
  2. Search for the password in:
    • the guest’s /proc/*/environ, /home, /workspace, /tmp, agent.db and the Chrome profile;
    • the next state snapshot;
    • the api and agent journals. No match. A positive control matches.
  3. A fill aimed at an origin that doesn’t match the active tab is refused, and nothing is typed.
  4. The box agent and a local MCP client both list their tools, and neither has a fill tool.

21. Residential or datacenter exit per box

Proposed 2026-10-01. Some boxes should reach the internet through a residential exit, for sites that block or throttle datacenter addresses; the rest keep the node’s datacenter exit. The choice exists only at the sandboxd layer. Inside every box the proxy is still 127.0.0.1, and nothing in the guest says which exit it has.

21.1 Pieces

21.2 Provider facts

21.3 Order and acceptance

Order:

  1. #279 lands in sandbox; cbsbx is upgraded to it.
  2. Forwarder unit on the bench host, with its credential file, a port-scoped internal allow entry, and a dedicated bridge address. It never binds 0.0.0.0 or the host network.
  3. api setting plus Deliver/PATCH.
  4. Docs.

Hot path: none on claim, screen or edge. Residential latency applies only to the boxes that chose it.

Acceptance on the bench host, with dev on residential and a throwaway claim on datacenter:

  1. curl https://ifconfig.me inside each box shows a residential address for dev and the node’s address for the other. Both boxes’ proxy env says 127.0.0.1.
  2. The provider credential appears in no guest, claim env read, api log or usage/audit record.
  3. Stopping the forwarder makes dev’s requests fail with 502, never a direct dial.
  4. Flipping dev back to datacenter by PATCH changes its exit for new connections without a restart.

22. One cocoonbox user, one gw key, one sandboxd tenant

Built 2026-10-02 (22.4 records what was built and the decisions made that night). Proposed 2026-10-01. gw and sandboxd are both multi-tenant. cocoonbox-api keeps a per-user record that links a user to exactly one gw key and one sandboxd tenant, and it hides both from the user: clients hold only their cocoonbox user token. gw then shows each user’s total model spend, and sandboxd shows each user’s boxes and caps them.

22.1 What changes from today

  Today Proposed
User record users in api.yaml (name and token hash), edited by hand, api restart A users table in the api’s PostgreSQL (the instance Connect uses): user id, token hash, gw ak_id, sealed gw key, sandboxd tenant name, sealed tenant token, state
gw key One per machine (owner = machine id), issued on every new machine and revoked on Rebase One per user (owner = user id), issued once at sign-up and revoked when the user is deleted. Every machine of the user gets the same key in its claim’s guest: false env
Model budget Per key = per machine; the tenant pool is shared Per key = per user: /admin/usage by key is the user’s total; the user’s day budget is a per-user override in the one tenant (§22.2, gateway#144)
sandboxd One tenant-less root token; boxes found by cb_owner metadata One sandboxd tenant per user (name = user id, its own token, max_claims = the user’s box quota, §19.3). The api creates and lists a user’s claims with that user’s tenant token, so a bug in the api cannot list or claim across users. Root-only surfaces (per-id reads, GET /v1/info) keep the root token
Onboarding Hand edit plus restart An admin call on the api (later the web console) creates the user, issues the gw key and adds the tenant; deleting reverses all three

The two invariants hold:

Sealed values (as built, 22.4): AES-256-GCM directly under the master key (a 32-byte file on the bench host), additional data users|<id>|<column>. There is no per-user data key, no envelope and no master key rotation path yet; the §16.1.1 envelope replaces this when Connect lands.

22.2 What it needs upstream

22.3 Order and acceptance

Order:

  1. sandboxd runtime tenants (upstream).
  2. The api users table, the admin create/delete calls, the per-user gw key (the current gwkeys client issues by user instead of by machine), and Deliver injecting the user’s key.
  3. The api moves to the user’s tenant token for claim and list.
  4. Migrate dev and test.
  5. Delete the users list from api.yaml.

Hot path: none on claim, screen or edge. Deliver reads one sealed key per provision.

Acceptance on the bench host:

  1. Create a user through the admin call, with no restart of api or sandboxd. Its first Ensure works; gw lists one key with owner = the user id; sandboxd lists the claim under the user’s tenant.
  2. A second box for the same user carries the same gw key (same ak_id in gw’s usage), and the user’s max_claims refuses one box more than the quota.
  3. gw’s usage for that key equals the sum of both boxes’ model calls.
  4. Delete the user: the key is gone (401), the tenant is gone, the claims are released, and the state prefix stays for the retention window.
  5. User A’s token cannot reach user B’s box at the edge, and the api’s tenant-scoped list for A never shows B’s claims.

22.4 As built (2026-10-02)

Decided and built:

Hot path: none. Token checks read an in-memory index, updated by every write; the root CLI sends NOTIFY cocoonbox_users after its writes, and the api, which LISTENs, reloads the index on each notification and after each (re)connect of its listening connection. Ensure unseals one tenant token (microseconds); provisioning unseals the gw key once per new machine.

22.5 State key and chunk store per user (decided 2026-10-02, built 2026-10-02)

§11 says per-user keys and one prefix per user; the code drifted to one key and one prefix per box (keyring.Key(ctx, box) reads /var/lib/cocoonbox-api/keys/<box>.json, and api/boxes/statelayer.go:404-407 keys every object by box). With one box per user the two are the same. With several boxes per user (§19) they are not, so the per-user form was decided:

Hot path: none. Save and restore read one sealed key per request instead of one keyring file.

As built (api 185e1bf, b8de45e; switched 2026-10-02):

Acceptance on the bench host:

  1. After the migration, the two existing boxes each restore (a same-image Rebase of one, the other on its own upgrade) from <user>/boxes/<box>/head.json, with the same snapshot id and file count as before.
  2. With §19, a second box of dev seeded with a file already in its first box uploads no chunk for that file.
  3. Deleting a test user removes its state_key; its snapshots can no longer be opened, and its prefix goes with the retention sweep.

23. Sharing a box

Decided 2026-10-02, revised 2026-10-03. A box owner shares one box through a share link. Whoever holds the link watches the screen in a web page and can connect their own TUI coding agents to the box over MCP. Nobody takes over the screen through a share. The same change stops tracking the owner’s own MCP sessions as agents.

23.1 Decisions

23.2 Pieces

Hot path: the edge adds one share lookup (in memory, like the token index) per call that does not come from the box’s owner. Owner calls are unchanged.

23.3 Acceptance on the bench host

  1. qa1 creates an open share of its box. With only the link, a browser shows qa1’s screen and ignores mouse and keys. An agent CLI configured with the share lists only the read-only tools, and a shell call is refused.
  2. qa1 turns on write. The next shell call from that agent CLI runs. Lifecycle tools and FillCredential stay refused. Turning write off refuses the next shell call.
  3. A named share for qa2 refuses the link without qa2’s token and accepts it with qa2’s token. qa3’s token is refused.
  4. qa1’s app lists the share session under External with its calls. qa1’s own agent CLI session creates no agent on the box.
  5. A Rebase of qa1’s box waits for a share session’s running shell call before it quiesces.
  6. Revoking, regenerating and expiry each cut the share’s page and MCP sessions within a second, and the old link answers PermissionDenied afterwards.

23.4 As built (2026-10-03, stage 1)

Decisions made while building, on top of 23.1 and 23.2:

Hot path:

Deploy:

Decided 2026-10-03. Before this, the api kept only the SHA-256 of a share secret and the app kept the secret in memory. After an app restart, the Share sheet found the live share but could not show its link. The owner’s only way to get a link was Regenerate, which ended the link that guests already used. Now the api keeps the secret sealed and returns it to the box’s owner. Nothing goes in the Mac keychain.

Hot path: none. The edge lookup is unchanged; List reads the secret from the in-memory index, opened once when the api loads the shares.

Deploy: migration 0006 runs at start. The api before this change (7a3c012) runs on a database with 0006 applied: it names its columns and ignores secret_sealed. Rolling back leaves sealed secrets that the old api does not update on Regenerate, so the rollback sets secret_sealed to NULL on every row; those shares then show “Regenerate to get a link” after the next deploy.

23.6 A read share reads files only under /workspace

Decided 2026-10-03. A read share’s read_file and list_dir reached every file the desktop user can read: the Chrome profile (cookies, Login Data) and the box agent CLIs’ logins (~/.claude, ~/.codex and the third CLI’s config directory). A link holder could take the owner’s web sessions and agent subscriptions, far more than seeing the screen.

Hot path: owner and write calls add one match on the principal. A read share’s file call adds one open of /workspace.

Deploy: ships in the agent of cbdesk:v20; the app and the console wording ship with their next build.

24. User data on a persistent data disk

Decided 2026-10-02. An upgrade today saves the user’s files to the store and restores them into a new machine. That is fast now for the save, but the restore moves every byte: a box of 96,949 chunks (9.3 GB) restores in about 87 s. Instead, the user’s data lives on a persistent data disk that the box carries from machine to machine; an upgrade replaces only the system disk.

24.1 Layout

24.2 Phases

  1. Node-local disk. A raw sparse file on the node’s XFS (reflink-capable), attached through cocoon’s runtime disk attach. Needs cocoonstack/sandbox#300 (per-owner persistent volumes: runtime API, attach to warm claims, single writer, hibernate, node placement). cocoon stays loosely coupled: its runtime vm disk attach/detach is enough, and sandboxd detaches the volume (after the guest unmounts it) before a hibernate or snapshot and re-attaches and re-mounts it after the wake (cocoonstack/cocoon#280 closed as control-plane logic). The state layer stays as the off-node backup and for node loss and moves; it saves from the mounted disk, unchanged. A box is bound to its disk’s node.
  2. S3-backed block device. The same volume API, with the disk’s blocks in S3 behind a node-local write-back cache, so a box attaches on any node and node loss costs only unflushed cache. Not S3FUSE for /home: Chrome profiles, SQLite and git need locking, fsync, atomic rename and mmap that FUSE-on-S3 does not give. At this phase the state layer becomes snapshots of the volume.

24.3 Open

Hot path: none on the screen or edge path. Wake gains one disk attach and mount; upgrade loses the restore.

25. Time capsule

Decided 2026-10-03. The user picks a point in time and gets back the files they had then. The state layer (§11) already keeps every save, so the history is in the store; what is missing is retention, garbage collection and a way to restore part of a snapshot into the running box.

25.1 Today (verified 2026-10-03)

25.2 Decisions

25.3 Retention and garbage collection

Metadata:

Both steps below run in one background pass in the api, per user: prune every box of the user, then sweep the user’s chunks. The pass runs once at start and then every state.gc.interval.

Prune (per box, under the box’s lock):

  1. Apply the rule to the timeline. The head and every pinned snapshot are kept.
  2. Write the timeline without the dropped snapshots.
  3. Delete their records, then their manifests, and every record or manifest the timeline does not name (a commit that failed after Upload).

A crash between 2 and 3 leaves orphan records; the next prune finds them by listing snapshots/. Every reader (restore, browse, the mark) takes the kept set from the timeline, never from the listing.

Sweep (per user, after the prune):

  1. Mark, without a lock. Read every box’s timeline and the chunk list of every record it names. Records are immutable and pruning only removes, so what is marked stays true; only saves that commit later add references. List <user>/chunks/.
  2. Take the user’s lock exclusively, then the index lock. Every save holds the user’s lock shared from its start to its commit, so no save is between its index read and its commit while the sweep runs. Re-list the user’s boxes, re-read the timelines and mark the records committed since step 1.
  3. Garbage = the index and the chunk listing, minus the marked chunks.
  4. Still under the lock: write the index without the garbage, then delete the garbage objects (DeleteObjects, state.gc.delete_batch per call, at most 1000), then release.
    • The index goes first. A crash after it leaves only unindexed objects: a save that needs one uploads it again, and the next sweep deletes the rest.
    • The reverse order would leave index entries for deleted chunks, and a save would skip uploading a chunk that is gone.
    • The deletes stay under the lock: once the lock is released, a save may upload a chunk from the garbage set again, and a late delete would remove it.

Why it is safe against the agent’s stamp reuse: the agent’s known chunks are those of its last upload, which is the box’s head, and the head is always kept. If they are ever ahead of the index, the existing heal applies (Upload refused, plan again without a parent, §22.5).

Cost: the mark reads every kept record, about 5 MiB each at that box’s size, so it runs once a day and never under the lock. The lock covers the new records, one index write and the deletes; that user’s saves wait for it. The locks are in-process, like today’s save locks; several api replicas need them in PostgreSQL first.

The store interface gains Delete (S3 DeleteObjects), and List, which the migration already used, moves into it and returns each object’s size.

As built (2026-10-03):

25.4 File and folder restore

  1. The app shows the timeline (ListSnapshots). The user picks a snapshot and browses its tree (BrowseSnapshot, one directory per call). A hibernated box is woken first.
  2. The api, under the box’s lock, checks that the snapshot is in the timeline. It calls the agent’s Restore with a scope: the paths, and a target folder name.
  3. The agent reads the manifest and selects the entries under the paths. Its first answers list the chunks they reference, in pages of 2048. The api checks that each one is in the record, and presigns and pages only those.
  4. The agent writes the entries as user into ~/Restored/.cb-restore-<id>/ and renames that to ~/Restored/<timestamp>/ at the end, so a failed restore leaves only a leftover that snapshots already exclude.
    • <timestamp> is the snapshot’s save time; a second restore from the same snapshot gets a -2 suffix.
    • <path> is the box path without its leading slash, for example ~/Restored/2026-10-02T1608Z/workspace/app/main.go.

What a scoped restore does not do: quiesce, stop the desktop, close Chrome, touch agent.db, or write outside ~/Restored. A path must lie under one of the agent’s roots (/home/user, /workspace) and contain no .., and the target is one folder name. Anything else is refused.

The next save uploads no chunk for the restored files: their chunk ids are keyed hashes under the same user key, so they are already stored.

The agent keeps the decrypted manifest of the last Browse in memory for 10 min, so browsing a tree does not fetch it once per directory.

Decided and built 2026-10-03. Where it differs from the plan above:

Hot path: none on claim, screen or edge. A save pays one map entry per re-read file for changed_files. Browse and RestorePaths are their own calls.

25.5 Pieces and contract

Sketch; the fields are the contract to refine (as built: no State.Browse and no TransferProgress.needed; State.RestorePaths takes the scope; TransferProgress.target = 7; EntryKind values ENTRY_KIND_FILE|DIR|SYMLINK; BrowseSnapshotResponse.truncated past 5000 entries; see 25.4):

service Boxes {           // clients, through the edge
  rpc ListSnapshots(BoxRef) returns (ListSnapshotsResponse);                 // kept snapshots, newest first
  rpc BrowseSnapshot(BrowseSnapshotRequest) returns (BrowseSnapshotResponse);
  rpc RestorePaths(RestorePathsRequest) returns (Operation);
}
message Snapshot { string id = 1; int64 saved_at_ms = 2; bool quiesced = 3; uint64 files = 4;
                   uint64 logical_bytes = 5; optional uint64 changed_files = 6; uint64 added_bytes = 7; }
message BrowseSnapshotRequest { string box_id = 1; string snapshot_id = 2; string dir = 3; }   // a box path; "" lists the roots
message BrowseSnapshotResponse { repeated SnapshotEntry entries = 1; }
message SnapshotEntry { string path = 1; EntryKind kind = 2; uint64 size = 3; int64 mtime_ms = 4; }
message RestorePathsRequest { string box_id = 1; string snapshot_id = 2; repeated string paths = 3; }

service State {           // api only
  rpc Browse(BrowseRequest) returns (BrowseResponse);
}
message BrowseRequest { StateKey key = 1; PresignedGet manifest = 2; string dir = 3; }
message RestoreScope { repeated string paths = 1; string target = 2; }      // target: one folder name under ~/Restored
// RestoreRequest.scope = 4: unset restores the whole state, as today.
// TransferProgress.needed = 7 (repeated string): a scoped restore's chunk ids, in pages, before any data.
// PlanResponse.changed_files = 6.

25.6 Acceptance on the bench host

On a scratch user’s box, never a real user’s:

  1. Timeline. Five saves, each after writing one known file. ListSnapshots lists five, each with 1 changed file and added bytes near that file’s size.
  2. File restore. RestorePaths of /workspace/t/file-2 from the second snapshot writes ~/Restored/<timestamp>/workspace/t/file-2 with the sha256 written before that save.
    • The desktop session and Chrome keep their PIDs.
    • A walk of both roots before and after differs only under ~/Restored.
    • The api presigned only that file’s chunks (its log).
  3. Folder restore of /home/user/Documents works the same way. A path under /etc, a path with .. and a target with / are refused.
  4. Prune. state prune -now <40 days ahead> keeps exactly the rule’s set. The sweep deletes exactly the chunks referenced only by dropped snapshots (checked against a mark computed offline from the records). A file restore from every kept snapshot still works.
  5. Concurrent save. A sweep started during a large save of a second box of the same user waits for that save’s commit, and the new snapshot restores. A unit test with a fake store forces the interleavings (sweep between index read and commit; delete after release).
  6. Crash. Killing the api between the index write and the deletes: the next save that needs a garbage chunk uploads it, and the next sweep removes the leftovers.
  7. The first real prune of a long-lived box: object counts and bytes before and after, and a file restore from the oldest kept snapshot.

25.7 Open

Hot path: none on the claim, screen or edge path. A commit writes one more small object (timeline.json). Prune and the sweep run outside every request; while the sweep holds a user’s lock, that user’s saves and restores wait. The new RPCs are their own calls.

25.8 Slimmer snapshot metadata

Decided and built 2026-10-03. Each record repeated the full chunk list (5.3 MiB on the box of 25.1) and each manifest the full file list (3.2 MiB), so a save wrote about 8.5 MiB of metadata and every kept snapshot stored it again.

Hot path: none on claim, screen or edge. A delta save does less store I/O than before and a few ms more CPU; a restore of a compacted snapshot rebuilds and writes one manifest first. Compaction runs in the GC pass.

26. Several agents on one desktop

Decided 2026-10-03. Several agents (mode-A loops and MCP sessions) run in one box at once, and today they all drive one desktop. A user saw two agents typing into each other’s Chrome tab. Stage 1 gives each agent its own Chrome window on :0. Stage 2, private displays per agent, is designed here and built when several agents need non-browser GUI apps at the same time.

26.1 The problem (verified 2026-10-03)

26.2 Stage 1: an own Chrome window per agent

Hot path: one map lookup per browser call; no new process, no memory beyond the window’s tab.

26.3 The :0 policy

26.4 Stage 2: a private display per agent (design)

Display N (N ≥ 1):

Allocation and release:

Caps and fallback:

App view and take-over:

What changes, file by file:

Prototype (dev’s box, 2026-10-03, nothing installed). The image already has Xorg, the dummy driver and the TigerVNC module.

Acceptance on the bench host (dev’s box, never a real user’s):

  1. Two MCP sessions each call computer_act. Each gets its own display (:1, :2); XI2 listeners on :0, :1 and :2 count each session’s keys only on its own display.
  2. The app shows each agent’s display when it is selected. Taking over :1 refuses that agent’s next computer_act and leaves the agent on :2 running.
  3. The fourth concurrent agent’s computer_* gets “no private display free” with the holders listed; nothing runs on :0.
  4. Closing an MCP session stops its display’s units within a second of the close; MemAvailable returns to within 20 MB of before.
  5. Without the hand-over, computer_act on :0 is refused; after it, the chosen agent acts on :0 and a second agent is still refused.

Hot path: one map lookup per tool call in serve and in the toolhost; nothing per frame, the screen relay dials another port. Allocation runs once, at an agent’s first computer_* call (about 1 s: Xorg 0.2 s, cua-driver 0.75 s). Off the hot path, each held display keeps about 90 MiB resident, which also grows the hibernate image.

26.5 The Chrome profile constraint

27. Offload

Decided 2026-10-03. The contract is contracts.md “Offload”.

The user is about to leave the Mac. The task a local coding agent session works on moves into the user’s box, where the same kind of CLI, signed in inside the box, keeps working; on return the box’s commits and hand-over come home, and the local session continues. This is mode C, beside mode A (the in-box loop) and mode B (a local CLI over MCP).

27.1 Model

27.2 Decisions

# Decision
D3 The box CLI runs in bypass-permissions mode by default (--dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox); the offload sheet says so
D5, D14 The work tree minus git-ignored files, and .git without objects/, by rsync; history by bundle; ignored files are rebuilt in the box
D6 Merge back: fast-forward, else a clean three-way merge applied uncommitted, else a branch
D7 Share guests: covered by §23.6 (a read share reads only /workspace); a write share has shell, and the UI warns next to “Allow write”; box_agent_* tools, if they ship, are owner only
D9 The app shows the box CLI through Session.Exec with a PTY running tmux attach, in a Ghostty pane
D12 One box CLI session per (repo, kind), named by both; at most one runs per repo (the guard) and 3 per box
D13 Preflight outcomes as in contracts.md: blocks for a box older than v22, missing box tools, a signed-out box CLI, unreturned work, a full box, a non-git directory and macOS-only commands; warnings for local services, VPN-only hosts, secrets, LFS and submodules
D15 No push from the box until M5.2; commits come home with back
D18 The copy’s git-ignored directories are left out of state saves; the box packer lists them at each offload
D20 Hibernate and upgrade stay allowed; the app and the CLI warn first while a box agent works
D21, D22 The box CLI’s own login; the hand-over as a compaction
D23 The inbox reached through rsh is the agent-to-agent channel; MCP box_agent_send / box_agent_read wait until after v1
D24 Opted-in secrets live on tmpfs and never reach the state store; a node-local hibernate image carries them, accepted like the box login, and end wipes them
D25 --kind may pick another vendor for the box; the default is the local kind
D26 The image carries no box CLI. When the preflight finds the CLI missing, it offers “Install in box”, which runs the vendor’s official installer over rsh into /home/user, so the install persists through the state layer; the user then signs in and updates it like any program

27.3 Facts from O0 that shaped the contract

27.4 Entry points

Decided 2026-10-03. The app’s own local agents (the agent CLIs it launches) drive boxes over MCP; their work target is already the box, so their pane has no “Move to box” or “Bring back”. Offload is for the user’s own terminal sessions: a claude or codex TUI in any repo.

Hot path: none. The edge and the api do not change; the inbox is idle between messages; the event is one bus publish per outbox entry or status change. The Offloaded section asks each running box with a record for offload status (one Exec) when the app connects, comes to the front or selects a row, never on a timer.

Order: O2 agent and image (the box packer, the inbox and its unit, the events, the tmpfs mount, the state excludes), O3 macOS (the packer, the skill, the app), O4 end to end on the bench.

28. Flows follow the user

Decided 2026-10-09. A flow saved in one cbcore box replays in every cbcore box of the same user. The user can export flows to a file and import them again, so a client app can carry a working flow to another box, and later to another computer. Today a flow lives only in the box that saved it: gui_plan writes /workspace/flows/<name>.json and agents write scripts as /workspace/flows/<name>.mts.

28.1 Shape

box (cbcore)                      api                                 Mac
toolhost: /workspace/flows  ◄──►  serve: Syncer ──box door──►  flows.Library ◄── cocoonbox.control.v1.Flows ◄── cocoonbox flows
          (files, INDEX)          agent.db flow_sync          (state store, <uid>/flows/)         (user token)

28.2 Sync

28.3 Decisions

Question Decision
Conflicts Last write wins, in the order the api receives writes. Every write takes the library’s next rev, and a write of identical content changes nothing. A box never overwrites a local edit with a pull; its push then wins. The losing version stays in the git history of the box that had it (gui_plan commits every save, the pull commits after it writes). Per-flow merge or CAS buys nothing for a single-author GUI script.
Limits 128 KiB per flow, 200 flows and 3 MiB per user, so one Export answer fits gRPC’s 4 MiB default. A flow is typically 2–10 KiB. Over a limit: HTTP 413 at the door, RESOURCE_EXHAUSTED on gRPC.
Another app version Each library entry records the box and the image that wrote it last (box_id, image), and cocoonbox flows list shows them. Replay already resolves each step by its saved fingerprint, then by name, then by a decision; a step that still fails escalates with escalate.reason, the step and the screen it saw, as today. A replay that passes with re-resolved steps saves the flow, and the push carries the new fingerprints and image to the library. A flow therefore follows the image it last passed on.
Deletion cocoonbox flows rm NAME (gRPC Delete) deletes from the library; every box removes its unchanged copy at its next pull. Deleting the file in a box stays local: the box does not pull it again until the library has a newer version. This way an agent that clears /workspace cannot wipe the library. Removing the whole folder is a lost folder, not a delete of each flow: the rows hold only for the folder they were agreed in (.sync-id), so a new folder gets every library flow it lacks, as a new box does.
Privacy Flows hold typed values and params, so the library is owner-only: the door acts only for the machine’s owner, Flows only for the user token, and objects are sealed with the user’s key. A read share (§23.6) is refused /workspace/flows on a cbcore box (read_file, list_dir, symlinks included) and gets no “Saved flows” section in the guide. A write share has shell, which §23.1 already treats as handing over the box, so it can read the flows on that box like any other file. The share sheet’s disclosure should name saved flows (app follow-up).

28.4 Hot path

28.5 Export file

cocoonbox flows export [NAME...] [-o FILE] writes JSON:

{"version": 1, "exported_at_ms": 0, "flows": [{"name": "mousepad-save-fruit.json", "content": "<text>", "sha256": "<hex>", "rev": 3, "saved_at_ms": 0, "box_id": "cb-…", "image": "cbcore:v1"}]}

cocoonbox flows import FILE (- for stdin) sends name, content and image for each flow. The rest of each entry is informational.

28.6 Rollout

This site is open source. Improve this page.