The daemon reads this file when it starts. After an edit, quit it (pkill -f 'shuttle serve'; Windows: end shuttle.exe in Task Manager) and the next tool call other than status starts it with the new settings.
shuttle reads one TOML file:
| OS | Path |
|---|---|
| macOS, Linux | $XDG_CONFIG_HOME/loomhand/shuttle/config.toml, else ~/.config/loomhand/shuttle/config.toml |
| Windows | %APPDATA%\loomhand\shuttle\config.toml |
shuttle calls models through one OpenAI-compatible endpoint with one key. The default is OpenRouter, so the quickstart is:
export OPENROUTER_API_KEY=sk-or-... in the environment the daemon starts from (on macOS, see Keys).decide model, typesafe/jev-1.14, answers on OpenRouter’s Decisions API, so its public model list shows it under ?output_modalities=decisions and not in the default chat list.The desk runtime calls models in four roles, each a model name on that endpoint:
| Role | Default model | What it does | Used by |
|---|---|---|---|
decide |
typesafe/jev-1.14 |
Picks one element from candidates, or answers a yes/no question, with probabilities, over text | gui_plan window checks, expect steps, find, and the text pick when the screenshot pick fails; desk.decide in scripts |
verify |
the decide model |
Answers the same over a screenshot, within 5 s | gui_plan when a step’s target matches more than one element, and its check after a grounded click; desk.look in scripts |
ground |
qwen/qwen3-vl-235b-a22b-instruct |
Returns the point of a described element in a screenshot, as point_2d from 0 to 1000 |
gui_plan when the accessibility tree has no match, for a point where the accessibility tree has an element (an app with no tree, such as desktop Chrome without accessibility on Linux, goes to computer_act), not a window or document root, and inside the plan window or a window it owns, judged by the element’s ancestors when they name a window and by its process when they name none (a UWP app shows its content from another process than its frame’s); and not under another window, which desk.at reports as covered on Linux and macOS (on Windows the hit test is already the top window’s); desk.ground |
ask |
anthropic/claude-sonnet-5 |
Answers a free-form question about screenshots | gui_plan expect steps the text check does not confirm, and toggle states the accessibility tree does not report; desk.ask |
Every field is optional; this file spells out the defaults:
base_url = "https://openrouter.ai/api/v1" # POST {base_url}/chat/completions
decisions_url = "https://openrouter.ai/api/alpha/decisions"
api_key_env = "OPENROUTER_API_KEY"
timeout_ms = 60000
decide_api = "decisions" # decisions | chat
# decide_build = "typesafe/jev-1.14-########" # only these builds may answer decide and verify
[models]
decide = "typesafe/jev-1.14"
# verify = "..." # defaults to decide
ground = "qwen/qwen3-vl-235b-a22b-instruct"
ask = "anthropic/claude-sonnet-5"
POST {base_url}/chat/completions for ask, ground, decide and verify with decide_api = "chat", and POST {decisions_url} with {model, state, questions} for decide and verify with decide_api = "decisions". Chat bodies carry max_tokens (100 for ground, 400 for ask, 800 for decide over chat), which OpenRouter accepts for every model.base_url to its /v1 root, decide_api = "chat" and the model names it serves. A server without a key needs none: with no key shuttle sends no Authorization header.base_url to its OpenAI-compatible root and decisions_url to its decisions route, such as base_url = "https://gw.example/v1" and decisions_url = "https://gw.example/v1/decisions".decide_build pins the builds a decide or verify answer may come from: # stands for one digit and every other character for itself, so typesafe/jev-1.14-######## takes any dated jev-1.14 build and refuses an alias or another version. The build is the model the endpoint names in its reply; with the key set, an answer that names none, or another, fails the call with that build in the error. Unset, any build answers; an empty value fails the file.timeout_ms bounds each call; every screenshot judgment (desk.look, the verify role) caps it at 5 s. One call is one request: shuttle does not retry it.decide without a decisions endpointgui_plan was tuned against a decisions endpoint, which returns calibrated probabilities per answer. With decide_api = "chat", shuttle asks the chat model for the same answers as JSON and reads the probabilities it writes. Those numbers are self-reported, not calibrated, and the plan executor’s thresholds (a 0.7 bar for a pick, 0.5 for a set field, a 0.6 runner-up margin, 0.9 for the 640x400 crop check of a grounded click, 0.65 for its zoomed crop, 0.95 for its full-screen check) were set on calibrated ones. Expect more escalations and more wrong picks than with a decisions endpoint.
api_key in the file when set, else from the variable api_key_env names. Prefer the variable: the file then holds no secret.Shuttle.app through LaunchServices and does not see your shell’s environment: there, set the variable with launchctl setenv OPENROUTER_API_KEY ... (until logout) or put api_key in the file.api_key must be readable by its owner only (mode 0600); shuttle refuses it otherwise, as ssh refuses a readable private key. On Windows, %APPDATA% is per user; shuttle does not check its ACL.OPENROUTER_API_KEY and the configured api_key_env in their environment, and errors and logs print a key as ***. A script runs with your user’s rights, so it can read a key kept in this file; keep the key in the variable.gui_plan saves passing plans as flows in a flows directory next to config.toml, as one JSON file per flow, an INDEX, a run log .runs.jsonl, .pending.json (the steps of the last plans, which a later save stores) and a git history of the changes when git is installed (on macOS, when the Command Line Tools are: the flow history never runs the git stub that would open their installer; git that a script_run script runs is that script’s own call). Flows hold the values plans typed, so the directory is readable by its owner only (0700) and its files are created 0600.
[safety]
apps = [] # app allowlist; empty allows every app
kill_hotkey = "ctrl+alt+cmd+escape" # ctrl+alt+shift+escape off macOS; "off" registers none
indicator = true # menu-bar item; tray icon on Windows
confirm = "off" # "coordinate" or "every" asks first
[tools]
script_run = true # false turns script_run off, and the plugin tools with it; services stay
# plugins = "/etc/shuttle/tools.d" # plugin and service directory, relative to this file's; default tools.d
[driver]
overlay = true # false: no agent cursor; no effect on macOS
The safety controls live in this file, so while it fails to load every tool call except status answers with its error, desk calls included; only shuttle stop and resume still work. kill_hotkey takes modifiers (ctrl, alt, shift, cmd, which is the Windows key on Windows) and one key (a letter, a digit, f1–f12, escape, space, return or tab), or is off for no global hotkey; on macOS and Windows a value that does not parse fails the file like any other error. safety.md describes each control. They are guard rails for the desk, screenshot and gui_plan tools that hold only with script_run = false and no service plugin installed: a script_run script and a service plugin are your own user’s code and can change or bypass them, as safety.md explains.
The MCP server reads the plugin directory and script_run when the agent starts it: after adding, changing or removing a plugin, or turning script_run on or off, restart the MCP client (the agent) for its tool list to follow. The daemon reads them when it starts, and refuses a plugin call while script_run is off whatever the list shows. The same directory holds service plugins, long-lived programs the daemon starts with itself whatever script_run is; after adding, changing or removing one, restart the daemon, and the agent for its tool list.
[driver] overlay = false starts the bundled cua-driver with --no-overlay, for a desktop nobody watches, such as a cloud box. It has no effect on macOS, since the driver’s front-window tracking needs its overlay, and the daemon says so in its log.
[desk]
input = "foreground" # or "background", "auto"
input says how desk input reaches an app:
foreground, the default, works as a person does: a click brings its window to the front, and keys and typed text go to the focused window. On macOS, foreground input that names a window first brings that window to the front, and refuses if it cannot.background acts without moving your pointer or taking the focus. Keys, typed text, scrolls and drags go to the window that the call’s window or pid names, which is required. A click at a point goes to the window under it, or to the window the call names, without raising it. A press runs the element’s own action before any click, and a press or element click that does click goes to the element’s own window, even under another window. An action that cannot be done this way fails with the reason, and shuttle never falls back to the foreground: you chose not to be interrupted. focus still brings a window to the front; it is the only call that does. The computer actions refuse, since they send desktop input. gui_plan drives the window its app or wait step names, and the dialogs that window owns, instead of the front window; on Linux it refuses, since background plans are not measured there. Scripts use the desk API with a window or pid on each input call.auto sends input as background does, and takes the foreground only for an action that background input refused before anything reached the app: shuttle brings the action’s window to the front and runs that action once more as foreground, and the result says "input": "foreground". A drag on Windows runs in the foreground from the start. Input the driver could not confirm comes back with unverified, as in background, and is never sent twice. gui_plan drives its target window as in background; when typing the driver could not confirm left every text of the window unchanged and the front alone, it brings the window to the front and types that step once more. A plan step that background refuses before sending anything (a macOS menu-bar entry, an element found on the screen, a state read from pixels, an expect checked by vision) brings the plan’s window to the front and runs in the foreground. After any step of a plan took the foreground this way, or through a desk action, the rest of that plan runs in the foreground. The computer actions run in the foreground. On Linux, and with an [android] section, auto acts as foreground. safety.md lists the refusals that take the foreground.With an [android] section, background fails to load: input on a device always reaches its screen.
safety.md lists what background input cannot do.
[daemon]
socket_group = "agents"
socket_group, a group name or number, opens the daemon’s unix socket to the members of that group (on Windows, see the end of this section): the daemon gives the socket to the group with mode 0660 when it starts. Unset, the socket stays with its user (mode 0600). The daemon’s user must be a member of the group. The endpoint must be an absolute path with no link anywhere on it. Its directory must belong to the daemon’s user and be writable by no one else, and every directory above it must belong to root or the daemon’s user and be writable by no one else unless it carries the sticky bit (as /tmp does); otherwise the daemon refuses to start and names the directory, its owner and its mode, since a member who could write or rename there could swap the socket. A layout such as /run (root, 0755) above /run/agents (your user, group agents, 2750) passes. The group needs search permission on that directory and those above it, never write; for example a directory of mode 0750, owner your user, group the shared group, named by SHUTTLE_SOCKET when the default cache directory sits in a private home. The daemon binds the socket under a temporary name, sets its group and mode there, starts listening only then, and renames it into place, so no connection reaches it before its final mode. The endpoint must fit a unix socket path with two bytes to spare for the temporary .n name: at most 105 bytes on Linux and 101 on macOS, since the system’s 108- and 104-byte limits count the terminating NUL. A longer endpoint fails at start with that limit named. The lock file beside the socket stays 0600. On Windows the daemon’s named pipe admits SYSTEM and its own user only, local clients only; socket_group there names one more account, an account name such as NT SERVICE\agent or HOST\agents, or a SID string such as S-1-5-32-545, which may then read and write the pipe but not create instances of it. An account that does not resolve fails the daemon at start with that name. safety.md says what the group can then do.
[android]
adb = "adb" # the adb program; default adb on PATH
serial = "127.0.0.1:5555" # the device; unset takes adb's own choice, such as ANDROID_SERIAL
# ui_jar = "/path/to/shuttle-ui.jar" # the UI server, built from android/ui; unset takes android/shuttle-ui.jar beside the program
scrcpy_jar = "/usr/local/share/scrcpy/scrcpy-server.jar"
su = "/usr/local/share/scrcpy/su1000" # runs the paste server as the system uid; needs adb root; unset runs it as adb's shell uid
With an [android] section the daemon’s desk is that device instead of this desktop. The desk verbs windows, wait, snapshot, find, read, press, act, at, key, type, click, focus, close, launch, screen and shot answer from the device’s UiAutomation tree and screenshot (screencap when the UI server cannot give one), taps and single keys injected by the UI server, adb shell input for the rest, and a scrcpy clipboard paste; scroll, drag, still and unanswered answer unknown desk tool on a device. press on an element that is off the screen asks the element to scroll itself into view, and swipes its list when the element does not move. The daemon then starts no cua-driver, no indicator and no stop hotkey, since none of them reaches a phone. shuttle desk computer_state, desk computer_act, desk computer_end and desk look answer the device’s screen tools from the command line only: state, act and look write their screenshot to a shuttle-*.png file in the temporary directory, whose path follows the answer, and computer_end forgets a session’s element indexes, the default session’s when it names none. The screen size is what wm size reports, read once when the daemon first needs it: after a wm size override, restart the daemon. It is the natural orientation, so on a portrait phone turned sideways desk screen gives the axes swapped, desk shot without a rectangle crops to the portrait width and height, a full-screen window lists as a dialog, a list swipe is clamped to the portrait height, and a computer_act scroll starts at the portrait centre and moves a third of the portrait height. Taps and swipes at absolute points pass through unchanged. On a stock phone, which has no adb root, leave su out and point scrcpy_jar at the upstream scrcpy-server-v4.1 release: its server sets the clipboard as the shell uid, where a server built for the system uid is refused. The linux release archives carry the UI server as android/shuttle-ui.jar beside the program, which install.sh puts at ~/.local/lib/loomhand/shuttle/android/shuttle-ui.jar, or $SHUTTLE_PREFIX/lib/loomhand/shuttle/android/shuttle-ui.jar with a custom prefix. Without ui_jar the daemon takes that jar, and does not start when there is none; a set ui_jar wins. The app allowlist does not apply to a device yet: a configuration with [android] and a nonempty [safety] apps fails to load, since the daemon would neither limit the calls to those apps nor mask the others in screenshots.