Skip to main content
Working with an agent? Give them a link to this page as markdown.

Client

The Watcher client is a single watcher binary installed on each developer machine. It hooks into the supported coding agents (Claude Code, Codex) to record sessions, upload them to your Watcher deployment, and enforce blocking review. This page covers installing and updating it, where its settings live, the command reference, and diagnosing problems with watcher doctor and watcher logs. To run the client inside a container alongside a coding agent, see Containers, or Dev containers for a dev container that reuses the sign-in already on the developer's machine. For an untrusted workspace, review the dev container hardening options. To keep one directory's sessions off your deployment, see Excluding a directory with .watcher-ignore.

Install​

The client runs on macOS (Apple Silicon) and x86_64 Linux. Install it with the published installer, telling it which backend this machine reports to:

# Apollo's hosted cloud
curl -fsSL https://github.com/ApolloResearch/watcher-bin/releases/latest/download/install.sh | bash -s -- --cloud

# A backend your organization hosts (see Self-hosting)
curl -fsSL https://github.com/ApolloResearch/watcher-bin/releases/latest/download/install.sh | bash -s -- --watcher-api-url https://watcher.example.com

The installer reads the chosen backend's version and installs the client release matching the lower of that server or the installer itself. It verifies the release checksum, installs the binary as ~/.local/bin/watcher, records the chosen backend, and registers the agent hooks. The one consent prompt is for the Codex config.toml changes described on Supported agents; hook entries themselves, including Claude Code's, are installed without prompting. If ~/.local/bin is not on your PATH, the installer prints the command that adds it. Open a new terminal afterwards so watcher is found.

Client and server are mutually compatible within one minor version of each other (while on 0.x). The latest installer automatically caps the client at the configured server's version. To impose a lower cap, use that release's installer:

curl -fsSL https://github.com/ApolloResearch/watcher-bin/releases/download/v<version>/install.sh | bash -s -- --watcher-api-url https://watcher.example.com

To remove everything (hooks, autostart, and the binary), run watcher uninstall.

Update​

Before every upgrade, read the release pages and their full changelog links for every version newer than the installed version and no newer than the target, including versions you skip. Check behavior changes and complete the steps that apply to your deployment before updating. The update guide explains how to identify the version range.

Update with:

watcher --version # what is installed now
watcher update # install the release matching the configured server
watcher --version # confirm

watcher update reads the configured server's version and installs that release, in either direction. It keeps your sign-in, settings, and local session data. A client running through its login agent runs the same reconciliation automatically every two to two-and-a-half hours, so those machines follow the server on their own.

Installs that update differently:

  • Clients from releases that predate watcher update have no update command. Run watcher uninstall, then re-run the installer, once; later updates use watcher update. Sign-in and settings survive, because uninstall keeps ~/.apollo_monitor/ (see Per-machine settings).
  • Foreground-only clients (no login agent) have watcher update but no automatic check: run it yourself.
  • Mirror-backed installs do not self-update. Publish the approved release to the mirror and redeploy through your fleet tooling (see internal mirrors).

To move back to a specific release during a server rollback, run watcher downgrade --version <version>. The requested version must be lower than the installed client and must match the configured server. Releases predating watcher update are unsupported downgrade targets.

If an update fails or leaves the client unhealthy, recovery is the same uninstall-and-reinstall: run watcher uninstall, re-run the installer, then check with watcher doctor. Local state under ~/.apollo_monitor/ survives both steps.

First run and sign-in​

Run watcher with no arguments to start the client's local services. The first such run installs the login agent and turns start-on-login on, then starts in the background and returns control to the terminal. It restarts after a crash and starts again when you log in. The local UI is at http://localhost:8228 (or the port set by WATCHER_BACKEND_PORT), where escalated tool calls wait for approval and per-machine settings are edited. Use watcher --foreground to keep it attached to the current terminal without enabling start-on-login.

Later runs read the start-on-login setting rather than turning it back on, so watcher autostart disable holds. While it is off, watcher starts in the current terminal and says so.

Any run that starts the client in the current terminal first stops a client the login agent is already running on that same port, so the terminal one can take it. The login agent stays installed, and it starts the client again at your next login unless start-on-login is off. A terminal pointed at some other port with WATCHER_BACKEND_PORT leaves the background client alone, since that client is not the one holding the port being asked for.

How you sign in depends on your deployment's authentication mode, which the client discovers from the backend:

  • SSO deployments (Apollo cloud, and self-hosted in SSO mode): sign in once from the local UI (the same address as above), which hands off to your identity provider in the browser. Starting watcher does not open the browser for you; on macOS the menu bar icon shows a "Sign in" entry until you have signed in. Recording starts once you have, and the running client keeps the credentials refreshed.

    On a machine with no browser, such as a headless server or a VM you reach over SSH, run watcher login instead: it prints a URL and a code you approve from a browser on any other device, and stores the same credentials. Machines nobody signs in to use an organization API key. See Signing in without a browser.

  • Header-proxy deployments: the client attaches headers taken from WATCHER_HEADER_* environment variables, which must be set in the environment that runs the installer (or watcher hooks install); there is no browser sign-in. Setup captures the values into a per-deployment file that later launches read, so they persist until you re-run watcher hooks install with new values, or remove the variables from the environment and run watcher hooks clear-headers (the command refuses while the variables are still set, because the next launch would re-capture them). See self-hosting.

A client that was working and later cannot authenticate says so rather than failing quietly. On SSO deployments the local UI shows a banner reporting that uploads and grading are paused, with a Sign in button; the banner clears itself once the client can authenticate again, whether you signed in from the banner or elsewhere. In Claude Code the statusline carries the same warning: Watcher signed out: uploads paused when the machine has no usable credentials, and Watcher auth error: uploads paused when the deployment is rejecting the credentials it has. watcher doctor reports the same condition in its auth check. In Codex, an approval request that arrives while the client's hooks have stepped aside shows a warning with the request, and Codex's own approval flow decides it. Hooks step aside when the machine has no usable credentials, when the local Watcher service is not running, or after repeated failures have paused grading. The warning suggests watcher login when the client is signed out, and watcher doctor otherwise. A deployment that is merely unreachable for one grade does not trigger this. That grade follows the configured failure decision, described under When grading fails.

Agent hooks​

watcher hooks install (run by the installer, and idempotent to re-run) registers Watcher with every supported agent on the machine. What the hooks do in each agent, the consent involved, and how blocking review coverage differs between agents is covered on Supported agents.

By default, these hooks are stored in user-writable configuration: ~/.claude/settings.json for Claude Code and ~/.codex/hooks.json for Codex ($CODEX_HOME/hooks.json when CODEX_HOME is set). The user, or a process running as that user, can edit or remove them. For managed deployments, see protecting hook configuration with an MDM.

Hooks are not tied to the client running. watcher hooks remove and watcher uninstall remove their user-level registration. A re-run of watcher hooks install also repairs hooks that a doctor check reports as out of date.

How sessions reach your deployment​

Recorded sessions do not stay on the machine. The hooks upload the conversation directly to the backend recorded at install time: after tool calls and at the end of each agent turn, so a session is visible in the Analyzer while it is still running. There is no periodic batch export to wait for.

The one exception is a directory an administrator has allowed you to exclude with a .watcher-ignore file. Sessions that file covers are not uploaded at all, so they stay on the machine and never reach the Analyzer.

A conversation can also be refused by the backend. When the Aperture gateway or another Watcher client has already recorded the same Claude Code or Codex conversation under a different Watcher user, the backend rejects this machine's copy (see Two recorders). The client then stops uploading that conversation, shows the agent a one-time notice, and lists it in watcher doctor. Blocking review still grades and answers its tool calls.

Which backend that is, is a per-install setting:

watcher target set --url https://watcher.example.com # your own deployment
watcher target set --cloud # Apollo's hosted cloud

watcher build-info prints the version and the backend target a launch would resolve, and watcher doctor (below) verifies the target is reachable and accepting this machine's requests.

Per-machine settings​

Blocking review resolves from settings on each machine: which graders run, the per-tool flag and deny thresholds, and the enforcement mode. Admins can distribute and lock these org-wide with managed client settings instead of configuring every machine by hand. The concept pages explain what each setting does; on the machine they live in layered YAML files under ~/.apollo_monitor/:

  • settings.yaml is the admin baseline. Values set here apply machine-wide, and the enforcement mode is read only from this file. On deployments that distribute managed client settings via Watcher, the running client pulls the org's document every few minutes and replaces this file with it, so hand edits do not stick; otherwise an administrator or an MDM profile writes it. Admins can lock individual values so local edits cannot override them.
  • settings.local.yaml holds the developer's own overrides. Most settings, thresholds included, are edited from the local Watcher UI rather than by hand.
  • Each settings profile is an entry under profiles in whichever of those two files defines it, so a profile's admin values live in settings.yaml. A developer's own edits made while a non-default profile is active still land in settings.<profile>.local.yaml.

A settings profile is a named bundle of these review settings that can be switched as a unit. The client ships with Default, Auto, Human in the Loop, and Skip Permissions profiles, and a deployment can add its own. watcher profile list shows what is available, watcher profile current the active one, and watcher profile set <name> switches; it accepts a profile's id, its display name, or any of its alternative names. Human in the Loop's id is permissive; you can also write human-in-the-loop. Both work in a settings file's active_profile, though a profile an organization pins for every machine is named there by its id.

Beyond settings, ~/.apollo_monitor/ holds the client's other local state: a local database of sessions and pending decisions backing the live UI, client logs, and the sign-in credential and auth-header files, written with owner-only (0600) permissions. watcher uninstall intentionally keeps this directory and prints a reminder; remove it manually to wipe a machine's local Watcher data (nothing already uploaded to the deployment is affected).

Environment variables​

Almost everything about the client is configured through the local UI or the settings files above. A few things are not, because they have to be in place before the client can read a settings file at all: which backend to talk to, which port to listen on, and how to trust your TLS.

Where you set them decides whether they take effect. The agent hooks run as short-lived subprocesses that the coding agent spawns, and they inherit nothing from your shell. A variable reaches them only because watcher hooks install bakes it onto the hook command lines as it writes them. So for anything in the first group below, set the variable in the environment that runs the installer (or watcher hooks install): your shell's startup file, or ~/.config/environment.d/watcher.conf on Linux. Re-run watcher hooks install after you change it. Changing the variable alone leaves the already-installed hooks on the old value.

Read by the hooks, so they must be set when hooks are installed:

VariableDefaultWhat it does
WATCHER_API_URLthe recorded targetThe backend this machine reports to. Overrides whatever watcher target set recorded, both for this launch and for the hooks installed by it. A value with no scheme is read as https://. Prefer watcher target set for a lasting change; use this to point one launch somewhere else.
WATCHER_BACKEND_PORT8228The port the local services listen on, which is also the port the local UI is served from. Change it when something else on the machine already owns 8228.
SSL_CERT_FILEthe system trust storePath to a CA bundle, for a deployment behind an internal CA or a self-signed certificate. Forwarded to the hooks only when it is set.
WATCHER_API_KEYnoneAn organization API key for the machine to authenticate with, instead of a browser sign-in. Mainly for unattended installs such as containers; a developer machine signs in instead. Captured authentication headers and a stored sign-in token take precedence over this key. Unlike the variables above it is not baked onto the hook command lines, so it must be set in the environment the agent runs in, where the hooks inherit it.
WATCHER_HEADER_*noneOne variable per header the client sends to your deployment, on header-proxy deployments. These are the exception to the rule above: rather than being baked onto the command lines, where they would show up in the process list, they are captured into an owner-only file that later launches read. See Sending additional headers.

Read by the client itself, so it must be set in whatever starts it:

VariableDefaultWhat it does
WATCHER_MENUBAR1Set to 0 and the macOS menu bar icon never starts. No effect on Linux.
WATCHER_BACKEND_WORKERS2Number of local HTTP worker processes. Must be a positive integer. Set to 1 to reduce memory use. A second worker can answer new requests while one worker is busy; background synchronization still runs once. Restart the client after changing this value.
WATCHER_BACKEND_RELOAD0Source development only: set to 1 for automatic reload when source files change. Reload mode uses one request worker and overrides WATCHER_BACKEND_WORKERS. No effect on the packaged client.

The menu bar icon's own "Hide Menu Bar Icon" entry closes it for the current run only; it is back at the next start. WATCHER_MENUBAR=0 is how you keep it away.

When the client starts on login it is given almost no environment: by default the login agent runs the binary directly with little beyond PATH. If the running client needs environment variables (WATCHER_MENUBAR, WATCHER_BACKEND_WORKERS, and also WATCHER_BACKEND_PORT and SSL_CERT_FILE where you have changed them, since the client uses those at runtime as well as at hook-install time), install the login agent with --launcher pointing at a wrapper script that exports them. Exporting a variable only in an interactive shell does not change the login agent's environment. See MDM rollout § Start on login.

Command reference​

Running watcher with no arguments starts the client under its login agent, enabling start-on-login on the first run. Every command supports --help.

CommandWhat it does
watcherStart the client in the background under the login agent, installing it and turning start-on-login on if this is the first run. Starts in the current terminal instead while start-on-login is off. watcher live start and watcher all start remain foreground aliases.
watcher --foregroundStart the client in the current terminal without installing or enabling the login agent. Stops a client the login agent is already running so this one can take the local port.
watcher --versionShow the installed client version.
watcher updateInstall the client release matching the configured server.
watcher downgrade --version <version>Install the configured server's explicitly selected older release.
watcher build-infoShow build details and the backend target a launch would resolve.
watcher whoami [<session ID>]Print the Analyzer link for a Claude Code session or Codex thread: the one the command runs in, or the ID you pass. Exits non-zero with a one-line reason when that session is not in Watcher.
watcher login / watcher auth loginSign in from the terminal without a local browser: prints a URL and a code to approve from a browser on any other device (SSO deployments; needs a server that offers device login). See Signing in without a browser.
watcher auth helper set / show / check / refresh / clearConfigure and diagnose a trusted CLI that supplies proxy bearer tokens. See External credential helpers.
watcher target set --cloud / --url <url>Record the backend this install reports to.
watcher hooks installInstall or repair the agent hooks for Claude Code and Codex. --yes skips the consent prompt (for unattended installs).
watcher hooks removeRemove the agent hooks for both agents.
watcher hooks clear-headersRemove the captured WATCHER_HEADER_* headers for the selected backend target (header-proxy deployments). Refuses while WATCHER_HEADER_* variables are still set in the environment: remove them first.
watcher profile list / current / set <name>List, show, or switch settings profiles.
watcher autostart enable / disableTurn starting-on-login on or off in settings.
watcher autostart start / stopStart or stop the login agent now, without waiting for login.
watcher autostart enabledExit 0 if autostart is enabled in settings, 1 otherwise (for scripts).
watcher autostart install [--launcher <script>] / uninstallInstall or remove the login agent itself (macOS LaunchAgent, Linux systemd user unit). By default the binary is its own launcher; pass --launcher to run a wrapper script that sets environment variables at login instead.
watcher doctorRead-only health checks for this install (see below).
watcher doctor dumpWrite a support bundle with best-effort redaction.
watcher logs [hooks|daemon|backend]Show client logs, merged across sources by default (see below). -n sets lines, -f follows, --since 2h sets a window, --errors filters.
watcher uninstallRemove hooks, autostart, and the binary.

watcher doctor​

watcher doctor is the first thing to run when sessions stop appearing in the Analyzer or blocking review stops intervening. It does not change configuration. On a deployment using an external credential helper, its authentication checks can invoke that helper and update the token cache.

What it checks​

The report has six sections, and each can be run alone (watcher doctor auth):

SectionChecks
targetA backend target is recorded, and any environment override agrees with it.
authBackend reachability, authentication mode, the selected credential, sign-in refresh status, and a live authenticated request where supported. API-key authentication on SSO deployments skips that live check.
hooksThe installed agent hook config matches what the current client would install.
daemonThe local services respond on their port, and whether autostart is installed and enabled.
enforcementNo open agent session runs with an approval policy that bypasses blocking review (for example a Codex session launched with --dangerously-bypass-approvals-and-sandbox). Such a session is still recorded, but Watcher cannot grade or block its tool calls before they run.
uploadsNo conversation was refused because a different Watcher user already records it.

Reading the output​

Each check reports ok, a warning, or an error, with a suggested fix in the message or on a Fix: line under it. By default only problems are shown, with the passing sections named on one line; --verbose shows passing checks too, and --json emits the full report for scripts. The exit code summarizes the report: 0 healthy, 1 at least one error, 2 warnings only.

watcher X.Y.Z — target: https://watcher.example.com — auth mode: workos
Auth
✗ No sign-in credentials for this target. Start watcher and sign in via the
browser, or run `watcher login` on a machine without one.

Agent Hooks
⚠ Agent hook config is out of date. Run: watcher hooks install

✓ Target, Daemon, Enforcement, Uploads
1 error, 1 warning, 7 ok

Common failures​

FindingWhat it meansFix
No backend target recordedThe install never had a backend selected, so nothing can upload.watcher target set --url https://<your-watcher-host> (or --cloud).
Cannot reach the backendWrong URL, no network route, or TLS fails against an internal CA.Check the URL and network. For an internal CA, set SSL_CERT_FILE to your CA bundle.
Cannot read captured hook headers / duplicate authentication fieldsThe saved headers file is malformed or contains the same authentication header more than once with different casing. Doctor skips backend authentication probes. Support dumps omit log excerpts when credential or header files cannot be read or parsed for redaction.Repair the file named in the finding, or remove the WATCHER_HEADER_* exports and run watcher hooks clear-headers.
No sign-in credentials for this targetNobody has signed in on this machine (or the target changed, which needs a fresh sign-in). When the finding adds that the hooks are disabled, nothing else on the machine can authenticate either: sessions are not recorded or graded, permission questions are not answered, and the coding agent's own permission prompt is all that gates its tool calls.Start watcher and sign in from the local UI, or run watcher login where there is no browser (see Signing in without a browser). An unattended install can set WATCHER_API_KEY instead.
Authenticated probe failed with 401The backend rejects the selected credential: expired or revoked credentials, or stale auth headers overriding the sign-in.Follow the finding for the selected authentication method: sign in again for browser credentials, or correct the configured authentication headers and proxy setup. Signing in does not replace captured authentication headers.
Credentials last refreshed N minutes agoThe running client should refresh credentials every few minutes and has stopped.Restart watcher; if it recurs, capture a watcher doctor dump.
Agent hook config is out of dateThe hooks in the agent config predate the current client or were edited.watcher hooks install.
No backend responding on localhostThe client services are not running, so there is no UI and no sign-in refresh.Start watcher (or watcher autostart start if the login agent is installed). watcher logs daemon shows why the last start failed.
Conversation is recorded under a different Watcher user; this machine's copy is not storedThe Aperture gateway or another client recorded this Claude Code or Codex conversation first, under another Watcher user. Usually your Aperture login and your Watcher sign-in are different users. The agent showed the same notice once, and this machine no longer uploads the conversation. Blocking review still works.Sign Watcher in as the same user as your Aperture login (watcher login, or sign in from the local UI). On a proxy-authenticated deployment, which identifies you by headers instead, make the identity headers your gateway or your WATCHER_HEADER_* settings send name that user, then run watcher hooks install. Either step lets refused conversations upload again.
Session runs with approval_policy "never"The agent was launched with approvals bypassed (or resumed a session that was), so it never asks for approval and Watcher cannot intervene — it can only record. While such a session is open and Watcher is signed in, the macOS menu bar icon shows a warning glyph (⚠) — the sign-in and auth-error warnings take priority when they apply — and its menu gains a "Watcher found problems" item that opens this doctor report.Remove the bypass flag from however the agent is launched and start a fresh agent session; resumed sessions keep their original policy.

Support bundle​

watcher doctor dump writes a text bundle: the full doctor report plus recent client logs. With an external credential helper, authentication is a local snapshot: the dump does not invoke the helper or probe upstream authentication. No session content is collected, and known secrets are redacted, but redaction is best-effort: review the file before sharing it, as the command's own output reminds you. --since 2h widens the log window (default one hour) and --output sets the file path. Attach it when reporting a client problem.

watcher logs​

watcher logs prints the client's own logs, so diagnosing a problem does not require knowing where the log files live. It covers three sources:

SourceWhat it holds
hooksThe agent hooks: one line per tool call evaluated, plus upload and sync errors.
daemonOutput captured from the client the login agent runs, including why a background start failed.
backendThe local client services' own log.
watcher logs # last 50 lines from all sources, merged
watcher logs hooks -n 200 # one source, more history
watcher logs -f # keep streaming new lines (Ctrl-C to stop)
watcher logs --since 2h --errors # warnings and errors from the last 2 hours

With no source argument, lines from all sources are merged in time order and prefixed with their source name. --level sets a minimum level (debug|info|warning|error|critical); --errors is shorthand for --level warning. The logs are read from files on disk, so the command works while the client is not running; the daemon source is the place to look when the client will not start. When output is piped rather than shown in a terminal, lines pass through as they appear in the log files. For JSON processing use watcher logs backend, the one source that holds only JSON records: the daemon source is captured process output that can mix JSON with plain text, the hook log is plain text, and the merged view prefixes every line with its source.

watcher logs output is not redacted. To share diagnostics outside the machine, prefer watcher doctor dump above, which redacts known secrets.