Gateway deployment reference
Gateway hooks connect an LLM proxy to the
Watcher API without a Watcher client
on each developer machine. The proxy routes model traffic; Watcher reviews
the conversation; the coding agent executes its own tools. While
agent inventory is on, both integrations also
record which clients send traffic, without storing any request content. Each
person the gateway names under sessions:write:any becomes a Watcher user,
even when Watcher records none of their traffic.
See the self-hosted deployment reference for Watcher's components, authentication paths, and internal network connections.
| Integration | Supported traffic | Effect on client tools | Setup |
|---|---|---|---|
| Tailscale Aperture observation | Captures from the supported coding agents. | Checks the conversation after the provider responds; cannot stop tools. | Configure Aperture. |
| Tailscale Aperture enforcement | Captured tool calls in supported provider formats; see the format limits. | Holds pending calls for review; replaces each refused call with a notice, removes it, or blocks the pending callback. | Configure Aperture enforcement. |
| LiteLLM with Apollo's experimental callback | Anthropic Messages, OpenAI Responses and OpenAI Chat Completions, including streaming. | Streams content up to each tool call, then holds that call and later content until Watcher permits it. | Install the callback that calls the raw pre-tool-use hook. For GitHub Copilot clients, follow Native Copilot through LiteLLM. |
Tailscale Aperture: asynchronous observation
- The agent sends a model request through Aperture. Aperture forwards the provider's response to the agent without waiting for Watcher review.
- Aperture sends an
entire_requestevent containing the request, response, and metadata toPOST /api/v1/hooks/aperture. Configure bothrequest_bodyandresponse_bodysend types. - Watcher returns HTTP
202 Accepted. In the background, it checks new tool calls and saves the conversation and results. Text-only turns are saved too.
Scope the Aperture grant to the intended users or device tags and provider
keys. Developer attribution comes from metadata.login_name. Check the
capture requirements for your client
and provider configuration.
Tailscale Aperture: opt-in enforcement
Configure a tool_call_pre_response hook targeting the same URL,
POST /api/v1/hooks/aperture; Watcher tells the stages apart by
metadata.stage. In enforce mode, Aperture holds
selected calls while Watcher evaluates them. Allowed calls pass unchanged.
Denied or escalated Bash, bash, exec_command and exec calls become harmless
notice commands with the same tool name and ID. A refused call to another tool
is renamed to the client's own shell tool and prints the same kind of notice.
Denial notices ask the agent to try another approach. Escalation notices ask it
to stop for a labelled human choice; the gateway itself cannot grant approval.
Allowed calls in the same response still run.
A denied call no shell tool can carry is removed while another call still reaches the client, and a notice for another refused call names it when there is one; otherwise it blocks the whole pending callback, as does an escalated call no shell tool can carry and any refused Gemini call. Earlier released calls cannot be retracted. Blocks end the current turn in Claude Code, Codex and OpenCode. For autonomous recovery, re-prompt the same conversation with the denial message at most twice per task. Stop for a person if the task remains incomplete or makes no progress. This path does not resample the model.
Keep entire_request observation alongside enforcement for the full transcript
and tool results. Previously judged calls retain their decisions without extra
grading. Subscribe the observation hook to pre_request too, so Watcher adds
its notice, unless managed client instructions already contain it. The two
hooks share the URL but stay separate entries, because Aperture sets the
failure policy per hook: observation and the notice fail open, enforcement
fails closed. In observe mode, a pre_response delivery is allowed
immediately and graded in the background.
Use a scoped grant, fail_policy: "fail_closed" and a 310s enforcement hook
timeout for Watcher's 300-second evaluation budget. Set block_response and
fail_response to "message" so a block or an unavailable hook reaches the
agent as an answer rather than an HTTP error it may retry. The notice hook uses
fail_open. Follow the configuration and verification steps.
LiteLLM: decisions before client tool delivery
This path needs a callback installed in the proxy; installing the Watcher SDK does not install one. The LiteLLM example README documents setup, configuration and limits for Apollo's experimental callback. For the tested native Copilot configuration, including GitHub account mapping, enforcement on managed devices and the optional Kubernetes reference, follow Native Copilot through LiteLLM. Other client configurations remain experimental.
- The agent sends an HTTP request to LiteLLM using a configured model alias. LiteLLM calls the model provider. A streamed reply reaches the agent until its first tool call. The callback holds that call and all later content.
- Once the call is complete, the callback sends it and its context to
POST /api/v1/hooks/pre-tool-usewithformat: "raw". Watcher parses the provider format and returns a decision. Earlier released calls are kept in assistant history rather than selected for evaluation again. - An allowed call and following content are released up to the next tool
call. This repeats in stream order. Messages calls can be checked at
content_block_stop; Responses calls atresponse.output_item.done. Chat Completions has no per-call completion event, so its tool calls wait for the choice'sfinish_reason. Argument fragments can interleave.
A denial withholds the pending call and asks the model for an alternative,
telling it that Watcher checks every call, so it can continue another way.
The callback can make two such requests after the original proposal. Each
alternative also streams content and holds tools for checking. The client sees
one continuous response with only approved executable calls, plus plain text
describing blocked calls and Watcher's public reasons. An escalation is not
retried: the callback has no approval interface, so one last request asks the
model to stop and ask the user how to proceed. It offers to allow the call
only when Watcher's grader escalated it; a rule or threshold would escalate
the same call again, whatever the user answers. The same last
request follows a third blocked proposal. Any tool call in that last reply is
withheld. A fixed question to the user follows the reply when it proposed a
call, had no text or was cut off, and replaces it when that last request fails
or runs out of time. Before that last request, missing decisions, transport
errors and malformed content end with an error instead.
Each request to Watcher states that refused calls are withheld, so Watcher
records them as blocked. When the callback records the response the client
received, it names the calls it withheld from refused proposals. See
withheld calls.
If denial recovery ends after three refused proposals, the agent can end its turn. For autonomous workloads that must continue, re-prompt the same conversation with the denial message and ask the agent to use an allowed approach. Retry at most twice per task, then stop for a person if the task is still incomplete. Stop sooner if the agent repeats a blocked operation or makes no progress. Resumed calls are evaluated normally.
After an escalation, stop the autonomous workload for a person to decide how to proceed. Do not automatically resume it with an alternative approach.
The callback also adds a fixed system notice to every request. It tells the model that Watcher's notices in its earlier replies are genuine and how to continue after a denial or an escalation. The README quotes it.
Earlier allowed calls may already have run and cannot be retracted. The alternative request tells the model that their execution results are unknown and they must not be repeated. Reusing the same tool name and arguments is blocked during recovery, even with a new call ID. Denied operations are also remembered. This does not detect different arguments with the same effect. The agent's own permissions still apply after release.
Non-streaming replies are checked as whole proposals. One refused call withholds all calls in that proposal. Only an accepted alternative's calls are returned, together with text notices. An escalation, or a third refused proposal, ends with the same last text-only request as for streams.
In the Analyzer, each call Watcher refused appears just before the response the client received, with its arguments and grades, marked Blocked by gateway. An allowed call dropped with a refused one is marked Cancelled by gateway. The calls are grouped under numbered proposals. Each links to the original proposal on its own branch through View original proposal.
The callback sends SSE comment keepalives every 10 seconds for all three
APIs, including before the first provider event. Each normal tool evaluation
has a 300-second wall-clock deadline, excluding earlier provider generation.
Once a call is refused, its existing deadline also bounds the rest of
recovery: draining that proposal, generating alternatives and checking them.
Each alternative uses the remaining time, not a fresh 300 seconds. Expiry
ends the stream with a timeout error, except during the last request after an
escalation or a third refusal, which ends with the fixed question instead. Watcher may finish grading and record
usage after the caller stops waiting. Client-visible usage covers only the
first provider attempt, not the total across alternatives.
A grading failure inside Watcher is different: Watcher returns a normal
decision chosen by policy_monitors.on_grading_failure. With allow, the
callback releases tools that were never fully graded. Set it to deny if
those must be withheld; see
delivery limits. The
raw request section lists
the checks a callback must make before it releases a response.
For enforcement, allow developer inference traffic only through the checked
/v1/messages, /v1/responses, and /v1/chat/completions paths. Restrict
direct provider access, and apply a separate access policy to administrative
routes. Realtime, WebSockets, batches, files, and LiteLLM's provider
pass-through routes bypass the callback and are not checked. Tools that run
on the provider's servers, such as hosted web search, have already run by the
time the response reaches the callback. The native Copilot gateway exposes
only its own /copilot/ routes and refuses provider-hosted tools.
Apollo's experimental LiteLLM callback also rejects requests it cannot evaluate
before the provider runs, with HTTP 400: any choice count (n) other than
1, Chat Completions audio output, and Responses requests that rely on history
stored by the provider (previous_response_id, conversation, or
background). Clients must send the full conversation in each request.
The callback uses x-watcher-session-id when supplied, otherwise native Claude
Code, Codex or OpenCode conversation identity. Missing or invalid selected
identity returns HTTP 400 before provider generation. Custom integrations must
use distinct IDs for children they want stored separately; a shared ID groups
those requests. See the identity contract
for validation and precedence, and subagent coverage
for capture limits.
Identity is not authentication; preserve the proxy's normal access controls.
The native Copilot gateway supplies identity from Copilot's own session headers.
Credentials and the Watcher connection
| Connection | Credentials |
|---|---|
| Developer → proxy | Aperture's access grants or LiteLLM's client credentials. The native Copilot gateway uses the developer's own GitHub login instead. |
| Proxy → model provider | The provider authentication configured in Aperture or LiteLLM. |
| Proxy → Watcher | Separate Watcher credentials, sent to the deployment's HTTPS endpoint on TCP 443. In SSO mode, use an organization API key as x-api-key. In proxy mode, use the trusted connection described below. |
Aperture requires sessions:write:any to attribute captures to developers.
With an API key, a LiteLLM callback can send the proxy key's user_email as
the developer, which needs the same permission. Otherwise, recording uses the Watcher
credential's write scope and can produce unattributed
sessions. See
hook authentication.
The LiteLLM example also supports WATCHER_AUTH_MODE=proxy. The gateway sends
its server-owned proxy token and the authenticated account's verified email
to Watcher. Configure the token, identity headers and access restrictions as
described in the
example README.
Developers need no Watcher API key. This mode scopes each conversation to its
LiteLLM account, keeping different accounts separate even when they supply the
same conversation ID. These scoped IDs do not merge with local client records.
Watcher uses its own configured model providers for grading. Its deployment, storage, and retention are described in the self-hosted deployment reference and security reference. The proxy's logging and retention are configured separately.
Modes and verification
The administrator's enforcement mode, supplied through managed client settings, controls hook behavior even when no Watcher client is installed.
| Mode | Aperture entire-request | Aperture pre-response | LiteLLM |
|---|---|---|---|
enforce | Checks tools but cannot stop them. | Replaces refused calls with notices (renamed into the client's shell tool when needed) and releases allowed siblings; removes or blocks refused calls no notice can carry. | Streams permitted calls in order. Denials can trigger two checked alternatives. An escalation, or a third refused proposal, ends with the model asking the user; no further call is released. |
observe | Checks tools but cannot stop them. | Releases immediately; checks tools in the background. Transport errors can still withhold them. | Each tool call waits for a deferred decision before release. A parse error or Watcher outage can still stop pending calls. |
paused | Transcript ingestion without hook judgments or grades. | Releases tools unchanged and records only the transcript. Transport errors can still withhold them. | Records without hook judgments or grades. Tool calls still wait for the server's response, so an outage can stop them. |
Verify a scoped coding task in the Analyzer: find its session, check attribution, and inspect decisions and available grades. Follow the Aperture verification steps.
Prefer one monitoring integration per conversation. Combining a local Watcher client with gateway hooks can repeat grading.
LiteLLM captures carry a LiteLLM source badge and match the Gateway category in the Analyzer. Incremental checks and recovery may record partial proposals alongside the delivered history as extra trajectory branches in the same session. Do not assume every recorded branch reached the client or that one reply has one stored trajectory.