---
description: >-
  How Watcher reviews a live session: blocking review gates tool calls before
  they run, trailing review grades the trajectory as the session progresses.
---

> Part of the Watcher docs. Index of all pages, with summaries: https://watcher-docs.apolloresearch.ai/llms.txt
> Before upgrading Watcher, read the release pages and full changelogs for every version crossed: https://watcher-docs.apolloresearch.ai/releases/index.md
> Found an issue with this page? Report it: https://github.com/ApolloResearch/watcher-bin/issues/new?title=Docs%20feedback%3A%20Blocking%20and%20trailing%20review&body=Page%3A%20Blocking%20and%20trailing%20review%0AURL%3A%20https%3A%2F%2Fwatcher-docs.apolloresearch.ai%2Fconcepts%2Fblocking-and-trailing-review%2F%0A%0A

# Blocking and trailing review

Watcher reviews a running [session](sessions-and-trajectories.md) in two ways.
**Blocking review** evaluates tool calls before they run and can stop them.
**Trailing review** grades the session's
[trajectory](sessions-and-trajectories.md) as it grows and surfaces findings
after the fact. The dividing line is simple: if a single action could cause
irreversible harm, it belongs to blocking review. Patterns that emerge across
turns belong to trailing review.

Blocking review runs from the [Watcher client](../client/index.md) on the
developer's machine. You configure it through the client's settings: which
[graders](graders.md) run, the thresholds, and the enforcement mode. These
resolve on each machine, but you do not have to set them machine by machine.
Admins can distribute and lock them across the organization with
[managed settings](organization-settings.md#managed-client-settings). Gateways
can use the same evaluation pipeline through
[gateway hooks](../api/hooks.md); the caller then applies the returned
decisions. Trailing review runs on the Watcher server over ingested sessions
and needs no client configuration.

## Blocking review

Blocking review evaluates the tool calls a coding agent proposes before they
run. Coverage depends on the [agent](../supported-agents.md). With Claude Code,
every tool call is evaluated. With Codex, review runs through the agent's
approval flow, so a call Codex does not ask about executes without a
pre-execution decision. That call is still recorded, and graded after the fact.
Codex asks about the commands it considers dangerous, and about anything that
needs to reach outside its sandbox. That is why Watcher keeps the sandbox on
when it is enforcing.

Subagent tool calls can also reach blocking review through the coding agent's
hooks. See [Subagent coverage](subagent-coverage.md) for an example and the
limits on the context available to that review.

The agent is paused while a call is evaluated, so the pipeline is staged from
fastest to slowest. Most calls never reach the slow stages:

1. **Terminal thresholds.** A tool whose thresholds are set to a terminal
   value is decided instantly: always-allow (both thresholds disabled),
   always-deny, or always-flag. These outrank everything that follows,
   including command rules. Setting a tool to always-allow also switches off
   that tool's command rules, deny rules included.
2. **Command rules.** Match rules that allow, deny, require grading, or send
   a call straight to human review.
3. **Triage.** A fast, cheap grader scores the call. Triage can only
   auto-approve: a low-scoring call proceeds immediately, and anything else
   falls through to gateway review. It never denies or escalates.
4. **Gateway review.** The gateway grader scores the call from 1 to 10. The
   score is compared against the tool's configured thresholds: at or above
   the deny threshold the call is denied, at or above the flag threshold it
   is escalated, otherwise it is allowed.
5. **Human escalation.** An escalated call waits for a person. The developer
   approves or denies it in the Watcher client's UI, or answers the coding
   agent's own permission prompt, whichever comes first.

### Command rule examples

Default command rules can approve routine shell commands without an LLM grade.
Examples include `rg --files | sort -u | uniq -c`, `gh pr checks 123`,
and `gh run view 456 --log-failed`.

These are examples, not a complete list. Approval depends on the command's
flags and pipeline stages. For example, `head` or `tail` in these pipelines
needs a numeric line count and no filename. Quoted text that looks like one of
these pipelines can also trigger review.
Review the effective rules in the client's
settings or your organization's [managed settings](organization-settings.md#managed-client-settings)
to check which rules apply on a machine.

Variable expansions and command substitutions need grading because they can change
what runs after a rule has matched the text. Directory-stack commands and shell
dialects that the rules cannot parse also need grading. Output redirects are
checked against file-write rules, including quoted paths and forced overwrites.

Allow rules approve shell commands only for the `Bash` tool. Watcher treats
other tools as shells too when they have a shell name, such as `shell` or
`exec_command`, or when their input has a `command` or `cmd` field. That
includes MCP tools. `Edit`, `Write` and `apply_patch` are never treated as
shells. Deny and human-review rules still apply to these other shell calls, but
allow rules cannot approve them, so they go to grading.

Valid deny and human-review rules still apply if another active rule is invalid.
Watcher withholds rule-based approvals until the invalid rule is fixed. This
prevents a broken restriction from leaving a broader approval rule in force.
When managed settings are saved, each active rule must have complete fields
and a valid regex of at most 16,384 characters. This includes rules in profiles.

If rule matching times out or a shell command is longer than 131,072
characters, the call
goes to grading on both the client and the gateway. In enforce mode, when the
client skips grading, such as while [snoozed](#snoozing), it asks for human
review instead. A deny or human-review match already found still applies.
Terminal thresholds take precedence.

The default Bash policy sends suspicious commands to review. It does not deny
them automatically. Admins can configure deny rules and thresholds through
managed settings.

### When grading fails

Every outcome is recorded as a decision on the session, so escalations and
denials are visible in the [Analyzer](analyzer.md) transcript afterward.

If grading fails or exceeds its time budget, Watcher applies the configured
failure decision: escalate, deny, or allow. Escalate is the default: a human
decides, through the Watcher client's UI or the coding agent's own permission
prompt.

Watcher retries some failures before applying the decision, and applies it
right away for others. The line is whether the backend answered:

- **Retried once (by default).** A transient failure where the backend
  answered but could not grade: it returned a server error or was rate
  limited, an upstream model provider failed, or the grade came back empty. A
  short pause, then one more attempt.
- **Applied immediately.** A refusal that another attempt cannot change: an
  authentication error, a grader model your deployment does not serve, a usage
  limit reached, or a transcript too long for the grading model.
- **Not retried: a grade that runs out of time.** One attempt may use the
  whole time budget, so there is nothing left to retry with. And a grader slow
  enough to exhaust the budget would only be slow again.
- **Not retried: a backend that does not answer at all.** The connection is
  checked before grading starts, so this is the one case above that never
  reaches a grade to retry. A refused or unanswered connection applies the
  failure decision within a couple of seconds, rather than holding the call
  for the full time budget.

The time budget is for a backend that answers slowly, where a gateway review may
legitimately need most of it. When triage fails, Watcher first checks whether
the deployment is answering before it spends what is left of the budget on gateway
review. If the deployment is answering, gateway review still runs. The two stages
use different models, so one failing says nothing about the other. If the
deployment is not answering, the failure decision applies straight away.

### When failures repeat

Repeated failures pause a stage rather than the client. Triage and gateway review
are counted separately, so an outage affecting one leaves the other working. If
gateway review is failing, triage keeps auto-approving the calls it can, and only
the rest take the failure decision.

The exception is a failure decision of `allow`. With gateway review paused, every
call would be allowed whichever way triage scored it. So calls are allowed
immediately and not graded at all until gateway review resumes.

Failures that show the deployment itself is unreachable are counted against the
deployment instead. Enough of those pause grading on the machine for a minute,
and each gated call goes to the coding agent's own permission prompt.

Every pause lifts by itself.

A machine with no usable credentials, such as a signed-out one, does not grade
at all. Its hooks step aside, and the coding agent's own permission flow
decides each call. See the [Watcher client](../client/index.md) for how the
client reports this.

There is one exception to the configured decision. On an SSO-mode deployment,
a machine that has never signed in, or has signed out, can still grade when it
holds an API key: one set in `WATCHER_API_KEY`, or one captured from the coding
agent's own requests. If the deployment rejects that grade as unauthenticated,
the call is blocked with a prompt to sign in, not with the configured decision.
The block covers that call only. Grading continues, and later calls are blocked
the same way while the rejections continue. After five rejections in a row the
client stops sending grades, apart from an occasional probe, until one
succeeds. Calls then follow the configured decision, or go to the agent's own
permission prompt when the key was captured. The remedy depends on the key.
A key set in `WATCHER_API_KEY` is superseded by signing in. A captured key
is used ahead of a stored sign-in, so it must be replaced or removed first.
`watcher doctor` shows which credential is selected and whether grading is
paused, but it does not validate an API key against an SSO deployment. The
first accepted grade is what confirms recovery. Any other failure kind on such
a machine still follows the configured decision above.

### Denial limits

An agent that keeps being denied is stuck in a loop that more denials will not
resolve. So two limits turn a would-be gateway denial into a human
escalation instead:

- After two consecutive gateway denials, the next one escalates.
- After ten gateway denials since the developer last interacted (a new
  prompt, or answering an escalation), the next one escalates. This applies
  even when allows were interspersed; only a human interaction resets that
  count.

Denials from terminal thresholds and command rules never count toward either
limit, and always stand. Both limits apply in enforce mode only. They are
tunable in the client settings' admin-only `enforcement` block:
`escalate_after_consecutive_denies` (default 2) and
`escalate_after_accumulated_denies` (default 10).

### Enforcement modes

The client's enforcement mode controls what blocking review does with its
judgments:

- **Enforce.** Decisions take effect: denials block the call, escalations
  wait for a human.
- **Observe.** The pipeline still grades calls and records what it would
  have decided, but never intervenes. With Codex, observe-mode grading
  happens after each call has run. Observe mode is useful for tuning
  thresholds before turning enforcement on.
- **Paused.** LLM grading stops, but deterministic checks still apply: a
  command rule or an always-deny tool threshold still denies. Calls that
  would have needed grading or escalation fall through to the coding
  agent's own permission flow.

For [gateway hooks](../api/hooks.md#enforcement-modes), paused means transcript
ingestion only: thresholds, rules, and monitors do not run, and no tool-call
decision is recorded.

A [`.watcher-ignore`](../client/watcher-ignore.md) file skips LLM grading the
same way for the sessions it covers, and keeps them from being uploaded. It
never turns off these deterministic checks.

### Snoozing

Snooze temporarily skips LLM grading without changing the enforcement mode.
In enforce mode, command-rule denials and requests for human review still
apply. Calls without a rule decision use the coding agent's own permission
flow. Observe and paused modes keep the behavior described above.

## Trailing review

Trailing review runs asynchronously on the Watcher server, on a periodic
schedule over recently active sessions. Nothing on the developer's machine
triggers or configures it. Two built-in jobs run today:

- **Summaries.** The built-in Summarize grader produces a title and short
  summary for each session: an early first summary while the session is
  still active, an updated one once it goes quiet, and periodic refreshes
  for sessions that never go quiet.
- **Deep Review.** Sessions the blocking pipeline scored highly get a
  second, more thorough security review once they settle.

A session without a deep review shows a **Generate report** button in the
Analyzer to its owner and to users who may act on others' sessions
(`sessions:write:any`, which admins hold). It queues a deep review of the
session's latest transcript, and the report appears
when the review finishes, which can take several minutes. Asking again for an
unchanged transcript returns the same review. A queued or running review
shows as **Report queued** or **Grading in progress** to everyone who opens
the session. When a session's report predates its latest activity, the same
users see an **Update report** button beside it; while the newer review is
queued or running the header shows **Updated report queued** or **Updating
report**, and the old report stays until the new one replaces it. If
a review fails, the same users can **Retry** it, a few times per transcript.
A requested review that overlaps a scheduled deep review of the same session
waits for it and reuses its report.

Trailing review never blocks the agent and never feeds anything back into
the agent's context. Its grades appear in the Analyzer, alongside grades
from any other grader. Sessions whose scores reach your organization's flag
threshold are flagged for review, and can trigger a Slack notification (see
[organization settings](organization-settings.md)).

### What drives the schedule when you self-host

A `xylon-grading-scheduler` container triggers trailing review every five
minutes by default. It asks the API to grade each organization's recent
sessions. The API performs the grading. The scheduler reads no session
content, holds no provider keys, and contacts no provider. It runs the
`xylon-api` image with a different entrypoint, so it is not a separate
image to deploy or review.

The scheduler runs trailing review on its periodic schedule.
Reports requested with **Generate report** are produced by the
`xylon-operation-worker` container instead, which the `operation-worker`
profile starts and which requires PostgreSQL 17 or later; without it, those
requests stay queued. The `async-grading` Compose profile starts the
scheduler, and `.env.example` enables both profiles for new installs. Without the scheduler, a deployment still records
sessions, and clients or gateway hooks can evaluate individual tool calls, but
trailing review does not run. See
[Update Watcher](../self-hosting/updates.md) for update instructions.

## Choosing what goes where

| | Blocking review | Trailing review |
| --- | --- | --- |
| Evaluates | one tool call, before it runs | the trajectory so far |
| Can interrupt the agent | yes | no |
| Latency budget | seconds; the agent is paused | generous; runs behind the session |
| Suited to | irreversible single actions: destructive commands, secret exposure, dangerous infrastructure changes | cross-turn patterns: wrong diagnoses, premature completion claims, ignored instructions, scope creep |
| Results appear in | the Watcher client (live) and the Analyzer (as decisions) | the Analyzer (as grades) |