Skip to content

Weekly governance review

Editions: Cloud, Enterprise. This feature ships only with the Preloop Enterprise governance_review plugin. The open-source edition has no review, no console page and no API for it.

The weekly governance review checks each week's agent sessions against your account's AI usage policy. It reports what it finds and never blocks work. This page is for operators. It covers how to write the policy, what the review measures, what it reads and sends, what it costs, and how to choose the analysis model.

In the console the review is Audit > Weekly review. The policy is Policies > AI usage policy.

Defaults and account testing

The weekly report is enabled by default and runs deterministic checks for every account, including accounts with no review settings row. Account report opt-outs and recipient preferences still apply. Quiet exception weeks send no email.

Model analysis is off by default in 1.0.0 and is planned to return in 1.1.0. The global GOVERNANCE_REVIEW_MODEL_ANALYSIS_ENABLED setting defaults to false. Each account's governance_review_settings.model_analysis_override is nullable: true enables analysis, false disables it, and null inherits the global default. An account value wins even when the global default differs. Account admins choose Model analysis for this account on the review settings page; platform admins use the same selector on the account details page. This permits testing on one account while the global default remains off.

When the effective value is off, no analysis model is resolved, no model budget is reserved, and no session work is queued. Deterministic reports are complete and say “Model analysis is not enabled.” The same gate covers session digests, semantic policy findings and follow-up checks, the AI usage policy summary, and optional exception-email synthesis. The latter retains its separate account opt-in and caps. When analysis is enabled, existing model selection, consent, allowance admission and daily/weekly limits apply; unknown allowances refuse summaries, and exhausted allowances do not prevent deterministic findings.

The account settings API returns model_analysis_override, model_analysis_default, the effective model_analysis_enabled, and model_analysis_override_available. Change only the override with PUT /api/v1/governance-review/model-analysis, body {"model_analysis_override": true}; use false for off or null for inherit. This requires manage_account. Platform superusers can GET/PUT /api/v1/admin/accounts/{account_id}/governance-review/settings; ordinary account admins cannot use that route. Writes are audited and do not immediately run models.

Enabling the account override also enables its analysis mode if it was off. Existing budget pauses remain subject to allowance admission. Model selection and the account's independent analysis mode remain available when the effective flag is on. An older core pin without the new column safely inherits the default and exposes the selector as unavailable until the core migration and pin update land.

Privacy when model analysis is enabled

With the platform model, scrubbed session excerpts and digests are sent to the platform's model provider. Full transcripts are not sent.

In detail, a session summary request contains:

  • Counts measured from gateway usage rows: models, requests, tokens, tools attached and used, failures, retries, cache behaviour.
  • Your active policy text (up to 4,000 characters, shortened further if the request would be too large) and the rules compiled from it.
  • The previous rolling summary of the same session, if one exists.
  • A sample of turns: the first two, the last two and up to four of the longest user turns. Each sampled turn is cut to 1,500 characters.

The sample is scrubbed for secrets before it is cut, so a cut cannot expose part of a credential. A request is limited to 6,000 input tokens. Middle samples are dropped first when it would be larger. The model returns intent, outcome, notable points, triviality, task class and up to 10 policy observations, each with a quote of at most 280 characters. The model cannot overwrite measured counts.

What is never sent: full transcripts, unscrubbed text, and any turn longer than 1,500 characters in full.

When an account uses its own analysis model, the same content goes to that model's provider through your gateway, under your provider agreement. On a self-hosted install there is no platform model; text goes only to the model the operator configures.

To stop all analysis, set the mode to off (see Settings). Stored summaries are kept until the account is deleted or retention removes them.

How it works

Deterministic checks run weekly regardless of model analysis. When model analysis is enabled, it adds two stages.

Stage A: session summaries, hourly. A sweep runs at minute 0 of every hour (UTC). It summarises a segment of a session when the session has closed, has been idle for 60 minutes, has about 50,000 new transcript tokens, or reaches the weekly cut-off on Monday at 07:00 UTC with new content. A session that lives for weeks gets one segment per trigger, so each week reads only new content. The sweep handles at most 200 sessions per account per run. Measured counts are always stored. The model call adds the summary and the policy observations.

Stage B: weekly review, Monday 07:00 UTC. Stage B reads the week's summaries, the measured signals and your whole policy document, never raw transcripts, and writes findings.

The first review runs on the first scheduled Monday after the review is enabled, not when it is enabled.

What the review reports

Each finding has evidence excerpts (at most 280 characters, scrubbed), links to the sessions and audit events, and one concrete suggestion. A finding is one of:

  • a violation: something your policy calls a requirement was broken,
  • an inefficiency: a preference was missed. It costs time or money. The suggestion quotes the rule.
  • an anomaly: unusual activity your policy does not cover yet, such as a spend outlier, a first-seen agent, tool or client, or an error-rate jump.

The page states its coverage at the top: "Week of ... (UTC). Reviewed n of m sessions." When summaries are missing (the first week, or after a budget stop), findings from measured usage still appear, the page says "Not yet reviewed" instead of showing zeros, and a run stopped by its budget is labelled partial and keeps the findings it has.

Members see violations and anomalies with view_policies and inefficiencies with view_cost. A finding can be dismissed as Expected, Fixed or Not applicable, and restored, with manage_policies.

The policy document

Each account has one active policy document. Write it in plain language, or paste the policy you already have. There is no required syntax: the review model reads all of it. Rules you call a requirement ("must", "never", "only") are reported as violations. Rules you call a preference ("prefer", "should") are reported as inefficiencies.

The review uses stable references for plain-language requirements and preferences in both stages; no headings or rule prefixes are required. Findings quote the relevant policy sentence.

The first time the document is read, the default template is stored as version 1. It covers approved agents and models (the agents onboarded to the account and the models it allows through the gateway), no secrets or customer personal data in prompts, a forbidden-technology list for you to fill in, cost expectations and good practice for retries, caching and tools. Every edit or restore creates a new version, and history is never deleted.

What the review measures

Some checks are counted, not read. Their numbers are in What the review measures, next to the policy editor. Most teams keep the defaults.

Check Setting Default
Repeated errors Repeats in one session / sessions with the same error 5 / 3
Error rate Error rate above 10%
Prompt caching Cached share below / system prompt changes / minimum requests 0.2 / 3 / 20
Unused tools Unused for 5 sessions
Model per task Triage / summaries a small model
Long tool loops Tool calls in a row 40
Oversized context Context above / share of the model window 150,000 tokens / 80%

Each field shows where its value comes from: your saved setting or the shipped default.

The API, under /api/v1/governance-review/:

Method and path Permission Purpose
GET policy view_policies Active version and the rules compiled from it
PUT policy manage_policies Save a new version
GET policy/versions view_policies Version history, newest first, paged
POST policy/versions/{version}/restore manage_policies Copy an old version into a new one
GET policy/watchlist view_policies Compiled rules of the active version
GET review-settings view_policies The measured checks, each value with its source
PUT review-settings manage_policies Save the measured checks; null restores the defaults

Every write is recorded in the audit log.

Analysis model and permissions

GET and PUT /api/v1/governance-review/settings read and change the account's mode and analysis model. Reading needs view_policies. Changing needs manage_policies. The response includes the privacy sentence above.

Mode Meaning
on_platform The review runs on the deployment's built-in model.
on_own_model The review runs on a model the account owns and can call through the gateway.
off_over_budget The budget rule turned the review off.
off Model analysis turned off by a user. Deterministic reports continue.

Accounts without a settings row receive deterministic reports. Semantic analysis requires an enabled effective feature flag and explicit analysis consent. A self-hosted deployment also needs a configured gateway model (see Self-hosted).

Cost and the allowance cut-off when model analysis is enabled

The platform model is the deployment's built-in default model. Governance code does not name a model. Spend is recorded with the purpose governance_session_digest (Stage A) or governance_review (Stage B). A run counts both purposes in its week.

Limits when the platform model is used depend on the plan:

  • Free and trial accounts stop Stage A at $0.05 daily and share a $0.25 weekly cap across both stages. Sessions over the cap keep their measured counts without a summary.
  • Paid accounts use their remaining built-in model allowance in both stages, without the small daily or weekly caps. Before each call, the worker checks the allowance, including reservations, against the safety threshold below. Stage A projects the next review from the week's governance spend plus the next call.
  • Every platform-funded call needs a known price and enough remaining allowance. A weekly run stopped by its budget is marked partial. Normal gateway allowance reservations and account budgets still apply to every call.

What happens after a run depends on the plan:

  • Free and trial plans. A run that spends more than $0.10 of platform-funded analysis turns the review to off_over_budget. The notice reads: "This review used $X of Preloop-funded analysis, over the $0.10 free allowance, so it is now off. Keep it on with your own model in Settings > Models."

  • Paid plans. The $0.10 rule does not apply. During Stage A and after a weekly run, if the remaining allowance is below the projected next-run cost plus GOVERNANCE_REVIEW_ALLOWANCE_SAFETY_MARGIN_USD (default $0.05), the review is turned off. The former percentage floor no longer applies.

    First, the account's primary user and admins receive an email. It gives the reason, the allowance figures (included, spent, reserved, remaining, projected next review, threshold) and two ways to continue: top up the allowance, or set an own analysis model. If the email cannot be sent, the review stays on and the step is retried.

Own model. Once an own analysis model is set, the cut-off never applies. The review keeps running on that model, paid by the account, and only the account's normal gateway budgets apply. POST /api/v1/governance-review/keep-on (permission manage_policies) switches to on_own_model. Without a model id it uses the account's default gateway model, which must be a model the account owns (Settings > Models).

Every mode change is recorded in the audit log as governance_review.mode_changed.

Deployment operators can tune the limits with environment variables on the worker:

Variable Default Applies to
GOVERNANCE_SESSION_DIGEST_DAILY_CAP_USD 0.05 Free and trial
GOVERNANCE_REVIEW_PLATFORM_HARD_CAP_USD 0.25 Free and trial
GOVERNANCE_REVIEW_PLATFORM_OFF_THRESHOLD_USD 0.10 Free and trial
GOVERNANCE_REVIEW_ALLOWANCE_SAFETY_MARGIN_USD 0.05 Paid

Each variable also accepts a PRELOOP_-prefixed fallback. Paid accounts do not need a larger cap configured to cover a busy week.

Self-hosted analysis model

A self-hosted Enterprise install has no platform model and no platform funding. Set the analysis model to a gateway alias that your deployment serves:

Helm value Environment variable Default
governanceReview.enabled GOVERNANCE_REVIEW_ENABLED unset (the application default is true)
governanceReview.modelAnalysisEnabled GOVERNANCE_REVIEW_MODEL_ANALYSIS_ENABLED unset (application default false; account override wins)
governanceReview.analysisModel PRELOOP_GOVERNANCE_ANALYSIS_MODEL unset 
governanceReview:
  analysisModel: example-flash

The chart sets both variables on the API, the workers and the scheduler, and only when the value is set. governanceReview.enabled: false removes the hourly sweep and the Monday review from the scheduler. When analysisModel is set, accounts with no stored settings default to on_own_model and call that alias through the gateway. Optional analysis also requires the effective model-analysis flag. An account can choose its own model or turn analysis off; deterministic reports continue unless the account opts out of the report itself.

Delivery Report recipients and roles

The Delivery Report policy (Account settings) gains a recipients section in Cloud and Enterprise. Every member has a report role, taken from their RBAC role unless an admin sets one:

RBAC role Report role Receives
owner, admin owner the full report link
tracker_manager lead the full report link (with per-member updates on, or when named as sponsoring lead)
editor, executor, no role member a short update (with per-member updates on)
analyst, viewer viewer nothing

The account owner always receives the full report link, and so does the sponsoring lead when the policy's recipients are set to "Sponsoring lead". Anyone in the policy's opt-out list receives nothing, owner included. Any member can opt out of, or back into, the report themselves.

API: GET/PUT /api/v1/delivery-report/recipients (view_policies to read, manage_policies to save) and PUT /api/v1/delivery-report/recipients/me for a member's own opt-out. The recipients, sponsoring lead, opt-outs and per-member opt-in are stored on the account's report policy. Role overrides are stored on the account. An old account metadata opt-in is honored until the policy opt-in is explicitly set. A completed run sends through this recipient pipeline, with successful messages deduplicated per account, period and user.