Skip to content

AI Model Gateway

The Preloop Model Gateway centralizes model traffic from managed runtimes. Instead of handing provider API keys to every local agent container, Preloop keeps credentials in its secret store and issues short-lived gateway tokens to enrolled runtimes.

Why Use the Gateway

Direct provider access Through Preloop Gateway
Raw API keys on every machine Short-lived runtime tokens
Spend scattered across providers Canonical ApiUsage ledger with attribution
No per-agent model limits Subject-scoped allowed-model lists
Hard to audit prompts/responses Normalized gateway events and session drill-down

The gateway is the companion to the MCP Safety Layer: tools go through /mcp/v1, model calls go through /openai/v1, /anthropic/v1, or /gemini/v1.

Supported Endpoints

Preloop exposes provider-compatible ingress on the gateway service:

Path Purpose
GET /openai/v1/models Model listing for OpenAI-compatible clients
POST /openai/v1/chat/completions Chat completions with SSE streaming
POST /openai/v1/responses Responses API with SSE streaming
POST /anthropic/v1/messages Anthropic-format clients such as Claude Code
Gemini-compatible routes under /gemini/v1beta Google/Gemini clients when configured

Authentication reuses the same runtime bearer tokens issued during managed agent onboarding. Preloop preserves API-key context and runtime-principal metadata for attribution.

What Gets Recorded

Every gateway request can persist:

  • token usage and estimated cost in ApiUsage
  • provider, model alias, managed agent, runtime session, flow, and API key attribution
  • normalized model_gateway_call events with redaction-aware payload capture
  • provider-neutral conversation previews for operator review

The console Cost, Runtime Sessions, and AI Models views read from this ledger.

Budget and Allowed Models

Before dispatching upstream, Preloop evaluates:

  1. Account-level gateway budgets
  2. Flow-level budgets when the call originates from a flow execution
  3. Subject-scoped allowed-model lists on the active API key or managed agent

Soft limits can annotate responses; hard limits deny the call before it reaches the provider.

Preloop Cloud and Preloop Enterprise add configurable budget policies, negotiated price overrides, and enforcement workflows through billing plugins. Preloop, the open-source edition, includes spend summaries and budget-health alerts from gateway limits. See Enterprise Billing & FinOps.

Managed Agent Onboarding

When you run preloop agents onboard openclaw or preloop agents onboard hermes, the CLI attempts to rewrite local model configuration so traffic targets the Preloop gateway URL instead of direct provider endpoints. MCP configuration is narrowed to the managed Preloop MCP entry at the same time.

If model rewriting is unsupported for a particular runtime build, MCP governance can still succeed while model traffic needs a manual follow-up edit.