Skip to content

Install Preloop Enterprise on minikube or Kubernetes

Use the Enterprise API and console images with the OSS Helm chart. This guide is for a single-node trial, with one flow at a time. For production, use an external managed PostgreSQL 16 database with pgvector or CloudNativePG, tested backups, TLS ingress, and a cluster administrator's resource and network review. The distribution README starts with the complete Docker Compose instructions.

Release example: 0.17.0-rc.1, Linux amd64. The registry images are not multi-platform. On an arm64 Mac, use locally built arm64 images for minikube as described below. Pull credentials do not grant access to the Enterprise source.

Get the install bundle

You need Docker, kubectl, Helm 3, minikube, 4 CPUs, 8 GB available to minikube, and disk space for images and a 5 GiB database volume. A local model needs additional memory outside this budget. You also need your read-only GitLab deploy token with read_registry, and access to the distribution repository. Use repository read credentials separately if your registry token lacks read_repository.

git clone https://gitlab.spacecode.ai/spacecode/preloop-ee-dist.git
cd preloop-ee-dist

Use chart.tgz from a released bundle. It includes the OSS chart and its NATS dependency, pinned to the release's OSS commit in RELEASE.json. The chart version can differ from the Enterprise image version. For this first release's source checkout, produce that same package with:

# From an EE source checkout with its preloop submodule initialized:
helm repo add nats https://nats-io.github.io/k8s/helm/charts
helm dependency build preloop/helm/preloop
helm package preloop/helm/preloop --destination /tmp
# Copy /tmp/preloop-<chart-version>.tgz into the bundle as chart.tgz.

Start minikube and create secrets

Calico enforces the agent NetworkPolicy. The default minikube CNI may accept NetworkPolicy objects without enforcing them. Use a separate kube context for this trial, and keep the namespace name preloop for the commands below.

minikube start -p apw-k8s --driver=docker --cpus=4 --memory=8192 --cni=calico
kubectl config use-context apw-k8s
kubectl create namespace preloop

# Enter your deploy token without putting its password in shell history.
read -r REGISTRY_USER
read -rs REGISTRY_PASSWORD; echo
REGISTRY_CONFIG=$(mktemp -d)
chmod 700 "$REGISTRY_CONFIG"
printf '%s' "$REGISTRY_PASSWORD" | docker --config "$REGISTRY_CONFIG" login \
  registry.spacecode.ai --username "$REGISTRY_USER" --password-stdin
unset REGISTRY_PASSWORD
kubectl -n preloop create secret generic registry-pull \
  --type=kubernetes.io/dockerconfigjson \
  --from-file=.dockerconfigjson="$REGISTRY_CONFIG/config.json"
rm -rf "$REGISTRY_CONFIG"
unset REGISTRY_CONFIG

# Flow Jobs use the namespace default ServiceAccount, not the chart account.
# Retain any other pull secrets if you adapt this to an existing namespace.
kubectl -n preloop patch serviceaccount default --type=merge \
  -p '{"imagePullSecrets":[{"name":"registry-pull"}]}'

# Keep generated credentials outside values and version control.
# Hex avoids URL-encoding problems in the connection string.
POSTGRES_PASSWORD=$(openssl rand -hex 24)
JWT_SECRET=$(openssl rand -hex 32)
kubectl -n preloop create secret generic preloop-app \
  --from-literal=jwt-secret="$JWT_SECRET" \
  --from-literal=postgres-password="$POSTGRES_PASSWORD" \
  --from-literal=database-url="postgresql+psycopg://postgres:$POSTGRES_PASSWORD@postgres:5432/preloop"
unset POSTGRES_PASSWORD JWT_SECRET

Secrets are namespaced. If you later move Jobs to a separate execution namespace, create registry-pull there too and patch that namespace's default account. The chart's imagePullSecrets applies to API, console, workers and Helm hook Jobs, but it does not configure dynamically created agent pods.

Install the signed Enterprise licence

Store the signed licence supplied by Preloop in a private local file, then create its Kubernetes Secret without putting the licence in command arguments:

kubectl -n preloop create secret generic preloop-license \
  --from-file=PRELOOP_EE_LICENSE_KEY=/secure/path/license.key

The supplied values-minikube.yaml includes an optional preloop-license secretRef in extraEnvFrom, alongside the optional preloop-network ConfigMap. Keep both entries in any private override. The chart passes these to backend processes and hooks. Without a valid licence, licensed capabilities are unavailable. After renewing the Secret, restart the backend deployments to load the changed environment. A superuser can check /api/v1/licence/status.

Install and migrate

The provided database is a trial StatefulSet with a PVC. It is deliberately separate from the chart, so there is no CloudNativePG operator prerequisite. The values use its URL from preloop-app, disable ingress and autoscaling, start the Jobs executor and limit orchestration to one concurrent flow.

kubectl -n preloop apply -f postgres-minikube.yaml
kubectl -n preloop rollout status statefulset/postgres --timeout=5m
helm upgrade --install preloop ./chart.tgz -n preloop \
  -f values-minikube.yaml --timeout 15m
kubectl -n preloop rollout status deployment/preloop-api --timeout=5m
kubectl -n preloop rollout status deployment/preloop-console --timeout=5m
kubectl -n preloop get pods,jobs,pvc
kubectl -n preloop port-forward svc/preloop-console 6230:80

Open http://localhost:6230. In another terminal:

curl -fsS http://localhost:6230/api/v1/health
helm -n preloop status preloop

The post-install hook runs scripts/init_db.py --force, which upgrades the schema, then synchronizes flow presets. Successful hooks are deleted; Helm reports a successful release only after they complete. Do not use --wait on the initial install: application readiness can depend on the schema the post-install hook creates. Inspect failed hooks with kubectl -n preloop logs job/preloop-init-db-job. On an upgrade, the pre-upgrade migration hook runs the lock-aware migration wrapper first.

Source-free candidate handoff

For an operator-supplied provisional candidate, use its CANDIDATE.json, image archive, checksum file and CANDIDATE-INSTALL.md. The manifest records the actual backend EE/OSS source commits separately from any newer installer-helper revision. Verify the supplied SHA-256 checksums before importing the archive:

sha256sum -c SHA256SUMS
docker load -i native-candidate-image.tar.gz
minikube -p apw-k8s image load "<backend-image-reference-from-CANDIDATE.json>"
helm upgrade --install preloop ./chart.tgz -n preloop \
  -f values-minikube.yaml -f candidate-values.yaml -f private-values.yaml \
  --timeout 15m

Keep private-values.yaml limited to your URL, network and secret overrides; check it does not override the candidate image reference. The handoff can keep the unchanged released console image. Its registry credentials and architecture requirements still apply. No Enterprise source checkout is required, and the candidate must retain its candidate tag instead of replacing a release tag. Check the handoff's acceptance status before customer use: an unmerged candidate is provisional even if its health check passes.

Native source candidate test

An Enterprise source checkout can build native images without publishing a release. Record both the Enterprise commit and the exact OSS submodule commit; use a clean checkout so the image corresponds to those commits. A candidate is not the published release, even when its application version is unchanged. On arm64, an emulated Compose test of the amd64 release does not validate native Kubernetes execution.

# From the EE source checkout, not the distribution checkout:
git status --short
EE_SHA=$(git rev-parse HEAD)
OSS_SHA=$(git -C preloop rev-parse HEAD)
CANDIDATE_TAG="ee-${EE_SHA}-oss-${OSS_SHA}"
docker build -f Dockerfile \
  --label "org.opencontainers.image.revision=$EE_SHA" \
  --label "ai.preloop.oss-revision=$OSS_SHA" \
  -t "preloop-ee-candidate:$CANDIDATE_TAG" .
docker build -f Dockerfile.frontend --build-arg BRAND=preloop-selfhosted \
  --label "org.opencontainers.image.revision=$EE_SHA" \
  --label "ai.preloop.oss-revision=$OSS_SHA" \
  -t "preloop-console-candidate:$CANDIDATE_TAG" .
minikube -p apw-k8s image load "preloop-ee-candidate:$CANDIDATE_TAG"
minikube -p apw-k8s image load "preloop-console-candidate:$CANDIDATE_TAG"
# Keep a registry-pull secret even for preloaded images. Dummy credentials
# may be used for this local-only test; they do not test registry authorization.
helm upgrade --install preloop ./preloop/helm/preloop -n preloop \
  -f distribution/values-minikube.yaml --timeout 15m \
  --set image.repository=preloop-ee-candidate --set-string image.tag="$CANDIDATE_TAG" \
  --set console.repository=preloop-console-candidate --set-string console.tag="$CANDIDATE_TAG"

Register, connect GitLab, run a flow

  1. Register the first user. It owns its account. After setup, change the REGISTRATION_ENABLED entry in your private values copy to "false", upgrade with that file, and invite colleagues from the console. Configure SMTP for invitations, password resets and email approvals.
  2. In Models, add a provider credential and enable gateway routing on the model used by your flow. For an OpenAI-compatible in-cluster Qwen service, use http://qwen.models.svc.cluster.local:8000/v1. Label its actual pods app=qwen in namespace models, or edit extraEgress to match them. The provided rule allows agent pods to that service's backend pods on TCP 8000; DNS and the Preloop API/gateway are allowed separately. The gateway also needs access to Qwen. This policy isolates agent pods, not the gateway. Separately, the model endpoint SSRF guard denies private addresses until an operator allowlists them. Allow only this Service IP, then restart the application Deployments so they read the optional preloop-network ConfigMap:
MODEL_IP=$(kubectl -n models get service qwen -o jsonpath='{.spec.clusterIP}')
kubectl -n preloop create configmap preloop-network \
  --from-literal=MODEL_ENDPOINT_ALLOWED_CIDRS="$MODEL_IP/32" \
  --dry-run=client -o yaml | kubectl -n preloop apply -f -
kubectl -n preloop rollout restart deployment -l app.kubernetes.io/instance=preloop
kubectl -n preloop rollout status deployment/preloop-api --timeout=5m

Do this before saving the model. Preserve the ConfigMap, and update its IP if the model Service is recreated. Do not allow all private ranges just to bypass the endpoint guard. 3. In Trackers, add GitLab, your instance URL, and a dedicated bot user's personal access token (or a group access token). Use api, read_repository and write_repository, with Maintainer access on the selected test project. Project access tokens cannot list groups. Complete the first sync. Confirm the webhook under GitLab Settings > Webhooks points at this installation. 4. In Flows, create a small manually triggered flow with your configured model and no repository writes, for example "Reply with INSTALL_OK". Trigger it, wait for a completed execution, and check its output. Then create a Pull Request Reviewer preset, bind the synced project, and test an MR opened/updated webhook. 5. Inspect a running agent pod. Kubernetes adds the default account's pull secret at admission, so inspect the pod, not only the Job template:

kubectl -n preloop get pods -l app=agent-execution \
  -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.spec.imagePullSecrets}{"\n"}{end}'
kubectl -n preloop get jobs -l app=agent-execution
kubectl -n preloop logs deployment/preloop-preloop-sync-worker-flow-execution

Agent images also need to support your node architecture. A private agent image needs a token authorized for its registry, added to the same ServiceAccount. image.repository and console.repository do not change the agent image. Use the chart's agents.codexImage override if your operator supplies one.

Provide a local built-in model

An operator can provision a global model through the source-free configure_local_model.py in the distribution. Run it inside the API container, which already has database credentials and the endpoint allowlist. This needs the core version that supports configurable built-in aliases; the original 0.17.1 core is rejected before writing. Use the explicitly identified candidate or a subsequent release, without relabeling it as the original 0.17.1 image.

First configure the narrow MODEL_ENDPOINT_ALLOWED_CIDRS entry described above. The provider must already serve the requested model. For a host-side Ollama server, use the host's reachable private address, not the pod's localhost, and permit only that destination and TCP port in the appropriate network policy. Configure and verify context limits at the model server; a model name or database description does not set its context window.

# Replace these example values with your operator-owned inference service.
kubectl -n preloop exec -i deployment/preloop-api -- python - \
  --name "Local Qwen" --model qwen-local \
  --endpoint http://10.0.0.5:11434/v1 --alias local/qwen \
  --context-window 98304 --max-output 8192 \
  --make-default < configure_local_model.py
# Review the preview, then repeat with --apply to persist it.

Set these limits to the values actually configured and verified on the model server. The helper stores model_parameters.context_window and model_parameters.max_output_tokens so the agent harness knows the same limits; it does not reconfigure Ollama. Verify the Codex job configuration contains model_context_window = 98304 and model_max_output_tokens = 8192 separately from verifying the Ollama allocation. Omit the flags only when preserving an existing configuration or deliberately leaving the harness limits unknown.

The script uses the CRUD layer, applies endpoint security checks, and makes no provider requests. The alias and upstream identifier remain separate and truthful. It declares operator-owned local inference, whose default inference price is zero; an existing explicit pricing override is preserved. The default is instance-wide; account-owned defaults continue to take precedence. Existing models with a matching alias are updated only if their upstream route matches. For an existing global row using an old compatibility alias, pass its explicit --model-id UUID to change the alias without breaking model-ID references. When migrating from Compose, add --clear-gateway-url only if the row has an old gateway URL override: this removes gateway.url so the new deployment chooses its normal gateway. Without the flag, that routing metadata is preserved. Account-owned models cannot be converted with this command. This helper targets no-auth local endpoints; configure authenticated endpoints through your normal credential management instead of putting secrets in command arguments.

After applying, check Models for the built-in/default badge, run provider model discovery, and execute a small flow. Verify its gateway alias, output, usage and effective price. Provisioning alone does not prove inference works.

Connect users and self-hosted GitLab

Choose one stable URL that users, the GitLab webhook sender and the execution workers can reach. Preloop and agent Jobs must also reach the GitLab API and HTTPS clone URLs. Public Internet exposure is optional.

For a private VPN or Tailscale deployment, keep the console port-forward bound to localhost and expose it through the approved private proxy. For example, with Tailscale already joined to the organization's tailnet:

# Keep this process supervised on the host, or running in a terminal.
kubectl -n preloop port-forward svc/preloop-console 6230:80
# In a second terminal on the same host:
tailscale serve --bg http://127.0.0.1:6230

Use the HTTPS URL reported by Tailscale as environment.preloopUrl. Confirm GitLab and cluster workloads can resolve and reach it through your private routing and ACLs; a browser on your laptop succeeding does not prove either path. If GitLab cannot join the VPN, provide an approved subnet router or webhook relay reachable from its network, or choose the public TLS ingress. Laptop tailnet access alone does not enable webhook delivery. Do not enable public Funnel for a private-only trial. If you use a private reverse proxy instead, bind its upstream locally and configure the same WebSocket and streaming timeout requirements below. A raw port-forward bound to 0.0.0.0 exposes the login over HTTP and requires a trusted network and explicit host firewall rules.

For a public HTTPS deployment, install your cluster's supported ingress controller, point DNS at it, and obtain a certificate. Put these overrides in a private values file, substituting your hostname and existing TLS Secret:

environment:
  preloopUrl: https://preloop.example.com
ingress:
  enabled: true
  className: nginx
  annotations: {}
  hosts:
    - host: preloop.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: preloop-tls
      hosts: [preloop.example.com]

Pass the private file after values-minikube.yaml on every Helm invocation. The TLS Secret must exist in namespace preloop, or use your organization's certificate controller annotations instead. For a short public trial, the approved tunnel below is another option.

A port-forward is local to the laptop. GitLab cannot send a webhook to your localhost. Keep the port-forward running and expose it using your approved HTTPS tunnel, for example:

cloudflared tunnel --url http://localhost:6230

Use the resulting HTTPS hostname as environment.preloopUrl in your private values file and repeat helm upgrade --install with all the same image and secret overrides. A stable named tunnel is preferable: changing the hostname requires updating tracker webhooks. GitLab must resolve and reach that URL and trust its certificate; Preloop and agent pods must reach the GitLab API and git HTTPS endpoint. Permit WebSocket upgrades for live execution logs and keep streaming proxy timeouts at least 900 seconds. No public tunnel was used for the local acceptance test.

For a private GitLab address, an administrator may need to allow local-network webhook requests. The agent policy excludes private ranges from general internet egress. Add a narrow extraEgress peer and port 443 for your GitLab's private CIDR; do not remove the database isolation to make it work. For a private CA, mount it using extraVolumes/extraVolumeMounts and set SSL_CERT_FILE and REQUESTS_CA_BUNDLE with extraEnv. Use a TLS ingress instead of a laptop tunnel for a persistent installation.

Single sign-on

SSO needs an external HTTPS origin in environment.preloopUrl and sso.enabled: true. On a fresh install the routes are off (sso.enabled: false). Setup, the other sso.* values and IdP guides are in Single sign-on.

Email and private certificate authorities

For invitations, password resets and email approval delivery, create a private file smtp.env containing SMTP_HOST, SMTP_PORT=587, SMTP_USERNAME, SMTP_PASSWORD, SMTP_FROM and SMTP_FROM_NAME. The SMTP relay must support STARTTLS and be reachable from the backend pods. Load it without placing the password in shell arguments:

chmod 600 smtp.env
kubectl -n preloop create secret generic preloop-smtp --from-env-file=smtp.env

Add secretRef: {name: preloop-smtp} to extraEnvFrom in your private values, retaining the existing preloop-license and preloop-network entries. Helm replaces lists rather than merging them. Upgrade with that file and restart backend deployments after rotating the Secret. Test delivery with an invitation before relying on email approvals. Store or remove the local credential file according to your organization's secret handling policy.

For private GitLab/model certificates, mount the approved CA bundle with extraVolumes and extraVolumeMounts, and set SSL_CERT_FILE and REQUESTS_CA_BUNDLE to its mounted path. Agent Jobs need the same trust chain in their execution image or configured runtime mount. Do not disable TLS verification to work around a private CA.

Restrict agent egress for local inference

The sample values allow public egress for normal Git and package workflows. For a local-only trial, explicitly override that default while keeping access to the API and model gateway:

agentExecution:
  networkPolicy:
    enabled: true
    allowExternalLLMAPIs: false
    allowPreloopAPI: true
    # Replace the sample direct-Qwen rule. Gateway-routed agents do not
    # need direct access to the model server.
    extraEgress: []

If the flow needs GitLab or package mirrors, replace the empty list with only the required destinations and ports. For example, one private HTTPS GitLab:

    extraEgress:
      - to:
          - ipBlock:
              cidr: 10.0.0.10/32
        ports:
          - protocol: TCP
            port: 443

Use the actual endpoint CIDR and review DNS, proxy and NAT behavior on your CNI. The gateway still needs its narrow route to the local model. These rules isolate agent pods; they do not impose a cluster-wide egress policy on API or gateway processes. Test that a real agent flow reaches the local gateway and that prohibited external destinations are blocked. Keep the overlay on every Helm upgrade.

Prepare an offline trial

Stage the exact release or candidate images and record their digests while connected: API, self-hosted console, PostgreSQL/pgvector, NATS, all chart hooks, the selected agent image, Kubernetes node images and Calico components. Review the rendered chart for every image reference. Also stage chart.tgz, the installation files, the licence, required CLI binaries and the model weights. An offline image build additionally needs its base images and package mirrors; prefer prebuilding and transporting the finished images.

Host Docker caches are separate from Minikube's runtime. Load each application and agent image with minikube -p apw-k8s image load IMAGE or use an internal registry reachable from the node. The pinned universal Codex agent image is large (approximately 32 GB unpacked in the tested trial); reserve disk space for both host and cluster storage. Verify image presence in Minikube before starting a flow. Keep private registry pull secrets on the execution ServiceAccount even when other application images are preloaded. A local model removes external AI-provider calls, but repository access, package installation inside agents and email still need approved local services or network paths. Test the complete flow with the intended offline restrictions.

Upgrade, back up and restore

Back up before upgrading, keep the old image version and chart, and preserve preloop-app (JWT and encryption material) in your secret manager. The database contains tracker credentials and audit data. Protect backups accordingly.

kubectl -n preloop exec postgres-0 -- pg_dump -U postgres -d preloop -Fc > preloop.dump
# Check the archive can be read. Rehearse a restore into a separate trial DB.
kubectl -n preloop exec -i postgres-0 -- pg_restore --list < preloop.dump

Fetch the next release bundle, review schema compatibility, update both image.tag and console.tag in your private values file, and run:

helm upgrade --install preloop ./chart.tgz -n preloop \
  -f values-minikube.yaml --timeout 15m
kubectl -n preloop rollout status deployment/preloop-api --timeout=5m
curl -fsS http://localhost:6230/api/v1/health

Keep every private override on each upgrade. The migration hook handles schema upgrades; Helm rollback does not undo a schema change. If a release requires recovery, stop all API/gateway/workers/scheduler/monitor Deployments, restore into an empty compatible PostgreSQL database using pg_restore --no-owner, restore the original secrets, and reinstall the matching old chart/images. Point database.urlFromSecret at the restored DB. Validate health and a flow before reopening traffic. For production use automated backups with restore drills and a retention policy, not only this manual dump.

helm uninstall preloop -n preloop removes the application, not the separate PostgreSQL StatefulSet. Do not delete PVCs or minikube delete -p apw-k8s until you have exported data you intend to keep.

Compose dispatch notes and Kubernetes troubleshooting

For a single Docker host, use the distribution's compose.yml, Caddyfile and .env.example; its README includes the complete Compose instructions. This uses Docker containers for flow execution rather than Kubernetes Jobs.

Compose grants the host Docker socket only to flow-worker. API, gateway, scheduler and general workers dispatch execute/resume tasks through NATS with FLOW_EXECUTION_WORKER_ENABLED=true; the general worker excludes those tasks. Do not mount the Docker socket into the API to repair dispatch. Kubernetes uses flowExecution.workerEnabled: true and creates agent Jobs through its ServiceAccount permissions, without mounting the host Docker socket.

Historical note (2026-11-04): only the immutable 0.17.1 Compose bundle needs an override setting FLOW_EXECUTION_WORKER_ENABLED=true on API, gateway, scheduler and general worker. The 1.0.0-rc.1 bundle sets this in the shared environment already; do not add the old override. Release tags stay immutable.

  • ImagePullBackOff: check the image tag, node architecture, secret namespace, token expiry and read_registry scope with kubectl describe pod. Recreate an expired pull secret and restart affected pods. docker login on your laptop does not configure Kubernetes credentials.
  • Hook fails: inspect the hook Job's logs and database readiness; check the secret URL and pgvector availability. Do not bypass a failed migration.
  • Flow fails: inspect the flow worker and agent pod logs, model gateway routing, ServiceAccount pull secret, agent image architecture, and policy selectors/ports. A rendered policy alone is not evidence of enforcement.
  • Billing mode: leave PRELOOP_HOSTED unset. Billing starts without Stripe, retains cost tooling and applies no Cloud subscription limits. The console uses frontend-selfhosted, opens login and has no Plan or pricing page. The Compose instructions explicitly set BILLING_ENFORCE_ENTITLEMENTS=false.

Console metadata on a custom domain

The preloop-selfhosted brand's domain in frontend/brands.yaml is a build-time value. It defaults to localhost because the distributable image cannot know an operator's hostname. The console uses relative application URLs, but generated public-page canonical URLs, sitemap, robots and other discovery metadata use that build-time domain. PRELOOP_URL does not rewrite those static files. If these public metadata files matter for your installation, set the brand's domain to your hostname and rebuild Dockerfile.frontend with --build-arg BRAND=preloop-selfhosted before loading the console image.