Prompt Management

Take the prompt out of the code

Prompts are versioned in a registry, published by label and distributed to the gateway as a signed bundle. Your application sends a prompt_id and variables; the template is filled in the gateway.

Immutable versions, movable labels
Gradual rollout through canaries
A version that fails the eval gate cannot ship

Request a demo Compare plans

< 30sp99 rollout

A prompt is configuration, not code

Embed a 4 KB instruction in application code and the only person who can change it is an engineer — and every change is a release. A prompt owner needs to be able to cut a version themselves, and rolling back a prompt that broke on a Friday evening should not require waking the release pipeline. The registry provides that; signed distribution keeps it from costing you any security.

What it does

Versions and labels

Versions are immutable, labels are movable. Dev, Staging and Production labels travel between versions; rollback is one call.

The variable contract

An unknown variable returns 400; a missing required variable returns 400. The contract between template and caller never breaks silently.

Canary

Give a new version 10% of traffic. The bucket is deterministic and pinned by conversationId, so a version never changes mid-conversation.

Eval gate

An unsigned eval report is rejected. If a metric the gate asks for is missing from the report, the result counts as a failure — an absent measurement is not a pass.

Prompt-aware policy

Thanks to the input.prompt block, policy can say "this prompt is regulated, pin it to a local model". The prompt is no longer the invisible part of the decision.

Cost per prompt

The decision trace carries prompt identity and version, so cost and cache savings can be broken down per prompt.

Request lifecycle

The 8 stages a request passes through

The order is not arbitrary. Prompt hydration, for instance, runs before policy evaluation: PII arrives inside the variables, so filling the template afterwards would mean scanning the wrong string.

How it works

Four steps from authoring to production — and when a prompt version changes, so does the cache namespace.

1
Create the version
Write the template and the variable schema. Once saved, a version never changes.
v1.5.0 · immutable
2
Put it through eval
A signed eval report must meet the metrics the gate asks for. The judge model cannot be the candidate model.
signed eval report
3
Publish by label
Move the staging label, then production. The production label cannot be granted without promotion approval.
promotion.decision
4
Roll out by canary
Start at 10% of traffic. If the canary is unhealthy, policy stops it itself; an explicit version pin overrides the canary.
10% → 100%

Technical summary

Call paths: /v1/chat/completions + modelion_prompt
Hydration latency: p99 ≤ 0.4 ms
Versions: Immutable · semver
Labels: Dev · Staging · Production (movable)
Distribution: Signed bundle, same path as the policy bundle · rollout p99 < 30 s
Canary: Deterministic bucket, pinned by conversationId
Rollback: One call
Cache effect: A new prompt version changes the cache namespace

The one ordering decision that matters

Prompt hydration runs BEFORE policy evaluation; prompt rendering runs AFTER routing. Reversing that breaks correctness first and latency second: PII arrives inside the variables, so any design that scans before substitution scans the wrong string — and filling the template after policy would force the decision point to run twice on every request.