A gateway with prompt management has an ordering question to answer: when does the template get filled in?
There are two options, and both look reasonable.
Option A: policy runs first, the template is filled in afterwards. Policy makes its decision, then the prompt is built and sent to the provider.
Option B: the template is filled in first, policy runs afterwards. The prompt exists in full, and policy decides while seeing it.
Option A looks cleaner at first: decision first, work second. But it is wrong.
PII arrives inside the variables
A prompt template looks like this:
Assess the credit limit for customer {{customer_name}}
({{customer_id}}). Current limit: {{current_limit}}.
There is no personal data in the template itself. The template sits in a repository, versioned, reviewed, entirely harmless.
The personal data is in the variables that arrive at call time:
{ "customer_name": "Ayşe Yılmaz", "customer_id": "10000000146" }
Now consider Option A. What does policy have when it runs? The template and the variables, separately. The guardrail pipeline wants to scan the prompt — which text does it scan? Scanning the template is pointless, because there is no PII in it. Scanning the variables individually is possible but stripped of context.
Any design that scans before substitution is scanning the wrong string.
Hence: hydration before policy
The ordering in Modelion is:
- Auth and rate limit
- Prompt hydration — template and variables are combined
- Signal hydration — the guardrail pipeline reads the real text and writes signals
- Policy evaluation — consumes the signals, makes the decision
- Routing — the decision's constraint intersects the candidate list
- Execution
When the guardrail pipeline runs at step 3 it has the full text: Assess the credit limit for customer Ayşe Yılmaz (10000000146).... The checksum runs, a signal is produced, and policy sees it.
But rendering after routing
There is a fine distinction here. Hydration happens before policy, but final rendering — the production of the text that will actually go to the provider — happens after routing.
The reason is that redaction is an obligation. When policy says "mask the detected types", the text reaching the provider must be the masked version. If you freeze the final text before policy, you have to regenerate it in order to apply the mask.
Reversing it has a second cost: filling the template in after policy forces the policy engine to run twice per request — once for the decision, once to evaluate the new signals of the masked text.
In short
Ordering is not an aesthetic preference. The wrong ordering breaks correctness first: the scanner scans the wrong text, no PII signal is produced, and policy has nothing to mask. Then it breaks latency: the PDP runs twice.
In decisions like this, one side looking "cleaner" is usually misleading. What decides it is which door the data enters through.