Failover is unambiguously good in infrastructure. If the first candidate does not answer you move to the second, and the user sees no outage. Lengthening the candidate chain increases resilience.
On a compliance path that logic inverts.
The shape of the problem
Suppose you write a rule: requests where a national ID is detected may only go to a model in the local region. You wrote it, you enforced it, and the regional distribution in your audit report looks clean.
Then one day the first candidate errors for thirty seconds.
What happens if the fallback chain is on? The request drops to the second candidate. If that candidate is a provider abroad, then for those thirty seconds regulated data leaves the region. Nobody notices, because the system behaved exactly as designed: the first candidate failed, the second took over, the user saw no outage.
In the audit report those thirty seconds appear as a line. You have to explain that line.
The silence is the real problem
The bad thing in this scenario is not that data went out — the bad thing is that it went out and was recorded as a success. The failover mechanism handled an error condition and resolved it. There is no error in the logs. No alert. Not even a spike in the metrics, because the request completed successfully.
Compliance violations reach their most expensive form exactly here: violations that look like the system working normally.
How Modelion settles it
A rule that raises a residency constraint also closes the fallback chain:
then:
effect: route
route:
candidates: [gpt-4o-mini]
fallbackToOriginal: false # ← this line
fallbackToOriginal: false means: when this rule wins, the candidate list is exactly the list the rule gave. Other candidates defined in the combo are not evaluated.
And if no candidate is left? The gateway returns 503.
The "what if the service stops" question
This comes up in every public-sector and fintech conversation. The answer is: yes, the service stops. And that is deliberate.
Failing open — permitting on error — must never widen the compliance boundary. An outage is expensive, measurable and explainable. Regulated data that went to the wrong jurisdiction is not only expensive; you also have to answer when, how many requests, and which data.
The right way to reduce that risk is not to open the fallback chain — it is to define several candidates inside the region. Two different models in the same region, or your own vLLM cluster plus a hosted model. The chain still exists, but all of it sits inside the compliance boundary.
Why policy can know this
In an architecture where the circuit breaker is pod-local and lives inside the routing layer, policy cannot see provider health. And because it cannot see it, policy cannot say "this provider is broken, but this request is in scope for data protection, so it still must not go abroad."
Injecting health through input.runtime makes that possible. Policy now knows two things at once: the state of the provider and the regulatory context of the request. It decides by looking at both.
One detail matters: if all providers are broken, no bypass happens. Failing open cannot be a justification for widening a constraint.