In public services, a generative answer does not merely summarize information; it silently constructs a decision surface.
What the phenomenon looks like
Eligibility rules are often conditional, evidence-based, and exception-heavy. Yet synthesis tends to compress them into a binary answer: eligible or not eligible, entitled or not entitled, compliant or non-compliant.
Why it happens
The model fills gaps by borrowing the nearest stable pattern from public discourse, documentation, and training priors. The result is often coherent, but coherence here comes from inference, not from authorized interpretation.
Why it matters
That binary output can mislead citizens and institutions alike. What should remain a procedural assessment becomes an apparently objective truth, even though the underlying rule required context, proof, and human review.
What must be governed
- Separate guidance from decision and preserve the evidentiary steps required before eligibility can be affirmed.
- Expose conditions, exceptions, and appeal paths instead of collapsing them into a yes/no output.
- Treat uncertainty as a legitimate state, not as a gap the model should close on its own.
