Role of the matrix
The interpretive variability matrix distinguishes normal wording variation from variation that changes the understanding of an entity.
It does not by itself produce a public score. It provides a qualification frame: which axes vary, which outputs change in substance, and which evidence allows the variation to be called acceptable, fragile, or problematic.
Minimum axes
1. Retrieval axis
Do cited or activated sources change across contexts? Low source overlap indicates that the answer depends heavily on the retrieval layer.
2. Selection axis
Do the same sources exist, but are some ignored, demoted, or replaced by secondary sources?
3. Representation axis
Does the entity keep the same role, category, limits, and proof?
4. Recommendation axis
Is the entity merely mentioned, actually recommended, recommended first, or replaced by a competitor?
5. Formulation axis
Does query reformulation change only the response style or the substance of the response?
6. Cross-system axis
Do systems produce compatible readings, or do they stabilize competing versions of the same entity?
7. Temporal axis
Is the variation punctual, recurrent, or progressive?
8. Regional or contextual axis
Does the variation appear only in certain test environments, regions, languages, or session states?
9. Delivery and cache axis
Does the observed output come from a fresh generation, a routed answer, a cached answer, an approved answer, an older retrieval state, or a non-visible orchestration path?
This axis prevents stability from being confused with fidelity. An identical answer across calls may indicate robust interpretation, but it may also indicate delivery-layer fixation.
Qualification
An observation should be classified in at least one of these states:
- stable: style variation without meaning displacement.
- fragile: details vary, but canon remains recognizable.
- unstable: category, proof, or recommendability is displaced.
- drifted: qualified contradiction with baseline or canon.
- inconclusive: insufficient evidence, weak sample, or uncontrolled context.
- fixed: variation appears to have been removed by a delivery layer, without sufficient evidence that the repeated answer reflects the model’s current native behavior.
Evidence rule
A single test is insufficient. A robust conclusion must state the query, system, language, context, date, cited sources, raw output, baseline used, and qualification reason.
Without those elements, variability remains anecdotal. With them, it becomes an object of interpretive observability.
When outputs are unusually repetitive, the evidence must also distinguish the model’s native reconstruction from the application-delivered reconstruction.