Multimodality, PDFs, images, tables, and video: opaque surfaces of authority
A growing share of information available to AI systems no longer appears as clean, linearly structured text. It appears as PDFs, screenshots, images, tables, diagrams, maps, visual excerpts, and video. For humans, these formats can remain perfectly legible. For synthesis systems, however, they often constitute opaque surfaces: surfaces where authority is visible, yet not immediately reconstructible.
The important point is not only that such formats are “harder to read”. The doctrinal point is that they alter the relationship between source, proof, and restitution. HTML text often makes titles, sections, dates, links, and neighborhoods explicit. A screenshot or a PDF can expose the same information while making hierarchy, context, version, or exact scope far more uncertain.
This page extends the logic of synthesis surfaces and directly meets the question of proof of fidelity. When the source is only partially interpretable, restitution may sound faithful while silently drifting away from the canon.
1. Why these formats are not “just another kind of text”
A PDF, screenshot, or video is not merely a different container. It shifts several critical markers.
- Context may depend on the previous page, the crop of the screenshot, or the exact moment in a video.
- Hierarchy may be carried by layout, size, color, legend, overlay, or visual order.
- Version may remain implicit, hidden in a filename, timestamp, or corner of an interface.
- Citation may become approximate when the source offers neither stable textual anchors nor directly addressable fragments.
In other words, the difficulty is not only recognition. It is the loss of some of the markers that allow us to know what prevails, within what scope, and from which exact origin.
2. Drift mechanisms specific to opaque surfaces
Several mechanisms are recurrent.
a) Structural flattening
A table is restituted as a sentence. A legend becomes a central fact. A footnote disappears. The system keeps the visible element while losing the relation that gave it meaning.
b) Loss of framing
A screenshot shows a state, but not necessarily the path, status, date, or perimeter of that state. An image can prove that an interface existed; by itself, it does not prove what that interface authorized.
c) Confusion between reading and inference
A chart, map, or diagram often requires interpretation. The system may turn that interpretation into an assertive statement without distinguishing what was observed from what was deduced.
d) Dissolved temporality
An archival PDF, old brochure, undated video, or reused visual may continue to govern synthesis because it remains legible, even though it is no longer applicable.
e) Masked origin
The excerpt circulates without the full source page. The image or PDF becomes the apparent “source”, even when it is only a secondary reuse. This meets the problem developed in Media, citation, and the disappearance of origin.
3. Why the problem is doctrinal, not merely technical
It may seem that better OCR, stronger vision parsing, or improved table extraction would be enough. Such advances matter. They do not resolve the central issue.
The central issue is that a partially legible source requires stronger discipline of restitution. The more opaque the source, the more cautious synthesis must be about what it asserts, how finely it cites, and where it places the boundary between observation and inference.
This is where the canon-output gap and canonical fragility become important. An output can look coherent while resting on an incomplete reading of its source. In these contexts, fluent prose is not evidence of fidelity. It can instead conceal the opacity of the originating material.
4. What governance should target
Governance of opaque surfaces does not mean banning such formats. It means declaring their status and bounding what they may govern.
At minimum, this requires:
- stating which formats are canonical, secondary, or purely illustrative;
- providing, for critical assertions, a more directly enforceable textual equivalent;
- attaching visuals, tables, and PDFs to a source page, version, and date;
- citing at the correct granularity: page, section, screen, frame, legend, rather than “document” in the abstract;
- requiring an interpretation trace whenever an opaque format supports a high-impact assertion.
In internal environments, this discipline also meets internal systems and silent delegation of authority. A screenshot or internal PDF must not become, through mere availability, an unqualified rule.
5. Interaction with the rest of the corpus
Opaque surfaces do not exist in isolation. They traverse several terrains already addressed on the site.
- In product sources, they appear as PDF docs, feature screenshots, price tables, or visual changelogs.
- In third-party platforms and local surfaces, they circulate as maps, listing cards, and reused excerpts.
- In media, they reinforce disappearance of origin when the visual excerpt survives better than the source publication.
- In synthesis interfaces, they encourage reformulations that sound safe even when the underlying material was only partially read.
This terrain therefore shows something simple: the more a source depends on its visual device, the more governance must make explicit what a system may read, infer, and cite from it.
6. What this page does not establish
This page does not say that a PDF, image, or video is inherently a weak source. Nor does it say that only HTML pages should count.
It establishes a more precise distinction: some formats make authority less directly reconstructible. When used as a basis for synthesis, they require stricter conditions of restitution, citation, and proof than ordinary linear text.