Discoverability vs training

Why this distinction has become necessary

In interpreted environments, the same site can be consulted for several purposes that must not be confused.

A system can:

• discover that a resource exists;
• read that resource in order to answer;
• reuse its content for training, alignment, or consolidation purposes.

When these three regimes are merged, an organization thinks it is governing one thing while it is actually trying to bound different uses.

Discoverability

Discoverability means that a resource can be found, explored, or retained as a reading candidate.

It mostly concerns:

• the public existence of a surface;
• its technical accessibility;
• its structural clarity;
• its probability of being mobilized in a reading path.

Discoverability does not yet tell us:

• whether the resource will be cited;
• whether its content will be faithfully rendered;
• whether its text will be reused for training purposes.

Reading for answer generation

A second regime appears when the system does more than discover a resource and actually uses it to construct an answer.

At that point, the relevant question is no longer only “can the system see me?” but rather:

• which surface is actually read;
• which part is retained;
• which source hierarchy is applied;
• under which conditions the response remains legitimate.

That regime should be read with Indexing, answer generation, and training, Signal, proof, and compliance, and the Evidence layer.

Training

Training belongs to a third regime.

It is not equivalent to discovering a resource or citing it once. It concerns the use of a corpus to modify parameters, behaviors, synthesis preferences, or probability distributions.

That is why training must not be described as if it were a simple reading event.

A discoverability signal is not a training signal. An access signal is not proof of reuse. A governance artifact is not a guarantee of obedience.

Why confusion persists

Confusion persists for three reasons.

1. Public vocabulary remains blurred

The market often mixes:

• AI visibility;
• crawler access;
• answer citation;
• training use;
• declared compliance.

2. Policy surfaces do not govern the same level

robots.txt, llms.txt, meta directives, headers, governance manifests, and contextual pages do not govern the same layer of the problem. See Machine policy surfaces.

3. Systems themselves do not always expose the regime they are using

A system may discover without citing, cite without fidelity, or rely on a resource without making that use legible. Hence the need for a doctrine that distinguishes regimes instead of collapsing them.

Doctrinal consequence

An organization that wants to govern its machine presence correctly must publish enforceable distinctions between:

• being discovered;
• being read for answering;
• being reused for training.

That distinction is not cosmetic. It conditions the validity of published policies, the interpretation of observations, and the ability to avoid abusive conclusions.

Consequence for applied surfaces

An applied surface such as Better Robots.txt can materialize part of that distinction on WordPress. It must not be read as if it exhausted the full doctrine of these regimes.

The proper reading order is therefore:

1. doctrine of regimes;
2. clarification of limits;
3. applied surface;
4. bounded proof repository.

For the concrete implementation layer, see the Better Robots.txt applied surface.