llms.txt: proposal and limits
Avoid treating an optional convention as a universal ranking standard.
The proposal
Some initiatives propose an llms.txt file listing site resources for model-using tools. It does not replace HTML pages, essential metadata, links or a documented API.
What Google says
Google Search Central says its search systems, including generative features, do not require llms.txt or special AI markup. Vendors promising guaranteed rankings from this file overstate the documented evidence.
Why experiment anyway
A compact text index can help specific software explicitly configured to use it. Verify the consuming product, access rules, link accuracy and upkeep rather than assuming universal adoption.
Evaluation protocol
Compare identical tasks with and without the file, measuring actual page discovery and answer quality. Separate experimental findings from forward-looking interpretation.
What the file actually proposes
The llms.txt proposal provides a possible listing of text resources for certain consumers. It has no universal discovery mechanism and does not replace ordinary URLs or APIs. Google Search Central states that indexing and AI search features do not require a special llms.txt file. Claims of automatic visibility go beyond that documentation.
How to test a specific consumer
If software explicitly claims to use llms.txt, compare identical tasks with and without the file in a controlled setting. Measure discovery of correct pages, link quality and observed limits. Do not generalize results to another engine or agent without evidence of support. Source pages remain the editorial authority.
Core steps
- 01Identify actual consumers
- 02Compare with and without the file
- 03Verify source links
- 04Avoid universal search promises
Three questions to ask
Is this file required by Google Search?
Which software actually consumes it?
Can the observed impact be reproduced?
Read the source documents
Versions, reuse terms and document status can change. Check them on the original sites.