What can llms.txt realistically do for AI visibility, and what should it not be relied on to do?

Treat llms.txt as an optional publisher-provided discovery/context aid, not as a universal AI indexing or ranking control.

The format can be useful for pointing automated systems toward important documentation, canonical reference pages, or a concise map of a site's information architecture. For documentation-heavy sites, that can be operationally useful even before there is broad ecosystem adoption.

But do not assume every AI crawler, answer engine, or model honors it. It does not replace robots.txt, HTTP access controls, canonicalization, XML sitemaps, or well-linked crawlable pages.

A sensible implementation is small and curated. Include the site's canonical identity, a short description, and links to genuinely important resources such as documentation, key product/service references, policies, or structured knowledge hubs. Keep URLs stable and avoid dumping the entire site.

Maintain it like any other machine-facing file. Do not let it reference redirects, stale pages, private content, or abandoned documentation.

If you want to control crawler access, use the mechanisms actually supported by that crawler/vendor and your server configuration. If you want content removed from a specific product, follow that product's documented controls.

Measure llms.txt as an experiment, not a promise. Record implementation dates and monitor crawler logs, referrals, citations, and vendor behavior where observable.

The strategic mistake is spending more time on a speculative text file than on the fundamentals: authoritative content, coherent entities, crawlability, citations, and strong documentation.