Anything Engine Optimization The rolling record of the AI-search industry.
Bright yellow and red walls contrasting with a vivid blue sky in a minimalist style.
Photo by Jan van der Wolf on Pexels
Common Crawl Blog · Aug 31 ResearchAEOLLMEO

Common Crawl: two-thirds of llms.txt files are templates

What this answers

How many llms.txt files did Common Crawl analyze?
Common Crawl analyzed 584,107 llms.txt files gathered in its July 2026 crawl archive, finding roughly two-thirds were produced by templates or CMS plugins rather than written by hand.
Do llms.txt files contain prompt injection attacks?
Automated screening flagged thousands of files with promotional or steering language, but Common Crawl's manual review confirmed only 4 of the 584,107 files contained a genuine prompt injection attempt.

Common Crawl analyzed 584,107 llms.txt files gathered in its July 2026 crawl archive and found 68% were produced by templates or SEO plugins — Wix alone accounts for 41% of the corpus, with All in One SEO and Yoast contributing smaller shares — while only about half fully follow the specification’s structure of a title, summary line, and link sections. Automated screening flagged 3,793 files for steering language aimed at AI crawlers and 10 for “classic prompt injection” severity, but manual review confirmed only 4 files contained a genuine injection attempt.

Why it matters: The count suggests llms.txt adoption is running mostly on CMS-plugin autopilot rather than deliberate authoring — one more reason to doubt the file format is doing much to shape what AI crawlers see.

The record: llms.txt

Glossary: SEOAI crawlersllms.txt

Via Common Crawl Blog ↗

Posted to the wire August 31, 2026. Edited by Joe Balewski.