llms.txt is on 37% of the Inc 5000, and a third of that is Shopify's doing

We asked 12,713 company domains for three files: /llms.txt, /llms-full.txt and /robots.txt. The domains come from four lists we already track — the Inc 5000 (2026), Y Combinator companies, the Fortune 500 and unicorns.

A site counts as having an llms.txt only when the request returned 200 with a non-empty body that is not HTML. Plenty of servers answer every path with the homepage, so we drop anything whose content type says HTML or whose body starts with a doctype. The denominator for each group is the domains whose /robots.txt request came back with any HTTP status at all: 4,931 for the Inc 5000, 5,630 for YC, 459 for the Fortune 500, 1,058 for unicorns, 12,078 together.

Inc 5000 companies serve an llms.txt at 36.8%, the Fortune 500 at 12.2%

We expected the Fortune 500 to lead, since they have the staff. They came last by a wide margin.

314 Inc 5000 files are the same Shopify template, and 183 of those are in Consumer Products

Reading the files, we kept hitting the same one. It opens with # Agent Instructions, tells personal shopping agents to install https://shop.app/SKILL.md, then documents the store's Universal Commerce Protocol discovery and MCP endpoints. Only the store name and domain change. Shopify ships it; the merchant did nothing.

That accounts for 314 of the 1,815 Inc 5000 files. The Consumer Products industry cut is where it lands: 214 of 267 domains there have an llms.txt, and 183 of those 214 are this template. The 80.1% adoption we show for Consumer Products is a Shopify deployment, not 214 decisions.

Drop the template and Inc 5000 adoption falls from 36.8% to 30.4%, which is (1,815 - 314) / 4,931. The template shows up 47 times in YC, 16 times among unicorns and once in the Fortune 500.

YC adoption runs from 22.2% in the oldest batches to 34.7% in 2024 and later

Cut YC by batch year and the line is monotonic: 22.2% for 2014 and earlier, 23.8% for 2015-2017, 25.2% for 2018-2019, 28.8% for 2020-2021, 29.4% for 2022-2023, 34.7% for 2024 and later. Blocking moves the same way — 5.8% of the oldest batches block at least one AI bot fully, against 10.5% of the 2024-and- later batch, on a base of 1,833 companies. The newest companies both publish more for crawlers and shut more of them out.

280 Inc 5000 robots.txt files name PerplexityBot and 7 disallow it

Perplexity gets written into robots.txt far more often than it gets blocked. 282 readable Inc 5000 robots files mention Perplexity, 280 of them name PerplexityBot, and 7 disallow it entirely. The bots that actually get the Disallow: / are CCBot (185 domains), Bytespider (180), GPTBot (170), Amazonbot (162), Google-Extended (160), ClaudeBot and Applebot-Extended (154 each). Whoever writes these files is willing to block training crawlers and scrapers, and wants the answer engines in.

The 56 Fortune 500 files include nvidia.com, salesforce.com, target.com and cvshealth.com

Others in that set: paypal.com, dell.com, gm.com, att.com, metlife.com, labcorp.com, thermofisher.com, carvana.com, autozone.com, acehardware.com, paloaltonetworks.com and usbank.com. Among unicorns the file is common in developer tooling — databricks.com, clickhouse.com, circleci.com, algolia.com, fivetran.com, getdbt.com, calendly.com, deel.com, brex.com, flexport.com.

Inc 5000 adoption sits between 29.7% and 43.9% across every headcount band

We thought bigger companies would publish more. They do not. Companies with 1-10 employees are at 43.9% (base 440), 11-50 at 36.2% (2,037), 51-100 at 36.4% (1,488), 101-250 at 37.1% (531), 501-1000 at 35.7% (224), 1001-5000 at 29.7% (175). The spread is 14 points and the middle four bands are within one point of each other.

We could read robots.txt on 92.1% of Inc 5000 domains and 72.3% of YC domains

That gap matters for every robots.txt number above: the YC blocking rates are computed over the domains whose file we could read, so a dead or WAF-protected YC site is simply absent. Some of those returned 403 to us at the edge. We also count only a full Disallow: / as blocking, so a site that disallows GPTBot from /blog/ and nothing else reads as not blocking. And a file at the homepage says nothing about whether its contents are accurate, current, or of any use to a model.

Cite this: Installmap, "llms.txt is on 37% of the Inc 5000, and a third of that is Shopify's doing", 2026-09-04. The full per-domain data is in the CSV.