# Method

## What is measured

For each company, whether the visible text of its homepage mentions AI.

## Matching rule

Visible text is the archived HTML with `script`, `style`, `noscript` and `svg`
blocks removed, all remaining tags replaced by a space, HTML entities decoded and
whitespace collapsed. Attributes (`alt`, `aria-label`, `meta content`) are not
visible text and do not count.

A homepage counts as a mention when the visible text matches either regex:

```python
re.compile(r"(?<![A-Za-z0-9])AI(?![A-Za-z0-9])")            # standalone AI, case-sensitive
re.compile(r"artificial\s+intelligence|(?<![A-Za-z0-9])A\.I\.", re.I)
```

So `AI`, `AI-powered` (the hyphen is a boundary), `artificial intelligence` and
`A.I.` count. `AIR`, `said`, `email` and `OpenAI` do not: `OpenAI` is one
alphanumeric run and the lookarounds reject it.

`title_mention` applies the same two regexes to the text of the `<title>`
element only.

## Record selection

The archive holds up to two records per domain: a plain fetch and a rendered
fetch. One record is used per listed domain:

1. the rendered record when it has html (rendered records store the first-hop
   status, often a redirect, so status is not used to filter them);
2. otherwise the plain record when its status is 200 and it has html;
3. otherwise the domain is dropped.

Drop reasons: `no_record` (no archived record), `no_html` (record with empty
html), `status_not_200` (only a plain record, not a 200), `short_text` (visible
text under 300 characters, which is a block page, an empty cgi page or a
redirect stub, not a homepage), `not_the_site` (the archived page is the local
network's router status page returned instead of the homepage; one record in
the whole set, atmeta.com, matched on the literal strings "Your Spectrum Router
Info" and "Download the My Spectrum app").

Some file names appear twice in the archive: the homepage record in the shard
directory, and a record of a different kind under a `paths` directory that
carries a `paths` dict and no homepage html. `paths` directories are skipped.

Duplicate domains on a list are scored once. The Fortune 500 archive holds
records for domains that are not on the 2026 list; only domains on the list are
scored.

## Funnel

| list | on list | record found | html present | visible text >= 300 chars | kept |
|---|---|---|---|---|---|
| Fortune 500 (2026) | 500 | 472 | 472 | 461 | 460 |
| Inc 5000 (2026) | 4969 | 4929 | 4888 | 4834 | 4834 |
| Y Combinator | 6133 | 5969 | 5385 | 4992 | 4992 |
| Unicorns | 1111 | 1000 | 1000 | 966 | 966 |

## Limits

- Homepage only. A company with AI on a product page and not on its homepage
  counts as no mention.
- Archived HTML is truncated at 1,500,000 characters. Kept homepages sitting
  exactly at that cap may have had visible text cut off, so a mention that
  appears only past the cap is missed. A page truncated in the middle of a
  script or style block loses the rest of that block, which is removed as an
  unclosed block; a few such pages fall under `short_text` and are dropped
  rather than counted as no mention.

| list | kept homepages at the 1,500,000 character cap |
|---|---|
| Fortune 500 (2026) | 18 |
| Inc 5000 (2026) | 236 |
| Y Combinator | 208 |
| Unicorns | 72 |

- Crawl date: the `fetched_at` stamps on the records used run from 2026-09-04 to
  2026-09-04.
- One fetch per homepage, no retries on a block page. A homepage behind a bot
  wall is dropped, not counted as no mention.
- Geography and language vary: some homepages resolve to a localised version,
  so the wording scored is whatever that fetch returned.
- No segment share is printed where the base is under 20 crawled domains.
- Counts under 5 print as `<5`.

Data: CC BY 4.0, Installmap, 2026-09-04.
Source: Installmap, September 2026.