AI Discovery Index: how fast AI finds new pages

A live data page from Mojo Dojo, an SEO and AI-search agency, that measures how long it takes search engines and AI crawlers to first visit a page after it is published. It updates daily; this page describes the snapshot read on 19 October 2026 (the page said “Updated 20 Oct”, its own time zone being ahead). Tracking started on 6 June 2026. Publish times come from the agency’s CMS and first visits from its own server logs. A hit counts only if the bot checks out against its operator’s published IP ranges or reverse DNS, and any URL that was crawled before it went live is dropped: 219 of 330 pages, leaving 111.

The headline: across all time, the median first visit from any bot and from any AI crawler were both about 14 hours. Bingbot got to new pages first most often (47%), ClaudeBot most often among AI crawlers (32%), and 73% of new pages saw an AI crawler inside a day. PerplexityBot’s median was about 30 hours, OpenAI’s two crawlers about 38, Meta’s and Apple’s around four days. The fetches made live on a user’s behalf are a different story: ChatGPT-User reached 41 of the 111 pages, with a median of about three days, and Claude-User reached five. The page marks crawlers with only a handful of pages as anecdotes, and before 12 October it knew only the publish day, so those first visits are estimated at midday.

It is free to reuse (CC BY 4.0), comes with range toggles, a “race” chart of medians, coverage and trend charts and a per-crawler table, and it is also the agency’s shop window: the free-audit offers run down the page.