Applebot-Extended: the robots.txt token for Apple AI training

Operator

Apple

robots.txt token

Applebot-Extended

User agent

No request user agent of its own; fetching is done by Applebot

Purpose

Usage control token (no crawler of its own)

Honours robots.txt

Its operator documents robots.txt as the control

Published IP ranges

https://search.developer.apple.com/applebot.json

Reverse DNS

*.applebot.apple.com (Applebot)

Official documentation

Apple: About Applebot

Verified on

2026-10-05

What Applebot-Extended does

Apple says publishers can opt out of their website content being used to train Apple's general-purpose foundation models, which power generative AI features across Apple products, by disallowing Applebot-Extended. [1]

Apple states that Applebot-Extended does not crawl webpages. Applebot, Apple's web crawler, does the fetching, and Applebot-Extended is only used to determine how the data Applebot collects may be used. [1]

Allow or block Applebot-Extended in robots.txt

Add one of these groups to the robots.txt at the root of each host (each scheme and port counts separately) it should apply to. This group controls how Apple may use content its crawler collects. It does not change what Applebot may fetch; that is decided by whichever group applies to the crawler doing the fetching.

User-agent: Applebot-Extended
Allow: /

User-agent: Applebot-Extended
Disallow: /

What blocking Applebot-Extended changes

Apple states that pages disallowing Applebot-Extended can still be included in search results, and that the rule is not considered in ranking. As long as Applebot itself may crawl the site, the content remains discoverable through Spotlight, Siri, Safari and other system-wide features. [1]

Apple's AI-generated answers that draw on broad world knowledge have a separate control: Apple says publishers opt out of those with the nosnippet meta tag. [1]

Apple says that in general search crawls Applebot follows robots.txt directives aimed at it, that when robots.txt names Googlebot but not Applebot it follows the Googlebot rules, and that it does not follow crawl-delay. Apple does not say whether Applebot-Extended inherits rules the same way, so name it explicitly. [1]

How to verify a request really is Applebot-Extended

Apple says Applebot's traffic is generally identified by reverse DNS in the applebot.apple.com domain, and shows both lookups: hostname to address, and address back to hostname. Apple also publishes Applebot's IP ranges as a JSON list. [1][2]

A crawler policy is one part of being readable by assistants. The AI search readiness guide covers the rest, and robots.txt and crawl access covers the firewall and bot-protection rules that block crawlers before robots.txt is ever read. A full SEO report covers AI search readiness alongside the rest of a site's technical health.