Amazonbot: Amazon's crawler, user agent and robots.txt rules

Operator

Amazon

robots.txt token

Amazonbot

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36

Purpose

Model training

Honours robots.txt

Yes, its operator states it does

Published IP ranges

https://developer.amazon.com/amazonbot/ip-addresses/

Reverse DNS

None documented

Official documentation

Amazon: About Amazonbot

Verified on

2026-10-05

What Amazonbot does

Amazon says Amazonbot is used to improve its products and services, helping it provide more accurate information to customers, and that it may be used to train Amazon AI models. [1]

Amazon labels the user agent above an example. Its Chrome/W.X.Y.Z reads as a version placeholder, which Amazon does not explain, so a log or firewall filter should match on Amazonbot. [1]

Allow or block Amazonbot in robots.txt

Add one of these groups to the robots.txt at the root of each host (each scheme and port counts separately) it should apply to. A crawler that finds one or more groups naming its token follows those groups, merged into one, and ignores the User-agent: * group, so repeat in them any * rules you still want applied (RFC 9309, section 2.2.1).

User-agent: Amazonbot
Allow: /

User-agent: Amazonbot
Disallow: /

What blocking Amazonbot changes

Amazon states that its automated crawling respects the Robots Exclusion Protocol, honoring user-agent and allow/disallow directives, and that it does not support crawl-delay. [1]

Amazon gives 2 timings for a change. It says each agent's setting may take about 24 hours for its systems to reflect, and also that its agents fetch robots.txt per host or use a cached copy from the last 30 days. [1]

Training also has a page-level control: Amazon honors the noarchive robots meta tag as "do not use the page for model training", alongside noindex, none and rel=nofollow. [1]

Amazon documents 2 more agents, each set independently. Amzn-SearchBot is a search crawler that, Amazon says, does not crawl content for generative AI training; when robots.txt does not mention it but allows other search bots, it follows their rules. Amzn-User fetches for users and may not follow all robots.txt directives. [1]

How to verify a request really is Amazonbot

Amazon publishes the addresses Amazonbot crawls from on an IP address page. It documents no reverse-DNS hostname, so that list is the check. [1][2]

A crawler policy is one part of being readable by assistants. The AI search readiness guide covers the rest, and robots.txt and crawl access covers the firewall and bot-protection rules that block crawlers before robots.txt is ever read. A full SEO report covers AI search readiness alongside the rest of a site's technical health.