Page Indexability: Why Search Engines Skip Pages You Want Ranked

Indexability is the first gate. A page search engines decline to index cannot rank, cannot earn a snippet, and cannot appear in an AI answer, however good the writing is. The directives that control it are small and easy to move by accident: a noindex meta tag, an X-Robots-Tag header, a robots.txt rule, or a canonical pointing somewhere else. Most indexability damage traces back to a deploy. A staging build ships with its blanket noindex intact. A CDN rule adds X-Robots-Tag: noindex across a whole path. A CMS setting flips for one template and quietly removes 200 product pages. All of it returns a healthy 200, so uptime monitoring stays green while traffic drains over weeks. The checks below read the directives the way a crawler does, in the two places that often disagree: the HTML you serve and the headers wrapped around it. A sitemap that lists blocked or noindexed URLs asks search engines to index pages you have simultaneously told them to skip; those checks live in the XML sitemaps guide, and they are worth reading next. Work through these in order. Confirm the homepage is open, confirm the header layer agrees with the HTML, then confirm the pages you actually care about are reachable.