Frequently asked questions
Straight answers about noindex tags, robots.txt, canonical tags, and how pages quietly fall out of Google's index.
What does it mean when a page gets "deindexed"?
A deindexed page has been removed from Google's search index, so it no longer appears in search results even though the URL may still load normally in a browser. It's different from a page simply ranking poorly -- a deindexed page isn't shown for any query at all.
What's the difference between robots.txt and a noindex tag?
robots.txt tells crawlers which URLs they're allowed to request -- it blocks crawling, not indexing. A page disallowed in robots.txt can sometimes still appear in search results (usually with no snippet) if Google finds links to it elsewhere. A noindex directive (in a meta tag or the X-Robots-Tag header) explicitly tells Google not to include the page in its index at all, and is the correct tool when you actually want a page removed from search results.
What is X-Robots-Tag and how is it different from a meta robots tag?
Both can carry the same directives (noindex, nofollow, and so on), but a meta robots tag lives in the page's HTML head, while X-Robots-Tag is sent as an HTTP response header. X-Robots-Tag is useful for non-HTML files (PDFs, images) or when you want to control indexing at the server level without touching the page template -- and it's easy to introduce by accident through a CDN, reverse proxy, or a misconfigured server rule.
Can a canonical tag cause a page to disappear from search results?
Yes. A canonical tag tells Google which URL is the authoritative version when similar content exists at multiple URLs. If a page points its canonical tag at a different URL, Google will typically index the canonical target instead and drop the original from search results -- even if that wasn't the intent. This commonly happens after a site migration, a templating bug, or a staging environment's canonical values leaking into production.
How long does it take Google to notice a noindex tag or deindex a page?
It depends on how often Google recrawls the page, which can range from hours for frequently-updated, high-authority pages to weeks for low-traffic ones. There's no fixed timeline, which is exactly why silent deindexing is dangerous: a page can sit invisible in search for a long time before anyone notices the traffic drop.
How do I check if my page is currently indexed by Google?
Search site:yourdomain.com/path in Google, or use the URL Inspection tool in Google Search Console, which shows the exact indexing status and any reason a page was excluded. Neither tells you the moment something changes, though -- that's the gap continuous monitoring is meant to close.
Why did my page get deindexed with no warning?
Most deindexing isn't announced anywhere you'd normally look. A deploy that carries over a staging noindex tag, a CDN rule that adds X-Robots-Tag headers, a CMS setting silently flipped, or a canonical tag pointed at the wrong URL can all happen without an error, a broken link, or any visible symptom on the page itself -- the only sign is a search traffic drop, discovered days or weeks later.
Does DeindexAlert prevent deindexing, or just alert me?
DeindexAlert checks robots.txt, meta robots, X-Robots-Tag, and canonical signals on your monitored pages on a recurring schedule and emails you the moment something changes. It doesn't modify your site -- it's a monitor, not a fix -- but catching the change within the same check cycle instead of weeks later is usually the difference between a five-minute fix and a month of lost organic traffic.