Skip to content

Toufiq SEO

Technical SEO

Noindex Tag: When to Use It and Mistakes That Remove Pages From Google

What noindex does, which pages should carry it and how to catch the accidental noindex that takes a whole site out of Google.

  • Published
  • 4 min read
Closed sign hanging in a shop window, representing a noindex tag
On this page
  1. How noindex works
  2. Noindex versus robots.txt versus canonical
  3. Pages that usually should be noindexed
  4. Pages that should almost never be noindexed
  5. Accidental noindex: the usual causes
  6. How to find noindexed pages
  7. Removing a noindex tag
  8. Noindex, follow and nofollow
  9. Frequently asked questions

Quick answer: A noindex tag tells search engines not to show a page in search results. Add it with a robots meta tag in the page head or an X-Robots-Tag HTTP header. Google must be able to crawl the page to see the tag, so do not also block the page in robots.txt. Use noindex for pages with no search value, such as thank-you pages, internal search results and staging sites, and check regularly that no important page carries it by mistake.

Noindex is one of the most powerful and most dangerous settings in SEO. One checkbox in a CMS can remove an entire site from Google. I find accidental noindex tags in a surprising number of audits, usually left over from a site launch or a plugin setting.

How noindex works

Google’s documentation on blocking indexing with noindex describes two methods:

<meta name="robots" content="noindex">

in the HTML head, or the HTTP response header:

X-Robots-Tag: noindex

The header works for non-HTML files like PDFs. When Googlebot next crawls the page and sees the directive, it drops the page from search results, even if other sites link to it.

Noindex versus robots.txt versus canonical

ToolWhat it doesUse when
noindexKeeps a page out of search resultsThe page should exist but not appear in Google
robots.txt disallowStops crawling, not indexingYou want to save crawl resources on large URL sets
canonicalSuggests a preferred version among duplicatesSeveral URLs show the same content
301 redirectSends users and search engines to a new URLThe page has moved or been replaced

The most common error is combining noindex with a robots.txt block. If Google cannot crawl the page, it never sees the noindex tag, and the URL can stay indexed based on links. My robots.txt guide and canonical tags guide go deeper on the other two.

Pages that usually should be noindexed

  • Thank-you and order confirmation pages.
  • Internal site search result pages.
  • Login, account and cart pages.
  • Staging and development sites (better still, password protect them).
  • Thin tag or author archives that duplicate category pages.
  • Landing pages built only for paid ads with duplicate content.

Pages that should almost never be noindexed

  • Service, product and category pages.
  • Location pages you want to rank.
  • Blog posts that answer real questions.
  • Paginated category pages, which help Google find products.

If a page is low quality, noindex hides the symptom. Improving, merging or redirecting it is often the better fix. See duplicate content for how to choose.

Accidental noindex: the usual causes

  1. WordPress “Discourage search engines” setting left on after launch. It is under Settings, then Reading.
  2. SEO plugin defaults that noindex a post type or taxonomy.
  3. Staging settings copied to live during a migration.
  4. Developer headers on the server that add X-Robots-Tag noindex to every response.
  5. Page builder templates with noindex in a global header.

How to find noindexed pages

  • Search Console Page indexing report: look at “Excluded by noindex tag”. Every URL listed should be one you meant to exclude.
  • URL Inspection: shows whether indexing is allowed for a specific page.
  • A crawler such as Screaming Frog reports meta robots and X-Robots-Tag for every URL.
  • View source on key pages and search for “noindex”.

Check after every launch, redesign, plugin change or migration. My Search Console guide shows where these reports are.

Removing a noindex tag

Remove the tag or header, confirm with URL Inspection that indexing is allowed, then click Request indexing for the most important pages. Make sure the pages are in your XML sitemap and linked internally. Re-indexing usually follows within days to a few weeks.

Noindex, follow and nofollow

“noindex, follow” asks Google not to index the page but to follow its links. Google has said that over time, long-term noindexed pages may be treated as noindex, nofollow, so do not rely on noindexed pages to pass link value. Keep important links on indexable pages.

If you suspect pages are missing from Google, a technical SEO review will check every source of noindex directives across your site.

Frequently asked questions

How long does it take for noindex to remove a page?

Until Google recrawls the page, which can take days or weeks. For urgent removals, use the Removals tool in Search Console as a temporary measure.

Does noindex affect rankings of other pages?

Not directly. It only removes the tagged page. Internal links from that page may lose value over time.

Should I noindex category or tag pages?

Categories with useful content usually should be indexed. Thin tag pages that duplicate categories are good noindex candidates.

Can I noindex a PDF?

Yes, with the X-Robots-Tag HTTP header, since PDFs cannot hold meta tags.

Sources: Google Search Central: Block Search indexing with noindex; Google Search Central: Robots meta tag, data-nosnippet and X-Robots-Tag specifications.

Toufiq-ur Rahman Kobir
Written by

Independent SEO consultant

Google, Maps and AI search

Toufiq-ur Rahman Kobir is an independent SEO consultant who helps US businesses get found on Google Search, Google Maps and in AI answers. He has led SEO as SEO Manager at a US agency, managing more than twenty webmasters, content managers and copywriters, and now works directly with a small number of clients on technical SEO, local SEO, content strategy and AI search visibility.

Works with
US businesses, remotely
Background
SEO Manager at a US agency
Focus
Technical SEO, content strategy, AI search
Status
Taking on a limited number of clients
Scroll to Top