Technical SEO

Noindex page still getting Google traffic: the 3 causes and how to fix each

A noindex page that still gets Google traffic usually means Google hasn't recrawled it, can't see the tag, or you're reading a lagging or canonical-assigned report. Here's how to tell which.

By SEOFix TeamPublished Updated
1,969 words · 9 min readMarkdown
Noindex page still getting Google traffic: the 3 causes and how to fix each

If a noindex page is still getting Google traffic, one of three things is going on. Google hasn't recrawled the page since you added noindex. Or Google can't see the noindex at all, because robots.txt blocks the page, a firewall answers instead of your server, or the tag only exists after JavaScript runs. Or the report you're reading is behind or counts clicks against a different URL than you think. Each has a quick check, and the fix depends on which one it is, so check before you change anything.

Before any of that, decide whether the noindex should be there. A page that earns clicks and has noindex is either about to lose that traffic on purpose, or carrying a tag left over from staging, a plugin setting or a shared layout. If it's the second, remove the tag and stop reading.

How Google processes noindex

Three rules from Google's documentation explain almost every case:

  • Google has to crawl the page to see the tag. "For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file, and it has to be otherwise accessible to the crawler" (Block indexing with noindex).
  • It only takes effect on the next crawl. The same page says that "depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page."
  • The meta tag and the HTTP header are equivalent. <meta name="robots" content="noindex"> in the HTML and X-Robots-Tag: noindex in the response headers do the same job, and the header also works for PDFs and images (robots meta tag spec). When rules conflict, "the more restrictive rule applies."

Once Google has processed it, the page shows up in Search Console's Page indexing report under "URL marked 'noindex'": "When Google tried to index the page it encountered a 'noindex' directive and therefore did not index it" (Page indexing report).

Cause 1: Google hasn't recrawled the page yet

The most common case and the least worrying. You added noindex last week, Googlebot last fetched the page a month ago, and Google is still serving the version it indexed then.

How to check. Open URL Inspection in Search Console for the page. The indexed result shows "Last crawl" ("the last time this page was crawled by Google") and "Indexing allowed?" (URL Inspection tool). If the last crawl is older than your change, this is your cause. Test live URL then fetches the page now and should report the noindex. If the live test doesn't see it either, skip to cause 2.

How to fix.

  • Click Request indexing in URL Inspection. Google calls this a request, not a guarantee. It usually takes "only a day or so, but can take much longer in some cases", and there's "a daily limit to how many index requests you can submit" (URL Inspection tool).
  • For many URLs, Request indexing doesn't scale. What we do is leave the pages in the XML sitemap until URL Inspection shows a "Last crawl" after the change, and remove them from the sitemap after that. A sitemap that lists noindexed pages is a contradiction you shouldn't keep long, but for a few weeks it's another pointer for Google to revisit them.
  • If the page has to disappear from results now (personal data, a leaked price list), use the Removals tool. It hides the URL "for about six months" and doesn't stop Google crawling it, so the noindex still has to be there when the block expires (Removals tool).

Then wait. Traffic falls away as Google recrawls, and low-priority pages go last.

Cause 2: Google can't see the noindex

This one doesn't fix itself. The page keeps its traffic until you change something, however long you wait.

robots.txt blocks the page

The classic mistake is adding noindex and a Disallow for the same path, to be thorough. Google's robots meta spec is explicit: "If a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored" (robots meta tag spec). Google adds that a blocked page "can still appear in search results, for example if other pages link to it" (block-indexing docs). In the Page indexing report this often shows up as "Indexed, though blocked by robots.txt", and Google's advice for it is to "remove the robots.txt block and use 'noindex'" (Page indexing report).

Check robots.txt for the path:

curl -s https://example.com/robots.txt | grep -i -n "disallow"

Fix: remove the Disallow for pages you want deindexed. Once Google has dropped them (they show as "URL marked 'noindex'"), you can add the Disallow back if you need to save crawl budget.

A firewall or login answers instead of the page

If a bot-management rule, rate limit or geo block sends Googlebot a challenge page or a 403, Google never gets your HTML and never sees the tag. URL Inspection's live test shows what Googlebot received. In SEOFix this shows up as BLOCKED_BY_FIREWALL on the affected pages. Our Cloudflare and Googlebot post covers how to check for it.

The noindex is added or removed by JavaScript

Google's JavaScript guidance says that when it "encounters the noindex tag, it may skip rendering and JavaScript execution", so using JavaScript to remove a noindex from the original HTML "may not work as expected" (JavaScript SEO basics). Adding noindex with JavaScript is allowed, but it only works when Google renders the page, which happens after the crawl and adds delay.

Put the tag in the server HTML. Check what the server actually returns, without a browser:

# robots meta tag and X-Robots-Tag header, as Googlebot's user agent sees them
curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  https://example.com/old-page | grep -i -o '<meta[^>]*robots[^>]*>'
curl -s -I -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  https://example.com/old-page | grep -i "x-robots-tag"

If neither command prints a noindex, Google doesn't get one from the first fetch.

In Next.js App Router, set it in the page's metadata so it renders on the server:

// app/old-page/page.tsx
import type { Metadata } from 'next'

export const metadata: Metadata = {
  robots: { index: false, follow: true },
}

The Next.js docs show the robots field rendering as <meta name="robots" content="..."> in the head. One caveat for generateMetadata: since 15.2, Next.js can stream metadata and append the tags to <body> after the initial HTML, for user agents it doesn't classify as "HTML-limited bots". The docs say Googlebot reads streamed metadata correctly. Other crawlers may not, so the curl check above is worth running anyway.

For files, or for a whole directory, send the header from the server. In nginx:

location /internal-reports/ {
    add_header X-Robots-Tag "noindex" always;
}

The tag is there, but not on every response

Some setups send noindex on some responses and not others. Common ones:

  • A CDN caches a version from before the change. Purge the paths.
  • One origin server or container still runs the old build.
  • The header is set by an app route, but static files bypass it.
  • The tag uses name="googlebot" on one template and name="robots" on another, and someone removed the wrong one while debugging.

Run the curl checks against several URLs from the same template, and a few times each.

Cause 3: the report isn't saying what you think

Sometimes Google has already dropped the page and the traffic you're looking at is old or belongs to a different URL.

The date range includes days before the change

Search Console's default views cover the last 28 days or 3 months. Google says collected data "should be available in 2-3 days" (About Search Console data). A page that went noindex ten days ago still shows weeks of clicks in a 28-day total. Switch the Performance report to daily data, filter on the page and look at the days after Google recrawled it (the crawl date from URL Inspection).

Clicks are assigned to Google's canonical

Search Console doesn't always report clicks against the URL the visitor landed on: "Click, impression, and position data for all variations of a page are assigned to the canonical URL that Google selects for each page" (What URL is my data assigned to?). If your noindexed URL is the one Google picked as canonical for a group of duplicates (say the http or trailing-slash variants of the same page), the whole group's clicks appear against it until Google re-evaluates.

The opposite also happens. You noindex /jobs?page=2 but give it a canonical to /jobs, and Google has to reconcile two signals pointing different ways. Google's canonicalization guide says it doesn't recommend "using noindex to prevent selection of a canonical page within a single site" (consolidate duplicate URLs). Pick one. Use a canonical if the pages are duplicates and the signals should go to one URL. Use noindex (with a self-referencing canonical or none) if the page shouldn't be in Google at all.

URL Inspection shows the "Google-selected canonical" for a URL. If it's a different URL from the one you inspected, the clicks you're seeing belong to that canonical.

A quick way to tell the three apart

What you seeLikely causeNext step
URL Inspection: last crawl before your change; live test sees noindex1: not recrawledRequest indexing; wait
Live test doesn't see noindex, or gets a 403 or challenge2: Google can't see itFix robots.txt, firewall or server HTML
curl shows the tag on some requests only2: inconsistent responsesPurge the CDN; check every origin
Page shows "URL marked 'noindex'", clicks only on days before the crawl3: date rangeNothing to fix
Google-selected canonical differs from the inspected URL3: canonical attributionMake canonical and noindex agree

How SEOFix flags it: NOINDEX_RECEIVES_TRAFFIC

SEOFix has a check for exactly this situation. From the indexability checks reference, NOINDEX_RECEIVES_TRAFFIC fires when:

  • the page answered 200 in the latest full audit,
  • the first <meta name="robots"> tag in its HTML contains noindex (case-insensitive), and
  • Search Console shows any clicks or impressions for the page over the last 28 days, matching any www, scheme or trailing-slash variant of the URL.

The issue records clicks_28d, impressions_28d and the meta_robots value. It's a warning, it only runs for verified sites with a Search Console connection, and it's re-evaluated at the next full audit rather than by Verify fix. SEOFix's 28-day window ends 3 days ago because of Google's data lag, so a page noindexed yesterday will still match. That's cause 3 in action, and why the check is a warning rather than an error.

It has limits worth knowing. It reads the robots meta tag only. A noindex sent in the X-Robots-Tag header, or only in a <meta name="googlebot"> tag, doesn't trigger it. It also can't tell you which of the three causes applies. It surfaces the pages so you can run the checks above. Two related checks help with the rest: BECAME_NOINDEX flags pages that gained noindex since the previous audit (the usual sign of a staging flag shipped to production), and NOINDEX_IN_SITEMAP flags noindexed pages still listed in your sitemap.

FAQ

How long does it take for a noindex page to drop out of Google? Until Googlebot recrawls it. For important pages that can be days. Google says it "may take months" for others. Request indexing speeds up individual URLs.

Should I block the page in robots.txt as well? Not until it's gone from the index. A robots.txt block stops Google seeing the noindex.

Is X-Robots-Tag better than the meta tag? Google treats them the same. Use the header for non-HTML files or when you can't edit the template. Use the meta tag when you want the rule visible in the page source.

The page shows "URL marked 'noindex'" but still has clicks. Why? Check the date range first. If the clicks are all from before the recrawl date, there's nothing to fix.


To find these pages across a whole site, verify it in SEOFix, connect Search Console and run an audit: every noindexed page that still gets clicks or impressions is listed by template. The first audit is free, up to 500 pages.

ShareXLinkedIn