Guides

The Page indexing report explained: every reason, which ones need action, and how to work past 1,000 examples

What each "Why pages aren't indexed" reason means, which ones you should fix and which mean Google is working as intended, and how to analyse a site with more than 1,000 affected URLs per reason.

By SEOFix TeamPublished Updated
2,111 words · 10 min readMarkdown
The Page indexing report explained: every reason, which ones need action, and how to work past 1,000 examples

The Page indexing report in Search Console shows how many URLs Google knows on your property, how many it indexed, and a reason for every URL it didn't. Most of those reasons are not errors. Google says so itself: "Remember that Not indexed is not necessarily bad" (Page indexing report help). The reasons that usually need work are server errors, redirect errors, unexpected 404s and noindex tags, soft 404s, 401/403 blocks, and "Crawled" or "Discovered - currently not indexed" on pages you care about. Duplicates, alternates and redirects are usually Google doing its job. On large sites each reason lists at most 1,000 example URLs, so you need sitemap filters and URL Inspection sampling to see the full picture.

Below: how the report is laid out, every reason with a verdict, the 1,000-row limit and how to work around it, and a workflow for sites with tens of thousands of URLs.

How the Page indexing report is laid out

The summary page has a chart of indexed and not indexed URLs and two tables (Google's description):

  • Why pages aren't indexed: the reasons that kept URLs out of the index. Click a row for the URLs and the issue's history.
  • Improve page experience: "issues that didn't prevent page indexing, but we recommend that you fix them". Today this holds warnings such as "Indexed, though blocked by robots.txt" and "Page indexed without content".
  • View data about indexed pages: the indexed count over time and up to 1,000 example indexed URLs.

Each reason row has a Source column, "Website" or "Google". Google's rule of thumb: "In general, you can fix only issues where the source is listed as 'Website'." Each row also has a Validation state. Validation is covered in depth in Search Console Validate fix vs Verify fix, so we skip it here.

The dropdown above the chart filters by sitemap:

FilterShows
All known pages (default)Every URL Google knows, in a sitemap or not
All submitted pagesURLs in a sitemap submitted in the Sitemaps report or listed in robots.txt
Unsubmitted pages onlyURLs not in any submitted sitemap
A specific sitemap URLURLs in that one sitemap or sitemap index

The specific-sitemap option is the main way around the 1,000-example cap, covered below.

Google also tells small sites they may not need the report at all: "If your site has fewer than 500 pages, you probably don't need to use this report", and suggests site: searches instead.

Every reason, and whether it needs action

Google's help page defines each reason. The verdict column is ours: what we'd do on a typical site.

ReasonWhat Google meansNeeds action?
Server error (5xx)"Your server returned a 500-level error when the page was requested."Yes, if it's a real page. Check host status in Crawl Stats.
Redirect errorA chain that was too long, a loop, a redirect URL over the max length, or "a bad or empty URL in the redirect chain"Yes, always
URL blocked by robots.txtGooglebot was not allowed to fetch the URLOnly if the block is a mistake
URL marked 'noindex'"it encountered a 'noindex' directive"Only if you want the page indexed
Soft 404A page that says "not found" (or looks empty) but returns 200Yes: return a real 404, or add real content
Blocked due to unauthorized request (401)Googlebot got an authorization requestYes for public pages; fine for private ones
Not found (404)The URL returned 404Only if you link to it or list it in a sitemap
Blocked due to access forbidden (403)"Googlebot never provides credentials, so your server is returning this error incorrectly"Usually yes: a WAF or bot rule
URL blocked due to other 4xx issueAny other 4xxYes, inspect it
Crawled - currently not indexed"The page was crawled by Google but not indexed."Yes for pages that should rank; ignore for thin or filter pages
Discovered - currently not indexedFound but not crawled yet, typically because crawling "was expected to overload the site"Yes if it's a large share of important URLs
Alternate page with proper canonical tagAn alternate (AMP, mobile, desktop) that points to an indexed canonicalNo
Duplicate without user-selected canonicalGoogle picked another URL as canonical and this page has no canonical tagUsually no; add a canonical if Google picked the wrong one
Duplicate, Google chose different canonical than userYou declared this URL canonical, Google prefers anotherCheck it: signals conflict
Page with redirect"a non-canonical URL that redirects to another page"No, unless your sitemap or internal links point at it

Google also lists Blocked by page removal tool and Duplicate, submitted URL not selected as canonical among the states the live test can't check (URL Inspection help), so you may see those on some properties.

We have separate posts on most of these: URL marked noindex, Discovered - currently not indexed, Crawled - currently not indexed on programmatic sites, soft 404, server error 5xx, not found 404, redirect error and page with redirect. This post covers how the reasons fit together; those posts cover the fixes.

The three that look scary and usually aren't

Page with redirect. Every http:// URL that redirects to https://, every old slug you 301'd, every trailing-slash variant ends up here. Google only indexes the target. A large and growing count after a migration is normal. It only needs work when your own sitemap or internal links still point at redirecting URLs.

Duplicate without user-selected canonical. Google calls this "not an error, but is working as intended, because Google does not serve duplicate pages". Parameter URLs (?sort=price, ?utm_source=…) land here. Inspect a few to see which URL Google kept. If it kept the right one, leave it.

Not found (404). Google recommends "fixing only 404 errors that you link to yourself or list in a sitemap", and the report "shows only URLs that have shown 404 errors in the past month". Deleted products with no replacement should stay 404.

The two that are easy to dismiss and often matter

Crawled - currently not indexed on templates that should rank (product, job, article pages) is a quality or duplication signal at template level. A few hundred filter URLs here is fine. Forty percent of your product pages here is not.

Discovered - currently not indexed at scale means Google knows the URLs but hasn't spent crawl on them. Google's crawl budget guide names sites "with a large portion of their total URLs classified by Search Console as Discovered - currently not indexed" as one of the cases it's written for.

The 1,000-example limit

Each reason's details page has a table of example URLs. Google: "The list does not necessarily show all URLs with that issue, and is limited to 1,000 rows." The indexed sample is capped at 1,000 too.

The counts are different. Google's About Search Console data page says most reports cover "a representative sample of URLs", with two exceptions: "the Page Indexing report and the Video pages report, which do provide comprehensive totals (but may still only list up to 1000 URLs)". So the number next to each reason is the full count. Only the URL list is a sample.

On a 300-page site that never bites. On a job board with 40,000 URLs under "Crawled - currently not indexed", the export gives you 1,000 of them and no guarantee they're representative of each template.

There's also no API for the report. The Search Console API has Search Analytics, Sitemaps, Sites and URL Inspection (API reference). Nothing returns the Page indexing table.

Working at scale: three ways past 1,000

1. Split sitemaps by template and filter by each one

Because the sitemap filter accepts "a specific sitemap URL", every sitemap you submit is its own slice of the report with its own 1,000-row example list per reason. Split by template:

https://example.com/sitemap-index.xml
  https://example.com/sitemaps/jobs-1.xml        (job pages, up to 50,000 URLs)
  https://example.com/sitemaps/jobs-2.xml
  https://example.com/sitemaps/companies.xml
  https://example.com/sitemaps/articles.xml
  https://example.com/sitemaps/categories.xml

Submit each child sitemap in the Sitemaps report (or just the index), then pick each one in the filter. Now "Crawled - currently not indexed" reads as "how many job pages, how many company pages", which is what you need to decide where to work. Smaller sitemaps give smaller slices: ten sitemaps of 5,000 URLs give you up to ten 1,000-row samples instead of one.

Google suggests the same trick for validation: "create and submit a sitemap containing only your most important pages, then filter the report by that sitemap before requesting a fix validation", because a validation against a subset "can complete faster".

The filter only sees submitted URLs. URLs Google found by crawling links stay under "Unsubmitted pages only", which is often where parameter and faceted URLs show up.

2. Sample with the URL Inspection API

The URL Inspection API returns, per URL, the same verdict and coverageState string the report uses ("Crawled - currently not indexed" and so on), plus lastCrawlTime, googleCanonical, userCanonical, robotsTxtState, indexingState and pageFetchState (UrlInspectionResult). The limits, as of November 2026, are 2,000 queries per day and 600 per minute per property (usage limits).

2,000 a day won't inspect a 200,000-page site, but 200 URLs per template across ten templates gives you a real indexing rate per template in a day. A minimal Python sampler, using a service account that you've added as a user on the property:

# pip install google-api-python-client google-auth
import collections, csv, random
from google.oauth2 import service_account
from googleapiclient.discovery import build

SITE = "sc-domain:example.com"   # or "https://www.example.com/" (trailing slash required)
creds = service_account.Credentials.from_service_account_file(
    "sa.json", scopes=["https://www.googleapis.com/auth/webmasters.readonly"])
gsc = build("searchconsole", "v1", credentials=creds)

def inspect(url):
    body = {"inspectionUrl": url, "siteUrl": SITE}
    r = gsc.urlInspection().index().inspect(body=body).execute()
    s = r["inspectionResult"]["indexStatusResult"]
    return s.get("coverageState"), s.get("lastCrawlTime"), s.get("googleCanonical")

# urls_by_template: {"/jobs/[slug]": [...], "/companies/[slug]": [...]} from your DB or sitemap
def sample(urls_by_template, per_template=200):
    rows = []
    for template, urls in urls_by_template.items():
        for url in random.sample(urls, min(per_template, len(urls))):
            state, crawled, canonical = inspect(url)
            rows.append((template, url, state, crawled, canonical))
    return rows

rows = sample(urls_by_template)
summary = collections.Counter((t, s) for t, _, s, _, _ in rows)
for (template, state), n in sorted(summary.items()):
    print(f"{template:30} {state:45} {n}")
with open("inspection-sample.csv", "w", newline="") as f:
    csv.writer(f).writerows(rows)

The inspect endpoint reports Google's indexed view only. Google's reference says "you cannot test the indexability of a live URL" with it, so a fix you deployed an hour ago won't show here until Googlebot recrawls.

3. Join it with a crawl of your own

The report tells you Google's reason. It doesn't tell you what the pages have in common. That needs a crawl: word counts, duplicate titles, canonical targets, click depth and internal links per template, matched against Google's verdicts.

This is what the SEOFix Google tab does. It inspects pages from your latest audit within the property's daily budget (2,000 a day on paid plans, 300 per weekly sync on Free), groups not-indexed pages by Google's reason and URL template, and shows the crawl signals most common in each group, such as "71% share a title with another page". Why aren't my pages indexed? then sorts the groups into worth indexing, low value, duplicates and excluded on purpose. Whatever tool you use, put Google's verdict per template next to your crawl data per template.

A workflow for the report on a large site

  1. Start with the indexed trend. A drop in indexed pages "without a corresponding increase in errors" usually means you're blocking existing pages "via robots.txt, 'noindex' or a required login" (Google's wording). Look for the reason that grew as indexed fell.
  2. Filter to "All submitted pages". These are the URLs you asked Google to index. Not-indexed URLs here are disagreements between your sitemap and your pages or Google's judgement.
  3. Sort reasons by Source = Website. Fix 5xx, redirect errors, 401/403, soft 404s and unexpected noindex first. They're mechanical and you control them.
  4. Per template, check the "not indexed" reasons that need judgement. Use per-template sitemaps and an inspection sample to see which templates sit in "Crawled" or "Discovered - currently not indexed".
  5. Leave the working-as-intended reasons alone. Page with redirect, alternates and parameter duplicates. Remove them from sitemaps if they're there.
  6. Fix at template level, verify, then validate. Confirm the fix on the live HTML first, then click Validate fix. Validation "typically takes up to about two weeks".

Common questions

Why is a URL in the report when URL Inspection shows no problem? Google's answer: "You might have fixed the error after the URL was last crawled by Google." Check the crawl date. The live test also skips duplicate and canonical conditions.

Why don't indexed + not indexed match my page count? Google says not to expect totals to match: the report counts every URL Google knows, including old, parameter and redirecting URLs, and "small discrepancies can occur".

Can I search for one URL in the report? No: "You can't search or filter by URL in the Page indexing report." Use URL Inspection.

How long does an issue stay in the table after I fix it? Until 90 days after the last instance is gone. "If ninety days pass without any recurrences, the issue is removed from the issues table."


If you want the template view without building the sampler yourself, SEOFix's first audit is free (up to 500 pages), and connecting Search Console adds Google's verdicts next to the crawl. Setup is in Connect Google Search Console.

ShareXLinkedIn