# Sitemap couldn't fetch in Search Console: how to find the cause and fix it > Couldn't fetch means Google never got your sitemap file. Test the exact fetch with URL Inspection and curl, then fix the 404, robots.txt rule, firewall or timeout. Source: https://seofix.ai/blog/sitemap-couldnt-fetch Published: 2026-11-12 Updated: 2026-11-02 Author: SEOFix Team Category: Technical SEO Tags: Technical SEO, Guides Length: 1,939 words, 9 min read "Couldn't fetch" in the Search Console Sitemaps report means Google never received the sitemap file. It's a fetch problem, not a content problem: Google didn't get far enough to parse a single URL. The usual causes are a wrong sitemap URL that returns 404, a robots.txt rule that blocks the file, a firewall or login in front of it, or a server that errors or times out while generating it. Run the sitemap URL through URL Inspection's live test, repeat the request with curl, fix whatever answers with something other than a `200` and XML, then resubmit. Google's [Sitemaps report help](https://support.google.com/webmasters/answer/7451001) (checked 2 November 2026) defines the three statuses: - **Success**: "The sitemap was fetched and read without any errors." - **Has errors**: "The sitemap could be fetched, but has one or more errors." - **Couldn't fetch**: "The sitemap could not be fetched for some reason." And it lists the possible reasons for the last one: - "The sitemap is blocked by your robots.txt file" - "Your site has a manual action" - "The URL provided for the sitemap is wrong (HTTP 404 error: sitemap not found)" - "Some other general error", such as the server being unavailable - "There is low crawl demand for the sitemap" Size and format problems (more than 50,000 URLs, over 50 MB uncompressed, invalid dates, bad XML) show up under **Has errors**, because Google fetched the file and then rejected what was in it. If your status is "Couldn't fetch", start with the request itself. ## Step 1: reproduce the fetch Google makes ### URL Inspection live test Google's own troubleshooting steps for this status: copy the sitemap URL, paste it into URL Inspection, click **Live test**, and read the **Page availability** section. You want "Crawl allowed?" = "Yes" and "Page fetch" = "Successful" ([Sitemaps report help](https://support.google.com/webmasters/answer/7451001)). This is the only test that runs from Google's infrastructure with Google's user agent, so it gets past (or fails at) your firewall exactly the way Googlebot does. If the live test fails, the reason it gives (blocked by robots.txt, not found, server error, access forbidden) points to one of the sections below. ### The same request with curl From your own machine, request the sitemap with Googlebot's smartphone user agent ([Google's published string](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers), with a real Chrome version filled in): ```bash UA='Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' # status, content type, size, time, and where redirects ended curl -sS -o /dev/null -L -A "$UA" \ -w '%{http_code} %{content_type} %{size_download}B %{time_total}s -> %{url_effective}\n' \ https://example.com/sitemap.xml # the first bytes: should start with https://example.com/sitemap.xml` followed by XML. Anything else is your lead. Two limits of this test. Your IP isn't Google's, so a firewall that verifies Googlebot by IP will block a spoofed user agent even when it lets the real one through. And a firewall that blocks real Googlebot may let your office IP through. Treat curl as a way to see what your server does with the request, and the live test as the verdict. Also check robots.txt, because the sitemap URL has to be crawlable: ```bash curl -sS https://example.com/robots.txt ``` ## Step 2: fix the cause | What you see | Likely cause | Fix | | --- | --- | --- | | `404` | Wrong URL submitted, or the generator isn't running | Submit the URL that serves the file | | Live test: blocked by robots.txt | A `Disallow` rule matches the sitemap path | Narrow the rule or add an `Allow` | | `403`, `401`, a challenge page | WAF, bot protection, basic auth | Allow verified Googlebot | | `5xx` or a long `time_total` | Sitemap built on every request, DB query too slow | Pre-generate or cache the file | | `200` with `text/html` | SPA fallback or a login page served instead of XML | Route the sitemap path before the catch-all | | Everything passes | Manual action, or low crawl demand | Check Manual actions; wait and resubmit | ### Wrong URL (404) Most often, the submitted URL isn't where the sitemap lives. Compare what you submitted against: - **Scheme and host.** `http://` vs `https://`, `www` vs bare domain. Submit the version your site redirects to, in the matching Search Console property. - **The path your platform uses.** WordPress core serves its sitemap index at `/wp-sitemap.xml`, not `/sitemap.xml`. SEO plugins usually replace it with their own path. Submit whichever one your site serves. - **The site isn't public.** WordPress core turns its sitemaps off when the site's `blog_public` option is off (the "Discourage search engines" checkbox), and the sitemap URLs then return a 404 ([WordPress source](https://github.com/WordPress/wordpress-develop/blob/trunk/src/wp-includes/sitemaps/class-wp-sitemaps.php)). A staging setting copied to production causes this, along with a site-wide `noindex`. - **Generated sitemaps that only exist after a build or cron job.** If a deploy wiped `public/sitemap.xml` and the job that writes it hasn't run, the file is gone. ### robots.txt blocks the sitemap Google applies your robots.txt to the sitemap URL like any other URL. The `Sitemap:` line in robots.txt "isn't tied to any specific user agent and may be followed by all crawlers, provided it isn't disallowed for crawling" ([Google's robots.txt spec](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec)). Rules meant for something else often catch it: ``` User-agent: * Disallow: /*.xml$ # meant for feed exports, also blocks /sitemap.xml Disallow: /sitemap # "old sitemap page", also blocks /sitemap.xml and /sitemap_index.xml Sitemap: https://example.com/sitemap.xml ``` Google uses the most specific matching rule, measured by path length, so a targeted `Allow` beats a broad `Disallow`: ``` User-agent: * Disallow: /*.xml$ Allow: /sitemap.xml Allow: /sitemaps/ Sitemap: https://example.com/sitemap.xml ``` Also check for a separate `User-agent: Googlebot` group. Googlebot only reads the most specific group that matches it and ignores `*` entirely, so a fix added under `*` may not apply. The robots.txt testing post shows how to check a rule against the exact URL. If robots.txt itself answers with a `5xx`, Google stops crawling the site for the first 12 hours and then works from the last good copy for up to 30 days ([robots.txt spec](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec)). That breaks more than the sitemap, so fix it first. ### Firewall, bot protection or a login Bot-fighting features, rate limits, geo blocks and HTTP basic auth on a staging-style setup all answer Googlebot with a `403`, a `401`, a `429` or a JavaScript challenge page. The sitemap is often hit hardest: it's one URL Googlebot requests repeatedly, and it's often served by a different route (a CDN worker, an object storage bucket, a separate service) with different rules than your pages. Look for Googlebot requests to the sitemap path in your server or CDN logs. If they're missing at the origin, the edge blocked them. Before you allowlist anything, confirm a request really came from Google: run `host` on the IP, check the name ends in `googlebot.com` or `google.com`, then run `host` on that name and confirm it returns the same IP. Google also publishes its crawler IP ranges as JSON files ([verifying Google's crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests)). ```bash host 66.249.66.1 # 1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com. host crawl-66-249-66-1.googlebot.com # crawl-66-249-66-1.googlebot.com has address 66.249.66.1 ``` If you're on Cloudflare, our post on [whether Cloudflare is blocking Googlebot](https://seofix.ai/blog/cloudflare-blocking-googlebot) walks through the security event log and the rules that usually cause it. ### Server errors and slow generation A sitemap built on request from a database is fine at 500 URLs and a liability at 200,000. The query runs on every fetch, the response takes many seconds, and under load it returns a `502` or `504` from the proxy in front of it. The curl `time_total` value tells you how close you are. Fixes, roughly in order of effort: 1. Cache the generated XML (a CDN cache rule, a framework cache, or a file written by a scheduled job) and serve the cached copy. 2. Split one huge sitemap into smaller files behind a sitemap index, so each request is cheap. The guide to large XML sitemaps covers the limits and how to split by template. 3. Generate the files at publish time instead of request time. ### HTML instead of XML Single-page apps and some hosting setups answer every unknown path with `index.html` and a `200`. If the sitemap route isn't registered before that fallback, Googlebot gets your app shell. A login redirect does the same thing on protected environments. The `head -c 300` check above catches it: you'll see `` instead of ``. These problems usually show as "Has errors" rather than "Couldn't fetch", but they're worth fixing while you're there. ### Everything passes and the status stays If the live test succeeds and curl gets a fast `200` with valid XML, two causes on Google's list remain: - **A manual action.** Check the Manual actions report in Search Console. - **Low crawl demand.** Google lists this as a reason a sitemap may not be fetched. On a new or small site with few signals, Google may not have requested it yet. Make sure the sitemap is also listed in robots.txt with a `Sitemap:` line, link your important pages internally, and give it time. ## Step 3: resubmit and wait Google says that "if a sitemap fetch fails, Google will retry for a few days, and then stop if the sitemap continues to be unavailable or has critical errors" ([Sitemaps report help](https://support.google.com/webmasters/answer/7451001)). So after a fix, don't count on Google coming back on its own: open the report, submit the same URL again, and watch the **Last read** column. It shows "the last time the sitemap was fetched by Google." Submitting is also only a hint. Google says it "doesn't guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site" ([Build and submit a sitemap](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap)). A "Success" status means the file was read, not that every URL in it will be crawled or indexed. ### Sitemap indexes If you submitted a sitemap index, the index URL can succeed while a child sitemap fails. The report adds the children's URLs into the index's **Discovered pages** count, so a number lower than you expect is a hint; run the curl checks above against each child URL. Child sitemaps "must be in the same directory as the sitemap index file, or lower in the site hierarchy", and on the same site unless you've set up cross-site submission ([Google's sitemap index docs](https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps)). A child on a CDN hostname or a storage bucket breaks that rule. ## Where SEOFix fits SEOFix doesn't see your Search Console sitemap status, but it fetches the same file. Every audit reads `/sitemap.xml` at the root of the site and follows a sitemap index to its child sitemaps, gzip included ([site structure and sitemap checks](https://seofix.ai/help/checks-structure)). When a child sitemap fails or can't be parsed, the report treats the sitemap as incomplete and skips the "not in sitemap" checks rather than guessing. For the sitemap it does read, it flags listed URLs that redirect or error (`SITEMAP_URL_NOT_200`), noindex pages in the sitemap and dates that can't be trusted. It also reports a robots.txt that answers `5xx` or doesn't answer at all as `ROBOTS_TXT_UNREACHABLE` ([status and access checks](https://seofix.ai/help/checks-status-and-access)). One limit: a sitemap declared only in robots.txt under another path isn't read. ## FAQ **My sitemap opens fine in the browser. Why can't Google fetch it?** Your browser isn't Googlebot. Firewalls, bot protection and geo rules treat the two differently, and you may be logged in. Use URL Inspection's live test, which fetches from Google. **Should I delete and resubmit the sitemap?** Resubmitting the same URL after a fix is enough. There's nothing to gain from deleting it first. **Does "Couldn't fetch" hurt rankings?** Not directly. It means Google can't use the sitemap to discover or recrawl URLs, so pages that are well linked internally are barely affected, while new or rarely linked pages get found later. **Can the sitemap be gzipped?** Yes, the [sitemaps protocol](https://www.sitemaps.org/protocol.html) allows gzip. The 50 MB limit applies to the uncompressed size. --- Check what's inside the sitemap once Google can read it: SEOFix flags sitemap URLs that redirect, 404, carry noindex or point their canonical elsewhere. The first audit is free (up to 500 pages).