Let SEOFix through your firewall

To let SEOFixBot past a firewall, add a rule that skips challenges for requests carrying the header X-SEOFix-Verify with your site's crawler secret as the value. The secret is on the site page under Manual firewall rule → Header value. SEOFixBot sends it only for verified sites, so verify the site first. On Cloudflare, Connect Cloudflare can create the rule for you.

#Why allowlist SEOFix

Many firewalls challenge or block automated traffic. SEOFixBot cannot solve a challenge, so a challenged page is never checked. The report marks those pages Blocked by firewall (never as broken pages) and warns when more than 20% of the audit was blocked. See Blocked pages in your report.

Search engine crawlers may be affected by the same rules, so a blocked audit is also worth a look on its own.

#The crawler secret header

The crawler secret is different from the verification token:

• The verification token is public. It sits in your DNS, your homepage tag or your verification file, so anyone can read it. Never build a firewall rule on it.
• The crawler secret is private. It is shown only to members of the site's team in the web app, and never returned by the /v1 API. Don't publish it or put it in your site's code.

SEOFixBot identifies itself with the user agent SEOFixBot/1.0 (+https://seofix.ai/bot). A user agent is easy to fake, so match the header, not the user agent. SEOFix does not publish a list of crawler IP addresses, so an IP-based allowlist is not possible.

#Cloudflare

Use Connect Cloudflare for an automatic rule, or Do it myself (no API token) for the manual steps. Full walkthrough: Connect Cloudflare.

The manual rule, in short:

1. Security → WAF → Custom rules → Create rule, named SEOFix crawler, placed First.

2. Expression:

   any(http.request.headers["x-seofix-verify"][*] eq "<crawler secret>")
   

3. Action Skip, ticking: All remaining custom rules, All managed rules, All Super Bot Fight Mode rules (Pro plan and above), and under "More components to skip": Browser Integrity Check, Security Level, User Agent Blocking, Hotlink Protection, Zone Lockdown.

Alternatively, add and not any(http.request.headers["x-seofix-verify"][*] eq "<crawler secret>") to the rule that issues the challenge.

"All remaining custom rules" on its own skips only your other custom rules. Security Level, Browser Integrity Check, managed rules and Super Bot Fight Mode can still challenge SEOFixBot, which is why the rule ticks them too. Cloudflare's 1015 (rate limited) and 1020 (blocked by a rule) pages also count as blocked.

Leave rate limiting rules in place: SEOFixBot backs off when it gets 429 responses. On the Free plan, Bot Fight Mode cannot be skipped by a rule: turn it off while audits run.

#Other firewalls and CDNs

Sucuri, Akamai, your host's WAF or any other firewall: add a rule that allows (skips challenges for) requests whose X-SEOFix-Verify header equals your crawler secret. The Manual firewall rule panel on the site page has the header name and value. Where to add the rule depends on your provider: look for custom rules, allowlists or bypass rules based on a request header.

SEOFix recognises block and challenge pages from these providers, so their blocks are reported as Blocked by firewall:

A block from a firewall SEOFix does not recognise shows up as an ordinary HTTP error (for example a 403) on those pages.

#Check your rule

On the site page, Cloudflare → Do it myself (no API token) → Check my setup tests the rule. It works for any firewall in front of the site, not only Cloudflare, because it only looks at the answers:

• It loads the site's start URL twice as SEOFixBot, once with the crawler header and once without (10-second timeout).
• It follows up to 3 redirects that stay on the site (same host or its www / bare-domain twin; http → https is allowed). The header is never sent anywhere else.
• You can run it 10 times a minute.

#Rotate the secret

If the secret leaks, choose Rotate secret next to it on the site page and confirm with Rotate. SEOFix issues a new value and the old one stops working.

• With Connect Cloudflare, SEOFix updates the Cloudflare rule in place ("Your Cloudflare rule was updated too"). If Cloudflare refuses the update, nothing changes.
• With a hand-built rule (Cloudflare manual or any other WAF), update the value in your rule right away. Until you do, audits (including one already running) may hit challenges again.

#Plain-URL audits through the API

An audit started with a plain url instead of a registered site can carry a header value you choose:

curl -X POST https://api.seofix.ai/v1/crawls \
  -H "Authorization: Bearer $SEOFIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "verify_token": "my-own-firewall-value-1234"}'

verify_token is 16–128 characters of letters, digits, _ and -. SEOFixBot sends it as X-SEOFix-Verify to that site, and your firewall rule matches it the same way. It does not raise the crawl speed above 3 req/s, and it is not stored with the crawl. Never use your site's verification token here: it is public.

#Related

• Connect Cloudflare
• Blocked pages in your report
• Verify site ownership
• SEOFixBot