# Let SEOFix through your firewall

> Allowlist SEOFixBot in Cloudflare or any other WAF with the private X-SEOFix-Verify header, check that the rule works, and rotate the secret.

Source: https://seofix.ai/help/firewall-allowlisting · Category: Sites & firewalls · Updated: 2026-10-08

To let SEOFixBot past a firewall, add a rule that skips challenges for requests carrying the header `X-SEOFix-Verify` with your site's **crawler secret** as the value. The secret is on the site page under **Manual firewall rule → Header value**. SEOFixBot sends it only for verified sites, so verify the site first. On Cloudflare, [Connect Cloudflare](https://seofix.ai/help/connect-cloudflare.md) can create the rule for you.

## Why allowlist SEOFix

Many firewalls challenge or block automated traffic. SEOFixBot cannot solve a challenge, so a challenged page is never checked. The report marks those pages **Blocked by firewall** (never as broken pages) and warns when more than 20% of the audit was blocked. See [Blocked pages in your report](https://seofix.ai/help/blocked-pages.md).

Search engine crawlers may be affected by the same rules, so a blocked audit is also worth a look on its own.

## The crawler secret header

| | |
|---|---|
| Header name | `X-SEOFix-Verify` |
| Header value | Your site's crawler secret (a random 43-character value) |
| Where to find it | Site page → **Manual firewall rule** → **Header value** (choose the eye icon to reveal it, or the copy button) |
| When SEOFixBot sends it | Only for a verified site, only over HTTPS, only to the site's own host and its `www` / bare-domain twin |

The crawler secret is different from the verification token:

- The **verification token** is public. It sits in your DNS, your homepage tag or your verification file, so anyone can read it. Never build a firewall rule on it.
- The **crawler secret** is private. It is shown only to members of the site's team in the web app, and never returned by the `/v1` API. Don't publish it or put it in your site's code.

SEOFixBot identifies itself with the user agent `SEOFixBot/1.0 (+https://seofix.ai/bot)`. A user agent is easy to fake, so match the header, not the user agent. SEOFix does not publish a list of crawler IP addresses, so an IP-based allowlist is not possible.

## Cloudflare

Use **Connect Cloudflare** for an automatic rule, or **Do it myself (no API token)** for the manual steps. Full walkthrough: [Connect Cloudflare](https://seofix.ai/help/connect-cloudflare.md).

The manual rule, in short:

1. Security → WAF → Custom rules → **Create rule**, named **SEOFix crawler**, placed **First**.
2. Expression:

   ```text
   any(http.request.headers["x-seofix-verify"][*] eq "<crawler secret>")
   ```

3. Action **Skip**, ticking: All remaining custom rules, All managed rules, All Super Bot Fight Mode rules (Pro plan and above), and under "More components to skip": Browser Integrity Check, Security Level, User Agent Blocking, Hotlink Protection, Zone Lockdown.

Alternatively, add `and not any(http.request.headers["x-seofix-verify"][*] eq "<crawler secret>")` to the rule that issues the challenge.

"All remaining custom rules" on its own skips only your other custom rules. Security Level, Browser Integrity Check, managed rules and Super Bot Fight Mode can still challenge SEOFixBot, which is why the rule ticks them too. Cloudflare's 1015 (rate limited) and 1020 (blocked by a rule) pages also count as blocked.

Leave rate limiting rules in place: SEOFixBot backs off when it gets `429` responses. On the Free plan, Bot Fight Mode cannot be skipped by a rule: turn it off while audits run.

## Other firewalls and CDNs

Sucuri, Akamai, your host's WAF or any other firewall: add a rule that allows (skips challenges for) requests whose `X-SEOFix-Verify` header equals your crawler secret. The **Manual firewall rule** panel on the site page has the header name and value. Where to add the rule depends on your provider: look for custom rules, allowlists or bypass rules based on a request header.

SEOFix recognises block and challenge pages from these providers, so their blocks are reported as **Blocked by firewall**:

| Provider | Detected from (on a 403, 429 or 503 answer) |
|---|---|
| Cloudflare | The `cf-mitigated` header, or `Server: cloudflare` with a challenge or block page |
| Sucuri | The `x-sucuri-id` or `x-sucuri-block` header |
| DataDome | The `x-datadome` header |
| Akamai | `Server: AkamaiGHost` with an "Access Denied" page |

A block from a firewall SEOFix does not recognise shows up as an ordinary HTTP error (for example a 403) on those pages.

## Check your rule

On the site page, **Cloudflare → Do it myself (no API token) → Check my setup** tests the rule. It works for any firewall in front of the site, not only Cloudflare, because it only looks at the answers:

- It loads the site's start URL twice as SEOFixBot, once with the crawler header and once without (10-second timeout).
- It follows up to 3 redirects that stay on the site (same host or its `www` / bare-domain twin; `http` → `https` is allowed). The header is never sent anywhere else.
- You can run it 10 times a minute.

| Result | Meaning | What to do |
|---|---|---|
| Passed ("Your rule works") | With the header the page loads; without it SEOFixBot is challenged or refused | Nothing |
| Note: nothing blocks SEOFixBot right now | Both requests got through | Keep the rule: it matters if you tighten security later |
| Needs attention: still challenged | The request with the header is challenged | Check the expression was pasted exactly, the action is Skip with every box ticked, and the rule is first and deployed. On Cloudflare Free, turn off Bot Fight Mode. |
| Needs attention: rate limited | A `429` | Wait a minute and retry. Rate limiting is left on purpose. |
| Needs attention: refused with HTTP (code) | Refused, but not by a recognised challenge | Check your server or any other firewall in front of it |
| Needs attention: couldn't reach your site | Network error, or the start URL answered an error such as 404 or 5xx | Make sure the start URL is online and public |
| Needs attention: redirects off the site | The start URL redirects to another site, another port, from `https` to `http`, or more than 3 times | Set the site's start URL to the final address |
| Note: comparison request failed | The header request passed but the plain one failed for another reason | Try again in a minute |

## Rotate the secret

If the secret leaks, choose **Rotate secret** next to it on the site page and confirm with **Rotate**. SEOFix issues a new value and the old one stops working.

- With **Connect Cloudflare**, SEOFix updates the Cloudflare rule in place ("Your Cloudflare rule was updated too"). If Cloudflare refuses the update, nothing changes.
- With a hand-built rule (Cloudflare manual or any other WAF), update the value in your rule right away. Until you do, audits (including one already running) may hit challenges again.

## Plain-URL audits through the API

An audit started with a plain `url` instead of a registered site can carry a header value you choose:

```bash
curl -X POST https://api.seofix.ai/v1/crawls \
  -H "Authorization: Bearer $SEOFIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "verify_token": "my-own-firewall-value-1234"}'
```

`verify_token` is 16–128 characters of letters, digits, `_` and `-`. SEOFixBot sends it as `X-SEOFix-Verify` to that site, and your firewall rule matches it the same way. It does not raise the crawl speed above 3 req/s, and it is not stored with the crawl. Never use your site's verification token here: it is public.

## Related

- [Connect Cloudflare](https://seofix.ai/help/connect-cloudflare.md)
- [Blocked pages in your report](https://seofix.ai/help/blocked-pages.md)
- [Verify site ownership](https://seofix.ai/help/verifying-site-ownership.md)
- [SEOFixBot](https://seofix.ai/help/seofixbot.md)