# Crawl settings

> Every setting you can change for a site or a single audit, such as page limit, speed, monitoring and alerts, with defaults, allowed ranges and plan rules.

Source: https://seofix.ai/help/crawl-settings · Category: Audits & crawling · Updated: 2026-10-08

Each site has four settings: Page limit, Speed (req/s), Scheduled audits and Email alerts. Change them on the site's page, in "Audit settings" and "Monitoring". They apply to "Run audit now" and to scheduled audits. Agents can change them with `PATCH /v1/sites/{id}` or the MCP tool `set_monitoring`, and can override page limit and speed for one run.

## Site settings

| Setting (UI label) | API field | Default | Allowed | Rules |
|---|---|---|---|---|
| Page limit | `max_pages` | 500 on the Free plan, 10,000 on Starter and Growth | 1 up to your plan's limit | Above 500 needs a verified site. |
| Speed (req/s) | `max_rps` | 2 | 0.5 to 10 | Runs at 3 at most until the site is verified. |
| Scheduled audits + Frequency | `monitoring` | `off` | `off`, `weekly`, `daily` | Paid plans (Starter, Growth) and a verified site. |
| Email alerts | `alert_email` | on (`true`) | on or off | Used by monitoring alerts. |

Plan limits for the page limit:

| Plan | Largest page limit per audit |
|---|---|
| Free | 500 |
| Starter | 10,000 to 500,000: a quarter of your page tier |
| Growth | 250,000 to 500,000: a quarter of your page tier |

A quarter of the monthly tier means the tier always covers a weekly audit of the whole site. For example, Starter with 150,000 pages a month audits up to 37,500 pages at a time. The full table is in [Plans and credits](https://seofix.ai/help/plans-and-credits.md#pages-per-audit).

### Page limit

The most pages one audit of the site may crawl. When an audit starts, SEOFix reserves this many credits and refunds whatever the audit did not use.

- Setting it above your plan's limit is refused with `403 plan_limit_max_pages` ("Above your plan's page limit"); the error message states your limit.
- On a site that is not verified, audits stop at 500 pages, even if the saved limit is higher. Starting a single run above 500 pages on such a site is refused with `409 site_not_verified`.
- If you move to a smaller plan or page tier, the saved limit is kept, and an audit above the new limit stops at the new limit.

### Speed (req/s)

The highest request rate per host, in requests per second. The crawler also slows down by itself when your site slows down or returns errors, see /help/how-the-crawler-works.

- Up to 3 req/s works for any site.
- Above 3 req/s needs a verified site. Until then the field hint says "Capped at 3 until verified" and audits run at 3.
- If the site later loses verification (the daily ownership re-check fails), audits drop back to 3.

Raise it only if your server can take it. On many sites, especially JavaScript frameworks with server rendering, each page view triggers further backend calls.

### Scheduled audits and Email alerts

See /help/monitoring-and-alerts.

## In the app

1. Open Sites and pick the site.
2. In "Audit settings", change Page limit or Speed (req/s), then press "Save settings".
3. In "Monitoring", switch "Scheduled audits" on or off, pick a Frequency (Weekly or Daily), and switch "Email alerts".

## Over the API and MCP

Change a site's settings:

```bash
curl -X PATCH https://api.seofix.ai/v1/sites/42 \
  -H "Authorization: Bearer $SEOFIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"max_pages": 5000, "max_rps": 4, "monitoring": "weekly", "alert_email": true}'
```

MCP: `set_monitoring` with `site_id`, `monitoring` and optional `max_pages`, `max_rps`, `alert_email`.

Override page limit and speed for one run of a site, without saving them:

```bash
curl -X POST https://api.seofix.ai/v1/sites/42/crawls \
  -H "Authorization: Bearer $SEOFIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"max_pages": 200, "max_rps": 1}'
```

MCP: `start_site_audit` with `max_pages` and `max_rps`. The speed of one run can't exceed the site's saved Speed.

### Audits of a URL without a site

`POST /v1/crawls` (MCP `start_audit`) crawls any public URL. It accepts:

| Field | Default | Allowed | Notes |
|---|---|---|---|
| `url` | | public `http`/`https` URL | Required unless `site_id` is given. |
| `site_id` | | a site of your team | Uses the site's settings; `url`, if given, must be on the site's origin. |
| `max_pages` | 10,000, capped by your plan and at 500 without a verified site for that origin | 1 to your plan's limit | Above 500 without a verified site: `409 site_not_verified`. |
| `max_rps` | 2 | 0.5 to 10 | Above 3 needs your team's verified site with the same scheme, host and port, else `422 max_rps_requires_verified_site`. |
| `concurrency` | 2 | 1 to 20 | Requests in flight. The speed limit still applies. |
| `webhook_url` | none | `http`/`https` URL | Called when the audit ends. |
| `verify_token` | none | 16–128 characters `A-Z a-z 0-9 _ -` | Sent to your site as the `X-SEOFix-Verify` header so your firewall can recognise SEOFix. It does not raise the speed limit. Not allowed with `site_id`. |

## Settings you can't change

- The user agent (`SEOFixBot/1.0 (+https://seofix.ai/bot)`) and robots.txt handling.
- Crawl depth (25 links), request timeout (15 seconds) and the 5 MB response cap.
- Previews (50 pages, 2 req/s) and the free audit (500 pages, 2 req/s).
- Concurrency for site audits started from the app (2).

## Related

- /help/how-the-crawler-works
- /help/monitoring-and-alerts
- /help/plans-and-credits
- /help/verifying-site-ownership