SEO

Crawl Budget

How many pages a search engine will crawl on your site in a given window; it mostly matters once a site gets large.

Definition

Crawl budget is the number of pages a search engine is willing and able to crawl on your site over a period of time. It's a function of how often the engine wants to crawl you and how much crawling your server can handle without slowing down.

In depth

Search engines don't crawl every page every day. They allocate a rough budget based on your site's size, health, and how fresh and important your content seems. When a site has thousands of low-value or duplicate URLs, that budget gets wasted on pages that don't help you, which can slow indexing of the ones that should rank.

For most residential contractors with a few dozen pages, crawl budget isn't a real concern; everything you have gets crawled comfortably. It starts to matter for bigger sites with large project galleries, lots of location pages, or filter URLs that multiply into thousands of near-identical addresses. A clean XML sitemap and solid technical SEO keep that footprint manageable.

The mistake is letting junk URLs pile up, such as endless filter combinations, session parameters, or duplicate galleries, and leaving them all crawlable. We keep crawlable URLs limited to the pages that should rank, block the noise, and fix slow pages so the server can handle crawling, then watch the results in Google Search Console to keep attention on the pages that earn business.

Worked example

Example

A builder's site spawned thousands of filtered gallery URLs; we blocked the filter parameters so crawlers spend their budget on real service and project pages instead.

SEO

Want this run for you, not just read about?

Own the searches your buyers make right before they act, and compound the traffic over time.