Tier 1 · SEO
Crawl Budget
Definition
Crawl Budget
Crawl budget is the number of URLs a crawler will fetch from a site in a period. Large catalogues waste it on faceted and parameterised duplicates more often than on genuine pages.
Crawl budget is the number of URLs a crawler will fetch from a site in a period. Large catalogues waste it on faceted and parameterised duplicates more often than on genuine pages.