What is crawl scope?
The scope of a crawl determines which URLs are considered part of the site. For example, the scope can be set to
Root domain (example.com)
Subdomain and Subfolder (www.example.com/en/uk)
The scope is determined by the site field in General Campaign Settings the first time a campaign is created. By default, rank tracking, site audits, and backlink reports all share this same scope.
However, there may be times when you may want to use a different scope for site audits than for rank tracking or backlink reports. For example, you may want to include URLs from all subdomains in rank tracking (root domain scope) but only crawl the www subdomain (subdomain scope) for site audits.
To do this, you can set a separate scope for crawls.
How to update crawl scope
Navigate to Crawler Settings under Campaign Settings in the left navigation
By default, you'll see the crawl scope is set to Same as main site, and the value will match what is set under General Campaign Settings.
To change the scope without affecting rank tracking or backlink reports, change Scope to Use other scope
This will unlock the text and dropdown field. Now you can select the scope to instruct how our crawler should crawl your site.
There are 4 match types of Site Scope you can set up for a crawl:
Includes all URLs on this root domain
Subdomain / subfolder
Only include URLs on the entered subdomain and/or subfolder
Only include the entered URL
Root domain only
Only include URLs on this root domain that have no subdomain
As a reminder, you may exclude the scheme when entering as site, as both http and https will be included.
Track all URLs on a root domain: Include only the root domain (example.com)
Track all URLs on a single subdomain: Include the subdomain + root domain (www.example.com)
Track all URLs in a subfolder on any subdomain: Include the root domain + path (example.com/en/uk/)
Track only URLs in a single subdomain on a single subfolder: Include the subdomain + root domain + path (www.example.com/products)
Track a single URL: Include only the URL you want to track (www.example.com/products/19312981.html)
When you're finished, save your changes.
The changes will take effect next crawl, but you can always initiate a manual crawl immediately.