Dragon Metrics' Site Auditor will crawl your website and diagnose SEO issues. By default, our crawler (Dragonbot) makes requests at a moderate and respectful rate.

If you have a large site and a powerful server, you may want to crawl at a faster rate to finish the crawl in shorter amount of time. If you have a more sensitive server that may return HTTP 5xx errors for crawlers that make requests faster than a certain rate, you may want to adjust the crawl rate slower to avoid these blocked requests.

There are two places where you can adjust crawl speed:

Campaign Settings

Navigate to Crawler Settings under Campaign Settings in the left navigation

Click the dropdown next to Speed and choose your desired rate.

Click Save changes

The changes will take effect next crawl, but you can always initiate a manual crawl immediately.

Site Auditor

Navigate to the Site Auditor under Onsite in the left navigation

Click Crawl options at the top of the page

Click the dropdown next to Crawl speed and choose your desired rate.

Click OK to save changes

The changes will take effect next crawl, but you can always initiate a manual crawl immediately.

Did this answer your question?