Dragon Metrics' Site Auditor will crawl your website and diagnose SEO issues. By default, our crawler (Dragonbot) makes requests at a moderate and respectful rate.

If you have a large site and a powerful server, you may want to crawl at a faster rate to finish the crawl in shorter amount of time. If you have a more sensitive server that may return HTTP 5xx errors for crawlers that make requests faster than a certain rate, you may want to adjust the crawl rate slower to avoid these blocked requests.

There are two places where you can adjust crawl speed:

Campaign Settings

Navigate to Crawler Settings under Campaign Settings in the left navigation

Site Auditor

Navigate to the Site Auditor under Onsite in the left navigation

Click Crawl options at the top of the page

Either way, it will bring you to the Crawler Campaign Settings page.

Crawler Campaign Setting

Click the dropdown next to Speed and choose your desired rate.

Click Save changes

The changes will take effect next crawl, but you can always initiate a manual crawl immediately.

Did this answer your question?