Dragon Metrics' Site Auditor will crawl your website and diagnose SEO issues. By default, our crawler (Dragonbot) makes requests at a moderate and respectful rate of 1.3 URLs / second.

If you have a large site and a powerful server, you may want to crawl at a faster rate to finish the crawl in shorter amount of time. If you have a more sensitive server that may return HTTP 5xx errors for crawlers that make requests faster than a certain rate, you may want to adjust the crawl rate slower to avoid these blocked requests.

How to change crawl speed

To get started, navigate to Campaign Settings > Crawler in the bottom left navigation.

Scroll down to the option for Crawl Speed and select your desired speed.

Choosing a speed higher than 5 URLs / second will require verification that you own the site. Please use caution when choosing high speeds, so not to overload your server.

Click Save Changes when finished.

The changes will take effect next crawl, but you can always initiate a manual crawl immediately.

Did this answer your question?