By default, our crawler does not execute any Javascript, so any content rendered with Javascript will be not be included in site crawls.

Our new crawler uses a browser to crawl your site and will render Javascript and CSS just like a normal visitor would. The new Javascript crawler uses Chromium (the open source version of Google Chrome), which is the same as Googlebot uses when it crawls your site. This crawler tries to emulate how Google crawls your site as closely as possible.

Javascript crawling is currently in a private beta, so access is available through invite only. To request access, please get in touch with our support team via live chat.

As this option becomes available to more customers, more detailed documentation will be found here. In the meantime, please direct any questions about Javascript crawling to our support team who would be happy to assist.

Did this answer your question?