The meta robots tag is a line of code found in the head element of the page that specifies how search engines and other crawlers should treat the page content.

A typical meta robots tag:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

There are several values that could be included in the content attribute, but the most common are index, noindex, follow, and nofollow. Multiple values can be used, comma-separated.

  • index: Instructs search engines to index the page content. This is the default value, so it is not actually needed to be written explicitly.
  • noindex: Instructs search engines not to include the page in the search engine's index.
  • follow: Instructs search engines to follow all links on the page as normal. Again, this is the default behavior, so is not really needed.
  • nofollow: Instructs search engines not to follow the links on the page.
Did this answer your question?