A robot is another common name for a web crawler. The robots.txt file is named as such since it addresses excluded areas of the site for robots / spiders / crawlers.

Did this answer your question?