Googlebot is a search bot used by Google. It collects documents from the web to build a searchable index for the Google search engine. If a webmaster wishes to restrict the information on their site available to a Googlebot, or other well-behaved spider, they can do so by with the appropriate directives in a robots.txt file.
