|
|
» Marketing Terms |
| |
A program used by search engines to "crawl" the web by following links from page to page.
It makes copies of the web pages found and stores these in the search engine's index.
For example, Googlebot is the crawler that travels
the web finding and indexing pages for the Google search engine.
This is how most search engines "find" the web pages that they place in their index.
Crawlers are also known as bot or spiders.
Back to list of marketing terms |
|
|
|