How do Search Engines Work - Web Crawlers

Finally, a search engine to your site to the attention of customers' expectations. It is preferable to note that these search engines work and how to provide customer information to search.

There are two main types of search engines. Before the mechanism known as caterpillars or spiders.

Search engines use spiders to index websites. When you submit your website pages to the search engine by completing their required submission page, the spider search engines to index your entire site. Spider is a program managed by the automated search engine. Spiders visit site to read the content in the actual meta tags and site links between sites. Then return all of the spider back to a central repository of information to index. It will be a link to your website and index those sites as well. Some spiders will only index number of pages on your site to create a site that is not 500 pages!


Spider periodically return to the site to verify information with any changes. The frequency with which this occurs is determined by the administrator of the search engine.

Spider is almost like a book index to the actual content and links and references to all the sites found during the search and index in May is one million per day.