How do search engines work
All search engines consist of three main parts:
- The Spider (or worm)
- The Index
- The Search Algorithm
The spider (or worm), continuously ?crawls? web space, following links that leads either to different website or with in the limits of the website. The spider ?reads? all pages content and passes the data to the index.
The index is the next step of search engine after crawling. Index is a storage area for spidered web pages and is of a huge magnitude. Google index is said to consist of more than three billion pages.
For example Google?s index, is said to consist of more than three billion pages.
Search algorithm is more sophisticated and third step of a search engine system. Search algorithm is very complicated mechanism that sorts an immense database within a few seconds and produces the results list. The most relevant the search engine sees the web page the nearer the top of the list. So site owners or webmasters should therefore see site?s relevancy to keywords.
Algorithm is unique for each and every search engine, and is a trade secret, kept hidden from the public.
Most of the modern web search combines the two systems to produce their results