search engine Apache Systems 

Search engines like Google, MSN, Netscape, Excite, Lycos, AOL, use spiders or robots who go from link to link. The search engine sends out these spiders on a regular basis and collects all information on the pages and puts it in the search engine memory. Then the robots/spiders will search for links and will jump to those sites and collect information. Thus the robots go all around the worlds servers and have all the info. So when you search the search engine scans over these known pages.

Suppose a site is down and your browser cannot find it. There is a way to get to that site. Suppose you were searching on Google. Then look at the displayed results and under them you will see cached.

Click on Cached and that page will be displayed right from Googles Memory. So Google keeps a picture of all the pages it robots /spiders scan.

Yahoo uses Google as search engine. Yahoo by itself is just a directory. That means it has no spiders.

In Yahoo and DMOZ people just add websites. Only those stored websites are statically displayed.

Want to do a search on Honda CRV transmission oil.  You will get more specific results if you WRITE , Honda CRV + transmission Oil.