I was told I should use a web spider on my web site, but I don't know what that is. Can someone help me out, without making me feel stupid? Thank you!
2007-02-20 01:13:17 · 2 answers · asked by Anonymous in Computers & Internet ➔ Internet
Web spiders or web crawlers are bots or automated web services popularized by Google. They visit the sites that are listed or added on your site and then proceed to visit and enlist the sites that have links from the original set of sites on your site.
Thus, the scope of your database of websites increases without you having to enlist them manually.
2007-02-20 01:20:11 · answer #1 · answered by Amlan 2 · 0⤊ 0⤋
A web spider or crawler is is a tool used by search companies like Google / Yahoo / MSN etc to search out pages on the internet to index.
So when a spider hits your site, it attempts to hit every link and that is within your site and pick out what is important and relay that back to the indexer to make you searchable.
2007-02-20 01:19:53 · answer #2 · answered by Pitchy 5 · 0⤊ 0⤋