English Deutsch Français Italiano Español Português 繁體中文 Bahasa Indonesia Tiếng Việt ภาษาไทย
All categories

3 answers

This is a script to tell search engine bots from google and yahoo, etc, to either check your site. or provide specific information about them

See http://en.wikipedia.org/wiki/Robots.txt for full information.

2006-11-08 05:56:24 · answer #1 · answered by fr0z3nph03n1x 2 · 1 0

to help your site get listed on search engines and to stop ceratin web/meta crawllers from indexing certain pages on your website.

Its just best to use htaccess.

2006-11-08 13:58:29 · answer #2 · answered by arus.geo 7 · 0 0

to more accurately list your page on search engines

2006-11-08 13:56:37 · answer #3 · answered by cool_clearwater 6 · 0 0

fedest.com, questions and answers