English Deutsch Français Italiano Español Português 繁體中文 Bahasa Indonesia Tiếng Việt ภาษาไทย
All categories

If I want to prevent an entire subdomained from being indexed by the search engines, but I want to make sure the main domain is indexed (that is, I want http://subdomain1.foo.com out, but http://www.foo.com in), then is the proper lines in my robots.txt file:


User-agent: *
Disallow: http://subdomain1.foo.com/



Thanks

2007-02-13 10:43:51 · 2 answers · asked by Timmbbo 1 in Computers & Internet Internet

2 answers

In the head part of your main domain you should have this tag:

In the head of your subdomain place this tag:

That will stop them.
Regards, Zyfert http://xsitepro.cybersprout.com

2007-02-13 11:02:45 · answer #1 · answered by Zyfert 7 · 0 0

Disallow
The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved. For example, Disallow: /help disallows both /help.html and /help/index.html, whereas Disallow: /help/ would disallow /help/index.html but allow /help.html.

Any empty value, indicates that all URLs can be retrieved. At least one Disallow field needs to be present in a record.

2007-02-13 19:04:08 · answer #2 · answered by Anonymous · 0 0

fedest.com, questions and answers