How can I tell search engines and crawlers to stop checking my website at a certain page?

Answer 1

create a robots.txt in the root directory of your website

in the txt file put

User-agent: *
Disallow: *
Disallow: /*

that combined with



being put into the root index.html page should stop engines that care to follow the rules

Answer 2

look up robot.txt files.

You can also put a meta tag in your header "robots no index, no follow" - coulpled with a robot.txt file you should be fine.

Answer 3

http://www.uab.edu/it/question-answer/google/robots.php