http://www.robotstxt.org/
Google obeys, but other spiders may not. 2008/10/8 <[EMAIL PROTECTED]>: > 66.249.71.108 - - [08/Oct/2008:12:50:49 +1300] "GET > /ben/set002/pages/IMG_4491.html HTTP/1.1" 200 1515 "-" "Mozilla/5.0 > (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" > > Google and other search engines are running through my home web server and > now using up my data cap a bit fast. > > What's the best way to make apache look at the refer(?) and send it packing? Reject the IP # in your firewall. -- Sincerely etc. Christopher Sawtell
