I believe Google obeys the robots.txt rule http://www.robotstxt.org/wc/norobots.html
Peter On Mon, 2006-03-20 at 15:20 +0300, Ronny wrote: > Hi all,just a quick question how can one kick out google robots from > crawling some prohibited networks not sites but still access all google > tools am not anti-google ( I use it every hour) but just got a > revelation ;-) > Thanks > Ronny > ----------------------- Random verse of the day: And who, as I, shall call, and shall declare it, and set it in order for me, since I appointed the ancient people? and the things that are coming, and shall come, let them shew unto them. -- Isaiah 44:7 _______________________________________________ LUG mailing list [email protected] http://kym.net/mailman/listinfo/lug %LUG is generously hosted by INFOCOM http://www.infocom.co.ug/ The above comments and data are owned by whoever posted them (including attachments if any). The List's Host is not responsible for them in any way. ---------------------------------------
