Phil, > - it even obeys robots.txt
Well, of course, Asimov immediately came to mind when I saw "obey" and "robot", you know: <quote> One, a robot may not injure a human being, or, through inaction, allow a human being to come to harm. Two, a robot must obey the orders given to it by human beings except where such orders would conflict with the First Law. Three, a robot must protect its own existence as long as such protection does not conflict with the First and Second Laws. </quote> But then I checked the "crawler" web page to find the prosaic explanation. :-( Chris Mason ----- Original Message ----- From: "Phil Payne" <[EMAIL PROTECTED]> Newsgroups: bit.listserv.ibm-main To: <[email protected]> Sent: Thursday, 04 May, 2006 5:10 PM Subject: IBM and Google / Six Apart / Yahoo / whomever > IBM already has search technology which it uses internally - you never see a hit from an IBM > machine other than the Almaden spider. It's a very polite little IBM bot - it even obeys > robots.txt: > > 66.147.154.3 - - [03/Apr/2006:01:39:13 +0100] "GET /robots.txt HTTP/1.0" 200 1911 "-" > "http://www.almaden.ibm.com/cs/crawler [fc14]" > > -- > Phil Payne > http://www.isham-research.co.uk > +44 7833 654 800 ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
