On Fri, 18 Oct 2002, Ron Woodall wrote: > Hi All: > > Another far-fetched question: > > Is there anyway that a search engine spyder can be identified with analog? > I tried pulling out robots.txt but I get 600+ redirects on it but only two > hits on it, one from me? (go figure) Is there some sort of tell-tale > behaviour? Don't say that any host that retrieves a lot of files, that > could be any proxy server. I have some that swallow hundreds of files at a > time. >
There is nothing fully reliable. Looking at the user agent, and comparing it to a comprehensive list, is usually as good as anything. -- Stephen Turner, Cambridge, UK http://homepage.ntlworld.com/adelie/stephen/ "Reserve your 2 hour delivery time, which means you'll see the same special offers as you would instore" - Sainsbury's internet shopping instructions +------------------------------------------------------------------------ | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at | http://www.mail-archive.com/analog-help@;lists.isite.net/ | http://lists.isite.net/listgate/analog-help/archives/ | http://www.tallylist.com/archives/index.cfm/mlist.7 +------------------------------------------------------------------------
