-
From: Steve Vernon [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Tuesday, November 19, 2002 11:18 PM
Subject: Re: [PHP] catching a robot or spider - asking too
From what I think I heard on another group was that it dosent matter what
stylesheet you use, it will just read the file, getting words
User agent isn't very useful. Most web spiders can be set to
return what ever user agent needed. You are depending on the
writers/users of the spiders to follow commonly acceptable practices ie
looking for robots.txt and the like.
On Tue, 19 Nov 2002 14:54:49 +0100
Wilbert Enserink [EMAIL
From what I think I heard on another group was that it dosent matter what
stylesheet you use, it will just read the file, getting words and following
links. Cant remember where I read this, think it was a forum, and they were
argueing about robots and whether they follow links specified in
At 23:08 19.11.2002, Vincent Vandemeulebrouck said:
[snip]
does anybody know a way to make a distinction
between robots and users?
should I use the user agent? Or is this not a safe method.
If the visitor is a spider/robot I want to include some script
4 matches
Mail list logo
