On Sat, 8 Dec 2001, Mark Hedges wrote:
> > I checked in the FAQ and other sites but found no information about > whether most robots will follow a redirection to another page. We > only give shared hosting customers 1 IP address and they use redirect > scripts to direct their viewers into subdirectories for different > domains. I am trying to find out if most robots will follow the > redirect script into the subdirectory. Thanks for any info you have > to share. I think that most robots will follow an HTTP status 301 or 302 redirect; however I suspect that what you are referring to is an HTML META refresh directive or javascript function. I believe that my (somewhat defunct) robot would follow a META link; it would certainly follow the usual "if that didn't work, click here" link, adding one hop from the root document in the process (whereas an HTTP redirect would not incur a hop) - since it is not a global robot, it is restricted to N hops from an origin. It would ignore any javascript. I have read that certain robots will refuse to index a page that has a META refresh redirect, since it is in effect presenting one set of content to the robot and another to a human (whose browser jumps immediately to the next page). I am not sure if this would also affect the harvesting of links. In theory it should not, but in practice it may. -- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 [EMAIL PROTECTED] -- This message was sent by the Internet robots and spiders discussion list ([EMAIL PROTECTED]). For list server commands, send "help" in the body of a message to "[EMAIL PROTECTED]".
