This is (was) a patch to disregard robot.txt files when indexing
and many people disapprove its use. It not nice to index a site
when the site administrator doesn't want you to.
On Thu, 15 Mar 2001, Gabriele Bartolini wrote:
> Hi guys,
(snip)
> >Failure on 'RETR 3.1.5/0ld/Server.cc.0' command
> >Failed to get 3.1.5/0ld/Server.cc.0: 550 3.1.5/0ld/Server.cc.0: Permission
> >denied.
> >Failed to get file 550 3.1.5/0ld/Server.cc.0: Permission denied.
--jesse
--------------------------------------------------------------------
J. op den Brouw Johanna Westerdijkplein 75
Haagse Hogeschool 2521 EN DEN HAAG
Faculty of Engeneering Netherlands
Electrical Engeneering +31 70 4458936
-------------------- [EMAIL PROTECTED] --------------------
Linux - because reboots are for hardware changes
_______________________________________________
htdig-dev mailing list
[EMAIL PROTECTED]
http://lists.sourceforge.net/lists/listinfo/htdig-dev
