On Sat, Jun 08, 2002 at 10:58:16PM -0500, Amy Rupp wrote: > >> Why not just put "robots=off" in your .wgetrc?
>With all due respect, I *DID* read the documentation, which I quoted, >and I attempted to find the latest version available. I didn't post >the original question, and *MY* question still stands: if you robots=off does just that; it does neither read nor honor robots.txt. As for User Agent, most sites like to see a string with WinXX or IE or Explorer in them. If you are using windows, go to a friend's web site and then ask them to mail you the user agent string your Explorer sent and use that. If you are using Unix and run apache, look through your own logs to see what strings people send you. -- AlanE
