On Wed, 18 Jul 2007, Josh Williams wrote:
Is there any particular reason we don't have an option to ignore robots.txt?
There is no particular reason, so we do.
Maciej
On 7/18/07, Maciej W. Rozycki [EMAIL PROTECTED] wrote:
There is no particular reason, so we do.
As far as I can tell, there's nothing in the man page about it.
From: Josh Williams
As far as I can tell, there's nothing in the man page about it.
It's pretty well hidden.
-e robots=off
At this point, I normally just grind my teeth instead of complaining
about the differences between the command-line options and the commands
in the .wgetrc
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Steven M. Schweda wrote:
From: Josh Williams
As far as I can tell, there's nothing in the man page about it.
It's pretty well hidden.
-e robots=off
At this point, I normally just grind my teeth instead of complaining
about the
Micah Cowan wrote:
The manpage doesn't need to give as detailed explanations as the info
manual (though, as it's auto-generated from the info manual, this could
be hard to avoid); but it should fully describe essential features.
I can't see any good reason for one set of documentation to be
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Tony Lewis wrote:
Micah Cowan wrote:
The manpage doesn't need to give as detailed explanations as the
info manual (though, as it's auto-generated from the info manual,
this could be hard to avoid); but it should fully describe
essential
Micah Cowan wrote:
Don't we already follow typical etiquette by default? Or do you mean
that to override non-default settings in the rcfile or whatnot?
We don't automatically use a --wait time between requests. I'm not sure what
other nice options we'd want to make easily available, but there
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Tony Lewis wrote:
Micah Cowan wrote:
Don't we already follow typical etiquette by default? Or do you
mean that to override non-default settings in the rcfile or
whatnot?
We don't automatically use a --wait time between requests. I'm not
Micah Cowan [EMAIL PROTECTED] writes:
I think we should either be a stub, or a fairly complete manual
(and agree that the latter seems preferable); nothing half-way
between: what we have now is a fairly incomplete manual.
Converting from Info to man is harder than it may seem. The script
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Hrvoje Niksic wrote:
Micah Cowan [EMAIL PROTECTED] writes:
I think we should either be a stub, or a fairly complete manual
(and agree that the latter seems preferable); nothing half-way
between: what we have now is a fairly incomplete manual.
Micah Cowan [EMAIL PROTECTED] writes:
Converting from Info to man is harder than it may seem. The script
that does it now is basically a hack that doesn't really work well
even for the small part of the manual that it tries to cover.
I'd noticed. :)
I haven't looked at the script that
11 matches
Mail list logo
