RE: Curb maximum size of headers

2005-03-17 Thread Tony Lewis
Hrvoje Niksic wrote:

 I don't see how and why a web site would generate headers (not bodies, to
 be sure) larger than 64k.

To be honest, I'm less concerned about the 64K header limit than I am about
limiting a header line to 4096 bytes. I don't know any sites that send back
header lines that long, but they could. Who's to say some site doesn't have
a 4K cookie?

Since you already proposing to limit the entire header to 64K, what is
gained by adding this second limit?

Tony




Re: Curb maximum size of headers

2005-03-17 Thread Hrvoje Niksic
Tony Lewis [EMAIL PROTECTED] writes:

 Hrvoje Niksic wrote:

 I don't see how and why a web site would generate headers (not
 bodies, to be sure) larger than 64k.

 To be honest, I'm less concerned about the 64K header limit than I
 am about limiting a header line to 4096 bytes.

The 4k limit does not apply to header lines -- only the 64k limit to
the whole header section does.  The line length limit is used by the
code that reads server output line by line, currently only FTP code
reading server responses over control connection.  (Directory listings
don't count as server responses in this context.)

I should have made the *use* of individual limits clearer in the patch
commentary.  I guess I concentrated too much on the implementation
details and forgot that the readers of the list needn't know exactly
what code uses fd_read_line for what purpose.

 Since you already proposing to limit the entire header to 64K, what
 is gained by adding this second limit?

I hope this is now clearer; the second limit is used by an entirely
separate piece of code.  Malicious overlong FTP lines could presumably
be exactly as dangerous as malicious HTTP responses.  In fact, prior
to this patch, something as simple as `nc -lp21  /dev/zero' brings a
connecting Wget, along with the unsuspecting user's system, to its
knees.  That is just too damn easy, especially with there being a
whole bunch of people who check for this kind of bugs, publicizing
their findings as security holes.

I sincerely wish none of these limits were necessary.  If anyone
abhors arbitrary limits, it is surely me.


Re: Curb maximum size of headers

2005-03-16 Thread Hrvoje Niksic
Tony Lewis [EMAIL PROTECTED] writes:

 Hrvoje Niksic wrote:

 This patch imposes IMHO reasonable, yet safe, limits for reading server
 responses into memory.

 Your choice of default limits looks reasonable to me, but shouldn't
 wget provide the user a way to override these limits?

Not if there is never a need to.  Wget has a huge number of
command-line options already; one should try not to add new ones
without very good reasons to back them up.  Overriding these limits
would require *two* new cryptic command-line options that would
clutter the code and documentation and in all likeliness would never
be used, thus bringing no value to the user.

Except for malicious server output, I see no use in accepting lines
greater than 4k or HTTP response headers larger than 64k.  If I am
proven wrong, the limit can be raised, but I suspect that it will not
be the case.  (Note that the limit can safely be raised without any
penalty for the usual, short, responses.)