Can you cause an out of memory failure while using getResponseBodyAsStream() to process a response from a *conformant* HTTP server?
Sure, I could construct a "chunked" response that indicated as a chunk size a number on the order of (16^(2^30)), which would consume all available memory, but then the question would be how to detect that and fail the connection sooner - due to a non-conformant server. It would not be a question of merely prematurely terminating the chunk.
I don't see how ChunkedInputStream.exhaustInputStream() can actually consume all available memory. Rather I see that if the server never stops sending data, it will simply never exit. The latter condition suggests an abuse of the HTTP protocol, and an alternative "streaming" protocol would be more appropriate.
-Eric.
[EMAIL PROTECTED] wrote:
Eric,
adding a maxBytes parameter to getResponseBody seems to be a resonable idea to limit the actual number of bytes returned in the response body.
Nevertheless, you would still have to fix some other flaws lying around at lower levels that attempt to read until EOF or newline, such as ChunkedInputStream.close() and exhaustInputStream() - or HttpParser.readRawline().
I can easily provide test cases which will cause HttpClient to eat up all available memory throwing an OutOfMemoryError because of reading and reading from a never ending HTTP Response.
I would regard this behaviour as a bug.
The patch is intended as an "emergency brake" if methods at higher levels try to read until EOF. It is up to the user whether to set this hard limit or not.
Christian
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
