HttpParser triggers unfriendly OutOfMemoryError on challenging input
--------------------------------------------------------------------
Key: HTTPCLIENT-566
URL: http://issues.apache.org/jira/browse/HTTPCLIENT-566
Project: Jakarta HttpClient
Type: Bug
Components: HttpClient
Versions: 3.0.1
Reporter: Gordon Mohr
Many users of HttpClient use it to connect to servers which generate
challenging HTTP responses, such as responses which include an arbitrarily
large number of headers or headers of arbitrarily large size. (Sometimes such
headers are conformant with the spec, in that they contain legal characters in
plausible header formats; other times these are filled with binary content that
is a violation of the relevant specs. Even when technically legal, often such
giant headers are the inadvertent result of server-side bugs.)
As a Java execution environment always has a hard cap on the available heap
space, any parsing code which can use an arbitrary amount of memory risks
triggering an OutOfMemoryError, either in its own thread or even another thread
that happens to need memory after the parsing thread has exhausted it all.
Such OutOfMemoryErrors are a particularly unfriendly way to indicate that a
practical limit has been exceeded, compared to other options. They can hide the
thread of execution which is most to blame. It is hard and awkward to set up
handlers that catch and recover from OOMEs wherever they are most likely to
occur. Even with such handlers, the actual allocation triggering an OOME may
occur in another critical thread, even if that thread has minimal and
well-controlled memory needs.
HttpClient ought to provide one or more ways for a user to protect against such
OOMEs, and instead receive a more convenient/recoverable indication of an HTTP
response that is impossible to process with the HttpClient library within the
available resources. Many approaches are possible; the easiest would be to
allow a user of HttpClient to set their own optional, pragmatic limits on
header sizes and number. Then, just as a user may already cleanly cancel the
stream-reading of an arbitrarily-long content-body without fouling up their
application state, they would be able to cancel the parsing of oversized
response headers.
Similar issues have been discussed before, for example in Bugzilla bug #25468
(http://issues.apache.org/bugzilla/show_bug.cgi?id=25468) which was to "Provide
HttpParser plug-in mechanism." Though that issue is marked resolved/fixed,
there is no such plug-in mechanism allowing an OOME workaround in the 3.x
HttpClient, and it is not clear that a mechanism/work-around exists in whatever
4.0 work has been completed.
So my suggestion is that this new issue be used to uniquely track the OOME risk
in HttpParser, and only be considered "fixed" when some version of HttpClient
offers an alternative to throwing OOMEs as a way of dealing with challenging
HTTP responses. Alternatively, this could simply become the issue in the new
system for collecting user-contributed workarounds/patches.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
