Reduce fetcher doesn't verify HTTP status code of response
----------------------------------------------------------
Key: MAPREDUCE-3992
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1
Affects Versions: 1.0.1
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Currently, the reduce fetch code doesn't check the HTTP status code of the
response. This can lead to the following situation:
- the map output servlet gets an IOException after setting the headers but
before the first call to flush()
- this causes it to send a response with a non-OK result code, including the
exception text as the response body (response.sendError() does this if the
response isn't committed)
- it will still include the response headers indicating it's a valid response
In the case of a merge-to-memory, the compression codec might then try to
interpret the HTML response as compressed data, resulting in either a huge
allocation (OOME) or some other nasty error. This bug seems to be present in
MR1, but haven't checked trunk/MR2 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
