[
https://issues.apache.org/jira/browse/MAPREDUCE-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232303#comment-13232303
]
Steve Loughran commented on MAPREDUCE-3992:
-------------------------------------------
+1
* I understand why there aren't tests here, it's very hard to set up a test for
this.
* I've reviewed the code and it looks fairly straightforward -bails out early
on a reponse != 200.
* If there's no error code checking in one part of the system, there always the
risk the same thing has happened elsewhere, which is something that we need to
keep an eye out for.
> Reduce fetcher doesn't verify HTTP status code of response
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3992
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 0.23.1, 0.24.0, 1.0.1
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: mr-3992.txt
>
>
> Currently, the reduce fetch code doesn't check the HTTP status code of the
> response. This can lead to the following situation:
> - the map output servlet gets an IOException after setting the headers but
> before the first call to flush()
> - this causes it to send a response with a non-OK result code, including the
> exception text as the response body (response.sendError() does this if the
> response isn't committed)
> - it will still include the response headers indicating it's a valid response
> In the case of a merge-to-memory, the compression codec might then try to
> interpret the HTML response as compressed data, resulting in either a huge
> allocation (OOME) or some other nasty error. This bug seems to be present in
> MR1, but haven't checked trunk/MR2 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
