[
https://issues.apache.org/jira/browse/YARN-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329865#comment-15329865
]
Junping Du commented on YARN-1773:
----------------------------------
Hi [~varun_saxena], this JIRA has pending for a while. Do you still have plan
to fix it?
> ShuffleHeader should have a format that can inform about errors
> ---------------------------------------------------------------
>
> Key: YARN-1773
> URL: https://issues.apache.org/jira/browse/YARN-1773
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.3.0, 2.4.0
> Reporter: Bikas Saha
> Assignee: Varun Saxena
> Priority: Critical
>
> Currently, the ShuffleHeader (which is a Writable) simply tries to read the
> successful header (mapid, reduceid etc). If there is an error then the input
> will have an error message instead of (mapid, reducedid etc). Thus parsing
> the ShuffleHeader fails and since we dont know where the error message ends,
> we cannot consume the remaining input stream which may have good data from
> the remaining map outputs. Being able to encode the error in the
> ShuffleHeader will let us parse out the error correctly and move on to the
> remaining data.
> The shuffle handler response should say which maps are in error and which are
> fine, what the error was for the erroneous maps. These will help report
> diagnostics for easier upstream reporting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
