[
https://issues.apache.org/jira/browse/YARN-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329895#comment-15329895
]
Varun Saxena edited comment on YARN-1773 at 6/15/16 4:23 AM:
-------------------------------------------------------------
[~djp], sorry this fell off my radar.
Do not have bandwidth to fix it, in short term.
I have unassigned it in case somebody else wants to take it up. Will assign it
back to myself once I have bandwidth and if nobody else has taken it up.
was (Author: varun_saxena):
[~djp], sorry this fell off my radar.
Do not have bandwidth to fix it, in short term. You can take it up, if you want.
> ShuffleHeader should have a format that can inform about errors
> ---------------------------------------------------------------
>
> Key: YARN-1773
> URL: https://issues.apache.org/jira/browse/YARN-1773
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.3.0, 2.4.0
> Reporter: Bikas Saha
> Priority: Critical
>
> Currently, the ShuffleHeader (which is a Writable) simply tries to read the
> successful header (mapid, reduceid etc). If there is an error then the input
> will have an error message instead of (mapid, reducedid etc). Thus parsing
> the ShuffleHeader fails and since we dont know where the error message ends,
> we cannot consume the remaining input stream which may have good data from
> the remaining map outputs. Being able to encode the error in the
> ShuffleHeader will let us parse out the error correctly and move on to the
> remaining data.
> The shuffle handler response should say which maps are in error and which are
> fine, what the error was for the erroneous maps. These will help report
> diagnostics for easier upstream reporting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
