[
https://issues.apache.org/jira/browse/TEZ-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642481#comment-16642481
]
Jonathan Eagles commented on TEZ-3990:
--------------------------------------
[~kshukla]. Couple more very minor things I noticed that I should have caught
before.
- please add "final" to maxPenaltyTime
- please add "shuffle" to the tez runtime parameter if you think that is
correct?
Also, go ahead and file the subsequent jira along with the new penalties
feature for unordered.
> The number of shuffle penalties for a host/inputAttemptIdentifier should be
> capped
> ----------------------------------------------------------------------------------
>
> Key: TEZ-3990
> URL: https://issues.apache.org/jira/browse/TEZ-3990
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.9.1, 0.10.0
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Priority: Major
> Attachments: TEZ-3990.001.patch, TEZ-3990.002.patch,
> TEZ-3990.003.patch, TEZ-3990.004.patch
>
>
> In a scenario where the same mapId fetches fail, the penalty code allows
> adding the same Host/InputAttemptIdentifier over and over with revised
> penalty time that grows exponentially. It should at some point drop the
> retrying and report failure to the AM asap to allow the job to rectify the
> upstream output.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
