[
https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310850#comment-15310850
]
Hitesh Shah edited comment on TEZ-3271 at 6/1/16 6:38 PM:
----------------------------------------------------------
- The event generation for the failed tasks is the main issue and something
maybe that the edge manager or the VM could do? I am not sure if the VertexImpl
is the right place for this as datamovementevent payload are input/output
specific unless this is an explicit framework generated event that all
inputs/outputs impls should handle. It might be better to split the 2 issues
into different jiras - one for the event generation and I/O changes to handle
the new event for no more data event and next address the failure threshold
handling.
- Other general comments:
- this config should be a vertex level config and not an AM specific one
hence named and scoped accordingly?
- code regarding commit or not should probably be put in a common place?
- any diagnostics updates to indicate vertex succeeded as failure threshold
was not met?
- any recovery impact? given that the same transitions are used for
recovery I dont think there should be any impact but might be worth checking.
was (Author: hitesh):
- The event generation for the failed tasks is the main issue and something
maybe that the edge manager or the VM could do? I dont think the VertexImpl is
the right place for this as datamovementevent payload are input/output
specific. It might be better to split the 2 issues into different jiras - one
for the event generation and I/O changes to handle the new event for no more
data event and next address the failure threshold handling.
- Other general comments:
- this config should be a vertex level config and not an AM specific one
hence named and scoped accordingly?
- code regarding commit or not should probably be put in a common place?
- any diagnostics updates to indicate vertex succeeded as failure threshold
was not met?
- any recovery impact? given that the same transitions are used for
recovery I dont think there should be any impact but might be worth checking.
> Provide mapreduce failures.maxpercent equivalent
> ------------------------------------------------
>
> Key: TEZ-3271
> URL: https://issues.apache.org/jira/browse/TEZ-3271
> Project: Apache Tez
> Issue Type: New Feature
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-3271.1.patch, TEZ-3271.2.patch, TEZ-3271.3.patch
>
>
> mapreduce.map.failures.maxpercent
> mapreduce.reduce.failures.maxpercent
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
