[
https://issues.apache.org/jira/browse/TEZ-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311301#comment-15311301
]
Jonathan Eagles commented on TEZ-3271:
--------------------------------------
This job will show off the current functionality.
{noformat}
HADOOP_CLASSPATH="$TEZ_HOME/*:$TEZ_HOME/lib/*:$TEZ_CONF_DIR" yarn jar
$TEZ_HOME/tez-tests-*.jar mrrsleep -Dmrr.sleepjob.map.fatal.error=true
-Dmrr.sleepjob.map.error.task.ids=0 -Dtez.am.task.max.failed.attempts=1
-Dtez.am.failures.percent=0.25f -m 4 -ir 1 -r 1
{noformat}
{noformat}
16/06/01 22:51:40 INFO client.DAGClientImpl: DAG initialized:
CurrentState=Running
16/06/01 22:51:40 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
TotalTasks: 6 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
16/06/01 22:51:45 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 0%
TotalTasks: 6 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
16/06/01 22:51:47 INFO client.DAGClientImpl: DAG: State: RUNNING Progress:
16.67% TotalTasks: 6 Succeeded: 1 Running: 3 Failed: 0 Killed: 0
16/06/01 22:51:47 INFO client.DAGClientImpl: DAG: State: RUNNING Progress: 50%
TotalTasks: 6 Succeeded: 3 Running: 1 Failed: 1 Killed: 0 FailedTaskAttempts: 1
16/06/01 22:51:48 INFO client.DAGClientImpl: DAG: State: RUNNING Progress:
83.33% TotalTasks: 6 Succeeded: 5 Running: 0 Failed: 1 Killed: 0
FailedTaskAttempts: 1
16/06/01 22:51:48 INFO client.DAGClientImpl: DAG: State: SUCCEEDED Progress:
83.33% TotalTasks: 6 Succeeded: 5 Running: 0 Failed: 1 Killed: 0
FailedTaskAttempts: 1
16/06/01 22:51:48 INFO client.DAGClientImpl: DAG completed. FinalState=SUCCEEDED
{noformat}
Attached a screenshot of what the UI looks like. I would prefer to have the
diagnostic message regarding success more prominent and less "red".
> Provide mapreduce failures.maxpercent equivalent
> ------------------------------------------------
>
> Key: TEZ-3271
> URL: https://issues.apache.org/jira/browse/TEZ-3271
> Project: Apache Tez
> Issue Type: New Feature
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: Succeeded with Failures.png, TEZ-3271.1.patch,
> TEZ-3271.2.patch, TEZ-3271.3.patch, TEZ-3271.4.patch
>
>
> There is a certain category of work that need not have 100% of tasks succeed
> to cause the work to be considered a success. To meet that end, I propose we
> provide a tez equivalent of mapreduce.map.failures.maxpercent and
> mapreduce.reduce.failures.maxpercent. In this way a vertex will be considered
> a success if the number of failures is below a configured threshold.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
