[
https://issues.apache.org/jira/browse/TEZ-3097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612351#comment-15612351
]
Jason Lowe commented on TEZ-3097:
---------------------------------
Sorry, I should have been more clear. The unit test is trying to test DAG
commit. In order for a DAG to commit, all the vertices need to succeed.
Therefore we should not be trying to schedule tasks beyond the DAG completion
in the context of this test, since that means a vertex succeeded before it
started scheduling tasks. In other words, the unit test is not doing a good
job of emulating what a real DAG does when it tries to commit.
In practice, yes, a DAG can enter a terminal state while vertices could still
be asynchronously trying to schedule, but I don't think that's an error.
Eventually the vertices will receive kill events from the dag and they in turn
will issue kill events to any tasks that were scheduled. The history handler
should have no issue recording these events which may occur after the dag
completion event. It only causes an issue in this test because the test is not
prepared to receive history events after the dag completes (again because it
poorly emulates what a DAG does to get to the commit stage).
> Flaky test: TestCommit.testDAGCommitStartedEventFail_OnDAGSuccess
> -----------------------------------------------------------------
>
> Key: TEZ-3097
> URL: https://issues.apache.org/jira/browse/TEZ-3097
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.8.3
> Reporter: Jeff Zhang
> Assignee: Harish Jaiprakash
>
> {noformat}
> testDAGCommitStartedEventFail_OnDAGSuccess(org.apache.tez.dag.app.dag.impl.TestCommit)
> Time elapsed: 0.072 sec <<< ERROR!
> java.util.ConcurrentModificationException: null
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
> at java.util.ArrayList$Itr.next(ArrayList.java:831)
> at
> org.apache.tez.dag.app.dag.impl.TestCommit$MockHistoryEventHandler.verifyVertexCommitStartedEvent(TestCommit.java:2033)
> at
> org.apache.tez.dag.app.dag.impl.TestCommit.testDAGCommitStartedEventFail_OnDAGSuccess(TestCommit.java:1880)
> {noformat}
> https://builds.apache.org/job/Tez-Build-Hadoop-2.4/245/console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
