[
https://issues.apache.org/jira/browse/TEZ-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529778#comment-14529778
]
Jeff Zhang commented on TEZ-2410:
---------------------------------
bq. The new VertexGroupStatus enum is duplicating the logic that is being
encoded in successfulCommits because all the enums can be represented as
methods on VertexGroupInfo which use a combination of "committed" and
"successfulCommits". So it may be better to use new methods on VertexGroupInfo
instead of having 2 ways of representing the same thing which may go out of
sync as code evolves.
The original purpose of VertexGroupStatus is to track the status of commits in
VertexGroup (successfulCommits=0 can mean both COMMIT_NOT_STARTED or
COMMITTING), and the status will be used in DAGImpl#vertexReRunning.
But after TEZ-2413, this VertexGroupStatus can be removed. Maybe I can include
TEZ-2413 in this patch. [~bikassaha] Thoughts ?
bq. 1) For normal vertices we do not write a group commit finished event. (the
patch does not seem to cover any of these)
Do you mean to add statement like this ?
{code}
historyEventHandler.verifyVertexGroupCommitStartedEvent("v1", 0);
historyEventHandler.verifyVertexGroupCommitFinishedEvent("v1", 0); //
use vertex name to verify vertex does not cause VertexGroupCommitFinished
{code}
bq. 2) For groups we only write the group commit finished event once. (the
patch covers some of these)
3 cases are covered in TestCommit (any other case I missed ?)
* If the commit is onDAGSuccess, then VertexGroupFinished should not be logged
* If the commit is onVertexSuccess, then VertexGroupFinished should be logged
once
* If the commit is onVertexSuccess and vertex group has multiple outputs, then
VertexGroupCommitFinished should still be logged once
> VertexGroupCommitFinishedEvent is not logged correctly
> ------------------------------------------------------
>
> Key: TEZ-2410
> URL: https://issues.apache.org/jira/browse/TEZ-2410
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: TEZ-2410-1.patch
>
>
> VertexGroupCommitFinishedEvent may be logged for non-vertex group commits.
> VertexGroupCommitFinishedEvent may be logged for each member vertex of the
> group instead of once per group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
