[
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979629#comment-14979629
]
Jeff Zhang commented on TEZ-2581:
---------------------------------
>>> Didn't quite understand this. What is the reasoning behind this? Also does
>>> "vertexData.getVertexFinishedEvent() != null" imply that full recovery
>>> events have not been seen? Should it be null (based on the code in
>>> isVertexFinished())?
It should be "vertexData.getVertexFinishedEvent() == null", will fix it.
>>> Why are we separately looking at vertex group commit members? vertex group
>>> commit is a single operation that commits for all member vertices. Each
>>> member vertex does not have a separate commit operation.
Because it is not known whether this vertex belong to any vertex group when
parsing recovery logs. So here check both vertex level commit and vertex group
level commit.
>>> Log debug or remove?
Will make it as debug
>>> Why is it always _1 ?
This method (RecoveryParser#readRecoveryEvents) is only for testing (verify the
recovery events). Will add dagId to make it more clear.
>>> How is DagRecoveryData.isRecoverable() different from isDAGRecoverable()?
isDAGRecoverable is for checking recoverability based on the summary data.
DagRecoveryData.isRecoverable() is for checking for recoverability based on the
non-summary events. Will reorganize the code the make it easier to understand
> Umbrella for Tez Recovery Redesign
> ----------------------------------
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch,
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch,
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf,
> TezRecoveryRedesignV1.1.pdf
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
