[
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976367#comment-14976367
]
Bikas Saha commented on TEZ-2581:
---------------------------------
tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java
Why rename to setParallelism2?
logVertexReconfigureDoneEvent() - This should be logged in
doneReconfiguringVertex() and in setParallelism() this should be logged if
vertexToBeReconfiguredByManager == false. We should create a follow up jira to
change VertexManager.java to hide this part by always ensuring that
doneReconfiguringVertex() is always called.
Alternatively, what we are trying to achieve here is store the final
configuration of the vertex (numTasks, edges etc.) in case they were updated at
runtime. In that case, we could store the updated items when we send vertex
configured notification in maybeSendConfiguredEvent(). At this point the vertex
has its final definition. If anything was updated by its VM then we could store
it here in a VertexFinalConfiguration() event. Then, on recovery, we could use
this event in the same way as the patch is currently using
VertexDoneReconfigureEvent. This way the logic would be independent of the
vertex manager code flow.
{code}OG.info(vertex.getLogIdentifier() + ":" + vertex.getState()
+ ",numInitedSourceVertices=" + vertex.numInitedSourceVertices{code}
Remove or change to debug?
NoOpVertexManager instead of DummyVertexManager? Also why does it need a
serialized reconfigure done event payload? When setting the DummyVertexManager
we could assign all the items from the reconfigureDoneEvent. DummyVertexManager
could just start all tasks onVertexStarted(). Also, debug logs instead of info
would be better.
Why is RecoverTransition needed?
> Umbrella for Tez Recovery Redesign
> ----------------------------------
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch,
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch,
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf,
> TezRecoveryRedesignV1.1.pdf
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
