[
https://issues.apache.org/jira/browse/TEZ-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294409#comment-14294409
]
Hitesh Shah commented on TEZ-993:
---------------------------------
I think there are a couple of issues which need to be clarified first.
- One of the concerns [~bikassaha] had was that the decision of whether to
log recovery events for a DAG was not being done inside the DAGImpl but in the
recovery layer. Tomorrow, if we support a config that says a particular DAG (
not a pre-warm dag ) should not be recovered, how would this re-factor support
this?
- Apart from that, the recovery process needed to know the internals of the
DAGAppMaster such as dagCounter, etc. Currently, any relevant changes in the
DAGAppMaster will also need corresponding changes to the recovery restore
layer.
> Remove application logic from RecoveryService
> ---------------------------------------------
>
> Key: TEZ-993
> URL: https://issues.apache.org/jira/browse/TEZ-993
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Jeff Zhang
> Attachments: TEZ-993-3.patch, Tez-993-2.patch, Tez-993.patch
>
>
> Currently RecoveryService storage logic knows a lot about the DAG like which
> dag is pre-warm and does not need to be stored, which events needs special
> treatment etc. This kind of logic couples the DAG and the storage more than
> is probably necessary and can be a source of complications down the road. The
> storage should ideally be simply storing a sequence of arbitrary records
> delimited by a marker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
