[
https://issues.apache.org/jira/browse/TEZ-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296131#comment-14296131
]
Hitesh Shah commented on TEZ-993:
---------------------------------
bq. It should be easy to implement. Just need to add one flag in the first
event: DAGSubmittedEvent to tell RecoveryStore whether it is needed to recover.
And RecoveryStore would skipp the following events of this dag if the dag is
not required to recover.
This will imply that all the events will still be sent to the recovery service
( which adds some level of overhead ). Also, I am assuming that there are code
changes needed to make sure these events are filtered out before being added to
the queue?
> Remove application logic from RecoveryService
> ---------------------------------------------
>
> Key: TEZ-993
> URL: https://issues.apache.org/jira/browse/TEZ-993
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Jeff Zhang
> Attachments: TEZ-993-3.patch, Tez-993-2.patch, Tez-993.patch
>
>
> Currently RecoveryService storage logic knows a lot about the DAG like which
> dag is pre-warm and does not need to be stored, which events needs special
> treatment etc. This kind of logic couples the DAG and the storage more than
> is probably necessary and can be a source of complications down the road. The
> storage should ideally be simply storing a sequence of arbitrary records
> delimited by a marker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
