[
https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated TEZ-3362:
-----------------------------
Attachment: TEZ-3362.007.patch
Updated patch to delete the DAG only from ContainerLanchers(s), so that
AMNodeTracker is not involved. This removes the need for event notification to
the AMNode.
{{ContainerLauncherWrapper}} needs more work so that we don't do {{instanceof}}
checks but it would be nice to handle it once and for all in the follow up JIRA
for generic service plugin design which should move the {{dagComplete}} to
{{ContainerLauncher}} abstract class. Also, currently the health of the node is
not checked before deletion which would need exposing {{isUsable}} or similar
methods through the {{containerlauncherContext}}. Added as TODO.
Additionally, took out the {{DagDeleteRunnable}} piece to a new class which
should evolve in subsequent JIRAs.
> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
> Key: TEZ-3362
> URL: https://issues.apache.org/jira/browse/TEZ-3362
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Kuhu Shukla
> Attachments: TEZ-3362.001.patch, TEZ-3362.002.patch,
> TEZ-3362.003.patch, TEZ-3362.004.patch, TEZ-3362.005.patch,
> TEZ-3362.006.patch, TEZ-3362.007.patch
>
>
> Applications like hive that use tez in session mode need the ability to
> delete intermediate data after a DAG completes and while the application
> continues to run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
