[
https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated TEZ-3362:
-----------------------------
Attachment: TEZ-3362.004.patch
Adding updated patch addressing [~hitesh]'s comments 1) fully and comment 2)
partially.
Moved all the deletion code to AMNodeTracker.
We want to have deletion service work for dags and vertices, which would mean
that we could pass a path to delete and no translation would be needed when the
HTTP request is received. This implementation does not address that generic api
and I was thinking of addressing that as a separate JIRA. Long term having a
plugin instead of AMNodeTracker for DAG/vertex deletion would be better.
Appreciate more comments/review from [~jeagles] and [~hitesh]. Thanks again!
> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
> Key: TEZ-3362
> URL: https://issues.apache.org/jira/browse/TEZ-3362
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Kuhu Shukla
> Attachments: TEZ-3362.001.patch, TEZ-3362.002.patch,
> TEZ-3362.003.patch, TEZ-3362.004.patch
>
>
> Applications like hive that use tez in session mode need the ability to
> delete intermediate data after a DAG completes and while the application
> continues to run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
