[
https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-3425:
--------------------------------------
Sprint: (was: Cont' improve - 2022/03/01)
> Clean up spill path created by Hudi during uneventful shutdown
> --------------------------------------------------------------
>
> Key: HUDI-3425
> URL: https://issues.apache.org/jira/browse/HUDI-3425
> Project: Apache Hudi
> Issue Type: Task
> Components: compaction
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Critical
> Fix For: 0.11.0
>
>
> h1. Hudi spill path not getting cleared when containers getting killed
> abruptly.
>
> When yarn kills the containers abruptly for any reason while hudi stage is in
> progress then the spill path created by hudi on the disk is not cleaned and
> as a result of which the nodes on the cluster start running out of space. We
> need to clear the spill path manually to free out disk space.
>
> Ref issue: https://github.com/apache/hudi/issues/4771
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
