[
https://issues.apache.org/jira/browse/HUDI-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-3441:
----------------------------
Description:
For r0.9.0 and before, there is a bug that the markers from compaction action
are not cleaned up after the commit is complete. There is a fix going in after
r0.9.0 and before r0.10.0 regarding the marker cleanup for compaction:
[https://github.com/apache/hudi/pull/3576]. For tables with r0.9.0 and before,
the makers are going to exist without cleanup.
We should clean up the Hudi markers of completed commits inside .temp if they
are not cleaned up due to older release. This process can be inside upgrade
process and a cli command as well.
> Clean up markers of completed commits in .hoodie/.temp folder
> -------------------------------------------------------------
>
> Key: HUDI-3441
> URL: https://issues.apache.org/jira/browse/HUDI-3441
> Project: Apache Hudi
> Issue Type: Task
> Components: writer-core
> Reporter: Ethan Guo
> Assignee: Forward Xu
> Priority: Blocker
> Fix For: 0.11.0
>
>
> For r0.9.0 and before, there is a bug that the markers from compaction action
> are not cleaned up after the commit is complete. There is a fix going in
> after r0.9.0 and before r0.10.0 regarding the marker cleanup for compaction:
> [https://github.com/apache/hudi/pull/3576]. For tables with r0.9.0 and
> before, the makers are going to exist without cleanup.
> We should clean up the Hudi markers of completed commits inside .temp if they
> are not cleaned up due to older release. This process can be inside upgrade
> process and a cli command as well.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
