[
https://issues.apache.org/jira/browse/HUDI-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502978#comment-17502978
]
sivabalan narayanan commented on HUDI-3441:
-------------------------------------------
[~guoyihua] [~xushiyan] : we should keep track of all such tooling
enchancements we are doing and ensure we add it to docs or in our release
notes. Like the validator tool, cleaning up of residual marker files etc.
[~guoyihua] : if I am not wrong, you were invovled in most of them either as
author or reviewer. Can you please add a tracking ticket somewhere.
> Clean up markers of completed commits in .hoodie/.temp folder
> -------------------------------------------------------------
>
> Key: HUDI-3441
> URL: https://issues.apache.org/jira/browse/HUDI-3441
> Project: Apache Hudi
> Issue Type: Improvement
> Components: writer-core
> Reporter: Ethan Guo
> Assignee: Forward Xu
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
>
> For r0.9.0 and before, there is a bug that the markers from compaction action
> are not cleaned up after the commit is complete. There is a fix going in
> after r0.9.0 and before r0.10.0 regarding the marker cleanup for compaction:
> [https://github.com/apache/hudi/pull/3576]. For tables with r0.9.0 and
> before, the makers are going to exist without cleanup.
> We should clean up the Hudi markers of completed commits inside .temp if they
> are not cleaned up due to older release. This process can be inside upgrade
> process and a cli command as well.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
