zhangyue19921010 edited a comment on pull request #4489: URL: https://github.com/apache/hudi/pull/4489#issuecomment-1019408956
Hi @nsivabalan I agree with your opinion that let cleaner delete files and partitions. Just a little concern that how can we deal with the scenario that we trigger a delete partition action with a **async** cleaner + enable metadata table. -> Async cleaner started and finished before replaced committed : 1. Async cleaner finished to delete old replaced files. 2. Current replaced committed. 3. Meta table synced ==> partitions are deleted in meta data table. We maybe get different result between `getAllPartitions` form metadata table and number of physical partitions. Also wen could have a strict limit that delete partition only works with sync cleaner but wondering to know if it is possible. Or this kind of `out-of-sync` will cause no damage. updated: we could let cleaner sync metadata table and delete partitions in it. It could solve the consistency issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
