danny0405 commented on pull request #4821: URL: https://github.com/apache/hudi/pull/4821#issuecomment-1062485562
> @danny0405 : here is the scenario. Lets say multi-writer is enabled and hence rollbacks are lazy. there is a commit C5 which got committed to MDT, but crashed before committing to data table. and the user restarts the pipeline. due to multi-writer, there are more commits added, but rollback will be triggered lazily by cleaner. Lets say cleaner configs are very easy (say 100 commits). So, by this time, archival could clean up the partially failed commit. Can we disable lazy cleaner for restarts/bootstrap then ? The lazy cleaning makes sense for normal commit but it just make things complex for boostrap/restarts and it even does not gains much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
