nsivabalan commented on issue #17901: URL: https://github.com/apache/hudi/issues/17901#issuecomment-3923274891
Should we try moving rollbacks out of clean path. Essentially, `rollbackFailedWrites` warrants a separate table service in itself. We could call it across many places. Single writer: a. just before starting a new commit. Multi-writer: 1. just before starting a new commit optionally. 2. post commit -> clean, rollbackFailedWrites, archive. 3. Separate async table service. Now coming to your proposal: `hoodie.clean.failed.writes.policy` is meant to dictate how the clean code path should handle failed writes. I think we should stick to just 1 values in here (EAGER, LAZY), since there are only two modes, single writer or not. may be we could introduce another writer config to force the rollback failed writes before starting a new commit `hoodie.write.rollback.failed.writes.pre.write.enable` We might have to fix https://github.com/apache/hudi/blob/eb1d7729a1bbaebee4a51e5d2ba24b1c068cf295/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java#L1014 piece of code to accommodate this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
