vinothchandar commented on pull request #2342: URL: https://github.com/apache/hudi/pull/2342#issuecomment-749988417
>Do you suggest we sync the rollbacks only if the original instants for which the rollback was done were synced to the metadata table? Thinking out loud whether there are tricky cases here. Specifically, is there a chance of skipping a rollback instant incorrectly. Let's say we restored the last 11 commits (12 -> 2) with metadata synced upto C8 ``` data timeline : C1, rollback C12, rollback C11, ..... rollback C2 metadata timeline : C1, C2, ... C8 ``` data timeline archival does not move past C8, but can technically archive C8. Similarly metadata timeline is free to archive independently and lets say it archives upto C2. If it ends up looking like ``` data timeline : C1, rollback C12, rollback C11, ..... rollback C2 metadata timeline: C3, ..., C8 ``` It's now hard to determine whether or not `rollback C2` should be applied. (we could compare the latest instant on metadata timeline C8 and assume its applied?) I think this complexity is unwarranted for sake of failing-fast. We are better of defensively, ignoring deletes for files that were not logged before. @prashantwason Hope you can be convinced of this :) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
