deepakpanda93 commented on issue #9257: URL: https://github.com/apache/hudi/issues/9257#issuecomment-4845421636
Closing this out — it was never reproduced (multiple attempts by maintainers), it was on a 0.13.0 vendor build, and the relevant code paths have since been substantially reworked. On the substance: the failure was a clean that deleted files and then failed in HoodieActiveTimeline.transitionState during its metadata-table update, leaving the clean inflight with files already removed — and since it ran async, the failure didn't fail the job, so a later clustering broke on the missing file. In 1.x: - the timeline was rewritten (completion-time / LSM timeline in 1.0), so the transitionState path that threw IllegalArgumentException: null here no longer exists; - clean and archival are now wrapped in try-catch (HUDI-8804, [#12562](https://github.com/apache/hudi/pull/12562), 1.0.1+), so a clean failure is surfaced rather than silently succeeding; - clean robustness was further hardened (cap commits per clean, cleanup/rollback policy before writes, incremental clean-planning fixes), with the clean/archival interplay simplification ongoing ([#17279]). Given it wasn't reproducible and the clean + timeline code has been reworked, I'd recommend retrying on a recent open-source 1.x release. If you can reproduce a failed clean that deletes files (with the job still succeeding) on 1.x, please reopen with the .hoodie/ timeline around the failure and the clean plan, and we'll dig in. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
