rangareddy commented on issue #17169: URL: https://github.com/apache/hudi/issues/17169#issuecomment-5366104194
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9740). **Findings: needs a reproduction against a current build before it can be dispositioned.** The report is concrete: after an async index build (e.g. RLI via HoodieIndexer) is crashed mid-way and the cleaner rolls the instants back, hoodie.properties still lists the partition as an inflight MDT partition. Reproducing it looks tractable - the description gives an ordered recipe - start async index building on a decent-sized table, crash it once requested/inflight exist, run the cleaner, then inspect hoodie.properties. I have not called this from the source. The surrounding code has been rewritten substantially since this was filed - table versions 8 and 9, the `BufferedRecordMerger` family replacing payload-based merging, the file-group-reader migration, the indexer abstraction in #18348 / #18372, and the `hudi-common` package moves in #19193 / #19195. Inferring either "still broken" or "silently fixed" through that much churn would be a guess. There is also no commit anywhere in the tree referencing HUDI-9740, so if it was fixed it was incidental rather than deliberate - another reason to verify rather than assume. The consequence is worth stating on the ticket: a stale inflight entry makes the table look mid-index forever, which can block a later index build or confuse readers. Mirror image of #17208 (HUDI-9802), where the inflight set is empty when a partition is expected - both are the inflight MDT partition list drifting from the timeline, and a single owner for that invariant would fix both. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
