Revanth14 opened a new pull request, #2239: URL: https://github.com/apache/iceberg-go/pull/2239
## Problem Closes #2238. On v3, removing a data file through `RewriteFiles`, `ReplaceFiles`, `ReplaceDataFilesWithDataFiles` or `ReplaceDataFiles` left its live deletion vector in place unless the caller also named it, so the snapshot kept a DV whose `referenced_data_file` was no longer live. The delete and overwrite paths already handled this through `liveDeletionVectors.removeDataFile`. ## Fix The two data-only replace paths read the live DVs on v3 and remove each data file through `removeDataFile`. `replaceFiles` already scans every entry, so it marks a live DV whose referenced data file is being removed and that the caller did not name, with no extra read. Those DVs are removed like automatic removals: they do not feed the delete-rewrite sequence-number checks, and a stale explicitly named DV still fails as before. As on the delete paths, a commit that removes a DV fails on a CAS conflict instead of replaying. Java does this centrally, in `ManifestFilterManager.removeDanglingDeletesFor` during manifest filtering. This PR fixes the call sites instead because the Go producer's summary is built from its removal sets, not from the filter scan. Moving it into `overwriteFiles.scanManifest` would mean threading the dropped DVs through the initial and retry summaries. Happy to do that as a follow-up if preferred. ## Testing `TestRemovingDataFileRemovesItsUnnamedDV` covers all four entry points, each over separate transactions and staged in one transaction, and asserts on the raw delete manifests and the snapshot summary. `TestRewriteFiles_UnnamedDVRemovalKeepsSharedPuffinSibling` checks that a sibling DV in the same Puffin file survives. All nine cases fail without the fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
