harshilv17 opened a new pull request, #2521: URL: https://github.com/apache/iceberg-go/pull/2521
Closes #2520 ## What - Reject a row delta that adds a deletion vector whose partition spec or partition differs from the data file it references. ## Why - The spec requires a DV to match its data file's spec and partition. Scans match DVs by path only, so a DV built with the current spec after partition evolution committed and was applied as if valid. ## Implementation - In `RowDelta.Commit`, look up each added DV's referenced data file among the data files added in the same delta, then in the current snapshot's live data entries (stopping once all are found), and compare spec ID and partition with the same `partitionsMatch` helper the scan side uses. - A reference to a data file that is not found is left to the existing conflict checks. ## Tests - Mismatched partition rejected, matching partition accepted, a DV for a data file added in the same delta rejected, and a DV written with the evolved spec for a spec-0 data file rejected. All four rejection cases fail on `main`. ## Question: v2 position deletes The issue also covers v2 position deletes that set `referenced_data_file`. Applying the same check to them breaks `TestDynamicPartitionOverwriteKeepsDeleteForSurvivingOldSpecData` and its pruned variant (#2381), which commit a v2 position delete under the new spec (null partition) targeting an old-spec data file, and note that this is allowed. Since #2471 the scan side rejects that same combination. I left v2 position deletes out of this PR: should those fixtures move to the data file's spec so the check can cover v2 too, or is the cross-spec v2 case meant to stay valid? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
