laskoviymishka opened a new pull request, #2620: URL: https://github.com/apache/iceberg-go/pull/2620
After an identity partition source is promoted (int to long, float to double), manifests written before the promotion still decode its partition values as int32/float32, and newer ones as int64/float64. So a v2 merge-on-read Delete on an old file commits a position delete that the scan then rejects with "partition does not match" (#2605), and equality deletes in a promoted float partition skip the old files (#2610). This change widens int32 to int64 and float32 to float64 where partition values are compared: in `partitionsMatch`, which covers the position delete check and `matchEqualityDeletesToData`, and in the single-field key of the equality delete index, which already widened integers. Both widenings are exact. No other type pairs are widened, and `comparablePartitionKey`, which the writers use for their partition keys, is left alone. The new tests in `table/partition_promotion_deletes_test.go` write data, promote the partition source and then delete. They cover int to long and float to double with a v2 merge-on-read Delete, and float to double equality deletes on v2 and v3, including a delete in partition 4.6 that must not remove a row in 4.5. They fail on main and pass with the change. Two existing unit cases that pinned the old cross-width behavior now expect a match, and I added cases for differing values, NaN and signed zero across widths, and int vs double. `go build ./...`, `go vet ./...`, `go test ./table/...` and golangci-lint on `table` pass. Badal, you said you'd pick up #2610. The fix fell out of this change as one line in `equality_delete_index.go`, so I included it, but I'm happy to drop it and its tests if you'd rather take it. This doesn't overlap textually with #2600; whichever lands second keeps a redundant int widening that can be trimmed later. Reading old manifests with the current partition type, as Java does, remains the proper long-term fix. Closes #2605 AI-assisted, reviewed by a human. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
