nssalian opened a new pull request, #2653:
URL: https://github.com/apache/iceberg-go/pull/2653

   ## What
   
   Widen float32 partition values to float64 when building the spec-aware key 
that equality-delete cleanup uses to look up surviving data files.
   
   ## Why
   
   After an identity partition source is promoted from float to double, data 
files written before the promotion decode the partition value as float32 and 
equality deletes written after it carry float64. The survivor key printed 
`1000=1.1` while the delete key printed `1000=1.100000023841858`, so 
`DecideDeadEqualityDeletesWithSpecs` found no surviving file and classified a 
delete that still applies as dead. Compaction then drops that delete and the 
rows it covered come back: the new 
`TestCompactionKeepsEqualityDeleteAfterFloatPartitionSourcePromotion` compacts 
two unrelated files and reads ids `[1 2 3 4]` on main instead of `[2 3 4]`.
   
   The opposite case was wrong too: a float32 `1.1` data file and a double 
`1.1` delete printed the same key and matched, although they are different 
partitions after the promotion. That case only kept a dead delete around.
   
   This is the same root cause as #2605 and #2610 (fixed for scan delete 
matching in #2620) and the partitions table fix in #2637, in a code path 
neither of them touches.
   
   ## Implementation
   
   - Widening is limited to the unexported spec-aware key in `SurvivorSurvey`. 
`PartMinSeq` and `appendPartitionTuple` keep their `fmt`-compatible output, so 
the exported survey layout and its keys do not change.
   - int to long needed nothing: both widths already format the same.
   
   ## Tests
   
   - `TestCompactionKeepsEqualityDeleteAfterFloatPartitionSourcePromotion` (v2 
and v3): end to end through a rewrite. Fails on main.
   - Two promotion cases in `TestDecideDeadEqualityDeletesWithSpecs_Predicate`: 
a survivor written before the promotion keeps the delete alive, and a different 
double value no longer matches. Both fail on main.
   - `commitPartitionedEqualityDeletes` in 
`partition_promotion_deletes_test.go` now holds the equality-delete commit both 
promotion tests use.
   - A never-promoted float32 case, which fails if only the survivor side were 
widened, and `float32(1.1)` in `TestAppendPartitionTupleMatchesFmt`, which 
fails if the exported keys were widened.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to