LuciferYang opened a new pull request, #11110:
URL: https://github.com/apache/paimon/pull/11110

   ### Purpose
   
   fix #11109
   
   - The generated `RecordEqualiser` compared two `BinaryRow`s, or two 
`BinaryArray`s, by bytes whenever the compared type had no FLOAT/DOUBLE. Maps 
and multisets inside those values were then compared by entry order, while the 
field-wise path matches entries by key. The same `ARRAY<MAP<...>>` value could 
compare equal as generic arrays and unequal as binary arrays.
   - `supportsBinaryEquals` now also returns false for types holding MAP or 
MULTISET at any depth, for both the row shortcut in `EqualiserCodeGenerator` 
and the array shortcut in `generateArrayComparison`.
   - For those types, and for FLOAT/DOUBLE since #8534, the byte comparison is 
kept as a fast path that only answers equal: if both sides are binary and their 
bytes match, the result is equal; otherwise the fields are compared. Equal 
bytes always mean equal values, so unchanged values still cost one byte 
comparison.
   - Behavior change: maps nested in binary arrays or rows now match by key in 
every representation, as top-level MAP fields and 
`row-deduplicate-ignore-fields` already did. For distinct `collect`, maps that 
differ only in entry order are one element, so with `count-limit` they no 
longer take two slots. `collect` `count-limit` (#10540) is not released yet.
   
   ### Tests
   
   - 
`EqualiserCodeGeneratorTest#testNestedMapEqualiserIgnoresEntryOrderInEveryRepresentation`:
 `ARRAY<MAP>`, `ARRAY<ARRAY<MAP>>`, `ROW<MAP>`, `ARRAY<ROW<MAP>>`, `MULTISET` 
and `ARRAY<MULTISET>` with reordered entries are equal, and with a changed 
value are not, as generic rows, mixed rows, `BinaryRow`s, and binary field 
values inside generic rows.
   - `testIntKeyMapEqualiserIgnoresEntryOrder` and `testBinaryKeyMapEqualiser` 
now check reordered maps in every representation instead of generic rows only.
   - `FieldAggregatorTest#testFieldCollectAggDistinctNestedMapsInBinaryForm`: 
distinct `collect` with `count-limit=2` on `ARRAY<ARRAY<MAP<INT, INT>>>`, fed 
binary arrays as the write buffer does, keeps the first map and the third 
distinct one.
   - All of the above fail on master.
   
   Generated-by: Claude Opus
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to