LuciferYang opened a new pull request, #11110: URL: https://github.com/apache/paimon/pull/11110
### Purpose fix #11109 - The generated `RecordEqualiser` compared two `BinaryRow`s, or two `BinaryArray`s, by bytes whenever the compared type had no FLOAT/DOUBLE. Maps and multisets inside those values were then compared by entry order, while the field-wise path matches entries by key. The same `ARRAY<MAP<...>>` value could compare equal as generic arrays and unequal as binary arrays. - `supportsBinaryEquals` now also returns false for types holding MAP or MULTISET at any depth, for both the row shortcut in `EqualiserCodeGenerator` and the array shortcut in `generateArrayComparison`. - For those types, and for FLOAT/DOUBLE since #8534, the byte comparison is kept as a fast path that only answers equal: if both sides are binary and their bytes match, the result is equal; otherwise the fields are compared. Equal bytes always mean equal values, so unchanged values still cost one byte comparison. - Behavior change: maps nested in binary arrays or rows now match by key in every representation, as top-level MAP fields and `row-deduplicate-ignore-fields` already did. For distinct `collect`, maps that differ only in entry order are one element, so with `count-limit` they no longer take two slots. `collect` `count-limit` (#10540) is not released yet. ### Tests - `EqualiserCodeGeneratorTest#testNestedMapEqualiserIgnoresEntryOrderInEveryRepresentation`: `ARRAY<MAP>`, `ARRAY<ARRAY<MAP>>`, `ROW<MAP>`, `ARRAY<ROW<MAP>>`, `MULTISET` and `ARRAY<MULTISET>` with reordered entries are equal, and with a changed value are not, as generic rows, mixed rows, `BinaryRow`s, and binary field values inside generic rows. - `testIntKeyMapEqualiserIgnoresEntryOrder` and `testBinaryKeyMapEqualiser` now check reordered maps in every representation instead of generic rows only. - `FieldAggregatorTest#testFieldCollectAggDistinctNestedMapsInBinaryForm`: distinct `collect` with `count-limit=2` on `ARRAY<ARRAY<MAP<INT, INT>>>`, fed binary arrays as the write buffer does, keeps the first map and the third distinct one. - All of the above fail on master. Generated-by: Claude Opus -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
