LuciferYang opened a new issue, #11109: URL: https://github.com/apache/paimon/issues/11109
**Search before asking** - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. **Paimon version** master @ eb0c10562e **Compute Engine** Engine-agnostic Java code generation. **Minimal reproduce step** Generate a `RecordEqualiser` for `ARRAY<MAP<INT, INT>>` and compare `[{1:10, 2:20}]` with `[{2:20, 1:10}]`. Two `GenericArray` values compare equal, because the map comparison matches entries by key. Two `BinaryArray` values compare unequal, because `generateArrayComparison` compares them by bytes when the element type has no FLOAT/DOUBLE. The `BinaryRow` fast path in `EqualiserCodeGenerator` does the same for a top-level `MAP` or `MULTISET` field. A user-visible case: a `collect` field with `distinct=true` and `count-limit=2` on `ARRAY<ARRAY<MAP<INT, INT>>>`, written with rows `[{1:10, 2:20}]`, `[{2:20, 1:10}]`, `[{3:30}]` for one key. The write buffer passes binary arrays to `FieldCollectAgg`, so the first two count as different elements and the result is the first two rows. Merging the same rows as generic arrays keeps the first and the third. **What doesn't meet your expectations?** The result of the generated equaliser should not depend on whether a value is binary or generic. Maps should match by key in every representation, as they already do for top-level `MAP` fields read from columnar or generic rows. **Anything else?** #8534 disabled the same byte shortcuts for FLOAT/DOUBLE for the same reason. `count-limit` for `collect` (#10540) is not in a release yet. This was found while reviewing apache/paimon-rust#1352, which ports `count-limit` and has to pick one of the two behaviors. **Are you willing to submit a PR?** - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
