[
https://issues.apache.org/jira/browse/FLINK-41328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137795#comment-18137795
]
sepuri sai krishna commented on FLINK-41328:
--------------------------------------------
I have a fix and tests for this locally. Please assign it to me and I will open
a PR.
> GROUP BY, DISTINCT, UNION and a join on a MAP column split one value into
> several in streaming mode
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-41328
> URL: https://issues.apache.org/jira/browse/FLINK-41328
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 2.4.0
> Reporter: sepuri sai krishna
> Priority: Major
>
> A {{MAP}} column used as a grouping, {{DISTINCT}}, set operation or join key
> splits one value into
> several in streaming mode.
> {{t}} is a filesystem CSV table with three rows, {{a,1,b,2}} and {{b,2,a,1}}
> and {{a,1,b,2}}, and
> {{m}} is {{MAP[k1,v1,k2,v2]}}. All three rows hold the same map, key {{a}} to
> 1 and key {{b}} to 2,
> the middle one with its entries written in the other order.
> {code:sql}
> SELECT COUNT(*) FROM t GROUP BY m; -- two groups, counts 2 and 1, not one
> group of 3
> SELECT DISTINCT m FROM t; -- two rows, both printing {a=1, b=2}
> {code}
> {{=}} between the two orders returns TRUE. {{m['a']}}, {{m['b']}} and
> {{CARDINALITY(m)}} return
> {{1}}, {{2}} and {{2}} on all three rows, and grouping by {{m['a']}} gives
> the expected single group
> of 3. {{UNION}} of the two orders returns both rows, {{INTERSECT}} returns
> none, and an equi-join on
> {{m}} returns none.
> Batch mode rejects every one of those queries with "Type 'MAP<STRING, INT>
> NOT NULL' cannot be
> ordered, so it cannot be used as a key for sorting, grouping, or joining",
> except the join, which
> returns the correct rows. Streaming accepts them all and answers differently,
> at parallelism 1 and 4
> alike.
> Reproduced on master over a table source, not literals.
> Not FLINK-40945 or FLINK-41013: no FLOAT, DOUBLE or -0.0 here, and the result
> does not depend on
> parallelism.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
