sepuri sai krishna created FLINK-41328:
------------------------------------------
Summary: GROUP BY, DISTINCT, UNION and a join on a MAP column
split one value into several in streaming mode
Key: FLINK-41328
URL: https://issues.apache.org/jira/browse/FLINK-41328
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Affects Versions: 2.4.0
Reporter: sepuri sai krishna
A {{MAP}} column used as a grouping, {{DISTINCT}}, set operation or join key
splits one value into
several in streaming mode.
{{t}} is a filesystem CSV table with three rows, {{a,1,b,2}} and {{b,2,a,1}}
and {{a,1,b,2}}, and
{{m}} is {{MAP[k1,v1,k2,v2]}}. All three rows hold the same map, key {{a}} to 1
and key {{b}} to 2,
the middle one with its entries written in the other order.
{code:sql}
SELECT COUNT(*) FROM t GROUP BY m; -- two groups, counts 2 and 1, not one
group of 3
SELECT DISTINCT m FROM t; -- two rows, both printing {a=1, b=2}
{code}
{{=}} between the two orders returns TRUE. {{m['a']}}, {{m['b']}} and
{{CARDINALITY(m)}} return
{{1}}, {{2}} and {{2}} on all three rows, and grouping by {{m['a']}} gives the
expected single group
of 3. {{UNION}} of the two orders returns both rows, {{INTERSECT}} returns
none, and an equi-join on
{{m}} returns none.
Batch mode rejects every one of those queries with "Type 'MAP<STRING, INT> NOT
NULL' cannot be
ordered, so it cannot be used as a key for sorting, grouping, or joining",
except the join, which
returns the correct rows. Streaming accepts them all and answers differently,
at parallelism 1 and 4
alike.
Reproduced on master over a table source, not literals.
Not FLINK-40945 or FLINK-41013: no FLOAT, DOUBLE or -0.0 here, and the result
does not depend on
parallelism.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
