[
https://issues.apache.org/jira/browse/FLINK-41328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137825#comment-18137825
]
sepuri sai krishna commented on FLINK-41328:
--------------------------------------------
Two more clauses do the same thing, with the same three-row table as the
description.
Deduplication, where the three rows are one partition so one row is expected:
{code:sql}
SELECT tag FROM (
SELECT tag, ROW_NUMBER() OVER (PARTITION BY m ORDER BY ord) AS rn FROM t
) WHERE rn = 1; -- two rows
{code}
Top-N with {{rn <= 2}} expects two rows from that one partition and returns
three. Partitioning by
{{m['a']}} instead of {{m}} returns the expected single row in both cases.
Batch rejects both with
the same "cannot be ordered" error as the queries in the description.
For the record, {{MULTISET}} does not behave this way. Two multisets built by
{{COLLECT}} with their
elements inserted in different orders group together correctly in streaming, so
I have no wrong
result to show there, although batch rejects a {{MULTISET}} grouping key with
that same error.
I also checked the sink upsert materializer with a MAP column and found nothing
wrong: driven with
real retractions it matches the row and emits the delete, because the stored
row and its retraction
are produced by the same operator and so share a layout.
> GROUP BY, DISTINCT, UNION and a join on a MAP column split one value into
> several in streaming mode
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-41328
> URL: https://issues.apache.org/jira/browse/FLINK-41328
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 2.4.0
> Reporter: sepuri sai krishna
> Priority: Major
>
> A {{MAP}} column used as a grouping, {{DISTINCT}}, set operation or join key
> splits one value into
> several in streaming mode.
> {{t}} is a filesystem CSV table with three rows, {{a,1,b,2}} and {{b,2,a,1}}
> and {{a,1,b,2}}, and
> {{m}} is {{MAP[k1,v1,k2,v2]}}. All three rows hold the same map, key {{a}} to
> 1 and key {{b}} to 2,
> the middle one with its entries written in the other order.
> {code:sql}
> SELECT COUNT(*) FROM t GROUP BY m; -- two groups, counts 2 and 1, not one
> group of 3
> SELECT DISTINCT m FROM t; -- two rows, both printing {a=1, b=2}
> {code}
> {{=}} between the two orders returns TRUE. {{m['a']}}, {{m['b']}} and
> {{CARDINALITY(m)}} return
> {{1}}, {{2}} and {{2}} on all three rows, and grouping by {{m['a']}} gives
> the expected single group
> of 3. {{UNION}} of the two orders returns both rows, {{INTERSECT}} returns
> none, and an equi-join on
> {{m}} returns none.
> Batch mode rejects every one of those queries with "Type 'MAP<STRING, INT>
> NOT NULL' cannot be
> ordered, so it cannot be used as a key for sorting, grouping, or joining",
> except the join, which
> returns the correct rows. Streaming accepts them all and answers differently,
> at parallelism 1 and 4
> alike.
> Reproduced on master over a table source, not literals.
> Not FLINK-40945 or FLINK-41013: no FLOAT, DOUBLE or -0.0 here, and the result
> does not depend on
> parallelism.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
