[ 
https://issues.apache.org/jira/browse/FLINK-40997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18146705#comment-18146705
 ] 

Mate Czagany commented on FLINK-40997:
--------------------------------------

Hi [~gustavodemorais] - I think it should be on 1.20 LTS. The latest minor 
release is 2.3, so I think it also makes sense to backport it to 2.2, as that 
will lose support soon when 2.4 is released. So unless I misunderstand 
something, we are well within the update policy. 

Now looking at the ticket, you are right that it was not correctly marked as 
"Critical", which the update policy asks for when backporting. I've fixed that. 
Considering that this bug can break upgrades, it should be a critical priority 
either way.

> Regular join picks a random state key among equal-length upsert keys
> --------------------------------------------------------------------
>
>                 Key: FLINK-40997
>                 URL: https://issues.apache.org/jira/browse/FLINK-40997
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Planner
>    Affects Versions: 2.0.2, 2.1.3, 2.2.2, 1.20.6, 2.3.1
>            Reporter: Mate Czagany
>            Assignee: Mate Czagany
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 2.4.0, 1.20.7, 2.2.3, 2.3.2
>
>
> The same SQL can produce different {{leftUpsertKeys}} / {{rightUpsertKeys}} 
> order on {{StreamExecJoin}} nodes.
> If the input has two or more upsert keys that tie for the fewest columns, and 
> none of them is contained in that side's join key, otherwise 
> {{JoinKeyContainsUniqueKey}} would be used.
> Example:
> {code:sql}
> CREATE TABLE items (id INT, score INT, brand STRING, PRIMARY KEY (id) NOT 
> ENFORCED) WITH (...);
> CREATE TABLE z (k STRING, z STRING) WITH (...);
> INSERT INTO snk
> SELECT t.id, t.rn, z.z
> FROM (SELECT * FROM (SELECT *, ROW_NUMBER() OVER (ORDER BY score DESC) AS rn 
> FROM items)
>       WHERE rn <= 3) t
> JOIN z ON t.brand = z.k;
> {code}
> The left input of the join is {{{}(id, brand, rn){}}}. It has two upsert 
> keys, {{id}} and {{{}rn{}}}, and the join key {{brand}} contains neither.
> After compiling this several times, {{leftUpsertKeys}} of the join switches 
> between [[0],[2]] (state keyed by {{{}id{}}}) and [[2],[0]] (keyed by 
> {{{}rn{}}}). This can break state restore using a previous snapshot with a 
> {{{}StateMigrationException{}}}.
> With id {{BIGINT}} instead of {{{}INT{}}}, both keys are {{{}BIGINT NOT 
> NULL{}}}, a state restore can succeed. Retractions then look up the wrong 
> key, and stale rows stay in the join state and keep joining. No error is 
> reported.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to