sepuri sai krishna created FLINK-41532:
------------------------------------------

             Summary: Union list state created with a State V2 descriptor is 
restored as split state
                 Key: FLINK-41532
                 URL: https://issues.apache.org/jira/browse/FLINK-41532
             Project: Flink
          Issue Type: Bug
          Components: Runtime / State Backends
    Affects Versions: 2.3.0, 2.2.0, 2.1.0, 2.0.0
            Reporter: sepuri sai krishna


Reported by Artemiy Lazarev on the dev mailing list: 
https://lists.apache.org/thread/tpmtp7mt51dfdo442wl430f1p21rlo4d

A list state created with {{OperatorStateStore#getUnionListState}} and a State 
V2 {{ListStateDescriptor}} is stored as {{SPLIT_DISTRIBUTE}} instead of 
{{UNION}}. After a restore, the entries are split across subtasks. The same 
state created with the V1 descriptor gives every subtask all of them.

Savepoint taken at parallelism 2, each subtask stores two entries, then 
restored:

||restore parallelism||V1 union list state||V2 union list state||
|2|every subtask gets all 4 entries|2 and 2|
|3|every subtask gets all 4 entries|1, 1 and 2|

This also shows up in SQL. With {{table.exec.async-state.enabled}} set to true, 
a one-phase TUMBLE window aggregate stores its watermark state as 
{{SPLIT_DISTRIBUTE}} in the savepoint. With the option off, the same query 
stores it as {{UNION}}.

Checked on 2.3.0 and on master, JDK 17.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to