sepuri sai krishna created FLINK-41532:
------------------------------------------
Summary: Union list state created with a State V2 descriptor is
restored as split state
Key: FLINK-41532
URL: https://issues.apache.org/jira/browse/FLINK-41532
Project: Flink
Issue Type: Bug
Components: Runtime / State Backends
Affects Versions: 2.3.0, 2.2.0, 2.1.0, 2.0.0
Reporter: sepuri sai krishna
Reported by Artemiy Lazarev on the dev mailing list:
https://lists.apache.org/thread/tpmtp7mt51dfdo442wl430f1p21rlo4d
A list state created with {{OperatorStateStore#getUnionListState}} and a State
V2 {{ListStateDescriptor}} is stored as {{SPLIT_DISTRIBUTE}} instead of
{{UNION}}. After a restore, the entries are split across subtasks. The same
state created with the V1 descriptor gives every subtask all of them.
Savepoint taken at parallelism 2, each subtask stores two entries, then
restored:
||restore parallelism||V1 union list state||V2 union list state||
|2|every subtask gets all 4 entries|2 and 2|
|3|every subtask gets all 4 entries|1, 1 and 2|
This also shows up in SQL. With {{table.exec.async-state.enabled}} set to true,
a one-phase TUMBLE window aggregate stores its watermark state as
{{SPLIT_DISTRIBUTE}} in the savepoint. With the option off, the same query
stores it as {{UNION}}.
Checked on 2.3.0 and on master, JDK 17.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)