[
https://issues.apache.org/jira/browse/FLINK-31593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713535#comment-17713535
]
Yanfei Lei commented on FLINK-31593:
------------------------------------
[~mapohl] Sorry for the late reply, after reproducing this issue locally, I
think the root cause is that `
StatefulJobSavepointMigrationITCase` partially moves the snapshot files to a
new
[directory|https://github.com/apache/flink/blob/master/flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/SnapshotMigrationTestBase.java#L337-L342].
All state backends in `StatefulJobSavepointMigrationITCase` are
non-incremental, all files are placed in chk-x folder. But the files of
changelog state backend are not completely placed under chk-x, some files are
placed under taskowned folder.
{code:java}
├── chk-2
│ ├── 5487d0fd-a361-4085-8ee0-7364ffd4511a
│ ├── _metadata
│ └── d3596cf7-3c6e-4081-b37b-f5a3e1a40086
├── shared
└── taskowned
├── 01aefc31-8ee1-41a8-9cd3-a94ccf85052f
├── 02bf09d3-73db-4c45-b6a1-15987659e3e6
├── 0c456b9b-9f90-4696-a2be-16e5938358ae {code}
This also explains why this issue didn't show up earlier:
1. If the version <= 1.15, changelog state backend is disabled.
2. If the version >= 1.16, change state backend is randomly turned on, when the
changelog is turned off, this issue would not be triggered.
So I have two questions:
# Whether the incremental rocksdb state backend should be tested here?
# Do we need to change the move function to support testing of changelog state
backend ?
> Update reference data for Migration Tests
> -----------------------------------------
>
> Key: FLINK-31593
> URL: https://issues.apache.org/jira/browse/FLINK-31593
> Project: Flink
> Issue Type: Sub-task
> Reporter: Matthias Pohl
> Assignee: Matthias Pohl
> Priority: Major
> Labels: pull-request-available
> Attachments:
> FLINK-31593.StatefulJobSavepointMigrationITCase.create_snapshot.log,
> FLINK-31593.StatefulJobSavepointMigrationITCase.verify_snapshot.log
>
>
> # Update {{CURRENT_VERSION in TypeSerializerUpgradeTestBase}} with the new
> version. This will likely fail some tests because snapshots are missing for
> that version. Generate them, for example in
> {{TypeSerializerUpgradeTestBase.}}
> # (major/minor only) Update migration tests in master to cover migration
> from new version: (search for usages of FlinkV{{{}ersion{}}})
> ** AbstractOperatorRestoreTestBase
> ** CEPMigrationTest
> ** BucketingSinkMigrationTest
> ** FlinkKafkaConsumerBaseMigrationTest
> ** ContinuousFileProcessingMigrationTest
> ** WindowOperatorMigrationTest
> ** StatefulJobSavepointMigrationITCase
> ** StatefulJobWBroadcastStateMigrationITCase
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
