Myasuka commented on a change in pull request #18391:
URL: https://github.com/apache/flink/pull/18391#discussion_r794735029
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogStateBackend.java
##########
@@ -271,7 +273,7 @@ public StateBackend configure(ReadableConfig config,
ClassLoader classLoader)
keyedStateHandle instanceof
ChangelogStateBackendHandle
? (ChangelogStateBackendHandle)
keyedStateHandle
: new ChangelogStateBackendHandleImpl(
-
singletonList(keyedStateHandle),
+
singletonMap(UUID.randomUUID(), keyedStateHandle),
Review comment:
As discussed offline, the two cases which switch jobs from not enable
change-log to enabled:
> When up-scaling, the same handle (file) may get multipiple keys; and
therefore can be discarded at some point while still in use
If we could introduce `UUID getIdentifier()` to `KeyedStateHandle`, that
could resolve our problems.
> The original checkpoint does not register it's private state at all
(CompletedCheckpoint.registerSharedStatesAfterRestored); so the original
problem remains for the migration case
I think this is only true for `CLAIM` mode. `LEGACY` and `NO_CLAIM` mode
would set the restored checkpoint property `discardOnSubsumed` as `false`,
which means it will not discard the restored checkpoint, thus the private state
would not be discarded then.
Since `CLAIM` mode is not default mode, this problem is not so serious. And
I create the ticket FLINK-25872 to track this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
