rkhachatryan commented on a change in pull request #18391:
URL: https://github.com/apache/flink/pull/18391#discussion_r794390136
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogStateBackend.java
##########
@@ -271,7 +273,7 @@ public StateBackend configure(ReadableConfig config,
ClassLoader classLoader)
keyedStateHandle instanceof
ChangelogStateBackendHandle
? (ChangelogStateBackendHandle)
keyedStateHandle
: new ChangelogStateBackendHandleImpl(
-
singletonList(keyedStateHandle),
+
singletonMap(UUID.randomUUID(), keyedStateHandle),
Review comment:
I see two potential problems here:
1. When up-scaling, the same handle (file) may get multipiple keys; and
therefore can be discarded at some point while still in use
1. The original checkpoint does not register it's private state at all
(`CompletedCheckpoint.registerSharedStatesAfterRestored`); so the original
problem remains for the migration case
Am I missing something?
The only solution I see is to move this logic (`new
ChangelogStateBackendHandleImpl`) to the JM.
However, this doesn't sound right because the logic is backend-specific.
edit: here, I'm talking about recovering with changelog from a checkpoint
which was made without changelog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
