curcur commented on a change in pull request #15420:
URL: https://github.com/apache/flink/pull/15420#discussion_r654911814
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogStateBackend.java
##########
@@ -96,10 +99,11 @@ public ChangelogStateBackend(StateBackend stateBackend) {
kvStateRegistry,
ttlTimeProvider,
metricGroup,
- stateHandles,
+ extractMaterializedState(stateHandles),
cancelStreamRegistry);
// todo: FLINK-21804 get from Environment.getTaskStateManager
InMemoryStateChangelogStorage changelogWriterFactory = new
InMemoryStateChangelogStorage();
+ // todo: apply state changes from non-materialized part of stateHandles
Review comment:
Should this be included as part of the recovery PR, or could you please
open a ticket for this.
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogStateBackend.java
##########
@@ -96,10 +99,11 @@ public ChangelogStateBackend(StateBackend stateBackend) {
kvStateRegistry,
ttlTimeProvider,
metricGroup,
- stateHandles,
+ extractMaterializedState(stateHandles),
cancelStreamRegistry);
// todo: FLINK-21804 get from Environment.getTaskStateManager
InMemoryStateChangelogStorage changelogWriterFactory = new
InMemoryStateChangelogStorage();
+ // todo: apply state changes from non-materialized part of stateHandles
Review comment:
Also, after recovery
The state of the changelog state backend includes two parts if I understand
correctly:
1). The delegated backend state, and
2). Changelog State
=====
1). is recovered here + TODO
2). What do we do for 2)? Do we need to recover at least the last seq no?
Maybe it is in the recovery part; Put a marker here.
If we do not, and always create a new log starting from 0; then how
materialized data is used to identify what part of the log has already been
materialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
