rkhachatryan commented on a change in pull request #17462:
URL: https://github.com/apache/flink/pull/17462#discussion_r746869320
##########
File path: .idea/vcs.xml
##########
@@ -20,6 +20,5 @@
</component>
<component name="VcsDirectoryMappings">
<mapping directory="$PROJECT_DIR$" vcs="Git" />
- <mapping directory="$PROJECT_DIR$/docs/themes/book" vcs="Git" />
</component>
-</project>
+</project>
Review comment:
I guess this change shouldn't belong to PR, please remove it if so.
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/AbstractChangelogState.java
##########
@@ -90,6 +90,10 @@ public void setCurrentNamespace(N namespace) {
return
delegatedState.getStateIncrementalVisitor(recommendedMaxNumberOfReturnedRecords);
}
+ public void resetMetaDataWritten() {
Review comment:
1. add javadoc?
2. **nit**: a more abstract name, something like `onChangelogTruncated`? (so
that details aren't exposed, and it shouldn't be renamed if they change in the
future)
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogKeyedStateBackend.java
##########
@@ -600,8 +600,14 @@ private ChangelogSnapshotState completeRestore(
System.currentTimeMillis(),
streamFactory,
CHECKPOINT_OPTIONS),
- // TODO: add metadata to log FLINK-23170.
- upTo));
+ upTo);
+
+ // log metadata after materialization is triggered
+ keyValueStatesByName
+ .values()
+ .forEach(t -> ((AbstractChangelogState)
t).resetMetaDataWritten());
Review comment:
1. Please use `ChangelogState` instead of `AbstractChangelogState`
2. Please add a call for `priorityQueueStatesByName` (item above will force
to implement it for PQ states)
3. Although this doesn't happen very often, it blocks processing, so I'd
rather use for-loop
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/test/java/org/apache/flink/state/changelog/ChangelogStateBackendTestUtils.java
##########
@@ -158,16 +155,15 @@ public static void testMaterializedRestore(
keyedBackend.setCurrentKey(2);
state.update(new StateBackendTestBase.TestPojo("u2", 2));
- awaitMaterialization(keyedBackend, env.getMainMailboxExecutor());
+ // trigger materialization manually
+ periodicMaterializationManager.triggerMaterialization();
keyedBackend.setCurrentKey(2);
state.update(new StateBackendTestBase.TestPojo("u2", 22));
keyedBackend.setCurrentKey(3);
state.update(new StateBackendTestBase.TestPojo("u3", 3));
- awaitMaterialization(keyedBackend, env.getMainMailboxExecutor());
-
Review comment:
Could you explain this change? Shouldn't we make sure that
materialization is completed at this point?
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/test/java/org/apache/flink/state/changelog/ChangelogStateBackendTestUtils.java
##########
@@ -158,16 +155,15 @@ public static void testMaterializedRestore(
keyedBackend.setCurrentKey(2);
state.update(new StateBackendTestBase.TestPojo("u2", 2));
- awaitMaterialization(keyedBackend, env.getMainMailboxExecutor());
+ // trigger materialization manually
+ periodicMaterializationManager.triggerMaterialization();
Review comment:
The comment in code doesn't add much information IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
