danny0405 commented on a change in pull request #3928:
URL: https://github.com/apache/hudi/pull/3928#discussion_r745531613
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java
##########
@@ -221,13 +222,42 @@ public void notifyCheckpointComplete(long checkpointId) {
// the stream write task snapshot and flush the data buffer
synchronously in sequence,
// so a successful checkpoint subsumes the old one(follows the
checkpoint subsuming contract)
final boolean committed = commitInstant(this.instant, checkpointId);
+
+ if (tableState.scheduleCompaction) {
+ // if async compaction is on, schedule the compaction
+ CompactionUtil.scheduleCompaction(metaClient, writeClient,
committed);
+ }
+
+ if (committed) {
+ // start new instant.
+ startInstant(0L);
+ // sync Hive if is enabled
+ syncHiveIfEnabled();
+ }
+ }, "commits the instant %s", this.instant
+ );
+ }
+
+ @VisibleForTesting
+ public void notifyCheckpointComplete(long checkpointId, long milliseconds) {
+ executor.execute(
+ () -> {
+ // The executor thread inherits the classloader of the
#notifyCheckpointComplete
+ // caller, which is a AppClassLoader.
+
Thread.currentThread().setContextClassLoader(getClass().getClassLoader());
+ // for streaming mode, commits the ever received events anyway,
Review comment:
You should not add complex logic just for testing purpose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
