danny0405 commented on a change in pull request #3928:
URL: https://github.com/apache/hudi/pull/3928#discussion_r743484893
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java
##########
@@ -221,11 +223,15 @@ public void notifyCheckpointComplete(long checkpointId) {
// the stream write task snapshot and flush the data buffer
synchronously in sequence,
// so a successful checkpoint subsumes the old one(follows the
checkpoint subsuming contract)
final boolean committed = commitInstant(this.instant, checkpointId);
+
+ if (tableState.scheduleCompaction) {
+ // if async compaction is on, schedule the compaction
+ metaClient.reloadActiveTimeline();
Review comment:
can we move these 3 lines into the
`CompactionUtil#scheduleCompaction(metaClient)` ?
##########
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java
##########
@@ -221,11 +223,15 @@ public void notifyCheckpointComplete(long checkpointId) {
// the stream write task snapshot and flush the data buffer
synchronously in sequence,
// so a successful checkpoint subsumes the old one(follows the
checkpoint subsuming contract)
final boolean committed = commitInstant(this.instant, checkpointId);
+
+ if (tableState.scheduleCompaction) {
+ // if async compaction is on, schedule the compaction
+ metaClient.reloadActiveTimeline();
+ String compactionInstantTime =
CompactionUtil.getCompactionInstantTime(metaClient);
+ writeClient.scheduleCompactionAtInstant(compactionInstantTime,
Option.empty());
+ }
+
if (committed) {
- if (tableState.scheduleCompaction) {
- // if async compaction is on, schedule the compaction
- writeClient.scheduleCompaction(Option.empty());
- }
Review comment:
can we also add a UT here, for both commits with / without data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
