simonsssu commented on a change in pull request #2593:
URL: https://github.com/apache/hudi/pull/2593#discussion_r581620232
##########
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##########
@@ -208,12 +210,32 @@ public void bootstrap(Option<Map<String, String>>
extraMetadata) {
@Override
public void commitCompaction(String compactionInstantTime, List<WriteStatus>
writeStatuses, Option<Map<String, String>> extraMetadata) throws IOException {
- throw new HoodieNotSupportedException("Compaction is not supported yet");
+ HoodieFlinkTable<T> table = HoodieFlinkTable.create(config,
(HoodieFlinkEngineContext) context);
+ HoodieCommitMetadata metadata =
FlinkCompactHelpers.newInstance().createCompactionMetadata(
+ table, compactionInstantTime, writeStatuses, config.getSchema());
+ extraMetadata.ifPresent(m -> m.forEach(metadata::addMetadata));
+ completeCompaction(metadata, writeStatuses, table, compactionInstantTime);
}
Review comment:
Thanks danny, in that test case, seems CompactFunction is still a part
of flink operator, how to understand the async here ?
##########
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##########
@@ -208,12 +210,32 @@ public void bootstrap(Option<Map<String, String>>
extraMetadata) {
@Override
public void commitCompaction(String compactionInstantTime, List<WriteStatus>
writeStatuses, Option<Map<String, String>> extraMetadata) throws IOException {
- throw new HoodieNotSupportedException("Compaction is not supported yet");
+ HoodieFlinkTable<T> table = HoodieFlinkTable.create(config,
(HoodieFlinkEngineContext) context);
+ HoodieCommitMetadata metadata =
FlinkCompactHelpers.newInstance().createCompactionMetadata(
+ table, compactionInstantTime, writeStatuses, config.getSchema());
+ extraMetadata.ifPresent(m -> m.forEach(metadata::addMetadata));
+ completeCompaction(metadata, writeStatuses, table, compactionInstantTime);
}
Review comment:
I understand the planscheduler to generate plan, and then
commitCompaction will read the plan to see if it should commit, but I'm
confused to connect all these together, when to trigger the real compact ? if
we use CompactFunction , how to understand the async here ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
