Github user zhangminglei commented on a diff in the pull request:
https://github.com/apache/flink/pull/6375#discussion_r203995868
--- Diff:
flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java
---
@@ -722,9 +727,31 @@ public void notifyCheckpointComplete(long
checkpointId) throws Exception {
}
}
}
+ isBucketReady(partitionPaths);
}
}
+ @Override
+ public boolean isBucketReady(Set<Path> bucketPathes) {
--- End diff --
If we finish the for loop and haven't returned true yet, then return false.
I think this is reasonable. If a file end with pending or inProgress, This
means that the file is not ready. and we can not use it.
---
