igalshilman commented on a change in pull request #7161:
[FLINK-10963][fs-connector, s3] Cleanup tmp S3 objects uploaded as backups of
in-progress files.
URL: https://github.com/apache/flink/pull/7161#discussion_r237084141
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/filesystem/Bucket.java
##########
@@ -269,6 +288,29 @@ void onSuccessfulCompletionOfCheckpoint(long
checkpointId) throws IOException {
it.remove();
}
}
+
+ cleanupOutdatedResumables(checkpointId);
+ }
+
+ private void cleanupOutdatedResumables(long checkpointId) throws
IOException {
+ Iterator<Map.Entry<Long, RecoverableWriter.ResumeRecoverable>>
it =
Review comment:
If this is the place that you were referring earlier for optimization, that
I'd also suggest to consider
[TreeMap#headMap](https://docs.oracle.com/javase/8/docs/api/java/util/TreeMap.html#headMap-K-boolean-)
instead of HashMap, but I suspect that in reality the number of elements would
be relatively low, wouldn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
