Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/5982
I think this fix here might not work for S3, because a rename() with the S3
file systems will actually trigger a copy (or even a download and upload), so
it is not a cheap operation.
The we can fix this by adding a `create(...)` method (or mode) to the
FileSystem API that does not publish data in the file until `close()` is
called. For hdfs://, file://, this would be using a temp file with renaming,
for S3 we don't write to a temp file, because S3 makes the file only visible on
close() anyways.
---
