Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/5601
The main issue was actually described in FLINK-8801:
According to
https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel:
> Amazon S3 provides read-after-write consistency for PUTS of new objects
in your S3 bucket in all regions with one caveat. The caveat is that if you
make a HEAD or GET request to the key name (to find if the object exists)
before creating the object, Amazon S3 provides eventual consistency for
read-after-write.
Some S3 file system implementations may actually execute such a request for
the about-to-write object and thus the read-after-write is only eventually
consistent.
---
