I have a scenario where multiple small files need to be written on S3. I'm using TwoPhaseCommit sink since I have a specific scenario where I can't use StreamingFileSink.
I've notice that because the way the S3 write is done (sequencially), the checkpoint is timining out (10 minutes), because it takes too much time to write multiple files in S3. I've search for a bit and found this documentation, https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/asyncio.html Should this be the best way to try to write multiple files in S3 to not wait for one file to be completed, in order to write the next one ? Thanks!
