chris snow created FLINK-8939:

             Summary: Provide better support for saving streaming data to s3
                 Key: FLINK-8939
             Project: Flink
          Issue Type: Improvement
          Components: DataStream API
            Reporter: chris snow
         Attachments: 18652BC0-DD67-42C3-9A33-12F7BC10F9F3.png

Flink seems to struggle saving data to s3 due to the lack of a truncate method, 
and in my test this resulted in lots of files with a .valid-length suffix

I’m using a bucketing sink:
return new BucketingSink<Tuple2<String, Object>>(path)
.setBucketer(new DateTimeBucketer<Tuple2<String, Object>>(formatString));{code}


See also, the discussion in the comments here:

This message was sent by Atlassian JIRA

Reply via email to