chris snow created FLINK-8939:
---------------------------------
Summary: Provide better support for saving streaming data to s3
Key: FLINK-8939
URL: https://issues.apache.org/jira/browse/FLINK-8939
Project: Flink
Issue Type: Improvement
Components: DataStream API
Reporter: chris snow
Attachments: 18652BC0-DD67-42C3-9A33-12F7BC10F9F3.png
Flink seems to struggle saving data to s3 due to the lack of a truncate method,
and in my test this resulted in lots of files with a .valid-length suffix
I’m using a bucketing sink:
{code:java}
return new BucketingSink<Tuple2<String, Object>>(path)
.setWriter(writer)
.setBucketer(new DateTimeBucketer<Tuple2<String, Object>>(formatString));{code}
!18652BC0-DD67-42C3-9A33-12F7BC10F9F3.png!
See also, the discussion in the comments here:
https://issues.apache.org/jira/browse/FLINK-8543?filter=-2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
