Hi Kostas, Nice to see these features merged to the release-1.6 branch. Supporting Parque format in StreamingFileSink really makes me eager to apply it to some use cases in our company, but I'm still wondering if the current StreamingFileSink works well on S3 or I should wait util [FLINK-9752] done.
Anyway, I would still like to see this happen, because it let me have more alternative to upload data with parquet format. Thanks for your great efforts on this. Best Regards, Tony Wei https://issues.apache.org/jira/browse/FLINK-9752 2018-07-21 4:24 GMT+08:00 Kostas Kloudas <[email protected]>: > Hi all, > > We just merged on the master [FLINK-9753] (commit > 66b1f854a0250bdd048808d40f93aa2990476841) > and [FLINK-9903] (commit b56c75ca375049b1d2c80d2d0945ae1ae04eb39e). > > These two commits introduce: > 1) a big refactoring to the new StreamingFileSink which allows it to > support bulk formats in general (FLINK-9753) and more expressive rolling > and bucketing strategies, and > 2) support for Parque, based on the previous refactoring. > > Both these features were pretty popular in the mailing list. > > Given that these 2 features are self contained, as they mainly affect a > connector, we would like to cherry pick these commits to the release-1.6 > branch. But for this, we would like to have the OK from the community. > > Please let me know what you think, > Kostas > > https://issues.apache.org/jira/browse/FLINK-9903 < > https://issues.apache.org/jira/browse/FLINK-9903> > https://issues.apache.org/jira/browse/FLINK-9753 < > https://issues.apache.org/jira/browse/FLINK-9753>
