Dear community, this weeks community digest with updates on Flink 1.8.1 & Flink 1.9.0, another thread on our development process and a few API related threads. I still include older threads from time to time. Enjoy!
As always, please feel free to add additional updates and news to this thread! Flink Development =============== * [development-process] Ajoscha started a discussion on our FLIP process. In particular, he pointed out that we are currently not following our own rules, which require a "lazy majority" (3 binding +1s) to accept a FLIP. In reality, most FLIPs are "accepted" without a vote at all or with a "lazy approval". There has generally been a lot of positive feedback to enforce a "lazy majority" in the future. In addition, questions were raised on which changes require a FLIP, how large a FLIP can be and if we can use the voting process to scope FLIPs into releases. [1] * [releases] Jincheng has prepared a first release candidate for *Flink 1.8.1* to be voted on. Additional testing and votes are needed to move this forward. * [releases] The feature freeze for *Flink 1.9.0 *has been pushed by one week (5hth of July) to finish work on the Blink-based Table API Runner and fine-grained recovery. * [dataset-api] Vino Yang has proposed (FLIP-44) to support local aggregations in Flink by introducing a new concept of "local keyed state". There seems to be a need of further discussion. Specifically, the community has not yet agreed on the main purpose of the FLIP (local aggregations to mitigate effects of data skew vs introduction of local keyed state, which can also be used for the former) and consequentially there are also open questions regarding the API and implementation. [4] * [table-api] When a DynamicTable is created by a `flatAggregate` function it is currently not possible to connect it to an `UpsertStreamTableSink`, because after `flatAggregate` it is not clear, which unique keys to use for the upsert. Jincheng has started a discussion to mitigate this and provided a design document laying out different options to add the missing information on unique keys. [5] * [client-api] Jeff started a conversation a couple of weeks ago on improving Flink's Client API and better programmatic job and cluster control. This would make it easier for downstream projects (e.g. Zeppelin, Beam) to integrate with Flink. The initiative has genreally been well-received, but will require some additional upfront design discussions due to its complexity and the many Public APIs, which are involved. [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-META-FLIP-Sticking-or-not-to-a-strict-FLIP-voting-process-td29978.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-1-8-1-release-candidate-1-td29929.html [3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Feature-freeze-for-Apache-Flink-1-9-0-release-td29751.html [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Local-Aggregation-in-Flink-td29307.html [5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-Upsert-mode-for-Streaming-Non-window-FlatAggregate-tp29975.html [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-client-api-enhancement-for-downstream-project-td25844.html Notable Bugs =========== Nothing that came to my attention. So far focusing this on released versions only. Its worth mentioning that FLINK-11662 (covered in Weekly Community Update 2019/24 [7]) has been resolved. [7] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Weekly-Community-Update-2019-24-tp29630.html Events, Blog Posts, Misc ==================== * Jincheng is now part of the Apache Flink PMC. Congrats! [8] * There are a couple of meetups coming up in the coming week * 2019/07/01: *Paris *Apache Beam Meetup with a Flink talk by myself (Ververica) [9] * 2019/07/05: Apache Flink Meetup *Munich* with talks by Steffen Hausmann (AWS) and Michel David (Ryte) [10] [8] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jincheng-Sun-is-now-part-of-the-Flink-PMC-td29868i20.html#a29966 [9] https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/ [10] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/261282757/ Cheers, Konstantin (@snntrable) -- Konstantin Knauf | Solutions Architect +49 160 91394525 -- Data Artisans GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Data Artisans GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
