Dear community, happy to share this week's community update. While slowly approaching the planned feature freeze for Flink 1.10, things have calmed down a bit on the dev mailing list. Stil there are couple of interesting topics to cover. This week's digest includes an update on the Flink 1.8.3 release, a proposal for a KafkaMetricsReporter, a FLIP to improve the configuration of Table API connectors and a bit more.
Flink Development ============== * [releases] With respect to the release of *Flink 1.8.3* the main questions seems to be whether to wait for flink-shaded 9.0. If not, a first release candidate can be expected within the next week. [1] * [releases] Chesnay proposes to release *flink-shaded 9.0* soon and is looking for someone to manage this release. [2] * [connectors] Stephan has shared a quick update on the progress of *FLIP-27 (New Source Interface)*: a first version will likely be available in 1.10, but it will probably take another release until connectors are migrated and things settle down a bit. [3] * [connectors, sql] Jark has started a FLIP discussion to* improve the properties format* of Table API/SQL connectors. It contains a whole list of smaller and larger improvements and it seems this will be targeted for the 1.11 release. [4] * [metrics] Gyula has started a discussion on contributing a *MetricsReporter* to write Flink's metrics to *Apache Kafka*. [5] * [development process] Dian Fu proposes to introduce a *security@f.a.o* mailing list for users to report security-related issues. There is a lot of positive feedback, but also some concerns, e.g. because there is already a cross-project secur...@apache.org mailing list. [6] [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-3-tp34811.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Release-flink-shaded-9-0-tp35041.html [3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-27-Refactor-Source-Interface-tp24952.html [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-86-Improve-Connector-Properties-tp34922.html [5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSSION-Kafka-Metrics-Reporter-tp35067.html [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Expose-or-setup-a-security-flink-apache-org-mailing-list-for-security-report-and-discussion-tp34950p34951.html Notable Bugs ========== Not much happening here. So, let's look at two edge cases, which might help one or the other user. * [FLINK-13184] [1.9.1] [1.8.2] If you are deploying to YARN and start a lot of Taskmanagers (1000s), the Resourcemanager might be blocked/unresponsive quite some time, so that heartbeats of Taskmanagers start timing out. Yang Wang is working on a fix, which might get into 1.8.3. [7] * [FLINK-14066] If you try to *build* PyFlink on Windows, this will fail as we use a UNIX specific path to the local Flink distribution's build target. The fix is contained in this ticket. [8] [7] https://issues.apache.org/jira/browse/FLINK-13184 [8] https://issues.apache.org/jira/browse/FLINK-14066 Events, Blog Posts, Misc =================== * A missed this excellent *Flink Foward Europe *recap by my colleagues *Robert* and* Fabian* published 1st of November on the Ververica blog in the last weeks, so here it is. [9] * Upcoming Meetups * *Bowen Li* will speak about "The Rise of Apache Flink and Stream Processing" at the next Big Data Bellevue in Seattle on the 20th of November. [10] * The next edition of the Bay Area Apache Flink meetup will happen on the 20th of November with talks by *Gyula Fora (Cloudera)* and *Lakshmi Rao (Lyft)*.[11] * We will have our next Apache Flink Meetup in Munich on November 27th with talks by *Heiko Udluft & Giuseppe Sirigu*, Airbus, and *Konstantin Knauf* (on Stateful Functions). [12] * There will be an introduction to Apache Flink, use cases and best practices at the next Uber Engineering meeup in Toronto. If you live in Toronto, its an excellent opportunity to get started with Flink or to meet local Flink users. [13] [9] https://www.ververica.com/blog/flink-forward-europe-2019-recap [10] https://www.meetup.com/Big-Data-Bellevue-BDB/events/fxbnllyzpbbc/ [11] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/266226960/ [12] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/266072196/ [13] https://www.meetup.com/Uber-Engineering-Events-Toronto/events/266264176/ Cheers, Konstantin (@snntrable) -- Konstantin Knauf | Solutions Architect +49 160 91394525 Follow us @VervericaData Ververica <https://www.ververica.com/> -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng
