Dear community, happy to share this week's community update. The first real release candidate of Apache Flink 1.9.0 has been published and the developer community has moved more towards the 1.10 development cycle with more discussion threads popping up again.
As always, please feel free to add additional updates and news to this thread! *Personal Note:* There will be no weekly community update in the next three weeks as I will be on vacation :). The next update will be on the 8th of September, which will then cover four weeks instead of one. Flink Development =============== * [releases] The voting thread on *RC2 for Apache Flink 1.9.0* has just started. It will be open for at least 72 hours. [1,2] * [development] Chesnay has started a discussion on *splitting up Flink's main repository* to tackle build time issues and an ever increasing number of pull requests. So far, the majority of answers is in favor of keeping the mono repo and tackling the problems differently via a more sophisticated build system, project structure, etc. It seems these alternatives deserve a more detailed investigation. [3] * [development process] Stephan has raised the point of *post-feature-freeze contributions* to Flink 1.9.0 in a discussion thread on the mailing list last week. The thread also contains a few proposals e.g. to formalize the process in the future, which probably need to be discussed in separate threads in the future. [4] * [resource management] Xintong has startd a discussion on FLIP-49 to improve and unify the *memory configuration of TaskManagers*. Its goals are to unify the memory configuration for batch and stream processing, to simplify automatically configure RocksDB and to simplify the configuration of the different memory pools. [5] * [documentation] Marta has written a *style guide for our documentation* based on best practices and other successful Open Source projects. More feedback welcome! [6] * [documentation] Myself and Fabian have recently contributed a *docker-based playground* to the Getting Started section of the documentation. This lead to a follow up discussion of how to publish the required Docker Images. [7] * [development process] The *style and quality guide* has finally been published on the project website. [8] * [sources] Zhijang has started a discussion to start with the integration of the *SourceReader* with the mailbox of the StreamTask, which is proposed as part of FLIP-27 (Refactor Source Interface). [9] [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Apache-Flink-Release-1-9-0-release-candidate-2-tp31542.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Progress-updates-for-Apache-Flink-1-9-0-release-tp30565.html [3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Repository-split-tp31417.html [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Merging-new-features-post-feature-freeze-tp31469.html [5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-49-Unified-Memory-Configuration-for-TaskExecutors-tp31436.html [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discuss-FLIP-42-Documentation-Style-Guide-tp31370.html [7] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Docker-Playgrounds-tp31496.html [8] https://flink.apache.org/contributing/code-style-and-quality-preamble.html [9] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Integrate-new-SourceReader-with-Mailbox-Model-in-StreamTask-tp31549.html Notable Bugs =========== * [FLINK-13586] [1.8.1] The signature of org.apache.flink.api.java.ClosureCleaner#clean was changed between 1.8.0 and 1.8.1. Hence, no application calling this method compiled against Flink 1.8.0 can be run on a Flink 1.8.1 cluster. For example, this includes any application using the Kafka or Cassandra Connector. [10] * [FLINK-13537] [1.8.1] [1.7.2] [1.6.4] If an exactly-once Kafka Producer is rescaled and the "pool size" (# of transactionanl ids per subtask) is changed at the same time, the transactional ids of different subtasks might overlap, which should never happen. Fortunately, users can easily avoid such a scenario until this is fixed. [11] [10] https://issues.apache.org/jira/browse/FLINK-13537 [11] https://issues.apache.org/jira/browse/FLINK-13586 Events, Blog Posts, Misc ==================== * *Hequn* is now an Apache Flink Committer. Congratulations! [12] * Upcoming Meetups * *Roshan Naik* of Uber and *Sijie Guo* of StreamNative speak the next Seattle Flink Meetup on the 22nd of August about Kappa+ and Flink with Pulsar respectively. [13] * *Luca Giovagnoli *of Yelp will speak at the ACM San Francisco meetup about a machine learning application on Apache Flink. [14] * There is a full-day of Apache Pulsar talks at the Apache Pulsar Meetup Beijing on the 17th of August. This also includes a talk on state management in Apache Flink by *Yun Tang*. [15] [12] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Hequn-becomes-a-Flink-committer-tp31378.html [13] https://www.meetup.com/seattle-flink/events/263782233 [14] https://www.eventbrite.com/e/apache-pulsar-meetup-beijing-tickets-67849484635 [15] https://www.meetup.com/acm-sf/events/263768407/ Cheers, Konstantin (@snntrable) -- Konstantin Knauf | Solutions Architect +49 160 91394525 Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
