Dear community, happy to share this week's community digest with updates on the next release cycle, a set of proposal for Flink's web user interface, a couple of discussions around our development process and a bit more.
Flink Development ============== * [releases] Stephan proposes an "anticipated feature freeze date" for Flink 1.11 around the end of April, and hence a release in May. This would make the next release a short one, which seems to be generally well received. Piotr and Zhijiang will be our release managers for Flink 1.11. [1] * [releases] flink-shaded 10.0 has been released. [2] * [web ui] Yadong has split up the improvement proposal to Flink's web user interface (FLIP-75) into multiple votes. Each thread contains a live demo of the proposed feature. * [FLIP-98]: Highlight backpressure task in job graph [3] * [FLIP-99]: Make maximally shown exceptions configurable [4] * [FLIP-100] Show the attempt history of tasks [5] * [FLIP-101] Expose Information on Pending Slots in the Web User Interface [6] * [FLIP-102] More Taskmanager metrics, in particular memory resources [7] * [FLIP-103] List all log files and make them downloadable [8] * [FLIP-104] More Flink Master metrics, in particular memory resources [9] * [development process] Hequn Cheng has started discussion on improving the FLIP process, in particular the way we deal with discussion on the wiki, mailing list and Google Docs. The discussion is ongoing, but there seems to be consensus not to use Google Docs more than right now, but to maybe even eliminate it from the process all together. In the discussion David raised a related point, that the FLIP document is often not updated after its implementation and changes between the original plan and the final implementation are not documented anywhere. [10] * [development process] Xingtong proposes to update the PR description template as it is quite outdated. The discussion shows that the template is generally considered useful, but it indeed needs an update. [11] * [deployment] Aljoscha has started a discussion to either drop or extend the support for Flink's windows scripts. [12] * [connectors] The connector for ElasticSearch 2.x will be dropped in Flink 1.11. Elastic Search 5.x will still be supported in Flink 1.11 and revisited for Flink 1.12. [13] * [state] Stephan proposes to drop savepoint compatibility with Flink 1.2. A stateful upgrade from Flink 1.2 to Flink 1.11 would still be possible by first upgrading to an intermediate Flink version. [14] [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Kicking-off-the-1-11-release-cycle-tp37817.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-shaded-10-0-released-tp37815.html [3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-98-Better-Back-Pressure-Detection-tp37893.html [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-99-Make-Max-Exception-Configurable-tp37895.html [5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-100-Add-Attempt-Information-tp37896.html [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-101-Add-Pending-Slots-Detail-tp37897.html [7] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-102-Add-More-Metrics-to-TaskManager-tp37898.html [8] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-103-Better-TM-JM-Log-Display-tp37899.html [9] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-104-Add-More-Metrics-to-Jobmanager-tp37901.html [10] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Improvements-on-FLIP-Process-tp37785.html [11] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Discuss-Update-the-pull-request-description-template-tp37755.html [12] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Extend-or-maintain-shell-script-support-for-Windows-tp37868.html [13] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Drop-connectors-for-Elasticsearch-2-x-and-5-x-tp37471.html [14] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Drop-Savepoint-Compatibility-with-Flink-1-2-tp37908.html Notable Bugs ========== * [FLINK-16111] [1.10] Flink's Kubernetes deployment does not respect the "taskmanager.cpu.cores" configuration [15] * [FLINK-16115] [1.10] The OSS (Alibaba Cloud's Object Storage Service) filesystem does not work as a plugin. [16] [15] https://issues.apache.org/jira/browse/FLINK-16111 [16] https://issues.apache.org/jira/browse/FLINK-16115 Events, Blog Posts, Misc =================== * Jingsong Lee is now an Apache Flink committer. Congratulations! [17] * Seth has published a post on the Apache Flink blog on Flink SQL DDL: "No Java Required: Configured Sources and Sinks in SQL" [18] * Upcoming Meetups * On February 26th, Prateep Kumar will host an online event on log analytics with Apache Flink [19]. * On March 5th, Stephan Ewen will talk about Apache Flink Stateful Function at the Utrecht Data Engineering Meetup. [20] * Cloudera is hosting a couple of "Future of Data" events on stream processing with Apache Flink in * Budapest, (February 25th, meetup) [21] * Vienna (March 4th, full-day workshop) [22] * Zurich (March 10th, full-day workshop) [23] * New Jersey (May 5th, meetup) [24] [17] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jingsong-Lee-becomes-a-Flink-committer-tp37938p38006.html [18] https://flink.apache.org/news/2020/02/20/ddl.html [19] https://www.meetup.com/apache-flink-aws-kinesis-hyd-india/events/268907057/ [20] https://www.meetup.com/Data-Engineering-NL/events/268424399/ [21] https://www.meetup.com/futureofdata-budapest/events/268423538/ [22] https://www.meetup.com/futureofdata-vienna/events/268418974/ [23] https://www.meetup.com/futureofdata-zurich/events/268423809/ [24] https://www.meetup.com/futureofdata-princeton/events/268830725/ Cheers, Konstantin (@snntrable) -- Konstantin Knauf | Head of Product +49 160 91394525 Follow us @VervericaData Ververica <https://www.ververica.com/> -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng
