[ANNOUNCE] Weekly Community Update 2019/28
Dear community, happy to share this week's community update with news about a Flink Ecosystem website, new Jira permissions, Apache Flink 1.9.0 and a few more topics. As always, please feel free to add additional updates and news to this thread! Flink Development === [ecosystem] Daryl and Robert have pushed the first version of a *Flink ecoystem website* [1] to a staging environment for public testing. The website will contain a catalog of community maintained Apache Flink packages (e.g., connectors, metrics connectors, tooling, ...), which will be an easy way to share one's work with the wider community. Currently, Daryl and Robert are looking for feedback on the corresponding thread [2] or github project [3]. [development process] Following our new contributions guidelines [4], the* Jira permissions *have been changed so that only committers can assign users to tickets. The discussion in the original thread has moved to the topic of how to deal with the existing set of about 600 contributors in our Jira project. [5] [development process] It seems the discussion around initial *bylaws for Apache Flink* has pretty much converged already. Expecting a vote to start soonish unless new points are raised [6]. [releases] Gordon has prepared a RC0 for *Apache Flink 1.9.0* [7]. There will not be a vote on this release candidate. It is merely a reference point for ongoing release testing. To follow the release testing efforts checkout this Kanban board [8] containing all blockers for 1.9.0. [client] Zili Chen started a discussion on dropping the hardly used *o.a.f.api.common.Program* interface to facilitate future changes in the client code base. Users of o.a.f.api.common.Program please join the discussion. [9] [distributed coordination] Lamber-Ken asks for opinions about an ongoing Jira ticket [10] in the context of the *leader selection*. He proposes that the JobManager should not loose leadership if the connection to Zookeeper is temporarily suspended, but only once the connection is lost. [11] [1] https://flink-ecosystem-demo.flink-resources.org/ [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-Flink-ecosystem-website-tp27519.html [3] https://github.com/sorahn/flink-ecosystem/issues [4] https://flink.apache.org/contributing/contribute-code.html [5] https://issues.apache.org/jira/browse/INFRA-18644 [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-A-more-restrictive-JIRA-workflow-td27344.html [7] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-project-bylaws-tp30409.html [8] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-tp30583.html [9] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=328=FLINK=FLINK-13249 [10] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Drop-stale-class-Program-tp30744.html [11] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISSCUSS-Tolerate-temporarily-suspended-ZooKeeper-connections-tp30781.html Notable Bugs === Same as last week, very active Jira due to release testing, but only a few bugs for released versions being filed. [FLINK-13044] [1.8.1] A ticket I missed last week. Due to the way we currently shade the com.amazonaws packages in flink-s3-fs-hadoop certain configurations are not forwarded properly to the filesystem classes. The Jira ticket contains a good explanation of the problem. [12] [12] https://issues.apache.org/jira/browse/FLINK-13044 Events, Blog Posts, Misc * *Jinagje (Becket) Qin* is now an Apache Flink committer. Congratulations! [13] * The conference *schedule of Flink Forward Europe* has been announced. [14] The training schedule has already been available a for a few weeks [15]. * Upcoming Meetups * On 30th of July *Berecz Dániel *of Ekata talks about a "Parameter Server on Flink, an approach for model-parallel machine learning" at the Budapest Scala Meetup [16] [13] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jiangjie-Becket-Qin-has-been-added-as-a-committer-to-the-Flink-project-td30670.html [14] https://berlin-2019.flink-forward.org/conference-program [15] https://berlin-2019.flink-forward.org/training-program [16] https://www.meetup.com/budapest-scala/events/263025323/ Cheers, Konstantin (@snntrable) -- Konstantin Knauf | Solutions Architect +49 160 91394525 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
Re: [ANNOUNCE] Weekly Community Update 2019/28
As always, thank you for putting these nice summaries together! On Sun, Jul 14, 2019 at 7:34 PM Konstantin Knauf wrote: > Dear community, > > happy to share this weeks community update with Apache Flink 1.9, bylaws > for Apache Flink, Savepoints vs Checkpoints, Flink on ARM, and more. > > As always, please feel free to add additional updates and news to this > thread! > > Flink Development > === > > * [releases] The release branch for *Apache Flink 1.9.0 *has been cut > lasat Thursday and we are moving on to release testing. [1] > > * [development process] Following the discussion on our FLIP process. > Becket has initiated a discussion [2] on writing up *bylaws for Apache > Flink*, a set of rules governing the core processes of the community. The > plan to add bylaws was very well received and the discussion quickly moved > to the first draft [3] of these bylaws. > > [development process] The migration of our CI infrastructure [4] to a > non-ASF Travis account [5] has been implemented very quickly by Chesnay. > Updates to the *CI bot *are currently discussed in a new thread [6]. > > * [development process] Robert opened a PR [7] to add the *code style and > quality guide* to the contribution guide on the Apache Flink website. If > you have not checked out the guide yet, now is good time ;) > > * [docs] In the course of FLIP-42 Konstantin added a first version of a > *glossary* [8] to the Flink documentation defining frequently used terms > like Operators, Task, Partition, a.s.o. This can help us to use a common > terminology in our documentation as well as on the mailing lists. > > * [state management] Kostas has started a FLIP(-47) [9] on how to think > about *savepoints* *vs* *checkpoints* in the future. In a nutshell, the > FLIP proposes to waive the distinction between savepoints & checkpoints: > both would simply be snapshots. Each snapshot can have different properties > depending on its origin (system/user) and format (incremental, > universal,...). The topic has also been addressed by Yu Li in FLIP-45 [10] > recently. > > * [build] Xiyuan Wang has started a discussion about officially supporting > *Apache > Flink on ARM*-based systems. It looks like there are no big blockers on > the technical side and the discussion currently focuses on integrating an > ARM-based build into our CI infrastructure. [11] > > [1] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-tp30500.html > [2] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-project-bylaws-tp30409.html > [3] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120731026 > [4] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-solve-unstable-build-capacity-problem-on-TravisCI-tp29881.html > [5] https://travis-ci.com/flink-ci > [6] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/CiBot-Update-td30536.html > [7] https://github.com/apache/flink-web/pull/224 > [8] > https://ci.apache.org/projects/flink/flink-docs-master/concepts/glossary.html > [9] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/FLIP-47-Savepoints-vs-Checkpoints-tp30324.html > [10] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-45-Reinforce-Job-Stop-Semantic-tp30161.html > [11] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-ARM-support-for-Flink-td30298.html > > Notable Bugs > === > > Quite a lot of bugs are being opened right now, but these are basically > all related to release testing for Flink 1.9 or test instabilities - hence > not affecting released versions of Apache Flink. > > [FLINK-11654] [1.7.2] [1.8.1] I recently sumbled across this tickcet of > the (exatly-once) FlinkKafkaProducer, which was already created back in > February. When running multiple instances of the same Flink program with an > exactly-once FlinkKafkaProducer, the transational ids used by these > producers can clash (both jobs use the same ids) and the jobs crash > frequently. The resolution is currently under discussion. [12] > > [12] https://issues.apache.org/jira/browse/FLINK-11654 > > Events, Blog Posts, Misc > > > * *Flink Forward Europ*e early-bird ends on the 15th of July. [13] > * Upcoming Meetups > * On 18th of July *Christos Hadjinikolis* is speaking at the "Big > Data LDN Meetup" on "How real-time data processing is used for application > in customer experience?" [14] > * Rong Rong is now an Apache Flink Committer. Congratulations! [15] > > [13] https://berlin-2019.flink-forward.org/ > [14] https://www.meetup.com/big-data-ldn/events/262638878/ > [15] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Rong-Rong-becomes-a-Flink-committer-td30451.html#a30474 > > Cheers and have a nice evening, > > Konstantin (@snntrable) > > -- > > Konstantin Knauf | Solutions Architect > > +49 160 91394525 > > > -- > >
[ANNOUNCE] Weekly Community Update 2019/28
Dear community, happy to share this weeks community update with Apache Flink 1.9, bylaws for Apache Flink, Savepoints vs Checkpoints, Flink on ARM, and more. As always, please feel free to add additional updates and news to this thread! Flink Development === * [releases] The release branch for *Apache Flink 1.9.0 *has been cut lasat Thursday and we are moving on to release testing. [1] * [development process] Following the discussion on our FLIP process. Becket has initiated a discussion [2] on writing up *bylaws for Apache Flink*, a set of rules governing the core processes of the community. The plan to add bylaws was very well received and the discussion quickly moved to the first draft [3] of these bylaws. [development process] The migration of our CI infrastructure [4] to a non-ASF Travis account [5] has been implemented very quickly by Chesnay. Updates to the *CI bot *are currently discussed in a new thread [6]. * [development process] Robert opened a PR [7] to add the *code style and quality guide* to the contribution guide on the Apache Flink website. If you have not checked out the guide yet, now is good time ;) * [docs] In the course of FLIP-42 Konstantin added a first version of a *glossary* [8] to the Flink documentation defining frequently used terms like Operators, Task, Partition, a.s.o. This can help us to use a common terminology in our documentation as well as on the mailing lists. * [state management] Kostas has started a FLIP(-47) [9] on how to think about *savepoints* *vs* *checkpoints* in the future. In a nutshell, the FLIP proposes to waive the distinction between savepoints & checkpoints: both would simply be snapshots. Each snapshot can have different properties depending on its origin (system/user) and format (incremental, universal,...). The topic has also been addressed by Yu Li in FLIP-45 [10] recently. * [build] Xiyuan Wang has started a discussion about officially supporting *Apache Flink on ARM*-based systems. It looks like there are no big blockers on the technical side and the discussion currently focuses on integrating an ARM-based build into our CI infrastructure. [11] [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-tp30500.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-project-bylaws-tp30409.html [3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120731026 [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-solve-unstable-build-capacity-problem-on-TravisCI-tp29881.html [5] https://travis-ci.com/flink-ci [6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/CiBot-Update-td30536.html [7] https://github.com/apache/flink-web/pull/224 [8] https://ci.apache.org/projects/flink/flink-docs-master/concepts/glossary.html [9] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/FLIP-47-Savepoints-vs-Checkpoints-tp30324.html [10] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-45-Reinforce-Job-Stop-Semantic-tp30161.html [11] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-ARM-support-for-Flink-td30298.html Notable Bugs === Quite a lot of bugs are being opened right now, but these are basically all related to release testing for Flink 1.9 or test instabilities - hence not affecting released versions of Apache Flink. [FLINK-11654] [1.7.2] [1.8.1] I recently sumbled across this tickcet of the (exatly-once) FlinkKafkaProducer, which was already created back in February. When running multiple instances of the same Flink program with an exactly-once FlinkKafkaProducer, the transational ids used by these producers can clash (both jobs use the same ids) and the jobs crash frequently. The resolution is currently under discussion. [12] [12] https://issues.apache.org/jira/browse/FLINK-11654 Events, Blog Posts, Misc * *Flink Forward Europ*e early-bird ends on the 15th of July. [13] * Upcoming Meetups * On 18th of July *Christos Hadjinikolis* is speaking at the "Big Data LDN Meetup" on "How real-time data processing is used for application in customer experience?" [14] * Rong Rong is now an Apache Flink Committer. Congratulations! [15] [13] https://berlin-2019.flink-forward.org/ [14] https://www.meetup.com/big-data-ldn/events/262638878/ [15] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Rong-Rong-becomes-a-Flink-committer-td30451.html#a30474 Cheers and have a nice evening, Konstantin (@snntrable) -- Konstantin Knauf | Solutions Architect +49 160 91394525 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
