rmetzger commented on a change in pull request #375: URL: https://github.com/apache/flink-web/pull/375#discussion_r483434583
########## File path: _posts/2020-08-31-community-update.md ########## @@ -0,0 +1,222 @@ +--- +layout: post +title: "Flink Community Update - August'20" +date: 2020-08-31T08:00:00.000Z +categories: news +authors: +- morsapaes: + name: "Marta Paes" + twitter: "morsapaes" + +excerpt: Ah, so much for a quiet August month. This time around, we bring you some new Flink Improvement Proposals (FLIPs), a preview of the upcoming Flink Stateful Functions 2.2 release and a look into how far Flink has come in comparison to 2019. +--- + +Ah, so much for a quiet August month. This time around, we bring you some new Flink Improvement Proposals (FLIPs), a preview of the upcoming [Flink Stateful Functions](https://ci.apache.org/projects/flink/flink-statefun-docs-master/) 2.2 release and a look into how far Flink has come in comparison to 2019. + +{% toc %} + +# The Past Month in Flink + +## Flink Releases + +### Getting Ready for Flink Stateful Functions 2.2 + +The details of the next release of [Stateful Functions](https://ci.apache.org/projects/flink/flink-statefun-docs-master/) are under discussion in [this @dev mailing list thread](http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Next-Stateful-Functions-Release-td44063.html), and the feature freeze is set for **September 10th** — so, you can expect Stateful Functions 2.2 to be released soon after! Some of the most relevant features in the upcoming release are: + +* **DataStream API interoperability**, allowing users to embed Stateful Functions pipelines in regular [DataStream API](https://ci.apache.org/projects/flink/flink-docs-stable/dev/datastream_api.html) programs with `DataStream` ingress/egress. + +* **Fine-grained control over state** for remote functions, including the ability to configure different state expiration modes for each individual function. + +As the community around StateFun grows, the release cycle will follow this pattern of smaller and more frequent releases to incorporate user feedback and allow for faster iteration. If you’d like to get involved, we’re always looking for [new contributors](https://github.com/apache/flink-statefun#contributing)! + +### Flink 1.10.2 + +The community has announced the second patch version to cover some outstanding issues in Flink 1.10. You can find a detailed list with all the improvements and bugfixes that went into Flink 1.10.2 in the [announcement blogpost](https://flink.apache.org/news/2020/08/25/release-1.10.2.html). + +<hr> + +## New Flink Improvement Proposals (FLIPs) + +The number of FLIPs being created and discussed in the @dev mailing list is growing week over week, as the Flink 1.12 release takes form and some longer-term efforts are kicked-off. Below are some of the new FLIPs to keep an eye out for! + +<table class="table table-bordered"> + <thead> + <tr> + <th><center>#</center></th> + <th></th> + </tr> + </thead> + <tbody> + <tr> + <td><a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741";>FLIP-131</a></td> + <td><ul> + <li><b>Consolidate User-Facing APIs and Deprecate the DataSet API</b></li> + <p>The community proposes to deprecate the DataSet API in favor of the Table API/SQL and the DataStream API, in the long run. For this to be feasible, both APIs first need to be <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741#FLIP131:ConsolidatetheuserfacingDataflowSDKs/APIs(anddeprecatetheDataSetAPI)-ProposedChanges">adapted and expanded</a> to support the additional use cases currently covered by the DataSet API.</p> + <p> The first discussion to branch out of this "umbrella" FLIP is around support for a batch execution mode in the DataStream API (<a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-134%3A+Semantics+of+Bounded+Applications+on+the+DataStream+API";>FLIP-134</a>).</p> + </ul> + </td> + </tr> + <tr> + <td><a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-135+Approximate+Task-Local+Recovery";>FLIP-135</a></td> + <td><ul> + <li><b>Approximate Task-Local Recovery</b></li> + <p>To better accommodate recovery scenarios where a certain amount of data loss is tolerable, but a full pipeline restart is not desirable, the community plans to introduce a new failover strategy that allows to restart only the failed task(s). Approximate task-local recovery will allow users to trade consistency for fast failure recovery, which is handy for use cases like online training.</p> + </ul> + </td> + </tr> + <tr> + <td><a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-136%3A++Improve+interoperability+between+DataStream+and+Table+API";>FLIP-136</a></td> + <td><ul> + <li><b>Improve the interoperability between DataStream and Table API</b></li> + <p>The Table API has seen a great deal of refactoring and new features in recent releases, but the interfaces to and from the DataStream API haven't been updated accordingly. The work in this FLIP will cover multiple known gaps to improve interoperability and expose important functionality also to the DataStream API (e.g. changelog handling).</p> + </ul> + </td> + </tr> + <tr> + <td><a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-139%3A+General+Python+User-Defined+Aggregate+Function+Support+on+Table+API";>FLIP-139</a></td> + <td><ul> + <li><b>Support Stateful Python UDFs</b></li> + <p>Python UDFs have been supported in PyFlink <a href="https://flink.apache.org/news/2020/02/11/release-1.10.0.html#pyflink-support-for-native-user-defined-functions-udfs";>since 1.10</a>, but were so far limited to stateless functions. The community is now looking to introduce stateful aggregate functions (UDAFs) in the Python Table API.</p> + <p><b>Note: </b>Pandas UDAFs are covered in a separate proposal (<a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-137%3A+Support+Pandas+UDAF+in+PyFlink";>FLIP-137</a>).</p> + </ul> + </td> + </tr> + </tbody> +</table> + +For a complete overview of the development threads coming up in the project, check the [Flink Wiki](https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals) and follow the feature discussions in the [@dev mailing list](https://lists.apache.org/[email protected]). Review comment: You could also link to the list of 1.12 features in the wiki here: https://cwiki.apache.org/confluence/display/FLINK/1.12+Release ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
