snuyanzin commented on code in PR #22146: URL: https://github.com/apache/flink/pull/22146#discussion_r1133775105
########## docs/content/release-notes/flink-1.17.md: ########## @@ -0,0 +1,203 @@ +--- +title: "Release Notes - Flink 1.17" +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Release notes - Flink 1.17 + +These release notes discuss important aspects, such as configuration, behavior or dependencies, +that changed between Flink 1.16 and Flink 1.17. Please read these notes carefully if you are +planning to upgrade your Flink version to 1.17. + + +### Clusters & Deployment + +#### Only one Zookeeper version is bundled in `flink-dist` +##### [FLINK-30237](https://issues.apache.org/jira/browse/FLINK-30237) +The Flink distribution no longer bundles 2 different Zookeeper client jars (one in lib/opt +respectively). Instead, only 1 client will be bundled within the flink-dist jar. This has no +effect on the supported Zookeeper server versions. + + +### Table API & SQL + +#### Incompatible plan change of event time temporal join on an upsert source +##### [FLINK-29849](https://issues.apache.org/jira/browse/FLINK-29849) +A correctness issue when do event time temporal join with a versioned table backed by an upsert +source was resolved. When the right input of the join is an upsert source, it no longer generates +a ChangelogNormalize node for it. This is an incompatible plan change compare to 1.16.0. + +#### Incompatible plan change of filter after temporal join +##### [FLINK-28988](https://issues.apache.org/jira/browse/FLINK-28988) +After the patch applied, the filter will not be pushed down into both inputs of the event time +temporal join. Note this may cause incompatible plan changes compare to 1.16.0, e.g., when left +input is an upsert source(use upsert-kafka connector), the query plan will remove the +ChangelogNormalize node from which appeared in 1.16.0. + +### Connectors & Libraries + +#### Remove cassandra connector from master branch +##### [FLINK-30312](https://issues.apache.org/jira/browse/FLINK-30312) +The Cassandra connector has been externalized and is no longer released as part of the main Flink +release. Downloads can be found at https://flink.apache.org/downloads.html and the +source code at https://github.com/apache/flink-connector-cassandra. + +#### Remove Pulsar connector from master branch +##### [FLINK-30397](https://issues.apache.org/jira/browse/FLINK-30397) +The Pulsar connector has been externalized and is no longer bundled and released as part of the +main Flink release. Downloads can be found at https://flink.apache.org/downloads.html and the +source code at https://github.com/apache/flink-connector-pulsar. + +#### Remove HCatalog +##### [FLINK-29669](https://issues.apache.org/jira/browse/FLINK-29669) +The HCatalog connector has been removed from Flink. You can use the Hive connector as a replacement. + +#### Remove Gelly +##### [FLINK-29668](https://issues.apache.org/jira/browse/FLINK-29668) +Gelly has been removed from Flink. Current users of Gelly should not upgrade to Flink 1.17 but +stay on an older version. If you're looking for iterations support, you could investigate +[Flink ML Iteration](https://nightlies.apache.org/flink/flink-ml-docs-stable/docs/development/iteration/) +as a potential successor. + +#### Support watermark alignment of source splits +##### [FLINK-28853](https://issues.apache.org/jira/browse/FLINK-28853) +Since Flink 1.17, source connectors have to implement watermark alignment of source split in order +to use the watermark alignment feature. The required methods to implement are: +`SourceReader#pauseOrResumeSplits` and `SplitReader#pauseOrResumeSplits`. + +If you are migrating from +Flink <= 1.16.x, and you were using watermark alignment, but at the same time you are not able to +upgrade/modify your connector, you can disable per split alignment via setting +`pipeline.watermark-alignment.allow-unaligned-source-splits` to true. Note that by doing so, +watermark alignment will be working properly only when your number of splits equals to the +parallelism of the source operator. + +#### Remove deprecated MiniClusterResource +##### [FLINK-29548](https://issues.apache.org/jira/browse/FLINK-29548) +The deprecated `MiniClusterResource` in `flink-test-utils` has been removed. The +`MiniClusterWithClientResource` is a drop-in replacement. + +#### Kinesis connector doesn't shade jackson dependency +##### [FLINK-14896](https://issues.apache.org/jira/browse/FLINK-14896) +Shade and relocate transitive Jackson dependencies of `flink-connector-kinesis`. If your Flink job +was transitively relying on the these, you may need to include additional Jackson dependencies into +your project. + +### Runtime & Coordination + +#### Speculative execution take input data amount into account when detecting slow tasks +##### [FLINK-30707](https://issues.apache.org/jira/browse/FLINK-30707) +The slow task detecting is improved for speculative execution. Previously, it only considers the +execution time of tasks when deciding which tasks are slow. It now takes the input data volume of +tasks into account. Tasks which has a longer execution time but consumes more data may not be +considered as slow. This improvement helps to eliminate the negative impacts of data skew on slow +task detecting. + +#### Use adaptive batch scheduler as default scheduler for batch jobs +##### [FLINK-30682](https://issues.apache.org/jira/browse/FLINK-30682) +Adaptive batch scheduler are now used for batch jobs by default. It will automatically decide the +parallelism of operators. The keys and values of related configuration items are improved for easy +of use. More details can be found in the +[document](https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/elastic_scaling/#adaptive-batch-scheduler). + +#### Simplify network memory configurations for TaskManager +##### [FLINK-30469](https://issues.apache.org/jira/browse/FLINK-30469) +The default value of `taskmanager.memory.network.max` has changed from `1g` to `Long#MAX_VALUE`, +to reduce the number of config options user needs to tune when trying to increase the network +memory size. This may affect the performance when this option is not explicitly configured, due to +potential changes of network memory size, and heap and managed memory sizes when the total memory Review Comment: ```suggestion potential changes of network memory size, heap and managed memory sizes when the total memory ``` Probably one `and` could be removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
