This is an automated email from the ASF dual-hosted git repository. guozhang pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push: new fb2b1313 KAFKA-13172: Add downgrade guidance note for 3.0 (#11184) fb2b1313 is described below commit fb2b131310ceff06008fbad3af1e99bba4bc2b1e Author: Guozhang Wang <wangg...@gmail.com> AuthorDate: Tue Aug 17 17:03:11 2021 -0700 KAFKA-13172: Add downgrade guidance note for 3.0 (#11184) Reviewers: Luke Chen <show...@gmail.com>, Anna Sophie Blee-Goldman <ableegold...@apache.org>, Bruno Cadonna <cado...@confluent.io> --- docs/streams/upgrade-guide.html | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html index 6a62731..febfc65 100644 --- a/docs/streams/upgrade-guide.html +++ b/docs/streams/upgrade-guide.html @@ -52,6 +52,22 @@ <li> restart all new ({{fullDotVersion}}) application instances </li> </ul> + <h3 class="anchor-heading"><a id="streams_notable_changes" class="anchor-link"></a><a href="#streams_notable_changes">Notable compatibility changes in past releases</a></h3> + <p> + Downgrading from 3.0.x or newer version to 2.8.x or older version needs special attention: + Since 3.0.0 release, Kafka Streams uses a newer RocksDB version whose on-disk format changed. + This means that old versioned RocksDB would not be able to recognize the bytes written by that newer versioned RocksDB, + and hence it is harder to downgrade Kafka Streams with version 3.0.0 or newer to older versions in-flight. + Users need to wipe out the local RocksDB state stores written by the new versioned Kafka Streams before swapping in the + older versioned Kafka Streams bytecode, which would then restore the state stores with the old on-disk format from the + changelogs. + </p> + + <p> + Kafka Streams does not support running multiple instances of the same application as different processes on the same physical state directory. Starting in 2.8.0 (as well as 2.7.1 and 2.6.2), + this restriction will be enforced. If you wish to run more than one instance of Kafka Streams, you must configure them with different values for <code>state.dir</code>. + </p> + <p> Starting in Kafka Streams 2.6.x, a new processing mode is available, named EOS version 2. This can be configured by setting <code>"processing.guarantee"</code> to <code>"exactly_once_v2"</code> for @@ -69,6 +85,8 @@ Afterward, you can downgrade your application to a pre-2.6.x version. </p> + <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that requires MacOS 10.14 or higher.</p> + <p> To run a Kafka Streams application version 2.2.1, 2.3.0, or higher a broker version 0.11.0 or higher is required and the on-disk message format must be 0.11 or higher. @@ -77,10 +95,8 @@ For Kafka Streams 0.10.0, broker version 0.10.0 or higher is required. </p> - <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that requires MacOS 10.14 or higher.</p> - <p> - Another important thing to keep in mind: in deprecated <code>KStreamBuilder</code> class, when a <code>KTable</code> is created from a source topic via <code>KStreamBuilder.table()</code>, its materialized state store + In deprecated <code>KStreamBuilder</code> class, when a <code>KTable</code> is created from a source topic via <code>KStreamBuilder.table()</code>, its materialized state store will reuse the source topic as its changelog topic for restoring, and will disable logging to avoid appending new updates to the source topic; in the <code>StreamsBuilder</code> class introduced in 1.0, this behavior was changed accidentally: we still reuse the source topic as the changelog topic for restoring, but will also create a separate changelog topic to append the update records from source topic to. In the 2.0 release, we have fixed this issue and now users can choose whether or not to reuse the source topic based on the <code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code>: if you are upgrading from the old <code>KStreamBuilder</code> class and hence you need to change your code to use @@ -89,11 +105,6 @@ More details about the new config <code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code> can be found in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-295%3A+Add+Streams+Configuration+Allowing+for+Optional+Topology+Optimization">KIP-295</a>. </p> - <p> - Note: Kafka Streams does not support running multiple instances of the same application as different processes on the same physical state directory. Starting in 2.8.0 (as well as 2.7.1 and 2.6.2), - this restriction will be enforced. If you wish to run more than one instance of Kafka Streams, you must configure them with different values for <code>state.dir</code>. - </p> - <h3><a id="streams_api_changes_300" href="#streams_api_changes_300">Streams API changes in 3.0.0</a></h3> <p> We improved the semantics of @@ -543,12 +554,6 @@ For more details please read <a href="https://issues.apache.org/jira/browse/KAFKA-8215">KAFKA-8215</a>. </p> - <h3 class="anchor-heading"><a id="streams_notable_changes_221" class="anchor-link"></a><a href="#streams_notable_changes_221">Notable changes in Kafka Streams 2.2.1</a></h3> - <p> - As of Kafka Streams 2.2.1 a message format 0.11 or higher is required; - this implies that brokers must be on version 0.11.0 or higher. - </p> - <h3 class="anchor-heading"><a id="streams_api_changes_220" class="anchor-link"></a><a href="#streams_api_changes_220">Streams API changes in 2.2.0</a></h3> <p> We've simplified the <code>KafkaStreams#state</code> transition diagram during the starting up phase a bit in 2.2.0: in older versions the state will transit from <code>CREATED</code> to <code>RUNNING</code>, and then to <code>REBALANCING</code> to get the first