This is an automated email from the ASF dual-hosted git repository.
guozhang pushed a commit to branch 3.0
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/3.0 by this push:
new 3f2c833 KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
3f2c833 is described below
commit 3f2c833cda512d5e7332f41e4c66e544114ae4d5
Author: Guozhang Wang <wangg...@gmail.com>
AuthorDate: Tue Aug 17 17:03:11 2021 -0700
KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
Reviewers: Luke Chen <show...@gmail.com>, Anna Sophie Blee-Goldman
<ableegold...@apache.org>, Bruno Cadonna <cado...@confluent.io>
---
docs/streams/upgrade-guide.html | 33 +++++++++++++++++++--------------
1 file changed, 19 insertions(+), 14 deletions(-)
diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html
index eb3b125..ecc8cb8 100644
--- a/docs/streams/upgrade-guide.html
+++ b/docs/streams/upgrade-guide.html
@@ -52,6 +52,22 @@
<li> restart all new ({{fullDotVersion}}) application instances </li>
</ul>
+ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"></a><a href="#streams_notable_changes">Notable
compatibility changes in past releases</a></h3>
+ <p>
+ Downgrading from 3.0.x or newer version to 2.8.x or older version
needs special attention:
+ Since 3.0.0 release, Kafka Streams uses a newer RocksDB version whose
on-disk format changed.
+ This means that old versioned RocksDB would not be able to recognize
the bytes written by that newer versioned RocksDB,
+ and hence it is harder to downgrade Kafka Streams with version 3.0.0
or newer to older versions in-flight.
+ Users need to wipe out the local RocksDB state stores written by the
new versioned Kafka Streams before swapping in the
+ older versioned Kafka Streams bytecode, which would then restore the
state stores with the old on-disk format from the
+ changelogs.
+ </p>
+
+ <p>
+ Kafka Streams does not support running multiple instances of the same
application as different processes on the same physical state directory.
Starting in 2.8.0 (as well as 2.7.1 and 2.6.2),
+ this restriction will be enforced. If you wish to run more than one
instance of Kafka Streams, you must configure them with different values for
<code>state.dir</code>.
+ </p>
+
<p>
Starting in Kafka Streams 2.6.x, a new processing mode is available,
named EOS version 2. This can be configured
by setting <code>"processing.guarantee"</code> to
<code>"exactly_once_v2"</code> for
@@ -69,6 +85,8 @@
Afterward, you can downgrade your application to a pre-2.6.x version.
</p>
+ <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that
requires MacOS 10.14 or higher.</p>
+
<p>
To run a Kafka Streams application version 2.2.1, 2.3.0, or higher a
broker version 0.11.0 or higher is required
and the on-disk message format must be 0.11 or higher.
@@ -77,10 +95,8 @@
For Kafka Streams 0.10.0, broker version 0.10.0 or higher is required.
</p>
- <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that
requires MacOS 10.14 or higher.</p>
-
<p>
- Another important thing to keep in mind: in deprecated
<code>KStreamBuilder</code> class, when a <code>KTable</code> is created from a
source topic via <code>KStreamBuilder.table()</code>, its materialized state
store
+ In deprecated <code>KStreamBuilder</code> class, when a
<code>KTable</code> is created from a source topic via
<code>KStreamBuilder.table()</code>, its materialized state store
will reuse the source topic as its changelog topic for restoring, and
will disable logging to avoid appending new updates to the source topic; in the
<code>StreamsBuilder</code> class introduced in 1.0, this behavior was changed
accidentally: we still reuse the source topic as the changelog topic
for restoring, but will also create a separate changelog topic to append the
update records from source topic to. In the 2.0 release, we have fixed this
issue and now users
can choose whether or not to reuse the source topic based on the
<code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code>: if you are upgrading from the
old <code>KStreamBuilder</code> class and hence you need to change your code to
use
@@ -89,11 +105,6 @@
More details about the new config
<code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code> can be found in <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-295%3A+Add+Streams+Configuration+Allowing+for+Optional+Topology+Optimization";>KIP-295</a>.
</p>
- <p>
- Note: Kafka Streams does not support running multiple instances of the
same application as different processes on the same physical state directory.
Starting in 2.8.0 (as well as 2.7.1 and 2.6.2),
- this restriction will be enforced. If you wish to run more than one
instance of Kafka Streams, you must configure them with different values for
<code>state.dir</code>.
- </p>
-
<h3><a id="streams_api_changes_300"
href="#streams_api_changes_300">Streams API changes in 3.0.0</a></h3>
<p>
We improved the semantics of
@@ -543,12 +554,6 @@
For more details please read <a
href="https://issues.apache.org/jira/browse/KAFKA-8215";>KAFKA-8215</a>.
</p>
- <h3 class="anchor-heading"><a id="streams_notable_changes_221"
class="anchor-link"></a><a href="#streams_notable_changes_221">Notable changes
in Kafka Streams 2.2.1</a></h3>
- <p>
- As of Kafka Streams 2.2.1 a message format 0.11 or higher is required;
- this implies that brokers must be on version 0.11.0 or higher.
- </p>
-
<h3 class="anchor-heading"><a id="streams_api_changes_220"
class="anchor-link"></a><a href="#streams_api_changes_220">Streams API changes
in 2.2.0</a></h3>
<p>
We've simplified the <code>KafkaStreams#state</code> transition
diagram during the starting up phase a bit in 2.2.0: in older versions the
state will transit from <code>CREATED</code> to <code>RUNNING</code>, and then
to <code>REBALANCING</code> to get the first
