This is an automated email from the ASF dual-hosted git repository.
guozhang pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new fb2b1313 KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
fb2b1313 is described below
commit fb2b131310ceff06008fbad3af1e99bba4bc2b1e
Author: Guozhang Wang <[email protected]>
AuthorDate: Tue Aug 17 17:03:11 2021 -0700
KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
Reviewers: Luke Chen <[email protected]>, Anna Sophie Blee-Goldman
<[email protected]>, Bruno Cadonna <[email protected]>
---
docs/streams/upgrade-guide.html | 33 +++++++++++++++++++--------------
1 file changed, 19 insertions(+), 14 deletions(-)
diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html
index 6a62731..febfc65 100644
--- a/docs/streams/upgrade-guide.html
+++ b/docs/streams/upgrade-guide.html
@@ -52,6 +52,22 @@
<li> restart all new ({{fullDotVersion}}) application instances </li>
</ul>
+ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"></a><a href="#streams_notable_changes">Notable
compatibility changes in past releases</a></h3>
+ <p>
+ Downgrading from 3.0.x or newer version to 2.8.x or older version
needs special attention:
+ Since 3.0.0 release, Kafka Streams uses a newer RocksDB version whose
on-disk format changed.
+ This means that old versioned RocksDB would not be able to recognize
the bytes written by that newer versioned RocksDB,
+ and hence it is harder to downgrade Kafka Streams with version 3.0.0
or newer to older versions in-flight.
+ Users need to wipe out the local RocksDB state stores written by the
new versioned Kafka Streams before swapping in the
+ older versioned Kafka Streams bytecode, which would then restore the
state stores with the old on-disk format from the
+ changelogs.
+ </p>
+
+ <p>
+ Kafka Streams does not support running multiple instances of the same
application as different processes on the same physical state directory.
Starting in 2.8.0 (as well as 2.7.1 and 2.6.2),
+ this restriction will be enforced. If you wish to run more than one
instance of Kafka Streams, you must configure them with different values for
<code>state.dir</code>.
+ </p>
+
<p>
Starting in Kafka Streams 2.6.x, a new processing mode is available,
named EOS version 2. This can be configured
by setting <code>"processing.guarantee"</code> to
<code>"exactly_once_v2"</code> for
@@ -69,6 +85,8 @@
Afterward, you can downgrade your application to a pre-2.6.x version.
</p>
+ <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that
requires MacOS 10.14 or higher.</p>
+
<p>
To run a Kafka Streams application version 2.2.1, 2.3.0, or higher a
broker version 0.11.0 or higher is required
and the on-disk message format must be 0.11 or higher.
@@ -77,10 +95,8 @@
For Kafka Streams 0.10.0, broker version 0.10.0 or higher is required.
</p>
- <p>Since 2.6.0 release, Kafka Streams depends on a RocksDB version that
requires MacOS 10.14 or higher.</p>
-
<p>
- Another important thing to keep in mind: in deprecated
<code>KStreamBuilder</code> class, when a <code>KTable</code> is created from a
source topic via <code>KStreamBuilder.table()</code>, its materialized state
store
+ In deprecated <code>KStreamBuilder</code> class, when a
<code>KTable</code> is created from a source topic via
<code>KStreamBuilder.table()</code>, its materialized state store
will reuse the source topic as its changelog topic for restoring, and
will disable logging to avoid appending new updates to the source topic; in the
<code>StreamsBuilder</code> class introduced in 1.0, this behavior was changed
accidentally: we still reuse the source topic as the changelog topic
for restoring, but will also create a separate changelog topic to append the
update records from source topic to. In the 2.0 release, we have fixed this
issue and now users
can choose whether or not to reuse the source topic based on the
<code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code>: if you are upgrading from the
old <code>KStreamBuilder</code> class and hence you need to change your code to
use
@@ -89,11 +105,6 @@
More details about the new config
<code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code> can be found in <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-295%3A+Add+Streams+Configuration+Allowing+for+Optional+Topology+Optimization";>KIP-295</a>.
</p>
- <p>
- Note: Kafka Streams does not support running multiple instances of the
same application as different processes on the same physical state directory.
Starting in 2.8.0 (as well as 2.7.1 and 2.6.2),
- this restriction will be enforced. If you wish to run more than one
instance of Kafka Streams, you must configure them with different values for
<code>state.dir</code>.
- </p>
-
<h3><a id="streams_api_changes_300"
href="#streams_api_changes_300">Streams API changes in 3.0.0</a></h3>
<p>
We improved the semantics of
@@ -543,12 +554,6 @@
For more details please read <a
href="https://issues.apache.org/jira/browse/KAFKA-8215";>KAFKA-8215</a>.
</p>
- <h3 class="anchor-heading"><a id="streams_notable_changes_221"
class="anchor-link"></a><a href="#streams_notable_changes_221">Notable changes
in Kafka Streams 2.2.1</a></h3>
- <p>
- As of Kafka Streams 2.2.1 a message format 0.11 or higher is required;
- this implies that brokers must be on version 0.11.0 or higher.
- </p>
-
<h3 class="anchor-heading"><a id="streams_api_changes_220"
class="anchor-link"></a><a href="#streams_api_changes_220">Streams API changes
in 2.2.0</a></h3>
<p>
We've simplified the <code>KafkaStreams#state</code> transition
diagram during the starting up phase a bit in 2.2.0: in older versions the
state will transit from <code>CREATED</code> to <code>RUNNING</code>, and then
to <code>REBALANCING</code> to get the first
