This is an automated email from the ASF dual-hosted git repository.
jlprat pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/kafka-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 321ee54fc MINOR Add upgrade instructiosn for 38 and 39 (#634)
321ee54fc is described below
commit 321ee54fcfaa273ecac88ac4b3059a3eaa1ae3b3
Author: Josep Prat <[email protected]>
AuthorDate: Tue Oct 8 15:05:06 2024 +0200
MINOR Add upgrade instructiosn for 38 and 39 (#634)
Signed-off-by: Josep Prat <[email protected]>
Reviewers: Chia-Ping Tsai <[email protected]>
---
38/upgrade.html | 61 ++++++++++++++++++++++++++++
39/upgrade.html | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 185 insertions(+)
diff --git a/38/upgrade.html b/38/upgrade.html
index be12f0441..2dcdba7e8 100644
--- a/38/upgrade.html
+++ b/38/upgrade.html
@@ -21,6 +21,67 @@
<h4><a id="upgrade_3_8_0" href="#upgrade_3_8_0">Upgrading to 3.8.0 from any
version 0.8.x through 3.7.x</a></h4>
+ <h5><a id="upgrade_380_zk" href="#upgrade_380_zk">Upgrading
ZooKeeper-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 2.1.x, please see the
note in step 5 below about the change to the schema used to store consumer
offsets.
+ Once you have changed the inter.broker.protocol.version to the latest
version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Update server.properties on all brokers and add the following
properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the
message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current
value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to
match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.7</code>, <code>3.6</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION
(See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this
configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have
not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.7</code>, <code>3.6</code>, etc.)</li>
+ </ul>
+ </li>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any
problems.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to
<code>3.8</code>.
+ </li>
+ <li>Restart the brokers one by one for the new protocol version to
take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the
cluster to an older version.
+ </li>
+ <li>If you have overridden the message format version as instructed
above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers
have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 3.8 on each broker and
restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format
introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a
href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+ </ol>
+
+ <h5><a id="upgrade_380_kraft" href="#upgrade_380_kraft">Upgrading
KRaft-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 3.3.0, please see the
note in step 3 below. Once you have changed the metadata.version to the latest
version, it will not be possible to downgrade to a version prior to
3.3-IV0.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the metadata.version by running
+ <code>
+ bin/kafka-features.sh upgrade --metadata 3.8
+ </code>
+ </li>
+ <li>Note that cluster metadata downgrade is not supported in this
version since it has metadata changes.
+ Every <a
href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java";>MetadataVersion</a>
+ after 3.2.x has a boolean parameter that indicates if there are
metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means
this version has metadata changes).
+ Given your current and target versions, a downgrade is only
possible if there are no metadata changes in the versions between.</li>
+ </ol>
+
<h5><a id="upgrade_380_notable" href="#upgrade_380_notable">Notable
changes in 3.8.0</a></h5>
<ul>
<li>MirrorMaker 2 can now emit checkpoints for offsets mirrored before
the start of the Checkpoint task for improved offset translation.
diff --git a/39/upgrade.html b/39/upgrade.html
index faa486215..d769cc70d 100644
--- a/39/upgrade.html
+++ b/39/upgrade.html
@@ -20,6 +20,69 @@
<script id="upgrade-template" type="text/x-handlebars-template">
<h4><a id="upgrade_3_9_0" href="#upgrade_3_9_0">Upgrading to 3.9.0 from any
version 0.8.x through 3.8.x</a></h4>
+
+ <h5><a id="upgrade_390_zk" href="#upgrade_390_zk">Upgrading
ZooKeeper-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 2.1.x, please see the
note in step 5 below about the change to the schema used to store consumer
offsets.
+ Once you have changed the inter.broker.protocol.version to the latest
version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Update server.properties on all brokers and add the following
properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the
message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current
value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to
match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.8</code>, <code>3.7</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION
(See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this
configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have
not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.8</code>, <code>3.7</code>, etc.)</li>
+ </ul>
+ </li>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any
problems.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to
<code>3.9</code>.
+ </li>
+ <li>Restart the brokers one by one for the new protocol version to
take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the
cluster to an older version.
+ </li>
+ <li>If you have overridden the message format version as instructed
above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers
have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 3.8 on each broker and
restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format
introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a
href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+ </ol>
+
+ <h5><a id="upgrade_390_kraft" href="#upgrade_390_kraft">Upgrading
KRaft-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 3.3.0, please see the
note in step 3 below. Once you have changed the metadata.version to the latest
version, it will not be possible to downgrade to a version prior to
3.3-IV0.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the metadata.version by running
+ <code>
+ bin/kafka-features.sh upgrade --metadata 3.9
+ </code>
+ </li>
+ <li>Note that cluster metadata downgrade is not supported in this
version since it has metadata changes.
+ Every <a
href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java";>MetadataVersion</a>
+ after 3.2.x has a boolean parameter that indicates if there are
metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means
this version has metadata changes).
+ Given your current and target versions, a downgrade is only
possible if there are no metadata changes in the versions between.</li>
+ </ol>
+
+
<h5><a id="upgrade_390_notable" href="#upgrade_390_notable">Notable
changes in 3.9.0</a></h5>
<ul>
<li>In case you run your Kafka clusters with no execution permission
for the <code>/tmp</code> partition, Kafka will not work properly. It might
either refuse to start or fail
@@ -45,6 +108,67 @@
<h4><a id="upgrade_3_8_0" href="#upgrade_3_8_0">Upgrading to 3.8.0 from any
version 0.8.x through 3.7.x</a></h4>
+ <h5><a id="upgrade_380_zk" href="#upgrade_380_zk">Upgrading
ZooKeeper-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 2.1.x, please see the
note in step 5 below about the change to the schema used to store consumer
offsets.
+ Once you have changed the inter.broker.protocol.version to the latest
version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Update server.properties on all brokers and add the following
properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the
message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current
value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to
match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.7</code>, <code>3.6</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION
(See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this
configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have
not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g.
<code>3.7</code>, <code>3.6</code>, etc.)</li>
+ </ul>
+ </li>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any
problems.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to
<code>3.8</code>.
+ </li>
+ <li>Restart the brokers one by one for the new protocol version to
take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the
cluster to an older version.
+ </li>
+ <li>If you have overridden the message format version as instructed
above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers
have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 3.8 on each broker and
restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format
introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a
href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+ </ol>
+
+ <h5><a id="upgrade_380_kraft" href="#upgrade_380_kraft">Upgrading
KRaft-based clusters</a></h5>
+ <p><b>If you are upgrading from a version prior to 3.3.0, please see the
note in step 3 below. Once you have changed the metadata.version to the latest
version, it will not be possible to downgrade to a version prior to
3.3-IV0.</b></p>
+
+ <p><b>For a rolling upgrade:</b></p>
+
+ <ol>
+ <li>Upgrade the brokers one at a time: shut down the broker, update
the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that
the cluster's behavior and performance meets expectations.
+ </li>
+ <li>Once the cluster's behavior and performance has been verified,
bump the metadata.version by running
+ <code>
+ bin/kafka-features.sh upgrade --metadata 3.8
+ </code>
+ </li>
+ <li>Note that cluster metadata downgrade is not supported in this
version since it has metadata changes.
+ Every <a
href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java";>MetadataVersion</a>
+ after 3.2.x has a boolean parameter that indicates if there are
metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means
this version has metadata changes).
+ Given your current and target versions, a downgrade is only
possible if there are no metadata changes in the versions between.</li>
+ </ol>
+
<h5><a id="upgrade_380_notable" href="#upgrade_380_notable">Notable
changes in 3.8.0</a></h5>
<ul>
<li>MirrorMaker 2 can now emit checkpoints for offsets mirrored before
the start of the Checkpoint task for improved offset translation.
