This is an automated email from the ASF dual-hosted git repository.
chia7712 pushed a commit to branch 4.0
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/4.0 by this push:
new e206d515916 KAFKA-18364 add document to show the changes of metrics
and configs after removing zookeeper (#18365)
e206d515916 is described below
commit e206d51591681f358b71727118ba05ba7da5bbb4
Author: Ken Huang <s7133...@gmail.com>
AuthorDate: Thu Jan 16 05:02:01 2025 +0800
KAFKA-18364 add document to show the changes of metrics and configs after
removing zookeeper (#18365)
This patch introduces a new page to document the configs and metrics that
have been removed in the transition to 4.0. While these removed items lack a
formal deprecation cycle as they are part of KIP-500, KIP-500 itself does not
provide an exhaustive list of all impacted configs and metrics. Therefore, this
new page aims to assist Kafka users in understanding the specific configs and
metrics that have been removed in the 4.0 release.
Reviewers: Chia-Ping Tsai <chia7...@gmail.com>
---
docs/upgrade.html | 6 ++
docs/zk2kraft.html | 159 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 165 insertions(+)
diff --git a/docs/upgrade.html b/docs/upgrade.html
index 92491a5920d..30505e6c92a 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -29,6 +29,12 @@
Finally, care also needs to be taken when it comes to kafka clients
that are not part of Apache Kafka, please see
<a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0";>KIP-896</a>
for the details.
</li>
+ <li>
+ Before updating your Kafka cluster which build on Zookeeper to
Kafka 4.0, you must understand what the updates and changes
+ are between Zookeeper mode and Kraft mode. See the following page
to learn about some of the significant changes in
+ latest Kafka releases.
+ <!--#include virtual="zk2kraft.html" -->
+ </li>
<li>A number of deprecated classes, methods, configurations and tools
have been removed.
<ul>
<li><b>Common</b>
diff --git a/docs/zk2kraft.html b/docs/zk2kraft.html
new file mode 100644
index 00000000000..c7ec880cc73
--- /dev/null
+++ b/docs/zk2kraft.html
@@ -0,0 +1,159 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<script><!--#include virtual="js/templateData.js" --></script>
+
+<script id="zk2kraft-template" type="text/x-handlebars-template"></script>
+
+<div class="p-zk2kraft">
+ <h4 class="anchor-heading">Significant Changes in Kafka 4.0 Release</h4>
+ <p>The following are some of the updates in Kafka 4.0 release:</p>
+ <h5 class="anchor-heading">Removal Zookeeper configs</h5>
+ <ul>
+ <li>
+ <p>
+ The password encoder-related configurations have been removed.
These configurations were used in
+ ZooKeeper mode to define the key and backup key for encrypting
sensitive data (e.g., passwords),
+ specify the algorithm and key generation method for password
encryption (e.g., AES, RSA), and control
+ the key length and encryption strength.
+ </p>
+ <ul>
+ <li><code>password.encoder.secret</code></li>
+ <li><code>password.encoder.old.secret</code></li>
+ <li><code>password.encoder.keyfactory.algorithm</code></li>
+ <li><code>password.encoder.cipher.algorithm</code></li>
+ <li><code>password.encoder.key.length</code></li>
+ <li><code>password.encoder.iterations</code></li>
+ </ul>
+ <p>
+ In Kraft mode, Kafka stores sensitive data in records, and the
data is not encrypted in Kafka.
+ </p>
+ </li>
+ <li>
+ <p>
+ Removed <code>control.plane.listener.name</code>. Kafka relies
on ZooKeeper to manage metadata, but some
+ internal operations (e.g., communication between controllers
(a.k.a., broker controller) and brokers)
+ still require Kafka’s internal control plane for coordination.
+ </p>
+ <p>
+ In KRaft mode, Kafka eliminates its dependency on ZooKeeper,
and the control plane functionality is
+ fully integrated into Kafka itself. The process roles are
clearly separated: brokers handle data-related
+ requests, while the controllers (a.k.a., quorum controller)
manages metadata-related requests. The controllers
+ use the Raft protocol for internal communication, which
operates differently from the ZooKeeper model. Use the
+ following parameters to configure the control plane listener:
+ </p>
+ <ul>
+ <li><code>controller.listener.names</code></li>
+ <li><code>listeners</code></li>
+ <li><code>listener.security.protocol.map</code></li>
+ </ul>
+ </li>
+ <li>
+ <p>
+ Removed graceful broker shutdowns-related configurations.
These configurations were used in ZooKeeper mode
+ to define the maximum number of retries and the retry backoff
time for controlled shutdowns. It can
+ reduce the risk of unplanned leader changes and data
inconsistencies.
+ </p>
+ <ul>
+ <li><code>controlled.shutdown.max.retries</code></li>
+ <li><code>controlled.shutdown.retry.backoff.ms</code></li>
+ </ul>
+ <p>
+ In KRaft mode, Kafka uses the Raft protocol to manage
metadata. The broker shutdown process differs from
+ ZooKeeper mode as it is managed by the quorum-based
controller. The shutdown process is more reliable
+ and efficient due to automated leader transfers and metadata
updates handled by the controller.
+ </p>
+ </li>
+ <li>
+ <p>
+ Remove the broker id generation-related configurations. These
configurations were used in ZooKeeper mode
+ to define the broker id, specify the broker id auto
generation, and control the broker id generation process.
+ </p>
+ <ul>
+ <li><code>reserved.broker.max.id</code></li>
+ <li><code>broker.id.generation.enable</code></li>
+ <li><code>broker.id</code></li>
+ </ul>
+ <p>
+ Kafka use the node id in Kraft mode to identify servers.
+ </p>
+ <ul>
+ <li><code>node.id</code></li>
+ </ul>
+ </li>
+ <li>
+ <p>
+ Removed Zookeeper related configurations.
+ </p>
+ <ul>
+ <li><code>zookeeper.connect</code></li>
+ <li><code>zookeeper.session.timeout.ms</code></li>
+ <li><code>zookeeper.connection.timeout.ms</code></li>
+ <li><code>zookeeper.set.acl</code></li>
+ <li><code>zookeeper.max.in.flight.requests</code></li>
+ <li><code>zookeeper.ssl.client.enable</code></li>
+ <li><code>zookeeper.clientCnxnSocket</code></li>
+ <li><code>zookeeper.ssl.keystore.location</code></li>
+ <li><code>zookeeper.ssl.keystore.password</code></li>
+ <li><code>zookeeper.ssl.keystore.type</code></li>
+ <li><code>zookeeper.ssl.truststore.location</code></li>
+ <li><code>zookeeper.ssl.truststore.password</code></li>
+ <li><code>zookeeper.ssl.truststore.type</code></li>
+ <li><code>zookeeper.ssl.protocol</code></li>
+ <li><code>zookeeper.ssl.enabled.protocols</code></li>
+ <li><code>zookeeper.ssl.cipher.suites</code></li>
+
<li><code>zookeeper.ssl.endpoint.identification.algorithm</code></li>
+ <li><code>zookeeper.ssl.crl.enable</code></li>
+ <li><code>zookeeper.ssl.ocsp.enable</code></li>
+ </ul>
+ </li>
+ </ul>
+ <h5 class="anchor-heading">Removal metrics</h5>
+ <ul>
+ <li>
+ <p>
+ Remove the following metrics related to ZooKeeper.
+ <code>ControlPlaneNetworkProcessorAvgIdlePercent</code>
+ is to monitor the average fraction of time the network
processors are idle. The other
<code>ControlPlaneExpiredConnectionsKilledCount</code>
+ is to monitor the total number of connections disconnected,
across all processors.
+ </p>
+ <ul>
+
<li><code>ControlPlaneNetworkProcessorAvgIdlePercent</code></li>
+ <li><code>ControlPlaneExpiredConnectionsKilledCount</code></li>
+ </ul>
+ <p>
+ In Kraft mode, Kafka also provides metrics to monitor the
network processors and expired connections.
+ Use the following metrics to monitor the network processors
and expired connections:
+ </p>
+ <ul>
+ <li><code>NetworkProcessorAvgIdlePercent</code></li>
+ <li><code>ExpiredConnectionsKilledCount</code></li>
+ </ul>
+ </li>
+ <li>
+ <p>
+ Remove the metrics which is monitoring the latency in
milliseconds for ZooKeeper requests from broker.
+ </p>
+ <ul>
+
<li><code>kafka.server:type=ZooKeeperClientMetrics,name=ZooKeeperRequestLatencyMs</code></li>
+ </ul>
+ <p>
+ In Kraft mode, Zookeeper is not used, so the metrics is
removed.
+ </p>
+ </li>
+ </ul>
+</div>
\ No newline at end of file
