sjhajharia commented on code in PR #668: URL: https://github.com/apache/kafka-site/pull/668#discussion_r1993107835
########## blog.html: ########## @@ -22,6 +22,286 @@ <!--#include virtual="includes/_nav.htm" --> <div class="right"> <h1 class="content-title">Blog</h1> + <article> + <h2 class="bullet"> + <a id="apache_kafka_400_release_announcement"></a> + <a href="#apache_kafka_400_release_announcement">Apache Kafka 4.0.0 Release Announcement</a> + </h2> + 17 March 2025 - David Jacot + + <p> + We are proud to announce the release of Apache Kafka® 4.0 This release contains many new + features and improvements. This blog post will highlight some of the more prominent features. + For a full list of changes, be sure to check the <a href="https://archive.apache.org/dist/kafka/4.0.0/RELEASE_NOTES.html">release notes</a>. + </p> + <p> + Apache Kafka 4.0 is a significant milestone, marking the first major release to operate entirely + without Apache ZooKeeper®. By running in KRaft mode by default, Kafka simplifies + deployment and management, eliminating the complexity of maintaining a separate ZooKeeper + ensemble. This change significantly reduces operational overhead, enhances scalability, and + streamlines administrative tasks. We want to take this as an opportunity to express our gratitude + to the ZooKeeper community and say, thank you! ZooKeeper was the backbone of Kafka for + more than 10 years, and it did serve Kafka very well. Kafka would most likely not be what it is + today without it. We don’t take this for granted and highly appreciate all the hard work the + community invested to build ZooKeeper. Thank you! + </p> + <p> + Kafka 4.0 also brings the general availability of <a href="https://cwiki.apache.org/confluence/x/HhD1D">KIP-848</a> introducing a powerful new consumer + group protocol designed to dramatically improve rebalance performance. This optimization + significantly reduces downtime and latency, enhancing the reliability and responsiveness of + consumer groups, especially in large-scale deployments. + </p> + <p> + Additionally, we are excited to offer <a href="https://cwiki.apache.org/confluence/x/i4ogF">early access</a> to + Queues for Kafka (<a href="https://cwiki.apache.org/confluence/x/4hA0Dw">KIP-932</a>), enabling Kafka to support traditional queue semantics directly. This + feature extends Kafka’s versatility, making it an ideal messaging platform for a wider range of + use cases, particularly those requiring point-to-point messaging patterns. + </p> + <p> + Kafka's major releases, such as version 4.0, remove APIs deprecated for at least 12 months to simplify the platform + and encourage the adoption of newer features. Notably, in Kafka 4.0, Kafka Clients and Kafka Streams require Java 11, + while Kafka Brokers, Connect, and Tools now require Java 17. This release also updated the minimum supported client + and broker versions (<a href="https://cwiki.apache.org/confluence/x/K5sODg">KIP-896</a>) and defined new baseline + requirements for supported upgrade paths. Further details are provided in subsequent sections of this document. + </p> + <p> + These highlights illustrate Kafka's continued evolution to meet the demands of modern, + scalable, and efficient data streaming and messaging architectures. + </p> + <p> + See the <a href="https://kafka.apache.org/documentation.html#upgrade_4_0_0">Upgrading to 4.0</a> section in the + documentation for the list of notable changes and detailed upgrade steps. + </p> + + <h3>Kafka Broker, Controller, Producer, Consumer and Admin Client</h3> + + <ul> + <li> + <a href="https://cwiki.apache.org/confluence/x/HhD1D">KIP-848: The Next Generation of the Consumer Rebalance Protocol</a> + <br> + Apache Kafka also says goodbye to “stop-the-world” rebalances with the general availability of the + next generation of the consumer rebalance protocol. It increases the stability and the performance + of consumer groups while simplifying clients. The new protocol is enabled by default on the server + side. Consumers must opt-in by setting <code>group.protocol=consumer</code>. + See <a href="https://kafka.apache.org/40/documentation.html#consumer_rebalance_protocol">here</a> for details. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/B40ODg">KIP-890: Transactions Server-Side Defense</a> + <br> + The second phase of KIP-890 has been completed. It reduces the chances of “zombie transactions” + during producer failures. See <a href="https://kafka.apache.org/40/documentation.html#transaction_protocol">here</a> for details. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/4hA0Dw">KIP-932: Queues for Kafka (Early Access)</a> + <br> + This KIP introduces the concept of a share group as a way of enabling cooperative consumption + using Kafka topics. It does not add the concept of a “queue” to Kafka per se, but rather that + introduces cooperative consumption to accommodate these queuing use-cases using regular Kafka + topics. Share groups make this possible. You can think of a share group as roughly equivalent to a + “durable shared subscription” in existing systems. + Please see the <a href="https://cwiki.apache.org/confluence/x/i4ogF">Early Access Release Notes</a> + for limitations and configurations. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/mpOzDw">KIP-966: Eligible Leader Replicas (Preview)</a> + <br> + KIP-966 introduces Eligible Leader Replicas (ELR) in preview in 4.0. ELR is a subset of the ISR + replicas guaranteed to have complete data up to the high-watermark. ELRs are safe for leader + election, preventing data loss. See <a href="https://kafka.apache.org/40/documentation.html#eligible_leader_replicas">here</a> for details. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/MYtEE">KIP-996: Pre-Vote</a> + <br> + KIP-996 introduces a "Pre-Vote" mechanism to reduce unnecessary KRaft leader elections in Apache + Kafka clusters. By allowing nodes to check their eligibility for leadership before initiating an election, + this approach minimizes disruptions caused by network partitions or transient issues. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/XA-OEg">KIP-1076: Metrics for client applications KIP-714 extension</a> + <br> + With <a href="https://cwiki.apache.org/confluence/x/XA-OEg">KIP-714</a>, cluster operators can now collect + client metrics directly from brokers using a plugin. This makes it much easier to monitor client behavior. + However, KIP-714 only gathers metrics from Kafka clients (admin, consumer, and producer) and doesn't cover + application-level metrics for embedded clients, such as Kafka Streams. To fully understand performance, + it's important to collect both client metrics and application-specific metrics. This KIP introduces a way + for applications that use Kafka clients to include their own metrics alongside the existing client metrics. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/NIyMEw">KIP-1106: Add duration based offset reset option for consumer clients</a> + <br> + Kafka consumer supports <code>auto.offset.reset</code> config option, which is used when there is no initial + offset in Kafka or if the current offset does not exist any more on the server. This config currently + supports earliest/latest/none options. Currently consumer resets might force applications to + reprocess large amounts of data from earlier offsets. With infinite storage, it's beneficial to have a + duration based offset reset strategy. This KIP allows applications to consume/initialise from a fixed + duration when there is no initial offset in Kafka. + </li> + <li> + <a href="https://cwiki.apache.org/confluence/x/XoowEg">KIP-1043: Administration of groups</a> + <br> + KIP-848 and KIP-932 introduce new group types, consumer and share respectively. The Admin Client + API has limitations when dealing with newer group types, sometimes returning errors or incorrectly + stating that groups don't exist. To address this challenge, a new command-line tool, kafka-groups.sh, Review Comment: Suggestion: `<code>kafka-groups.sh</code>` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org