Hi David,
we found an issue with Kafka Streams that definitely is a regression. A
commit fixed a leak, introduced a bug that prevented Kafka Streams from
re-initializing its transactional producer in case the producer is
fenced. The consequence of not re-initializing the transactional
producer is an infinite loop of ProducerFencedExceptions with
corresponding rebalances (approx every min).
I opened a PR that reverts the defective commit:
https://github.com/apache/kafka/pull/19078
We tested the revert in our internal soak and everything seems fine again.
May I cherry-pick the revert to 4.0?
Best,
Bruno
On 28.02.25 22:59, Ismael Juma wrote:
Given that this is effectively a documentation fix, a cherry-pick makes
sense.
Ismael
On Fri, Feb 28, 2025 at 1:30 PM Jun Rao <j...@confluent.io.invalid> wrote:
Hi, David,
We just fixed https://issues.apache.org/jira/browse/KAFKA-18864. Not sure
if you want to cherry-pick it to 4.0.
Thanks,
Jun
On Thu, Feb 27, 2025 at 7:22 PM Ismael Juma <m...@ismaeljuma.com> wrote:
Thanks for the testing Kirk. Can you please file an issue for the
repeated
log to make sure it doesn't get lost in this thread?
Ismael
On Thu, Feb 27, 2025 at 2:54 PM Kirk True <k...@kirktrue.pro> wrote:
Hi David,
I focused on testing the clients using Java 11:
My steps:
1. Start the broker via the native Docker image
(apache/kafka-native:4.0.0-rc0)
2. Wrote a simple client to write and read messages, using
kafka-clients
4.0.0 from https://repository.apache.org/content/groups/staging
3. Ran my client with group.protocol set to CLASSIC
4. I then ran the client with CONSUMER
5. Finally, I ran two clients instances in the same consumer group, one
with a group.protocol set to CLASSIC and the other with CONSUMER
Everything worked as expected.
I did notice the following message showing up repeatedly in the broker
logs after my tests completed:
Generated 0 tombstone records while cleaning up group metadata in 0
milliseconds.
(org.apache.kafka.coordinator.group.GroupCoordinatorShard)
Not sure if that's expected or not.
Thanks,
Kirk
On Wed, Feb 26, 2025, at 5:15 PM, Chia-Ping Tsai wrote:
hi David
I apologize for not testing the release candidate (RC) as I focused
on
reviewing the documentation. I have created tickets to address issues
in
the documentation [0][1][2][3], which will be merged into 4.0 as the
risk
is minimal.
Please let me know if you have any other options for those tickets.
[0] https://issues.apache.org/jira/browse/KAFKA-18869
[1] https://issues.apache.org/jira/browse/KAFKA-18868
[2] https://issues.apache.org/jira/browse/KAFKA-18850
[3] https://issues.apache.org/jira/browse/KAFKA-18849
Best,
Chia-Ping
On 2025/02/22 10:16:43 David Jacot wrote:
Hello Kafka users, developers and client-developers,
This is the first candidate for release of Apache Kafka 4.0.0. We
still have some remaining blockers but we figured that getting a
first
release candidate will help the community to test this major
release.
- This is the first release without Apache Zookeeper
- The Next Generation of the Consumer Rebalance Protocol is
Generally
Available
- The Transactions Server-Side Defense (Phase 2) is Generally
Available
- Queues for Kafka is in Early Access
- Kafka uses log4j2
- Drop broker and tools support for Java 11
- Remove old client protocol API versions
Release notes for the 4.0.0 release:
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/RELEASE_NOTES.html
*** Please download and test the release. Voting is not necessary
as
we still have blockers.
Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS
* Release artifacts to be voted upon (source and binary):
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/
* Docker release artifacts to be voted upon:
apache/kafka:4.0.0-rc0
apache/kafka-native:4.0.0-rc0 (Building the native image failed, I
need to investigate it)
* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/
* Javadoc:
https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc0/javadoc/
* Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag:
https://github.com/apache/kafka/releases/tag/4.0.0-rc0
* Documentation:
https://kafka.apache.org/40/documentation.html
* Protocol:
https://kafka.apache.org/40/protocol.html
* Successful CI builds for the 4.0 branch:
Unit/integration tests:
https://github.com/apache/kafka/actions/runs/13459676207
System tests: TBD
* Successful Docker Image Github Actions Pipeline for 4.0 branch:
Docker Build Test Pipeline (JVM):
https://github.com/apache/kafka/actions/runs/13471603921
Docker Build Test Pipeline (Native):
https://github.com/apache/kafka/actions/runs/13471605941
/**************************************
Thanks,
David
