hi Maros Thanks for the proposal! Would you mind creating a KIP to describe the details?
> Version coverage is also narrower since the framework only targets KRaft 3.9+ I am a bit concerned about this since our testing baseline starts from 2.0. For example, we still need to test Kafka 2.0 clients against 4.x servers. How would the new framework handle these older compatibility tests? Best, Chia-Ping Maroš Orsák <[email protected]> 於 2026年4月9日週四 下午5:53寫道： > Hello Kafka Devs, > > After years of working with Kafka system tests, I would like to propose > moving them from Python/Ducktape to Java/Testcontainers, building on top of > the @ClusterTest framework that contributors already know. > > The motivation is making system tests more effective and accessible for > everyone. Instead of learning a second language and a niche test runner, > contributors would write system tests the same way they write integration > tests just with a different execution mode. > > The benefits include a single language and framework with reduced > maintenance. Some tests also run significantly faster (e.g., compression > test dropped ~85s to ~12s), though that will not hold for every scenario. > > It is not without downsides. Ducktape can do things containers cannot > (e.g., > real VM isolation, Kibosh fault injection, multi-host setups). Version > coverage is also narrower since the framework only targets KRaft 3.9+, so > ZooKeeper-based versions cannot be tested. This is deliberate (i.e., KRaft > has been production-ready since 3.6.0, and ZooKeeper mode was removed in > 4.0, so investing in ZK-based test infrastructure isn't justified). Some > tests will likely stay in Ducktape, and that is fine. > > If the community agrees this direction makes sense, I will follow up with > the concrete details: > > > 1. *which *tests can be migrated now, > 2. *what *features @ClusterTest still needs, > 3. *how *to wire up CI so both frameworks run from one place. > > Looking forward to hearing your thoughts. > > Cheers, > > Maros >
