pjfanning opened a new pull request, #646: URL: https://github.com/apache/pekko-connectors-kafka/pull/646
### Motivation The testkit could start Confluent Platform 8.x (Kafka 4) images only far enough to skip the ZooKeeper container (#332); the broker itself was still configured in ZooKeeper mode, which Kafka 4 no longer supports, so the container never started. CI also only tested against a Kafka 3 server. ### Modification - `PekkoConnectorsKafkaContainer` detects 8.x image tags and configures the broker in KRaft mode (`CLUSTER_ID`, node id, process roles, controller listener and quorum voters) and runs `/etc/confluent/docker/ensure` so the KRaft storage is formatted; ZooKeeper mode is kept for older images. The KRaft env setup follows the approach of testcontainers-java's `KafkaContainer` (this class is already noted as a copy of it). - `KafkaContainerCluster` decides ZooKeeper vs KRaft from the actual Kafka image tag (instead of a system property, which did not reach forked test JVMs when only the env var was set), shares one cluster id and controller quorum across brokers, and uses `--bootstrap-server` for the readiness-check producer since Kafka 4 removed `--broker-list`. - `withBrokerNum` now always applies the `broker-<n>` network alias (previously broker number 1 never got its alias), which the KRaft quorum voters rely on. - `DEFAULT_CONFLUENT_PLATFORM_VERSION` falls back to the `CONFLUENT_PLATFORM_VERSION` env var so CI can select the server version. - CI runs the test job and the integration tests against both Confluent 7.9.2 (Kafka 3) and 8.3.1 (Kafka 4). ### Result Tests run against a Kafka 4 server in CI while Kafka 3 server coverage is retained. Testkit users can test against Confluent Platform 8.x images via `CONFLUENT_PLATFORM_VERSION` or the testcontainers settings. ### Tests - `CONFLUENT_PLATFORM_VERSION=8.3.1 sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.TimestampSpec"` — KRaft broker + schema registry, passed locally - `CONFLUENT_PLATFORM_VERSION=8.3.1 sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.ReconnectSpec"` — broker stop/start under KRaft, passed locally - `sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.TimestampSpec"` — default 7.9.2 ZooKeeper mode, passed locally - `sbt testkit/mimaReportBinaryIssues` — passed - `sbt javafmtAll` (JDK 17) and native `scalafmt` run on changed files - multi-broker KRaft coverage left to the new CI integration-test matrix (Confluent 8.3.1) ### References Fixes #301 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
