The GitHub Actions job "CI" on pekko-connectors-kafka.git/main has failed. Run started by GitHub user pjfanning (triggered by pjfanning).
Head commit for run: 6b970f1e2ed5766287fa8eff73ac8f65a3d5f2af / PJ Fanning <[email protected]> test with Kafka 4 server as well as Kafka 3 (#646) * test with Kafka 4 server as well as Kafka 3 Motivation: The testkit could start Confluent Platform 8.x (Kafka 4) images only far enough to skip the ZooKeeper container; the broker itself was still configured in ZooKeeper mode, which Kafka 4 no longer supports, so the container never started. CI also only tested against a Kafka 3 server. Modification: - PekkoConnectorsKafkaContainer detects 8.x image tags and configures the broker in KRaft mode (CLUSTER_ID, node id, process roles, controller listener and quorum voters) and runs /etc/confluent/docker/ensure so the KRaft storage is formatted; ZooKeeper mode is kept for older images. - KafkaContainerCluster decides ZooKeeper vs KRaft from the actual Kafka image tag (instead of a system property that did not reach forked test JVMs when only the env var was set), shares one cluster id and controller quorum across brokers, and uses --bootstrap-server for the readiness check producer since Kafka 4 removed --broker-list. - withBrokerNum now always applies the broker-<n> network alias (previously broker number 1 never got its alias), which KRaft quorum voters rely on. - DEFAULT_CONFLUENT_PLATFORM_VERSION falls back to the CONFLUENT_PLATFORM_VERSION env var so CI can select the server version. - CI runs the test job and the integration tests against both Confluent 7.9.2 (Kafka 3) and 8.3.1 (Kafka 4). Result: Tests run against a Kafka 4 server in CI while Kafka 3 server coverage is retained. Testkit users can test against Confluent Platform 8.x images via CONFLUENT_PLATFORM_VERSION or the testcontainers settings. Tests: - CONFLUENT_PLATFORM_VERSION=8.3.1 sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.TimestampSpec" (KRaft broker + schema registry, passed) - CONFLUENT_PLATFORM_VERSION=8.3.1 sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.ReconnectSpec" (broker stop/start under KRaft, passed) - sbt "tests/testOnly org.apache.pekko.kafka.scaladsl.TimestampSpec" (default 7.9.2 ZooKeeper mode, passed) - sbt testkit/mimaReportBinaryIssues (passed) - multi-broker int-tests left to the new CI matrix References: Fixes #301 * fix duplicate broker network alias breaking multi-broker KRaft quorum Motivation: The 3-broker KRaft integration tests failed in CI: the constructor-default broker-1 network alias was never removed when the broker number changed, because testcontainers' getNetworkAliases() returns a copy of the alias list, so remove() mutated a throwaway list. Every broker container then also carried the broker-1 alias, and Docker DNS resolved broker-1 round-robin across all brokers (visible in the CI container logs), which broke the controller quorum and the failover tests. Modification: withBrokerNum builds the new alias list and applies it via setNetworkAliases, which writes through to the container definition. Result: Each broker container carries exactly its own broker-<n> alias. Tests: - standalone check instantiating brokers 0..2: each container ends up with exactly its own broker-<n> alias - multi-broker KRaft covered by the CI integration-test matrix (8.3.1) References: Refs #301 Report URL: https://github.com/apache/pekko-connectors-kafka/actions/runs/33490751569 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
