nzw921rx opened a new pull request, #11442: URL: https://github.com/apache/seatunnel/pull/11442
### Purpose of this pull request Fix flaky tests in the Kafka E2E module by making startup and partition-discovery assertions wait for actual consumption readiness instead of assuming the source is ready immediately after the job reaches RUNNING. **Root cause:** The Kafka CI job intermittently fails with `ConditionTimeout` in tests that use `start_mode = latest`: ``` KafkaIT.testSourceKafkaRestoreWithLatestMode KafkaIT.testDynamicPartitionDiscovery ``` For latest-mode restore, records produced immediately after `awaitJobRunning` can be skipped if the Kafka source has not finished subscribing and seeking yet. For dynamic partition discovery, records sent to the new partition immediately after partition creation can be skipped before SeaTunnel discovers the new partition and starts consuming it. **Changes in `KafkaIT`:** - Add a probe record loop before the latest-mode assertions to confirm the running job is consuming newly produced records - Count first-run and restore records by their payload instead of relying on the full sink partition offset count - Add a probe record loop after increasing Kafka partitions to confirm SeaTunnel has discovered and consumed the new partition - Assert the dynamic-partition test against `new_partition_message_` records only, so probe records do not affect the result ### Does this PR introduce _any_ user-facing change? No. This change only affects E2E test stability. ### How was this patch tested? Ran: ``` ./mvnw spotless:apply ``` Per the flaky-CI fix workflow, no E2E tests were run locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
