kete1987 opened a new pull request, #56427: URL: https://github.com/apache/spark/pull/56427
### What changes were proposed in this pull request? Make the test \\ esolved start offset greater than end offset (without latest)\\ in \KafkaRelationSuite\ deterministic. The two messages per partition are now produced with explicit, increasing CreateTime timestamps (\ase\, \ase + 1\) via \RecordBuilder\, so the second message (offset 1) is deterministically resolved by \offsetsForTimes\. ### Why are the changes needed? The test is flaky. It produces two messages per partition with no explicit timestamp and uses the second message's CreateTime as the ending-offset timestamp, expecting \offsetsForTimes\ to resolve to offset 1. When both messages land in the same millisecond they share a timestamp, \offsetsForTimes\ resolves to offset 0, and the hardcoded \\ esolved end offset 1\\ assertion fails with \\ esolved end offset 0\\. The existing \eventually(60.seconds)\ does not help: produced timestamps are fixed at produce time, so retrying always resolves the same offset (~3800 retries before timeout). Introduced by SPARK-52096. ### Does this PR introduce _any_ user-facing change? No. Test-only change. ### How was this patch tested? A/B, 50 runs x 5 suite variants (V1 / V2 / WithAdminV1 / WithAdminV2) = 250 test instances per arm: - With the fix: 250/250 passed. - Forcing the race (both messages produced with the same timestamp) reproduces the failure deterministically: \\The resolved start offset 3 is greater than the resolved end offset 0 ...\\ vs the expected \\... resolved end offset 1 ...\\, with \eventually\ exhausting ~3800 retries. ### Was this patch authored or co-authored using generative AI tooling? Co-authored with Claude (Anthropic), used for analysis, code generation and review assistance. Generated-by: Claude Sonnet 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
