[
https://issues.apache.org/jira/browse/KAFKA-19811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18031159#comment-18031159
]
Andrew Schofield commented on KAFKA-19811:
------------------------------------------
I've had a quick look at the code, but I must admit that I'm a bit of a Scala
amateur.
I suggest using a ConsumerRebalanceListener to ensure that your test has been
assigned partitions to consume. Given the symptom about a test failing to
receive the first record fails to receive any of them, it sounds like its not
been assigned the partition to subscribe from. If this has occurred since the
move to Apache Kafka 4.x, it may be related to the difference in behaviour of
the `poll(long)` method which was removed and its replacement with
`poll(Duration)`.
> Acked record on new topic not immediately visible to consumer
> -------------------------------------------------------------
>
> Key: KAFKA-19811
> URL: https://issues.apache.org/jira/browse/KAFKA-19811
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: Erik van Oosten
> Priority: Minor
>
> h2. Steps to reproduce
> * program uses a single broker (we see the issue with an in-JVM embedded
> kafka server)
> * create a new topic with 1 partition
> * produce a record with {{acks=all}}
> * await acknowledgement from the broker
> * start a consumer (configured to read from beginning of topic)
> * spuriously, _the consumer never sees the record_
> The problem seems to occur more on a very busy server (e.g. a build server
> running many tests in parallel). We have not seen it on a modern laptop.
> A delay of 10ms after receiving the acknowledgement, and before starting the
> consumer, makes the record always visible.
> We observe this problem in ~1 in 6 runs of
> [zio-kafka|https://github.com/zio/zio-kafka]'s test suite, when run from a
> GitHub action. Since we run the test-suite for 3 different JVM versions, more
> than half of the zio-kafka builds fails due to this issue.
> h2. Expected behavior
> A record, produced with {{{}acks=all{}}}, of which producing was
> acknowledged, should be immediately visible for all consumers.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
