[
https://issues.apache.org/jira/browse/KAFKA-19811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18042606#comment-18042606
]
Erik van Oosten commented on KAFKA-19811:
-----------------------------------------
Okay, good to know this is not an expected situation/known problem. We _are_
using the java consumer under the hood, so we could enable this tracing. I will
also think about ways our layer on top of the java consumer could cause this
behavior.
BTW, this only happens when running lots of tests simultaneously on the free
tier GitHub runners, running against an in-JVM Kafka broker. We have not been
able to reproduce this on a developer machine, nor do I know of users
complaining about consumers being stuck.
> Acked record on new topic not immediately visible to consumer
> -------------------------------------------------------------
>
> Key: KAFKA-19811
> URL: https://issues.apache.org/jira/browse/KAFKA-19811
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: Erik van Oosten
> Priority: Minor
>
> h2. Steps to reproduce
> * program uses a single broker (we see the issue with an in-JVM embedded
> kafka server)
> * create a new topic with 1 partition
> * produce a record with {{acks=all}}
> * await acknowledgement from the broker
> * start a consumer (configured to read from beginning of topic)
> * spuriously, _the consumer never sees the record_
> The problem seems to occur more on a very busy server (e.g. a build server
> running many tests in parallel). We have not seen it on a modern laptop.
> A delay of 10ms after receiving the acknowledgement, and before starting the
> consumer, makes the record always visible.
> We observe this problem in ~1 in 6 runs of
> [zio-kafka|https://github.com/zio/zio-kafka]'s test suite, when run from a
> GitHub action. Since we run the test-suite for 3 different JVM versions, more
> than half of the zio-kafka builds fails due to this issue.
> h2. Expected behavior
> A record, produced with {{{}acks=all{}}}, of which producing was
> acknowledged, should be immediately visible for all consumers.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
