Jason Gustafson commented on KAFKA-4238:

We've debated a little bit on the current behavior of relying on 
{{metadata.max.age.ms}} when a subscribed topic doesn't exist. An alternative 
might be to retry fetching metadata more rapidly (say using 
{{retry.backoff.ms}}) in this case. It's a little annoying to have to tune down 
the metadata refresh interval when the consumer is really only waiting for 
initial topic creation. That would definitely help out in test cases where 
there is often a race condition with topic creation and consumer subscription.

> consumer-subscription not working, when accessing a newly created topic 
> immediately after its creation with the AdminUtils
> --------------------------------------------------------------------------------------------------------------------------
>                 Key: KAFKA-4238
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4238
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions:,
>            Reporter: Florian Witteler
> I created a test-project to reproduce the bug.
> https://github.com/FloWi/kafka-topic-creation-bug
> We use a docker container that creates a fresh topic before a testsuite gets 
> executed (see {{trait FreshKafkaTopics}}). That trait uses the AdminUtils to 
> create the topic. 
> If we access the newly created topic directly after its creation, the 
> subscriber is broken. It sometimes works though (<5%), so it seems to be a 
> race-condition.
> If I put a {{Thread.sleep(1000)}} after the topic-creation, everything's fine 
> though.
> So, the problem is twofold:
> - {{AdminUtils.createTopic}} should block until the topic-creation is 
> completed
> - {{new KafkaConsumer[String, 
> String](props).subscribe(util.Arrays.asList(topic))}} should throw an 
> exception, when the topic is "not ready"

This message was sent by Atlassian JIRA

Reply via email to