[ 
https://issues.apache.org/jira/browse/STORM-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kush Khandelwal updated STORM-3081:
-----------------------------------
    Description: 
A single thread is pushing some messages to kafka serially and we consume it at 
the storm's end using storm-kafka-client.
After a few requests, the consumer is not able to consume from the queue 
blocking the thread which waits for the response from storm's end. 
We added one more consumer with a different consumer group and there the 
messages are getting read properly.
So we know there is some problem at the storm kafka client consumer's end.
The kafka spout config is written like this - 

KafkaSpoutConfig<String, String> kafkaSpoutConfig = 
KafkaSpoutConfig.builder(kafkaProperties.getProperty("bootstrap.servers"), 
stormProperties.getProperty("TOPIC"))
                
.setFirstPollOffsetStrategy(KafkaSpoutConfig.FirstPollOffsetStrategy.LATEST)
                .setRecordTranslator(new MessageDeserializer(), arguments)
                .build();

I can't seem to figure out the issue.
Can someone please help me out?

Thanks.

  was:
A single thread is pushing some messages to kafka serially and we consume it at 
the storm's end using storm-kafka-client.
After a few requests, the consumer is not able to consume from the queue 
blocking the thread. 
We added one more consumer with a different consumer group and there the 
messages are getting read properly.
So we know there is some problem at the storm kafka client consumer's end.
The kafka spout config is written like this - 

KafkaSpoutConfig<String, String> kafkaSpoutConfig = 
KafkaSpoutConfig.builder(kafkaProperties.getProperty("bootstrap.servers"), 
stormProperties.getProperty("TOPIC"))
                
.setFirstPollOffsetStrategy(KafkaSpoutConfig.FirstPollOffsetStrategy.LATEST)
                .setRecordTranslator(new MessageDeserializer(), arguments)
                .build();

I can't seem to figure out the issue.
Can someone please help me out?

Thanks.


> Storm kafka client not consuming messages properly
> --------------------------------------------------
>
>                 Key: STORM-3081
>                 URL: https://issues.apache.org/jira/browse/STORM-3081
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka, storm-kafka-client
>    Affects Versions: 1.2.1
>            Reporter: Kush Khandelwal
>            Priority: Blocker
>
> A single thread is pushing some messages to kafka serially and we consume it 
> at the storm's end using storm-kafka-client.
> After a few requests, the consumer is not able to consume from the queue 
> blocking the thread which waits for the response from storm's end. 
> We added one more consumer with a different consumer group and there the 
> messages are getting read properly.
> So we know there is some problem at the storm kafka client consumer's end.
> The kafka spout config is written like this - 
> KafkaSpoutConfig<String, String> kafkaSpoutConfig = 
> KafkaSpoutConfig.builder(kafkaProperties.getProperty("bootstrap.servers"), 
> stormProperties.getProperty("TOPIC"))
>                 
> .setFirstPollOffsetStrategy(KafkaSpoutConfig.FirstPollOffsetStrategy.LATEST)
>                 .setRecordTranslator(new MessageDeserializer(), arguments)
>                 .build();
> I can't seem to figure out the issue.
> Can someone please help me out?
> Thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to