Jeff Fenchel created STORM-2107:

             Summary: Storm-kafka-client: Slow topology start breaks the Kafka 
Client with session timeouts 
                 Key: STORM-2107
             Project: Apache Storm
          Issue Type: Bug
            Reporter: Jeff Fenchel

Specifically CommitFailedExceptions are being thrown when an attempt to commit 
the offset is made. The problem is detailed here:

On topology start, the spout will start emitting tuples before all bolts are 
prepared. This means that tuples will hang around in the topology until 
everything is ready to start processing. This caused the max spout pending 
limit to be hit, and the spout to stop polling kafka for the duration of the 
timeout. It seems like there could be other use cases that cause this too. 

I propose we start polling on a timer kafka to avoid this.

This message was sent by Atlassian JIRA

Reply via email to