Jeff Fenchel created STORM-2107:
-----------------------------------
Summary: Storm-kafka-client: Slow topology start breaks the Kafka
Client with session timeouts
Key: STORM-2107
URL: https://issues.apache.org/jira/browse/STORM-2107
Project: Apache Storm
Issue Type: Bug
Reporter: Jeff Fenchel
Specifically CommitFailedExceptions are being thrown when an attempt to commit
the offset is made. The problem is detailed here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records
On topology start, the spout will start emitting tuples before all bolts are
prepared. This means that tuples will hang around in the topology until
everything is ready to start processing. This caused the max spout pending
limit to be hit, and the spout to stop polling kafka for the duration of the
timeout. It seems like there could be other use cases that cause this too.
I propose we start polling on a timer kafka to avoid this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
