[
https://issues.apache.org/jira/browse/STORM-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399079#comment-16399079
]
Stig Rohde Døssing commented on STORM-2994:
-------------------------------------------
Ok, thanks. I'm asking because the debug log you posted is definitely from
1.1.0, and versions prior to 1.0.6, 1.1.2 or 1.2.0 are known to suffer from a
number of issues.
Your best bet to figure out what's going on would be to enable trace logging
for this class
https://github.com/apache/storm/blob/v1.1.2/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/internal/OffsetManager.java.
The log you posted is here
https://github.com/apache/storm/blob/v1.1.2/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/internal/OffsetManager.java#L156,
and if you enable trace logging, the class will log all the message offsets
that are pending commit. That should make it pretty easy to spot why the
messages can't be committed.
You might also want to look at trace logging for
https://github.com/apache/storm/blob/v1.1.2/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpoutRetryExponentialBackoff.java#L264,
and the KafkaSpout class itself.
Regarding the different Kafka versions, I took a quick look at the release
notes for Kafka 0.10.2.0 and 0.10.2.1, and noticed
https://issues.apache.org/jira/browse/KAFKA-4547. The spout uses
kafkaConsumer.position a bunch, so maybe that's the issue? I believe the Kafka
consumer is backwards compatible with older brokers
https://cwiki.apache.org/confluence/display/KAFKA/KIP-97%3A+Improved+Kafka+Client+RPC+Compatibility+Policy.
Could you try running your topology with the kafka-client dependency set to
0.10.2.1 against your 0.10.1.1 Kafka cluster, and see if that resolves the
issue?
> KafkaSpout consumes messages but doesn't commit offsets
> -------------------------------------------------------
>
> Key: STORM-2994
> URL: https://issues.apache.org/jira/browse/STORM-2994
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka-client
> Affects Versions: 1.1.0, 1.1.2
> Reporter: RAbreu
> Priority: Major
>
> A topology that consumes from two different Kafka clusters: 0.10.1.1 and
> 0.10.2.1.
> Spouts consuming from 0.10.2.1 have a low lag (and regularly commit offsets)
> The Spout that consumes from 0.10.1.1 exhibits either:
> 1- Unknown lag
> 2- Lag that increments as the Spout reads messages from Kafka
>
> In DEBUG, Offset manager logs: "topic-partition has NO offsets ready to be
> committed", despite continuing to consume messages.
> Several configuration tweaks were tried, including setting maxRetries to 1,
> in case messages with a lower offset were being retried (logs didn't show it,
> though)
> offsetCommitPeriodMs was also lowered to no avail.
> The only configuration that works is to have
> ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG=true, but this is undesired since
> we lose processing guarantees.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)