[
https://issues.apache.org/jira/browse/STORM-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304829#comment-16304829
]
Erik Weathers edited comment on STORM-2844 at 12/27/17 10:05 PM:
-----------------------------------------------------------------
[~hmclouro]: I'm sympathetic to the problem in general, but I have a few
responses:
# This seems like a really bad problem from the description. Am I correct in
interpreting that it means users on 1.1.x branches cannot use EARLIEST with
Kafka? If so then choosing to not backport this fix is effectively killing
Storm 1.1.x for users of Kafka.
# The difficulty of backporting a fix is certainly a factor that should be
considered, but also the impact of the fixes. Specifically, the change that
introduced this problem *was* backported, so I'm not entirely sure why the fix
for that change wouldn't be backported too. i.e., the original change that
introduced the problem which STORM-2844 is fixing (STORM-2666) seemed like a
"nice to have" from the description (prevent some duplicates from flowing
through the topology), whereas this seems like a "must have" (crashing the
worker).
was (Author: erikdw):
[~hmclouro]: I'm sympathetic to the problem in general, but I have a few
responses:
# This seems like a really bad problem from the description. Am I correct in
interpreting that it means users on 1.1.x branches cannot use EARLIEST with
Kafka? If so then choosing to not backport this fix is effectively killing
Storm 1.1.x for users of Kafka.
# The difficulty of backporting a fix is certainly a factor that should be
considered, but also the impact of the fixes. Specifically, the change that
introduced this problem *was* backported, so I'm not entirely sure why the fix
for that change wouldn't be backported too.
> KafkaSpout Throws IllegalStateException After Committing to Kafka When First
> Poll Strategy Set to EARLIEST
> ----------------------------------------------------------------------------------------------------------
>
> Key: STORM-2844
> URL: https://issues.apache.org/jira/browse/STORM-2844
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka-client
> Affects Versions: 2.0.0, 1.2.0
> Reporter: Hugo Louro
> Assignee: Hugo Louro
> Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0, 1.2.0
>
> Time Spent: 18h 40m
> Remaining Estimate: 0h
>
> This
> [code|https://github.com/apache/storm/blob/1.x-branch/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java#L407-L409],
> which was committed to fix
> [STORM-2666|https://issues.apache.org/jira/browse/STORM-2666] throws
> IllegalStateException when the KafkaSpout commits to Kafka and is restarted
> with the same consumer group id and first poll strategy is set to EARLIEST.
> For example consider the following sequence:
> # KafkaSpout with consumer_group_id=TEST polls and commits offsets 1-5
> # KafkaSpout with consumer_group_id=TEST is restarted with first poll
> strategy set to EARLIEST
> ==> IllegalStateException will be thrown
> This bug could be a blocker. I am setting it to Critical because assigning a
> different consumer id serves as a workaround to the problem.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
