[ 
https://issues.apache.org/jira/browse/STORM-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462137#comment-16462137
 ] 

Stig Rohde Døssing commented on STORM-2974:
-------------------------------------------

[~aniket.alhat] I think you have a few options.

* As you mention, you could use the opaque spout instead. I believe we only 
have one pending issue for it right now 
https://issues.apache.org/jira/browse/STORM-3046, which I will make sure to 
backport. If you need to migrate your offsets so the new spout doesn't start 
processing from scratch, you might want to take a look at 
https://github.com/apache/storm/tree/master/external/storm-kafka-migration.
* You could wait for us to figure out how we're going to release 
storm-kafka-client. If we start releasing it separately from Storm, you should 
just be able to plug in the newest version of storm-kafka-client to your 
topology. Since I don't have a timeframe for this, you might want to pick one 
of the other options until this becomes relevant.
* I can't say whether this option will work, but as far as I'm aware, the 
consumer API used by storm-kafka to connect to Kafka brokers has only been 
deprecated, not removed. It's very possible that storm-kafka will still work 
with newer Kafka clusters. The API used by storm-kafka is 
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/SimpleConsumer.scala,
 which as you can see has not been deleted yet (as of Kafka 1.1.0). If you're 
not comfortable upgrading to storm-kafka-client yet, you might be able to stay 
on storm-kafka until Kafka removes that class.

If you migrate to storm-kafka-client, be aware that we made some breaking 
changes in the 1.0.6/1.1.2/1.2.0 releases, since we were unable to fix some 
bugs without changing the spout API. They're mostly changes to 
KafkaSpoutConfig, so they'll show up when you try to compile against a new 
storm-kafka-client version.

Regarding breaking changes in Storm, it's my impression that we try to stick to 
semantic versioning as much as possible, so there shouldn't be breaking changes 
between 1.0.2 and 1.2.x. The exception is storm-kafka-client as mentioned, 
because we sadly had to choose between sitting on fixes until Storm 2.0.0, or 
releasing breaking changes in non-major versions.

> Add a transactional non-opaque spout to storm-kafka-client
> ----------------------------------------------------------
>
>                 Key: STORM-2974
>                 URL: https://issues.apache.org/jira/browse/STORM-2974
>             Project: Apache Storm
>          Issue Type: Sub-task
>          Components: storm-kafka-client
>    Affects Versions: 2.0.0
>            Reporter: Stig Rohde Døssing
>            Assignee: Stig Rohde Døssing
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to