[
https://issues.apache.org/jira/browse/STORM-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462137#comment-16462137
]
Stig Rohde Døssing commented on STORM-2974:
-------------------------------------------
[~aniket.alhat] I think you have a few options.
* As you mention, you could use the opaque spout instead. I believe we only
have one pending issue for it right now
https://issues.apache.org/jira/browse/STORM-3046, which I will make sure to
backport. If you need to migrate your offsets so the new spout doesn't start
processing from scratch, you might want to take a look at
https://github.com/apache/storm/tree/master/external/storm-kafka-migration.
* You could wait for us to figure out how we're going to release
storm-kafka-client. If we start releasing it separately from Storm, you should
just be able to plug in the newest version of storm-kafka-client to your
topology. Since I don't have a timeframe for this, you might want to pick one
of the other options until this becomes relevant.
* I can't say whether this option will work, but as far as I'm aware, the
consumer API used by storm-kafka to connect to Kafka brokers has only been
deprecated, not removed. It's very possible that storm-kafka will still work
with newer Kafka clusters. The API used by storm-kafka is
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/SimpleConsumer.scala,
which as you can see has not been deleted yet (as of Kafka 1.1.0). If you're
not comfortable upgrading to storm-kafka-client yet, you might be able to stay
on storm-kafka until Kafka removes that class.
If you migrate to storm-kafka-client, be aware that we made some breaking
changes in the 1.0.6/1.1.2/1.2.0 releases, since we were unable to fix some
bugs without changing the spout API. They're mostly changes to
KafkaSpoutConfig, so they'll show up when you try to compile against a new
storm-kafka-client version.
Regarding breaking changes in Storm, it's my impression that we try to stick to
semantic versioning as much as possible, so there shouldn't be breaking changes
between 1.0.2 and 1.2.x. The exception is storm-kafka-client as mentioned,
because we sadly had to choose between sitting on fixes until Storm 2.0.0, or
releasing breaking changes in non-major versions.
> Add a transactional non-opaque spout to storm-kafka-client
> ----------------------------------------------------------
>
> Key: STORM-2974
> URL: https://issues.apache.org/jira/browse/STORM-2974
> Project: Apache Storm
> Issue Type: Sub-task
> Components: storm-kafka-client
> Affects Versions: 2.0.0
> Reporter: Stig Rohde Døssing
> Assignee: Stig Rohde Døssing
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)