Parth Brahmbhatt created STORM-631:
--------------------------------------
Summary: Simplifying kafka connector code
Key: STORM-631
URL: https://issues.apache.org/jira/browse/STORM-631
Project: Apache Storm
Issue Type: Bug
Components: storm-kafka
Reporter: Parth Brahmbhatt
Assignee: Parth Brahmbhatt
The current kafka connector was written when kafka did not support any APIs to
expose its internal structures. We already have STORM-590 to modify the kafka
connector to new kafka meta APIS.
The current codebase uses some marker interfaces to support reading kafka
partition/broker info from zookeeper or for users to specify the mapping them
self. Adding one more layer that reads this info using kakfa APIs would reduce
readability. Because we are using marker interfaces there are bunch of places
that has if statements with instanceof checks and each one creates a separate
code path to trace. We should delete these interfaces and their implementation
in favor of a single way to get kafka topic information, using kafka APIs. This
will be a backward incompatible change but should make future changes and usage
easy.
Current kafka connector also has 2 separate configs for trident and core
topologies. This configurations extend from a common interface but the
extensions are not interchangeable. I think it is intuitive to have a single
config with sane defaults for both core and trident spouts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
