P. Taylor Goetz created STORM-650:
-------------------------------------
Summary: Storm-Kafka Refactoring and Improvements
Key: STORM-650
URL: https://issues.apache.org/jira/browse/STORM-650
Project: Apache Storm
Issue Type: Improvement
Components: storm-kafka
Reporter: P. Taylor Goetz
This is intended to be a parent/umbrella JIRA covering a number of
efforts/suggestions aimed at improving the storm-kafka module. The goal is to
facilitate communication and collaboration by providing a central point for
discussion and coordination.
The first phase should be to identify and agree upon a list of high-level
points we would like to address. Once that is complete, we can move on to
implementation/design discussions, followed by an implementation plan, division
of labor, etc.
A non-exhaustive, initial list of items follows. New/additional thoughts can be
proposed in the comments.
* Improve API for Specifying the Kafka Starting point
Configuring the kafka spout's starting position (e.g. forceFromStart=true) is a
common source of confusion. This should be refactored to provide an easy to
understand, unambiguous API for configuring this property.
* Use Kafka APIs Instead of Internal ZK Metadata (STORM-590)
Currently the Kafka spout relies on reading Kafka's internal metadata from
zookeeper. This should be refactored to use the Kafka Consumer API to protect
against changes to the internal metadata format stored in ZK.
* Improve Error Handling
There are a number of failure scenarios with the kafka spout that users may
want to react to differently based on their use case. Add a failure handler API
that allows users to implement and/or plug in alternative failure handling
implementations. It is assumed that default (sane) implementations would be
included and configured by default.
* Configuration/General Refactoring (BrokerHosts, etc.) (STORM-631)
(need to flesh this out better) Reduce unnecessary marker interfaces/"instance
of" checks. Unify configuration of core storm/trident spout implementations.
Discussion Items:
* How important is backward compatibility?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
