Chris Riccomini created KAFKA-360:
-------------------------------------
Summary: Add ability to disable rebalancing in ZooKeeper consumer
Key: KAFKA-360
URL: https://issues.apache.org/jira/browse/KAFKA-360
Project: Kafka
Issue Type: Bug
Components: core
Reporter: Chris Riccomini
There is a need for a ZooKeeper-based Kafka consumer that does not re-balance.
This is needed because we may be handling partitioning outside of Kafka. For
example, I may have a stateful process that is meant to consume only from
Partition 7 of a given Kafka topic. When that process stops, I don't want
another consumer to take over partition 7.
The benefits we get from using the ZooKeeper-based consumer (vs the Simple
Consumer) without rebalancing is that offsets will still be handled by
Kafka/ZK, as will failover when a partition's leader disappears/fails.
I think the way to do this is to add a consumer config parameter that disables
a consumer group's rebalancing. That way, the first consumer in the group to
connect (when the ZK node is created) can specify if rebalancing should be
enabled for the consumer group. If rebalancing is disabled, the consumers
should be forced to supply a list of partition IDs that they wish to read from.
Perhaps this can be done during the createMessageStreams call?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
