[
https://issues.apache.org/jira/browse/KAFKA-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059796#comment-14059796
]
nicu marasoiu commented on KAFKA-167:
-------------------------------------
Hi,
So I would suggest this: for each consumer group, one broker can be elected as
partition/replica assignment HA coordinator using a standard election mechanism
on zookeeper already available in Curator.
I think it is nicer to distribute as much as possible, and, while recognizing
that the algorithm is more deterministic, stable and convergent when using a
centralized coordinator, there is no reason to pick the same coordinator for
different consumer groups.
Apart from the mail discussion thread, can you please indicate any discussion
on design, implementation, so that I can try to pick a small part and code
something for it?
Thanks,
Nicu Marasoiu
> Move partition assignment to the broker
> ---------------------------------------
>
> Key: KAFKA-167
> URL: https://issues.apache.org/jira/browse/KAFKA-167
> Project: Kafka
> Issue Type: New Feature
> Reporter: Jay Kreps
>
> Currently partitions are assigned to consumers for consumption via a
> co-ordination algorithm by the consumers. This means there is effectively no
> master broker node, which is nice in a way, but makes the consumer logic
> complex to implement. It would be good to move the co-ordination to the
> brokers. This would make implementation of a consumer client much much
> easier, since only minimal zk interaction would be required. There are a
> number of details that would have to be worked out to finalize the protocol.
> Discussion is here:
> http://mail-archives.apache.org/mod_mbox/incubator-kafka-dev/201109.mbox/%3ccafbh0q2adjxcbcyqgh8tb4jxbas3hq5rlxk9o9w76we-bwa...@mail.gmail.com%3e
--
This message was sent by Atlassian JIRA
(v6.2#6252)
