[
https://issues.apache.org/jira/browse/CURATOR-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265583#comment-15265583
]
Drew Kutcharian commented on CURATOR-321:
-----------------------------------------
Thanks. I wasn't really expecting you to build it, just putting out there to
see if it already exists and if there is interest for it.
I'll try to dig through Kafka "old" consumer code and see if I can put
something together. Will appreciate your code review though.
> Recipe for Group Membership with Partition/Shard Assignment
> -----------------------------------------------------------
>
> Key: CURATOR-321
> URL: https://issues.apache.org/jira/browse/CURATOR-321
> Project: Apache Curator
> Issue Type: Wish
> Components: Recipes
> Reporter: Drew Kutcharian
> Assignee: Jordan Zimmerman
> Priority: Minor
>
> It'd be great if Curator had a recipe for partition/shard assignment, similar
> to Kafka's consumer rebalancing. It could be thought of as the generalization
> of what Kafka consumers do using Zookeeper.
> Use Case:
> You have 5 worker machines that need to process data from 10 database shards,
> but you want each worker to be pinned to a single database so you don't have
> to worry about race conditions.
> Solution:
> A more advanced group management recipe that:
> 1. Has a concept of slots (or paritions). You can create a group with say
> 1000 slots.
> 2. When a node joins the group, it gets assigned a range of the slots. If
> it's the first node, then it will get all the slots. If there are already
> nodes registered in the group, then they will split the slots equally between
> them.
> 3. When a node leaves the group, the rest of the nodes rebalance so they each
> end up with a equal number of remaining slots.
> I am aware that something like this can be done using Storm et al but I feel
> that this primitive would be useful on its own in Curator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
