Drew Kutcharian created CURATOR-321:
---------------------------------------
Summary: Recipe for Group Membership with Partition/Shard
Assignment
Key: CURATOR-321
URL: https://issues.apache.org/jira/browse/CURATOR-321
Project: Apache Curator
Issue Type: Wish
Reporter: Drew Kutcharian
Assignee: Jordan Zimmerman
It'd be great if Curator had a recipe for partition/shard assignment, similar
to Kafka's consumer rebalancing. It could be thought of as the generalization
of what Kafka consumers do using Zookeeper.
Use Case:
You have 5 worker machines that need to process data from 10 database shards,
but you want each worker to be pinned to a single database so you don't have to
worry about race conditions.
Solution:
A more advanced group management recipe that:
1. Has a concept of slots (or paritions). You can create a group with say 1000
slots.
2. When a node joins the group, it gets assigned a range of the slots. If it's
the first node, then it will get all the slots. If there are already nodes
registered in the group, then they will split the slots equally between them.
3. When a node leaves the group, the rest of the nodes rebalance so they each
end up with a equal number of remaining slots.
I am aware that something like this can be done using Storm et al but I feel
that this primitive would be useful on its own in Curator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
