Hello,
I'm trying to use Helix (0.7.1) to manage our resource partitions on a
cluster. My scenario is, every 5 minutes a partition will be added; What I
do now is, get the ideal state, +1 for the partition number and then update
it with ZKHelixAdmin. With a rebalance action, the new partition will be
created and assigned to an instance. What the instance can get from Helix
is the resource name and partition id. To map the parition to my logical
data, I maintained a mapping table in our datastore, which looks like:
{
"resource_0": 201601010000_ 201601010005,
"resource_1": 201601010005_ 201601010010,
...
"resource_11": 201601010055_ 201601010100
}
Here it has 12 partitions; Now I want to discard some old partitions, say
the first 6 partitions; It seems in Helix the partitions must start from
0, so with an update on the IdealState, set # of partitions to 6, the new
partitions on the cluster would become to:
resource_0,resource_1, ..., resource_5,
To make sure the partitions wouldn't be wrongly mapped, I need update my
mapping table before the rebalance. While that may not ensure the atomic
between the two updates.
So my question is, what's the suggested way to do the resource partition
mapping? does Helix allow user to specify additional information on a
partition (if have this, I don't need maintain the mapping outside)? Can we
have some simple APIs like addPartition(String parititionName),
dropParitition(String partitionName), just like that for resource? The
numeric paritition id can be an internal ID and not exposed to user.
I guess many entry users will have such questions. Just raise this for a
broad discussion; Any comment and suggestion is welcomed. Thanks for your
time!
--
Best regards,
Shaofeng Shi
