Hi Kishore, from Helix's tutorial, I don't see a way to dynamically add/remove partition, which made me had thought the partitions need be fixed when adding the resource. If that can be adjusted, it would match my case very well. Could you give me a sample or the API name on that? Thanks!
Greg, thanks for putting that to a JIRA, I has watched it and looks forward for the fix in next release. 2016-02-03 3:05 GMT+08:00 Greg Brandt <[email protected]>: > Hey ShaoFeng, > > I think this is a bug in the UI... in > ResourceStateSpec#getResourceStateTable > < > https://github.com/apache/helix/blob/master/helix-ui/src/main/java/org/apache/helix/ui/api/ResourceStateSpec.java#L62 > >, > we should check the rebalance mode, and use the partitions from external > view if FULL_AUTO, as there's no ideal placement for them. > > I've created this ticket https://issues.apache.org/jira/browse/HELIX-627 > for that fix. > > Thanks, > -Greg > > On Tue, Feb 2, 2016 at 9:04 AM, kishore g <[email protected]> wrote: > > > Hi ShaoFeng*,* > > > > The rebalancer works ensures even distribution within a resource. Across > > resources, it uses hashing based on resource names. > > So if you add few more resources, you might start seeing the distribution > > change. > > > > Another option is to add one partition every 5 minutes to the same > > resource. > > > > thanks, > > Kishore G > > > > On Tue, Feb 2, 2016 at 6:43 AM, ShaoFeng Shi <[email protected]> > > wrote: > > > > > Hello Helix developers, > > > > > > I'm trying to use Helix (0.7.1) to manage some resources; every 5 > > minutes, > > > such a resource will be added to the cluster; Each resource only has 1 > > > partition, and it's state model is "LEADER_STANDBY" (the leader will do > > > some exclusive job); the rebalance mode is FULL_AUTO. Now the cluster > has > > > two instances. The interesting thing I found is, all the LEADERs are > > > assigned to the first node of my cluster, and the other cluster carries > > all > > > STANDBYs. Although when I disable the first node, all the LEADERs are > > > transferred to the second node automatically, I expect there could be > > some > > > load balance among them: half leaders on the first node, the others on > > the > > > second. Did I do something wrong, or this is expected? > > > > > > Another issue I faced is, on the helix-ui, it can show all resources, > but > > > when selecting a resource, the "partitions" tab couldn't show which > > > instances it be assigned to; the only message is "No partitions of > > > <resource_name> are assigned!" > > > > > > I checked the UI code, it indicates the reason is the "listFields" is > > > empty. That matches with what I see with the helix-admin.sh > > > --listResourceInfo: > > > > > > IdealState for Resource_Stream_1454391900000_1454392200000: > > > { > > > "id" : "Resource_Stream_1454391900000_1454392200000", > > > "mapFields" : { > > > "Resource_Stream_1454391900000_1454392200000" : { > > > } > > > }, > > > "listFields" : { > > > "Resource_Stream_1454391900000_1454392200000" : [ ] > > > }, > > > "simpleFields" : { > > > "IDEAL_STATE_MODE" : "AUTO_REBALANCE", > > > "NUM_PARTITIONS" : "1", > > > "REBALANCE_MODE" : "FULL_AUTO", > > > "REPLICAS" : "3", > > > "STATE_MODEL_DEF_REF" : "LeaderStandby", > > > "STATE_MODEL_FACTORY_NAME" : "DEFAULT" > > > } > > > } > > > > > > ExternalView for Resource_Stream_1454391900000_1454392200000: > > > { > > > "id" : "Resource_Stream_1454391900000_1454392200000", > > > "mapFields" : { > > > "Resource_Stream_1454391900000_1454392200000" : { > > > "kylin-dev3_8080" : "LEADER", > > > "kylin-dev4_8080" : "STANDBY" > > > } > > > }, > > > "listFields" : { > > > }, > > > "simpleFields" : { > > > "BUCKET_SIZE" : "0" > > > } > > > } > > > > > > I checked the code, it seems for FULL_AUTO, the values for "listFields" > > and > > > "mapFields" are empty (at least at the begining). Will it be updated at > > > some point of time, or how could I trigger that? > > > > > > Thanks in advance for your help; I'm new to Helix so may not get some > > > concepts correctly. Just correct me if I'm wrong, thanks! > > > > > > -- > > > Best regards, > > > > > > Shaofeng Shi > > > > > > -- Best regards, Shaofeng Shi
