We should add some docs for large number of partitions. Varun, you might also want to enable group message feature to for faster state transitions. On Nov 22, 2014 1:28 PM, "Varun Sharma" <[email protected]> wrote:
> I got hold of the EXTERNALVIEW znode for 500 partitions and it worked out > to 80kB. So, I am using 1 bucket per 1000 partitions ~ 160Kb. There are a > handful of cases where we need upto 4K partitions - hence I wanna use 4 > buckets for that usecase. > > On Sat, Nov 22, 2014 at 12:59 PM, Zhen Zhang <[email protected]> wrote: > >> Hi Varun, >> >> You can refer to this example: >> >> https://github.com/apache/helix/blob/helix-0.6.x/helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java >> >> The accessor handles bucket so HelixAdmin#setResourceIdealState should do >> the work also. >> May I know how many partitions you have and how large is your ideal state? >> >> Thanks, >> Zhen >> >> >> On Sat, Nov 22, 2014 at 11:53 AM, kishore g <[email protected]> wrote: >> >>> I dint realize that you had buckets. Let me check again >>> >>> On Sat, Nov 22, 2014 at 11:52 AM, kishore g <[email protected]> wrote: >>> >>>> >>>> http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/HelixAdmin.html >>>> public abstract void setResourceIdealState (String >>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html> >>>> clusterName, String >>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html> >>>> resourceName, IdealState >>>> <http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/model/IdealState.html> >>>> idealState) >>>> >>>> Set ideal state for a resource. >>>> >>>> That should work. >>>> >>>> >>>> On Sat, Nov 22, 2014 at 11:26 AM, Varun Sharma <[email protected]> >>>> wrote: >>>> >>>>> How do I add a resource with an ideal state and buckets since my >>>>> resource has extremely large number of partitions ? I can't find an >>>>> addResource call in HelixAdmin to do the same ? >>>>> >>>>> Thanks >>>>> Varun >>>>> >>>>> On Fri, Nov 21, 2014 at 8:07 PM, Kanak Biscuitwala < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Varun, >>>>>> >>>>>> Adding a resource is equivalent to adding an ideal state. So if you >>>>>> just create your own initial ideal state and set it, you've effectively >>>>>> created a resource with the properties that you want. >>>>>> >>>>>> Kanak >>>>>> >>>>>> ________________________________ >>>>>> > Date: Fri, 21 Nov 2014 17:08:36 -0800 >>>>>> > Subject: Re: Specifying Bucket size when adding resource with >>>>>> IDEAL_STATE >>>>>> > From: [email protected] >>>>>> > To: [email protected] >>>>>> > >>>>>> > Or should I break this up into 2 calls - one to add the resource and >>>>>> > then another to set the ideal states ? >>>>>> > >>>>>> > On Fri, Nov 21, 2014 at 5:03 PM, Varun Sharma >>>>>> > <[email protected]<mailto:[email protected]>> wrote: >>>>>> > Looking at the HelixAdmin API - I do not see an API for adding a >>>>>> > resource with a particular IdealState and the number of buckets to >>>>>> use >>>>>> > (znodes). >>>>>> > >>>>>> > Thanks >>>>>> > Varun >>>>>> > >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >
