How hard is this to change to '$' - its difficult for us to now change the partition names now. Is there a place to get the helix 0.6.4 source code, make changes and built it ?
Another related question - is the partition number mainly needed for hashing into the buckets - the partition name would still continue to be "resource_name_partition_num", right ? On Mon, Feb 23, 2015 at 11:52 AM, Zhen Zhang <[email protected]> wrote: > The bucketize logic is expecting partition name following the convention > {resource_name}_{partition_number}. In your case, you are using $ ? > ------------------------------ > *From:* Varun Sharma [[email protected]] > *Sent:* Monday, February 23, 2015 11:48 AM > > *To:* [email protected] > *Subject:* Re: Bucketized resources not working in Helix 0.6.4 > > Note that we had to prefix each partition with the resource name to > make the partitions unique. > > On Mon, Feb 23, 2015 at 11:48 AM, Varun Sharma <[email protected]> > wrote: > >> Here is the latest error I am getting: >> >> 2015-02-23 19:44:48,153 [controller-pool-0] >> (TerrapinControllerServiceImpl.java:129) WARN Resource creation failed for >> $terrapin$data$pinpin_board_join$1424377884722, rolling back. >> >> *java.lang.IllegalArgumentException: Could NOT parse partition# >> (join$1424377884722$0) in $terrapin$data$pinpin_board_join$1424377884722$0* >> >> *at >> org.apache.helix.ZNRecordBucketizer.getBucketName(ZNRecordBucketizer.java:72)* >> >> *at >> org.apache.helix.ZNRecordBucketizer.bucketize(ZNRecordBucketizer.java:105)* >> >> at >> org.apache.helix.manager.zk.ZKHelixDataAccessor.setProperty(ZKHelixDataAccessor.java:110) >> >> at >> org.apache.helix.manager.zk.ZKHelixAdmin.setResourceIdealState(ZKHelixAdmin.java:686) >> >> On Mon, Feb 23, 2015 at 10:40 AM, Zhen Zhang <[email protected]> wrote: >> >>> Not really. The bucketize logic just creates child znode under the >>> original ideal state, and each of the child znode holds a subset of the >>> total partitions. >>> ------------------------------ >>> *From:* Varun Sharma [[email protected]] >>> *Sent:* Monday, February 23, 2015 10:37 AM >>> >>> *To:* [email protected] >>> *Subject:* Re: Bucketized resources not working in Helix 0.6.4 >>> >>> One other question - would bucketizing add anything to the partition >>> name - just worried about any special symbols (we use _ and $) causing >>> issues there ? >>> >>> On Mon, Feb 23, 2015 at 10:35 AM, Zhen Zhang <[email protected]> >>> wrote: >>> >>>> Hi Varun, >>>> >>>> HelixAdmin#addResource was initially used for adding an empty >>>> resource only, followed by HelixAdmin#rebalance, which bucketize the ideal >>>> state, so the bucketize logic is missing from addResource. We should >>>> definitely fix this. We are also considering merging addResource and >>>> rebalance, since separating them are confusing. For now you may use >>>> setResourceIdealState as a workaround. >>>> >>>> Thanks, >>>> Zhen >>>> >>>> ------------------------------ >>>> *From:* Varun Sharma [[email protected]] >>>> *Sent:* Monday, February 23, 2015 10:20 AM >>>> *To:* [email protected] >>>> *Subject:* Re: Bucketized resources not working in Helix 0.6.4 >>>> >>>> Thanks, Zhen. >>>> >>>> What about just calling addResource with an idealState with the >>>> bucket size - is that expected to not work on its own ? >>>> >>>> Varun >>>> >>>> On Mon, Feb 23, 2015 at 10:18 AM, Zhen Zhang <[email protected]> >>>> wrote: >>>> >>>>> Hi Varun, >>>>> >>>>> Did you do the following? >>>>> HelixAdmin#addResource() // set bucket size >>>>> HelixAdmin#setResourceIdealState() // no bucket size >>>>> >>>>> When you set the ideal state, you still need set the bucket size in >>>>> the ideal state. This value will be used to bucketize the ideal state. >>>>> >>>>> Thanks, >>>>> Jason >>>>> >>>>> ________________________________________ >>>>> From: Varun Sharma [[email protected]] >>>>> Sent: Monday, February 23, 2015 1:51 AM >>>>> To: [email protected] >>>>> Subject: Bucketized resources not working in Helix 0.6.4 >>>>> >>>>> Hi, >>>>> >>>>> I am trying to bucketize a resource on Helix 0.6.4 - however, am >>>>> unable to do so. I am setting the bucketSize in the idealState and using >>>>> addResource() and setResourceIdealState and it ends up complaining that ZK >>>>> size of 1M has been exceeded. >>>>> >>>>> I also tried using the addResource() api which creates an empty ideal >>>>> state with a valid bucket size. However, when I tried to set the ideal >>>>> state on that bucket (this ideal state did not have the bucket size set), >>>>> it failed as well. >>>>> >>>>> Whats the right way to get bucketizing to work ? >>>>> >>>>> Thanks ! >>>>> Varun >>>>> >>>> >>>> >>> >> >
