The bucketize logic is expecting partition name following the convention
{resource_name}_{partition_number}. In your case, you are using $ ?
________________________________
From: Varun Sharma [[email protected]]
Sent: Monday, February 23, 2015 11:48 AM
To: [email protected]
Subject: Re: Bucketized resources not working in Helix 0.6.4
Note that we had to prefix each partition with the resource name to make the
partitions unique.
On Mon, Feb 23, 2015 at 11:48 AM, Varun Sharma
<[email protected]<mailto:[email protected]>> wrote:
Here is the latest error I am getting:
2015-02-23 19:44:48,153 [controller-pool-0]
(TerrapinControllerServiceImpl.java:129) WARN Resource creation failed for
$terrapin$data$pinpin_board_join$1424377884722, rolling back.
java.lang.IllegalArgumentException: Could NOT parse partition#
(join$1424377884722$0) in $terrapin$data$pinpin_board_join$1424377884722$0
at org.apache.helix.ZNRecordBucketizer.getBucketName(ZNRecordBucketizer.java:72)
at org.apache.helix.ZNRecordBucketizer.bucketize(ZNRecordBucketizer.java:105)
at
org.apache.helix.manager.zk.ZKHelixDataAccessor.setProperty(ZKHelixDataAccessor.java:110)
at
org.apache.helix.manager.zk.ZKHelixAdmin.setResourceIdealState(ZKHelixAdmin.java:686)
On Mon, Feb 23, 2015 at 10:40 AM, Zhen Zhang
<[email protected]<mailto:[email protected]>> wrote:
Not really. The bucketize logic just creates child znode under the original
ideal state, and each of the child znode holds a subset of the total partitions.
________________________________
From: Varun Sharma [[email protected]<mailto:[email protected]>]
Sent: Monday, February 23, 2015 10:37 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: Bucketized resources not working in Helix 0.6.4
One other question - would bucketizing add anything to the partition name -
just worried about any special symbols (we use _ and $) causing issues there ?
On Mon, Feb 23, 2015 at 10:35 AM, Zhen Zhang
<[email protected]<mailto:[email protected]>> wrote:
Hi Varun,
HelixAdmin#addResource was initially used for adding an empty resource only,
followed by HelixAdmin#rebalance, which bucketize the ideal state, so the
bucketize logic is missing from addResource. We should definitely fix this. We
are also considering merging addResource and rebalance, since separating them
are confusing. For now you may use setResourceIdealState as a workaround.
Thanks,
Zhen
________________________________
From: Varun Sharma [[email protected]<mailto:[email protected]>]
Sent: Monday, February 23, 2015 10:20 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: Bucketized resources not working in Helix 0.6.4
Thanks, Zhen.
What about just calling addResource with an idealState with the bucket size -
is that expected to not work on its own ?
Varun
On Mon, Feb 23, 2015 at 10:18 AM, Zhen Zhang
<[email protected]<mailto:[email protected]>> wrote:
Hi Varun,
Did you do the following?
HelixAdmin#addResource() // set bucket size
HelixAdmin#setResourceIdealState() // no bucket size
When you set the ideal state, you still need set the bucket size in the ideal
state. This value will be used to bucketize the ideal state.
Thanks,
Jason
________________________________________
From: Varun Sharma [[email protected]<mailto:[email protected]>]
Sent: Monday, February 23, 2015 1:51 AM
To: [email protected]<mailto:[email protected]>
Subject: Bucketized resources not working in Helix 0.6.4
Hi,
I am trying to bucketize a resource on Helix 0.6.4 - however, am unable to do
so. I am setting the bucketSize in the idealState and using addResource() and
setResourceIdealState and it ends up complaining that ZK size of 1M has been
exceeded.
I also tried using the addResource() api which creates an empty ideal state
with a valid bucket size. However, when I tried to set the ideal state on that
bucket (this ideal state did not have the bucket size set), it failed as well.
Whats the right way to get bucketizing to work ?
Thanks !
Varun
