I tested it out using a resource with 8192 partitions and 8 buckets and it worked out just fine - (disable and drop).
On Thu, Mar 12, 2015 at 4:08 PM, Varun Sharma <[email protected]> wrote: > I just deployed this to a server node and when the current states node is > moved over - the bucket nodes are no longer present - the node has executed > dropped transitions for all the nodes. Is it expected to not contain any > buckets ? > > On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <[email protected]> wrote: > >> Should be no problem. Controller and participant are compatible. >> >> ------------------------------ >> *From:* Varun Sharma [[email protected]] >> *Sent:* Thursday, March 12, 2015 1:29 PM >> *To:* [email protected] >> >> *Subject:* Re: Cutting a release 0.6.5 tonight >> >> Just to confirm - it wont be a problem if the controller is running >> 0.6.5 and participant runs 0.6.4 for a while due to this ? >> >> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <[email protected]> wrote: >> >>> Yes, I will think of a way to make it backwards compatible. >>> >>> Thanks >>> Kishore G >>> On Mar 12, 2015 1:21 PM, "Varun Sharma" <[email protected]> wrote: >>> >>>> Kishore, I will be testing it out today. Btw, did you also look at the >>>> issue with change of Abstract method signature createNewStateModel in >>>> StateModelFactory ? >>>> >>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <[email protected]> wrote: >>>> >>>>> Jason and I have checked in the changes needed. I am not sure if >>>>> TestSchedulerMsg is fixed. >>>>> >>>>> Last call for any other changes needed. >>>>> >>>>> Varun, it will be great if you can test the bug fixes for bucketing >>>>> before we cut the release. >>>>> >>>>> thanks, >>>>> Kishore G >>>>> >>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks Lei, I fixed the first two. >>>>>> >>>>>> Here is what caused the failure. I added the code to copy all >>>>>> simple fields from IS to EV when we update the ExternalView in >>>>>> controller. >>>>>> This is to enable compression in ExternalView if its set in IS. I could >>>>>> have copied only enableCompression variable but I thought its good to >>>>>> have >>>>>> partition number/replica etc in ExternalView as well. Let me know if you >>>>>> foresee any problem in this. Only thing I could think of is IdealState is >>>>>> deleted in which case I copy the simplefields from existing externalview. >>>>>> >>>>>> I dont understand why testschedulermsg test cases are failing. >>>>>> Uncommenting my code does not help either. >>>>>> >>>>>> >>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> The first two are concerning. I don't think the scheduler message >>>>>>> test failures should block releases. >>>>>>> >>>>>>> ---------------------------------------- >>>>>>> > From: [email protected] >>>>>>> > To: [email protected] >>>>>>> > CC: [email protected] >>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight >>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000 >>>>>>> > >>>>>>> > The test failed are: >>>>>>> > >>>>>>> > >>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates >>>>>>> > >>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource >>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3 >>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4 >>>>>>> > >>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints >>>>>>> > >>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue >>>>>>> > >>>>>>> > I got these from my local build too. (mvn clean install package on >>>>>>> helix-0.6.x) >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > Thanks >>>>>>> > Lei >>>>>>> > >>>>>>> > -- >>>>>>> > >>>>>>> > Lei Xia >>>>>>> > Software Engineer >>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage >>>>>>> > LinkedIn >>>>>>> > >>>>>>> > [email protected] >>>>>>> > www.linkedin.com/in/lxia1 >>>>>>> > >>>>>>> > ________________________________________ >>>>>>> > From: kishore g [[email protected]] >>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM >>>>>>> > To: [email protected] >>>>>>> > Cc: [email protected] >>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight >>>>>>> > >>>>>>> > Hi Lei, >>>>>>> > >>>>>>> > Can you point to the failures? >>>>>>> > >>>>>>> > thanks, >>>>>>> > Kishore G >>>>>>> > >>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia <[email protected]> >>>>>>> wrote: >>>>>>> > >>>>>>> >> Hi, Kishore >>>>>>> >> >>>>>>> >> I saw there are regression test failures from last two recent >>>>>>> commits >>>>>>> >> on 0.6.x branch, running from both local box and Linkedin's >>>>>>> hudson jobs. >>>>>>> >> Are we going to fix them before the release? >>>>>>> >> >>>>>>> >> >>>>>>> >> Thanks >>>>>>> >> Lei >>>>>>> >> >>>>>>> >> -- >>>>>>> >> >>>>>>> >> Lei Xia >>>>>>> >> Software Engineer >>>>>>> >> Data Infrastructure/Distributed Data Systems/Nuage >>>>>>> >> LinkedIn >>>>>>> >> >>>>>>> >> [email protected] >>>>>>> >> www.linkedin.com/in/lxia1 >>>>>>> >> >>>>>>> >> ________________________________________ >>>>>>> >> From: kishore g [[email protected]] >>>>>>> >> Sent: Wednesday, March 11, 2015 12:04 PM >>>>>>> >> To: [email protected] >>>>>>> >> Cc: [email protected] >>>>>>> >> Subject: Re: Cutting a release 0.6.5 tonight >>>>>>> >> >>>>>>> >> Will write a test case for migration and document the steps >>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma" <[email protected]> >>>>>>> wrote: >>>>>>> >> >>>>>>> >>> What would be the migration path from non-compressed buckets to >>>>>>> >> compressed >>>>>>> >>> non bucket resources ? It seems even the CURRENTSTATES are being >>>>>>> bucketed >>>>>>> >>> in this case, I thought that was not expected with bucketing. >>>>>>> Does the >>>>>>> >>> controller read these current states appropriately ? To migrate, >>>>>>> it seems >>>>>>> >>> that we would need to also rewrite the CURRENT STATES ? >>>>>>> >>> >>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g <[email protected]> >>>>>>> wrote: >>>>>>> >>> >>>>>>> >>>> Hi, >>>>>>> >>>> >>>>>>> >>>> I will work with Jason to cut a 0.6.5 release tonight. >>>>>>> >>>> >>>>>>> >>>> The new thing I added is to enableCompression while storing >>>>>>> data in >>>>>>> >>>> Zookeeper, this allows us to go up to 100k partitions per >>>>>>> resource >>>>>>> >> without >>>>>>> >>>> having to use bucketing feature. We also fixed few bugs with >>>>>>> bucketed >>>>>>> >>>> resource just in case some one needs it. >>>>>>> >>>> >>>>>>> >>>> The property store api needs some changes, I plan to get it in >>>>>>> today. >>>>>>> >>>> >>>>>>> >>>> Let me know if you need any other changes to be included. Are >>>>>>> there any >>>>>>> >>>> changes that went into 0.7.x branch that we need to merge it >>>>>>> back in to >>>>>>> >>>> 0.6.x ? >>>>>>> >>>> >>>>>>> >>>> thanks, >>>>>>> >>>> Kishore G >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>> >>>>>>> >> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >> >
