Sorry spoke too soon - I saw the "Resetting the RoutingTableProvider" bug again which caused the RoutingTableProvider to stop receiving updates.
Varun On Thu, Mar 12, 2015 at 5:24 PM, Varun Sharma <[email protected]> wrote: > I tested it out using a resource with 8192 partitions and 8 buckets and it > worked out just fine - (disable and drop). > > On Thu, Mar 12, 2015 at 4:08 PM, Varun Sharma <[email protected]> wrote: > >> I just deployed this to a server node and when the current states node is >> moved over - the bucket nodes are no longer present - the node has executed >> dropped transitions for all the nodes. Is it expected to not contain any >> buckets ? >> >> On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <[email protected]> wrote: >> >>> Should be no problem. Controller and participant are compatible. >>> >>> ------------------------------ >>> *From:* Varun Sharma [[email protected]] >>> *Sent:* Thursday, March 12, 2015 1:29 PM >>> *To:* [email protected] >>> >>> *Subject:* Re: Cutting a release 0.6.5 tonight >>> >>> Just to confirm - it wont be a problem if the controller is running >>> 0.6.5 and participant runs 0.6.4 for a while due to this ? >>> >>> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <[email protected]> wrote: >>> >>>> Yes, I will think of a way to make it backwards compatible. >>>> >>>> Thanks >>>> Kishore G >>>> On Mar 12, 2015 1:21 PM, "Varun Sharma" <[email protected]> wrote: >>>> >>>>> Kishore, I will be testing it out today. Btw, did you also look at the >>>>> issue with change of Abstract method signature createNewStateModel in >>>>> StateModelFactory ? >>>>> >>>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <[email protected]> >>>>> wrote: >>>>> >>>>>> Jason and I have checked in the changes needed. I am not sure if >>>>>> TestSchedulerMsg is fixed. >>>>>> >>>>>> Last call for any other changes needed. >>>>>> >>>>>> Varun, it will be great if you can test the bug fixes for bucketing >>>>>> before we cut the release. >>>>>> >>>>>> thanks, >>>>>> Kishore G >>>>>> >>>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks Lei, I fixed the first two. >>>>>>> >>>>>>> Here is what caused the failure. I added the code to copy all >>>>>>> simple fields from IS to EV when we update the ExternalView in >>>>>>> controller. >>>>>>> This is to enable compression in ExternalView if its set in IS. I could >>>>>>> have copied only enableCompression variable but I thought its good to >>>>>>> have >>>>>>> partition number/replica etc in ExternalView as well. Let me know if you >>>>>>> foresee any problem in this. Only thing I could think of is IdealState >>>>>>> is >>>>>>> deleted in which case I copy the simplefields from existing >>>>>>> externalview. >>>>>>> >>>>>>> I dont understand why testschedulermsg test cases are failing. >>>>>>> Uncommenting my code does not help either. >>>>>>> >>>>>>> >>>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> The first two are concerning. I don't think the scheduler message >>>>>>>> test failures should block releases. >>>>>>>> >>>>>>>> ---------------------------------------- >>>>>>>> > From: [email protected] >>>>>>>> > To: [email protected] >>>>>>>> > CC: [email protected] >>>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight >>>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000 >>>>>>>> > >>>>>>>> > The test failed are: >>>>>>>> > >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3 >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4 >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints >>>>>>>> > >>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue >>>>>>>> > >>>>>>>> > I got these from my local build too. (mvn clean install package >>>>>>>> on helix-0.6.x) >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > Thanks >>>>>>>> > Lei >>>>>>>> > >>>>>>>> > -- >>>>>>>> > >>>>>>>> > Lei Xia >>>>>>>> > Software Engineer >>>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage >>>>>>>> > LinkedIn >>>>>>>> > >>>>>>>> > [email protected] >>>>>>>> > www.linkedin.com/in/lxia1 >>>>>>>> > >>>>>>>> > ________________________________________ >>>>>>>> > From: kishore g [[email protected]] >>>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM >>>>>>>> > To: [email protected] >>>>>>>> > Cc: [email protected] >>>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight >>>>>>>> > >>>>>>>> > Hi Lei, >>>>>>>> > >>>>>>>> > Can you point to the failures? >>>>>>>> > >>>>>>>> > thanks, >>>>>>>> > Kishore G >>>>>>>> > >>>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia >>>>>>>> <[email protected]> wrote: >>>>>>>> > >>>>>>>> >> Hi, Kishore >>>>>>>> >> >>>>>>>> >> I saw there are regression test failures from last two recent >>>>>>>> commits >>>>>>>> >> on 0.6.x branch, running from both local box and Linkedin's >>>>>>>> hudson jobs. >>>>>>>> >> Are we going to fix them before the release? >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> Thanks >>>>>>>> >> Lei >>>>>>>> >> >>>>>>>> >> -- >>>>>>>> >> >>>>>>>> >> Lei Xia >>>>>>>> >> Software Engineer >>>>>>>> >> Data Infrastructure/Distributed Data Systems/Nuage >>>>>>>> >> LinkedIn >>>>>>>> >> >>>>>>>> >> [email protected] >>>>>>>> >> www.linkedin.com/in/lxia1 >>>>>>>> >> >>>>>>>> >> ________________________________________ >>>>>>>> >> From: kishore g [[email protected]] >>>>>>>> >> Sent: Wednesday, March 11, 2015 12:04 PM >>>>>>>> >> To: [email protected] >>>>>>>> >> Cc: [email protected] >>>>>>>> >> Subject: Re: Cutting a release 0.6.5 tonight >>>>>>>> >> >>>>>>>> >> Will write a test case for migration and document the steps >>>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma" <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >>> What would be the migration path from non-compressed buckets to >>>>>>>> >> compressed >>>>>>>> >>> non bucket resources ? It seems even the CURRENTSTATES are >>>>>>>> being bucketed >>>>>>>> >>> in this case, I thought that was not expected with bucketing. >>>>>>>> Does the >>>>>>>> >>> controller read these current states appropriately ? To >>>>>>>> migrate, it seems >>>>>>>> >>> that we would need to also rewrite the CURRENT STATES ? >>>>>>>> >>> >>>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g < >>>>>>>> [email protected]> wrote: >>>>>>>> >>> >>>>>>>> >>>> Hi, >>>>>>>> >>>> >>>>>>>> >>>> I will work with Jason to cut a 0.6.5 release tonight. >>>>>>>> >>>> >>>>>>>> >>>> The new thing I added is to enableCompression while storing >>>>>>>> data in >>>>>>>> >>>> Zookeeper, this allows us to go up to 100k partitions per >>>>>>>> resource >>>>>>>> >> without >>>>>>>> >>>> having to use bucketing feature. We also fixed few bugs with >>>>>>>> bucketed >>>>>>>> >>>> resource just in case some one needs it. >>>>>>>> >>>> >>>>>>>> >>>> The property store api needs some changes, I plan to get it in >>>>>>>> today. >>>>>>>> >>>> >>>>>>>> >>>> Let me know if you need any other changes to be included. Are >>>>>>>> there any >>>>>>>> >>>> changes that went into 0.7.x branch that we need to merge it >>>>>>>> back in to >>>>>>>> >>>> 0.6.x ? >>>>>>>> >>>> >>>>>>>> >>>> thanks, >>>>>>>> >>>> Kishore G >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>> >>>>>>>> >> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>> >> >
