Here is the relevant trace. Essentially, the following happens -
externalview is deleted by controller since no more nodes are reporting it
in the current state. After removing the rogue node, controller tries to
still subscribe to the zk data/child change and we get a NoNode exception.
The RoutingTableProvider receives a FINALIZE notification and we see the
"Resetting the RoutingTableProvider" notification. I hit this the very
first time I got a bucketed resource deleted correctly in 0.6.5 so it
should be reproducible.
*2015-03-13 00:10:30,172 [Thread-7] (ExternalViewComputeStage.java:161)
INFO Remove externalView for resource:
$terrapin$data$pinpin_board_join$1424982892102*
2015-03-13 00:10:30,190 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:352) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p911-p1821,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,190 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:358) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p911-p1821,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,193 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:139) INFO 16 START:INVOKE
/mainoffline_a/EXTERNALVIEW
listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
2015-03-13 00:10:30,193 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:223) INFO terrapinmainoffline-a-namenode001_9090
subscribes child-change. path: /mainoffline_a/EXTERNALVIEW, listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
*2015-03-13 00:10:30,207 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:223) INFO terrapinmainoffline-a-namenode001_9090
subscribes child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331*
2015-03-13 00:10:30,217 [Thread-7] (ExternalViewComputeStage.java:167)
INFO END ExternalViewComputeStage.process(). took: 229 ms
2015-03-13 00:10:30,217 [Thread-7] (GenericHelixController.java:301) INFO
END: Invoking controller pipeline for event: currentStateChange, took 1082
ms
2015-03-13 00:10:30,218 [Thread-7] (GenericHelixController.java:288) INFO
START: Invoking controller pipeline for event: messageChange
2015-03-13 00:10:30,218 [Thread-7] (ReadClusterDataStage.java:47) INFO
START ReadClusterDataStage.process()
*2015-03-13 00:10:30,218 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:304) WARN fail to subscribe child/data change. path:
/mainoffline_a/EXTERNALVIEW, listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331*
*org.I0Itec.zkclient.exception.ZkNoNodeException:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
NoNode for
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102*
at
org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
at
org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
at org.apache.helix.manager.zk.ZkClient.getChildren(ZkClient.java:209)
at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
at
org.apache.helix.manager.zk.CallbackHandler.subscribeForChanges(CallbackHandler.java:279)
at
org.apache.helix.manager.zk.CallbackHandler.invoke(CallbackHandler.java:202)
at
org.apache.helix.manager.zk.CallbackHandler.handleChildChange(CallbackHandler.java:391)
at org.I0Itec.zkclient.ZkClient$7.run(ZkClient.java:568)
at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1277)
at
org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99)
at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:212)
at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:209)
at
org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
... 7 more
2015-03-13 00:10:30,218 [Thread-7] (ClusterDataCache.java:83) INFO START:
ClusterDataCache.refresh()
2015-03-13 00:10:30,347 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:214) INFO 16 END:INVOKE /mainoffline_a/EXTERNALVIEW
listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
Took: 154ms
2015-03-13 00:10:30,348 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:352) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p5466-p6376,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,348 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:358) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p5466-p6376,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,350 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:352) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p3644-p4554,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,350 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:358) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p3644-p4554,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,351 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:352) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p2733-p3643,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,351 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:358) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p2733-p3643,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,352 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:139) INFO 16 START:INVOKE
/mainoffline_a/EXTERNALVIEW
listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
2015-03-13 00:10:30,352 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:227) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path: /mainoffline_a/EXTERNALVIEW, listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,362 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:227) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,362 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p7288-p8198,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p2733-p3643,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p5466-p6376,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p3644-p4554,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p0-p910,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p911-p1821,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p6377-p7287,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p6377-p7287,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p4555-p5465,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p1822-p2732,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:227) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe child-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p683-p1365,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p1366-p2048,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p0-p682,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb73312015-03-13
00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$popular_pins$1422378222676,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$recommendation_p2p_fullsig$1422422699753,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
2015-03-13 00:10:30,370 [ZkClient-EventThread-16-terrapinzk001a:2181]
(CallbackHandler.java:244) INFO terrapinmainoffline-a-namenode001_9090
unsubscribe data-change. path:
/mainoffline_a/EXTERNALVIEW/$terrapin$data$u2p_from_fb_friends$1422438987857,
listener:
com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
*2015-03-13 00:10:30,439 [ZkClient-EventThread-16-terrapinzk001a:2181]
(RoutingTableProvider.java:99) INFO Resetting the routing table.*
On Thu, Mar 12, 2015 at 7:22 PM, Varun Sharma <[email protected]> wrote:
> Sorry spoke too soon - I saw the "Resetting the RoutingTableProvider" bug
> again which caused the RoutingTableProvider to stop receiving updates.
>
> Varun
>
> On Thu, Mar 12, 2015 at 5:24 PM, Varun Sharma <[email protected]> wrote:
>
>> I tested it out using a resource with 8192 partitions and 8 buckets and
>> it worked out just fine - (disable and drop).
>>
>> On Thu, Mar 12, 2015 at 4:08 PM, Varun Sharma <[email protected]>
>> wrote:
>>
>>> I just deployed this to a server node and when the current states node
>>> is moved over - the bucket nodes are no longer present - the node has
>>> executed dropped transitions for all the nodes. Is it expected to not
>>> contain any buckets ?
>>>
>>> On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <[email protected]> wrote:
>>>
>>>> Should be no problem. Controller and participant are compatible.
>>>>
>>>> ------------------------------
>>>> *From:* Varun Sharma [[email protected]]
>>>> *Sent:* Thursday, March 12, 2015 1:29 PM
>>>> *To:* [email protected]
>>>>
>>>> *Subject:* Re: Cutting a release 0.6.5 tonight
>>>>
>>>> Just to confirm - it wont be a problem if the controller is running
>>>> 0.6.5 and participant runs 0.6.4 for a while due to this ?
>>>>
>>>> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <[email protected]> wrote:
>>>>
>>>>> Yes, I will think of a way to make it backwards compatible.
>>>>>
>>>>> Thanks
>>>>> Kishore G
>>>>> On Mar 12, 2015 1:21 PM, "Varun Sharma" <[email protected]> wrote:
>>>>>
>>>>>> Kishore, I will be testing it out today. Btw, did you also look at
>>>>>> the issue with change of Abstract method signature createNewStateModel in
>>>>>> StateModelFactory ?
>>>>>>
>>>>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Jason and I have checked in the changes needed. I am not sure if
>>>>>>> TestSchedulerMsg is fixed.
>>>>>>>
>>>>>>> Last call for any other changes needed.
>>>>>>>
>>>>>>> Varun, it will be great if you can test the bug fixes for
>>>>>>> bucketing before we cut the release.
>>>>>>>
>>>>>>> thanks,
>>>>>>> Kishore G
>>>>>>>
>>>>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks Lei, I fixed the first two.
>>>>>>>>
>>>>>>>> Here is what caused the failure. I added the code to copy all
>>>>>>>> simple fields from IS to EV when we update the ExternalView in
>>>>>>>> controller.
>>>>>>>> This is to enable compression in ExternalView if its set in IS. I could
>>>>>>>> have copied only enableCompression variable but I thought its good to
>>>>>>>> have
>>>>>>>> partition number/replica etc in ExternalView as well. Let me know if
>>>>>>>> you
>>>>>>>> foresee any problem in this. Only thing I could think of is IdealState
>>>>>>>> is
>>>>>>>> deleted in which case I copy the simplefields from existing
>>>>>>>> externalview.
>>>>>>>>
>>>>>>>> I dont understand why testschedulermsg test cases are failing.
>>>>>>>> Uncommenting my code does not help either.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> The first two are concerning. I don't think the scheduler message
>>>>>>>>> test failures should block releases.
>>>>>>>>>
>>>>>>>>> ----------------------------------------
>>>>>>>>> > From: [email protected]
>>>>>>>>> > To: [email protected]
>>>>>>>>> > CC: [email protected]
>>>>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight
>>>>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000
>>>>>>>>> >
>>>>>>>>> > The test failed are:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
>>>>>>>>> >
>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue
>>>>>>>>> >
>>>>>>>>> > I got these from my local build too. (mvn clean install package
>>>>>>>>> on helix-0.6.x)
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Thanks
>>>>>>>>> > Lei
>>>>>>>>> >
>>>>>>>>> > --
>>>>>>>>> >
>>>>>>>>> > Lei Xia
>>>>>>>>> > Software Engineer
>>>>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage
>>>>>>>>> > LinkedIn
>>>>>>>>> >
>>>>>>>>> > [email protected]
>>>>>>>>> > www.linkedin.com/in/lxia1
>>>>>>>>> >
>>>>>>>>> > ________________________________________
>>>>>>>>> > From: kishore g [[email protected]]
>>>>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM
>>>>>>>>> > To: [email protected]
>>>>>>>>> > Cc: [email protected]
>>>>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight
>>>>>>>>> >
>>>>>>>>> > Hi Lei,
>>>>>>>>> >
>>>>>>>>> > Can you point to the failures?
>>>>>>>>> >
>>>>>>>>> > thanks,
>>>>>>>>> > Kishore G
>>>>>>>>> >
>>>>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia
>>>>>>>>> <[email protected]> wrote:
>>>>>>>>> >
>>>>>>>>> >> Hi, Kishore
>>>>>>>>> >>
>>>>>>>>> >> I saw there are regression test failures from last two recent
>>>>>>>>> commits
>>>>>>>>> >> on 0.6.x branch, running from both local box and Linkedin's
>>>>>>>>> hudson jobs.
>>>>>>>>> >> Are we going to fix them before the release?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> Thanks
>>>>>>>>> >> Lei
>>>>>>>>> >>
>>>>>>>>> >> --
>>>>>>>>> >>
>>>>>>>>> >> Lei Xia
>>>>>>>>> >> Software Engineer
>>>>>>>>> >> Data Infrastructure/Distributed Data Systems/Nuage
>>>>>>>>> >> LinkedIn
>>>>>>>>> >>
>>>>>>>>> >> [email protected]
>>>>>>>>> >> www.linkedin.com/in/lxia1
>>>>>>>>> >>
>>>>>>>>> >> ________________________________________
>>>>>>>>> >> From: kishore g [[email protected]]
>>>>>>>>> >> Sent: Wednesday, March 11, 2015 12:04 PM
>>>>>>>>> >> To: [email protected]
>>>>>>>>> >> Cc: [email protected]
>>>>>>>>> >> Subject: Re: Cutting a release 0.6.5 tonight
>>>>>>>>> >>
>>>>>>>>> >> Will write a test case for migration and document the steps
>>>>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma" <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >>> What would be the migration path from non-compressed buckets to
>>>>>>>>> >> compressed
>>>>>>>>> >>> non bucket resources ? It seems even the CURRENTSTATES are
>>>>>>>>> being bucketed
>>>>>>>>> >>> in this case, I thought that was not expected with bucketing.
>>>>>>>>> Does the
>>>>>>>>> >>> controller read these current states appropriately ? To
>>>>>>>>> migrate, it seems
>>>>>>>>> >>> that we would need to also rewrite the CURRENT STATES ?
>>>>>>>>> >>>
>>>>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>> >>>
>>>>>>>>> >>>> Hi,
>>>>>>>>> >>>>
>>>>>>>>> >>>> I will work with Jason to cut a 0.6.5 release tonight.
>>>>>>>>> >>>>
>>>>>>>>> >>>> The new thing I added is to enableCompression while storing
>>>>>>>>> data in
>>>>>>>>> >>>> Zookeeper, this allows us to go up to 100k partitions per
>>>>>>>>> resource
>>>>>>>>> >> without
>>>>>>>>> >>>> having to use bucketing feature. We also fixed few bugs with
>>>>>>>>> bucketed
>>>>>>>>> >>>> resource just in case some one needs it.
>>>>>>>>> >>>>
>>>>>>>>> >>>> The property store api needs some changes, I plan to get it
>>>>>>>>> in today.
>>>>>>>>> >>>>
>>>>>>>>> >>>> Let me know if you need any other changes to be included. Are
>>>>>>>>> there any
>>>>>>>>> >>>> changes that went into 0.7.x branch that we need to merge it
>>>>>>>>> back in to
>>>>>>>>> >>>> 0.6.x ?
>>>>>>>>> >>>>
>>>>>>>>> >>>> thanks,
>>>>>>>>> >>>> Kishore G
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>
>>>>>>>>> >>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
