On Thu, Mar 17, 2016 at 8:10 AM, Flavio Junqueira <[email protected]> wrote: > >> On 17 Mar 2016, at 14:12, Patrick Hunt <[email protected]> wrote: >> >> On Thu, Mar 17, 2016 at 12:59 AM, Flavio Junqueira <[email protected]> wrote: >>> I was thinking server side actually. If the switch is off, any attempt to >>> reconfig is a no-op and the server would log a warning. If you we want to >>> propagate the error up, then yeah, we need to start worrying about >>> compatibility and such. >>> >> >> Are you saying someone wants to run with reconfig inaccessible, but >> doesn't want to have to enable ACL/auth? That actually does make sense >> to me from a "feature switch" perspective. >> > > I think you meant to disagree here and say "doesn't make sense". It's a good > point, but for clarity, are you suggesting we disable it via ACLs? >
I did mean "does make sense". I've been thinking about it a bit and I'm coming around to what you are saying, but from a security/auth perspective rather than quality. One thing in particular is "principal of least surprise". If you're running zk w/o security turned on and suddenly folks can do reconfig operations it's going to potentially be a problem. It's like how we used to allow "kill" 4lw but then removed it because we didn't want to expose that ability to clients. Rather than force people to turn on kerberos (etc...) we could instead have the feature off by default through a simple configuration. Likely we'd want to ship with it turned off as the default. Folks that want to enable reconfig support could do so simply. Patrick >>> I also would rather see us address it, although I can fully understand that >>> there are other fixes and improvements in 3.5 that people are interested in. >>> >> >> My thinking is more that reconfig changed a lot of core functionality. >> It's not a "surface" feature. As such just disabling access doesn't >> affect quality. I don't see this as a concern for reconfig alone, >> there were a number of such changes in 3.5 and that's really my >> concern - overall core quality vs quality of one specific >> change/feature. >> > > Agreed that disabling doesn't affect quality, but possibly decouples the > decision of moving from alpha to beta independent of the security review of > reconfig. > > -Flavio > >>> >>>> On 17 Mar 2016, at 00:46, Patrick Hunt <[email protected]> wrote: >>>> >>>> I'm not a huge fan of turning it off to be honest. Also just turning >>>> it off at the API level wouldn't be enough, we'd need to turn it off >>>> at the protocol level (otw it could still be accessed). >>>> >>>> I'd rather see us address it than kick it down the road. It's a major >>>> feature of 3.5. >>>> >>>> Patrick >>>> >>>> On Wed, Mar 16, 2016 at 2:46 PM, Flavio Junqueira <[email protected]> wrote: >>>>> The main issue to sort out is stability of the API. There is a security >>>>> concern around the fact that clients can freely reconfigure the ensemble. >>>>> If we follow the plan that Pat proposed some time ago: >>>>> >>>>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E >>>>> >>>>> <https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E> >>>>> >>>>> Locking the API is the main step to move it to beta. Sorting out bugs is >>>>> definitely necessary, but it isn't the main thing that is keeping 3.5 in >>>>> alpha. >>>>> >>>>> About making it experimental, I was raising the option of having a switch >>>>> that disables the API calls, not the code. The reason why that could work >>>>> is that anyone using 3.5 who uses the "experimental" API must explicit >>>>> turn on the switch and enable the calls. If they do it, they need to be >>>>> aware that the API can change. >>>>> >>>>> I must say that I haven't really looked closely into doing it, and I'm >>>>> not even entirely convinced that this is a good idea, but since Jason >>>>> raised the point, I'm exploring options. >>>>> >>>>> -Flavio >>>>> >>>>>> On 16 Mar 2016, at 20:59, Alexander Shraer <[email protected]> wrote: >>>>>> >>>>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only >>>>>> 3-4 >>>>>> are related to reconfig. Given this, and the fact that it is run in >>>>>> production since 2012 in multiple companies, I don't think its more >>>>>> unstable than any other part of ZooKeeper. >>>>>> >>>>>> There are multiple reconfig-related bugs that turned out really difficult >>>>>> to debug without access to the actual system or at least to the Hudson >>>>>> machines where some tests are failing. In the past, Michi and I >>>>>> physically >>>>>> went to Hortonworks to debug one such issue, but this is of course not a >>>>>> good method, and we weren't able to arrange such a visit again. >>>>>> >>>>>> Regarding making it optional - the reconfig logic has several different >>>>>> parts, some would be really difficult to disable using a configuration >>>>>> parameter. But the actual dynamic expansion of the cluster is triggered >>>>>> by >>>>>> the reconfig command, so it should not affect users who don't invoke it. >>>>>> >>>>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I suppose we could give it a try. How do other folks feel about it? >>>>>>> >>>>>>> -Flavio >>>>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <[email protected]> wrote: >>>>>>> >>>>>>>> So, you could enable the dynamic reconfiguration feature behind a >>>>>>>> config >>>>>>>> option, and document that it should only be enabled experimentally, use >>>>>>> at >>>>>>>> your own risk. Keep it off by default. Allow only static config by >>>>>>>> default, until it's stable? >>>>>>>> >>>>>>>> Jason >>>>>>>> >>>>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jason, >>>>>>>>> >>>>>>>>> The consumer in Kafka is pretty independent from the core (brokers), >>>>>>>>> that's how that project manages to make such a separation. We don't >>>>>>> have >>>>>>>>> the same with ZooKeeper as the feature we are talking about is part of >>>>>>>> the >>>>>>>>> server and the only way I see of doing what you say is to turn off >>>>>>>>> features. More specifically, we'd need to disable the reconfig API and >>>>>>> do >>>>>>>>> not allow any change to the configuration, even though the code is >>>>>>> there. >>>>>>>>> >>>>>>>>> Reconfig here refers to the ability of changing the configuration of >>>>>>>>> an >>>>>>>>> ensemble (e.g., changing the set of servers). >>>>>>>>> >>>>>>>>> -Flavio >>>>>>>>> >>>>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> So, it would seem sensible to me to have a release where all features >>>>>>>> are >>>>>>>>>> stable, except where noted. E.g. mark certain features as only >>>>>>> 'alpha >>>>>>>>>> quality', e.g. the 're-config feature'. (I assume it's possible to >>>>>>>>> happily >>>>>>>>>> use 3.5.X without exercising the unstable re-config bits?). >>>>>>>>>> >>>>>>>>>> There's precedent for doing this sort of thing in other projects, >>>>>>> e.g. >>>>>>>> in >>>>>>>>>> Kafka, they've had several release where a new "Consumer API" is >>>>>>>> shipped >>>>>>>>>> that is available for beta-testing, but you can still just use the >>>>>>>> older >>>>>>>>>> stable consumer api, etc. >>>>>>>>>> >>>>>>>>>> Jason >>>>>>>>>> >>>>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti >>>>>>>>> <[email protected] >>>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Doug, >>>>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or have >>>>>>>> you >>>>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being labeled as >>>>>>>>> alpha >>>>>>>>>>> might not be fair, since it is far more stable then what most people >>>>>>>>>>> associate an alpha release to be. >>>>>>>>>>> Perhaps if you do not use re-config feature may be it will just work >>>>>>>> for >>>>>>>>>>> you?. >>>>>>>>>>> There are many examples of 3.5.X being used in productions from my >>>>>>>>> limited >>>>>>>>>>> knowledge. >>>>>>>>>>> ThanksPowell. >>>>>>>>>>> >>>>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira < >>>>>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> None of us expected the reconfig changes to take this long to >>>>>>>> stabilize. >>>>>>>>>>> Until we get there, I don't think we can do anything else with 3.5. >>>>>>>> The >>>>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable rather >>>>>>>>> than >>>>>>>>>>> trying to work around it. >>>>>>>>>>> >>>>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if we >>>>>>>> get >>>>>>>>>>> everyone to contribute more patches and code reviews, we should be >>>>>>>> able >>>>>>>>> to >>>>>>>>>>> do it sooner. After all, it is a community based effort, so the >>>>>>>>> community >>>>>>>>>>> shouldn't rely on just 2-3 people doing the work. >>>>>>>>>>> >>>>>>>>>>> -Flavio >>>>>>>>>>> >>>>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Doug, I forgot to respond to your question about 3.4. Since 3.4 is >>>>>>>> the >>>>>>>>>>>> stable maintenance line, we are very conservative about >>>>>>> back-porting >>>>>>>> to >>>>>>>>>>>> it. Our policy is to limit back-ports to critical bug fixes and >>>>>>> not >>>>>>>>>>>> introduce any new features in the 3.4 line. This is a matter of >>>>>>>>> managing >>>>>>>>>>>> risk. >>>>>>>>>>>> >>>>>>>>>>>> Jason, your question about release cadence is a fair one. If it's >>>>>>>> any >>>>>>>>>>>> consolation, we are now taking the approach of trying to limit the >>>>>>>>> scope >>>>>>>>>>>> of anything new introduced in 3.5 too. That would allow us to >>>>>>> focus >>>>>>>> on >>>>>>>>>>>> stabilization: resolving blocker bugs and freezing public APIs. I >>>>>>>>> think >>>>>>>>>>>> this will help us accelerate the releases into beta and eventual >>>>>>> GA. >>>>>>>>>>>> >>>>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear >>>>>>>> thoughts >>>>>>>>>>>> from more experienced committers and PMC members about your >>>>>>> proposal >>>>>>>> to >>>>>>>>>>>> try to cut a stable release for a limited subset of what is in >>>>>>>>> branch-3.5 >>>>>>>>>>>> now. My instinct is that it would be challenging to cherry-pick >>>>>>> out >>>>>>>>>>>> pieces of branch-3.5 piecemeal at this point. This would become >>>>>>>>> another >>>>>>>>>>>> release line for an already resource-constrained volunteer staff to >>>>>>>>>>>> manage. I'd prefer to dedicate those limited resources to overall >>>>>>>> 3.5 >>>>>>>>>>>> stabilization. Also, a 3.5 release in which certain features >>>>>>>>> "vanished" >>>>>>>>>>>> because of not meeting some stability criteria would be >>>>>>> undesirable. >>>>>>>>>>>> >>>>>>>>>>>> --Chris Nauroth >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Chris, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than >>>>>>> others >>>>>>>>>>> (e.g. >>>>>>>>>>>>> if we don't care about certain new features, is it relatively >>>>>>>> stable)? >>>>>>>>>>>>> Would it be possible to cut out a version that only has the bits >>>>>>> we >>>>>>>>>>> think >>>>>>>>>>>>> are stable (and release that)? >>>>>>>>>>>>> >>>>>>>>>>>>> From that timeline, and the historic release cadence, it would >>>>>>> seem >>>>>>>> to >>>>>>>>>>> be >>>>>>>>>>>>> a >>>>>>>>>>>>> years away before we get to the stable release? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Jason >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth < >>>>>>>>>>> [email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hello Doug, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your interest in the SSL feature! >>>>>>>>>>>>>> >>>>>>>>>>>>>> At this point, I think we're still pretty far away from >>>>>>> declaring a >>>>>>>>>>>>>> stable >>>>>>>>>>>>>> release in the 3.5 line. I don't think we're close enough that >>>>>>>>> anyone >>>>>>>>>>>>>> can >>>>>>>>>>>>>> offer a reliable ETA. This is an earlier thread that describes >>>>>>> the >>>>>>>>>>>>>> high-level strategy for release planning in the 3.5 line: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://s.apache.org/ADK1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The next step is a 3.5.2-alpha release. We're working on >>>>>>>> resolving a >>>>>>>>>>>>>> few >>>>>>>>>>>>>> more blockers before we produce a release candidate. Hopefully >>>>>>>> that >>>>>>>>>>>>>> will >>>>>>>>>>>>>> get done in the next few weeks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> --Chris Nauroth >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <[email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I know it's only been a few months, but I was wondering if there >>>>>>>>> was a >>>>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is there >>>>>>>> any >>>>>>>>>>>>>>> chance >>>>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person >>>>>>>> looking >>>>>>>>>>> to >>>>>>>>>>>>>>> have >>>>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> View this message in context: >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat >>>>>>>>>>>>>> e >>>>>>>>>>>>>>> -tp7581744p7582136.html >>>>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >
