Hi Patrick, Flavio, Since there seems to be consensus on this, I can add this flag, unless someone else wants to. I assume that getConfig should still work regardless of the flag ? is there a security concern with clients knowing the list of servers?
Cheers, Alex On Mar 21, 2016 8:34 PM, "Patrick Hunt" <[email protected]> wrote: > On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <[email protected]> wrote: > > I gotta say that I'm not super excited about this option, but for some > reason I ended up carrying the flag. To recap, I just raised this option > because it seems that there are folks interested in features in 3.5 like > SSL and not necessarily in reconfiguration. SSL is important and to take > Kafka as an example, it sucks that we can't have a whole set up using SSL. > For ZK, the real questions are: > > > > 1- how fast can we make 3.5 stable? > > 2- would it be faster if we have a way of disabling reconfiguration? > > 3- would enough users care about a stable 3.5 that has reconfiguration > disabled? > > > > It is taking a long time to get 3.5 to beta. There has been some good > activity around 3.5.2 release, which is a great step, but it is unclear > when 3.5.3 is going to come and if we will be able to make 3.5 beta then. > Frankly, disabling reconfiguration sounds undesirable because it is an > important feature, but I currently don't use it in production, so from a > practical point of view, I can go both ways. Whether we go through the > trouble of doing 2 depends on users interested in that option and folks > willing to implement it. > > > > To answer your question, Powell, my pseudo-proposal is kind of a funny > option because once the feature is stable, then we wouldn't need a switch > any longer, so there is not need of a deprecation path, we just start > ignoring it from the first beta release. Until it is beta, I'd say that > default is disabled. > > I would argue that we need this even when it does become stable. To me > this is not a quality issue so much as it is an auth issue. We want to > make it simple for folks to run a vanilla/old ZK cluster and not worry > about the security implications of having reconfig enabled. > > Patrick > > > > > -Flavio > > > >> On 17 Mar 2016, at 17:44, powell molleti <[email protected]> > wrote: > >> > >> Hi Flavio, > >> > >> Generally do config options and command line args come under the same > SLA as API?. I was assuming as such hence my question. Perhaps if the > expectation is that this config option is temporary from get go then may be > it is ok. The default for re-config support will be enabled or disabled?. > >> > >> I am just thinking from provisioning point of view when people generate > config options etc. > >> > >> Thanks > >> Powell. > >> > >> > >> > >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira <[email protected]> > wrote: > >> Hi Powell, > >> > >> I was thinking config file and system property server side. What's your > concern with compatibility? The API itself wouldn't change, but the config > option wouldn't exist in previous versions and would not exist either in > later stable versions of 3.5. > >> > >> -Flavio > >> > >> > >> > >>> On 16 Mar 2016, at 22:08, powell molleti <[email protected]> > wrote: > >>> > >>> Will this option be supplied via config file/args/API?. Will this > option be a temporary thing i.e what about its compatibility?. > >>> > >>> thanks > >>> Powell. > >>> > >>> > >>> > >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira <[email protected]> > wrote: > >>> The main issue to sort out is stability of the API. There is a > security concern around the fact that clients can freely reconfigure the > ensemble. If we follow the plan that Pat proposed some time ago: > >>> > >>> > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > < > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > > > >>> > >>> Locking the API is the main step to move it to beta. Sorting out bugs > is definitely necessary, but it isn't the main thing that is keeping 3.5 in > alpha. > >>> > >>> About making it experimental, I was raising the option of having a > switch that disables the API calls, not the code. The reason why that could > work is that anyone using 3.5 who uses the "experimental" API must explicit > turn on the switch and enable the calls. If they do it, they need to be > aware that the API can change. > >>> > >>> I must say that I haven't really looked closely into doing it, and I'm > not even entirely convinced that this is a good idea, but since Jason > raised the point, I'm exploring options. > >>> > >>> -Flavio > >>> > >>> > >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <[email protected]> wrote: > >>>> > >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, > only 3-4 > >>>> are related to reconfig. Given this, and the fact that it is run in > >>>> production since 2012 in multiple companies, I don't think its more > >>>> unstable than any other part of ZooKeeper. > >>>> > >>>> There are multiple reconfig-related bugs that turned out really > difficult > >>>> to debug without access to the actual system or at least to the Hudson > >>>> machines where some tests are failing. In the past, Michi and I > physically > >>>> went to Hortonworks to debug one such issue, but this is of course > not a > >>>> good method, and we weren't able to arrange such a visit again. > >>>> > >>>> Regarding making it optional - the reconfig logic has several > different > >>>> parts, some would be really difficult to disable using a configuration > >>>> parameter. But the actual dynamic expansion of the cluster is > triggered by > >>>> the reconfig command, so it should not affect users who don't invoke > it. > >>>> > >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA <[email protected]> > wrote: > >>>> > >>>>> I suppose we could give it a try. How do other folks feel about it? > >>>>> > >>>>> -Flavio > >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <[email protected]> wrote: > >>>>> > >>>>>> So, you could enable the dynamic reconfiguration feature behind a > config > >>>>>> option, and document that it should only be enabled experimentally, > use > >>>>> at > >>>>>> your own risk. Keep it off by default. Allow only static config by > >>>>>> default, until it's stable? > >>>>>> > >>>>>> Jason > >>>>>> > >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>>> Hi Jason, > >>>>>>> > >>>>>>> The consumer in Kafka is pretty independent from the core > (brokers), > >>>>>>> that's how that project manages to make such a separation. We don't > >>>>> have > >>>>>>> the same with ZooKeeper as the feature we are talking about is > part of > >>>>>> the > >>>>>>> server and the only way I see of doing what you say is to turn off > >>>>>>> features. More specifically, we'd need to disable the reconfig API > and > >>>>> do > >>>>>>> not allow any change to the configuration, even though the code is > >>>>> there. > >>>>>>> > >>>>>>> Reconfig here refers to the ability of changing the configuration > of an > >>>>>>> ensemble (e.g., changing the set of servers). > >>>>>>> > >>>>>>> -Flavio > >>>>>>> > >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> So, it would seem sensible to me to have a release where all > features > >>>>>> are > >>>>>>>> stable, except where noted. E.g. mark certain features as only > >>>>> 'alpha > >>>>>>>> quality', e.g. the 're-config feature'. (I assume it's possible > to > >>>>>>> happily > >>>>>>>> use 3.5.X without exercising the unstable re-config bits?). > >>>>>>>> > >>>>>>>> There's precedent for doing this sort of thing in other projects, > >>>>> e.g. > >>>>>> in > >>>>>>>> Kafka, they've had several release where a new "Consumer API" is > >>>>>> shipped > >>>>>>>> that is available for beta-testing, but you can still just use the > >>>>>> older > >>>>>>>> stable consumer api, etc. > >>>>>>>> > >>>>>>>> Jason > >>>>>>>> > >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti > >>>>>>> <[email protected] > >>>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Doug, > >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. Or > have > >>>>>> you > >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being > labeled as > >>>>>>> alpha > >>>>>>>>> might not be fair, since it is far more stable then what most > people > >>>>>>>>> associate an alpha release to be. > >>>>>>>>> Perhaps if you do not use re-config feature may be it will just > work > >>>>>> for > >>>>>>>>> you?. > >>>>>>>>> There are many examples of 3.5.X being used in productions from > my > >>>>>>> limited > >>>>>>>>> knowledge. > >>>>>>>>> ThanksPowell. > >>>>>>>>> > >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira < > >>>>>>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> None of us expected the reconfig changes to take this long to > >>>>>> stabilize. > >>>>>>>>> Until we get there, I don't think we can do anything else with > 3.5. > >>>>>> The > >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable > rather > >>>>>>> than > >>>>>>>>> trying to work around it. > >>>>>>>>> > >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and if > we > >>>>>> get > >>>>>>>>> everyone to contribute more patches and code reviews, we should > be > >>>>>> able > >>>>>>> to > >>>>>>>>> do it sooner. After all, it is a community based effort, so the > >>>>>>> community > >>>>>>>>> shouldn't rely on just 2-3 people doing the work. > >>>>>>>>> > >>>>>>>>> -Flavio > >>>>>>>>> > >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth < > [email protected]> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Doug, I forgot to respond to your question about 3.4. Since > 3.4 is > >>>>>> the > >>>>>>>>>> stable maintenance line, we are very conservative about > >>>>> back-porting > >>>>>> to > >>>>>>>>>> it. Our policy is to limit back-ports to critical bug fixes and > >>>>> not > >>>>>>>>>> introduce any new features in the 3.4 line. This is a matter of > >>>>>>> managing > >>>>>>>>>> risk. > >>>>>>>>>> > >>>>>>>>>> Jason, your question about release cadence is a fair one. If > it's > >>>>>> any > >>>>>>>>>> consolation, we are now taking the approach of trying to limit > the > >>>>>>> scope > >>>>>>>>>> of anything new introduced in 3.5 too. That would allow us to > >>>>> focus > >>>>>> on > >>>>>>>>>> stabilization: resolving blocker bugs and freezing public > APIs. I > >>>>>>> think > >>>>>>>>>> this will help us accelerate the releases into beta and eventual > >>>>> GA. > >>>>>>>>>> > >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear > >>>>>> thoughts > >>>>>>>>>> from more experienced committers and PMC members about your > >>>>> proposal > >>>>>> to > >>>>>>>>>> try to cut a stable release for a limited subset of what is in > >>>>>>> branch-3.5 > >>>>>>>>>> now. My instinct is that it would be challenging to cherry-pick > >>>>> out > >>>>>>>>>> pieces of branch-3.5 piecemeal at this point. This would become > >>>>>>> another > >>>>>>>>>> release line for an already resource-constrained volunteer > staff to > >>>>>>>>>> manage. I'd prefer to dedicate those limited resources to > overall > >>>>>> 3.5 > >>>>>>>>>> stabilization. Also, a 3.5 release in which certain features > >>>>>>> "vanished" > >>>>>>>>>> because of not meeting some stability criteria would be > >>>>> undesirable. > >>>>>>>>>> > >>>>>>>>>> --Chris Nauroth > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <[email protected]> > wrote: > >>>>>>>>>> > >>>>>>>>>>> Chris, > >>>>>>>>>>> > >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than > >>>>> others > >>>>>>>>> (e.g. > >>>>>>>>>>> if we don't care about certain new features, is it relatively > >>>>>> stable)? > >>>>>>>>>>> Would it be possible to cut out a version that only has the > bits > >>>>> we > >>>>>>>>> think > >>>>>>>>>>> are stable (and release that)? > >>>>>>>>>>> > >>>>>>>>>>> From that timeline, and the historic release cadence, it would > >>>>> seem > >>>>>> to > >>>>>>>>> be > >>>>>>>>>>> a > >>>>>>>>>>> years away before we get to the stable release? > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Jason > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth < > >>>>>>>>> [email protected]> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hello Doug, > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks for your interest in the SSL feature! > >>>>>>>>>>>> > >>>>>>>>>>>> At this point, I think we're still pretty far away from > >>>>> declaring a > >>>>>>>>>>>> stable > >>>>>>>>>>>> release in the 3.5 line. I don't think we're close enough > that > >>>>>>> anyone > >>>>>>>>>>>> can > >>>>>>>>>>>> offer a reliable ETA. This is an earlier thread that > describes > >>>>> the > >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line: > >>>>>>>>>>>> > >>>>>>>>>>>> https://s.apache.org/ADK1 > >>>>>>>>>>>> > >>>>>>>>>>>> The next step is a 3.5.2-alpha release. We're working on > >>>>>> resolving a > >>>>>>>>>>>> few > >>>>>>>>>>>> more blockers before we produce a release candidate. > Hopefully > >>>>>> that > >>>>>>>>>>>> will > >>>>>>>>>>>> get done in the next few weeks. > >>>>>>>>>>>> > >>>>>>>>>>>> --Chris Nauroth > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <[email protected]> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if > there > >>>>>>> was a > >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is > there > >>>>>> any > >>>>>>>>>>>>> chance > >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another person > >>>>>> looking > >>>>>>>>> to > >>>>>>>>>>>>> have > >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! :) > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> View this message in context: > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat > >>>>>>>>>>>> e > >>>>>>>>>>>>> -tp7581744p7582136.html > >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at > Nabble.com. > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > > >
