I don't think that getConfig should be an admin functionality. It is essential for client-side re-balancing that we implemented (all clients shoudl be able to detect configuration changes via getConfig). It could be hidden somewhat by defining higher-level re-balancing policies (ZOOKEEPER-2016) but there hasn't been enough progress on that. Perhaps instead getConfig should be controlled by a separate flag ?
Alex On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <[email protected]> wrote: > On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <[email protected]> > wrote: > > Hi Patrick, Flavio, > > > > Since there seems to be consensus on this, I can add this flag, unless > > someone else wants to. I assume that getConfig should still work > regardless > > of the flag ? is there a security concern with clients knowing the list > of > > servers? > > > > We've always hidden that detail from users. We don't even expose which > server you're connected to today. I remember Ben (and perhaps Flavio?) > highlighting this as important to maintain although I'm not super > familiar with the specifics on why. It made sense to me though from > the perspective that we don't want clients to make assumptions that > probably shouldn't. > > My thinking is that we should 1) add a config option to enable > reconfig (off by default), 2) move reconfig specific functionality > from ZooKeeper.java (including getconfig) into an "admin" package, > within say a class ZooKeeperAdmin, 3) document/test use of ACLs for > when folks do want to enable reconfig and are also worried about auth. > (e.g. turn on kerb) > > Again, I don't see any of this as a quality issue personally. As such > I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we > were interested in doing such a release. Adjusting the API should be > done before we move to "beta" though. Although that seems like a > pretty mechanical (eclipse/idea) type refactoring? > > Patrick > > > Cheers, > > Alex > > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <[email protected]> wrote: > > > >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <[email protected]> > wrote: > >> > I gotta say that I'm not super excited about this option, but for some > >> reason I ended up carrying the flag. To recap, I just raised this option > >> because it seems that there are folks interested in features in 3.5 like > >> SSL and not necessarily in reconfiguration. SSL is important and to take > >> Kafka as an example, it sucks that we can't have a whole set up using > SSL. > >> For ZK, the real questions are: > >> > > >> > 1- how fast can we make 3.5 stable? > >> > 2- would it be faster if we have a way of disabling reconfiguration? > >> > 3- would enough users care about a stable 3.5 that has reconfiguration > >> disabled? > >> > > >> > It is taking a long time to get 3.5 to beta. There has been some good > >> activity around 3.5.2 release, which is a great step, but it is unclear > >> when 3.5.3 is going to come and if we will be able to make 3.5 beta > then. > >> Frankly, disabling reconfiguration sounds undesirable because it is an > >> important feature, but I currently don't use it in production, so from a > >> practical point of view, I can go both ways. Whether we go through the > >> trouble of doing 2 depends on users interested in that option and folks > >> willing to implement it. > >> > > >> > To answer your question, Powell, my pseudo-proposal is kind of a funny > >> option because once the feature is stable, then we wouldn't need a > switch > >> any longer, so there is not need of a deprecation path, we just start > >> ignoring it from the first beta release. Until it is beta, I'd say that > >> default is disabled. > >> > >> I would argue that we need this even when it does become stable. To me > >> this is not a quality issue so much as it is an auth issue. We want to > >> make it simple for folks to run a vanilla/old ZK cluster and not worry > >> about the security implications of having reconfig enabled. > >> > >> Patrick > >> > >> > > >> > -Flavio > >> > > >> >> On 17 Mar 2016, at 17:44, powell molleti <[email protected] > > > >> wrote: > >> >> > >> >> Hi Flavio, > >> >> > >> >> Generally do config options and command line args come under the same > >> SLA as API?. I was assuming as such hence my question. Perhaps if the > >> expectation is that this config option is temporary from get go then > may be > >> it is ok. The default for re-config support will be enabled or > disabled?. > >> >> > >> >> I am just thinking from provisioning point of view when people > generate > >> config options etc. > >> >> > >> >> Thanks > >> >> Powell. > >> >> > >> >> > >> >> > >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira < > [email protected]> > >> wrote: > >> >> Hi Powell, > >> >> > >> >> I was thinking config file and system property server side. What's > your > >> concern with compatibility? The API itself wouldn't change, but the > config > >> option wouldn't exist in previous versions and would not exist either in > >> later stable versions of 3.5. > >> >> > >> >> -Flavio > >> >> > >> >> > >> >> > >> >>> On 16 Mar 2016, at 22:08, powell molleti > <[email protected]> > >> wrote: > >> >>> > >> >>> Will this option be supplied via config file/args/API?. Will this > >> option be a temporary thing i.e what about its compatibility?. > >> >>> > >> >>> thanks > >> >>> Powell. > >> >>> > >> >>> > >> >>> > >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira < > [email protected]> > >> wrote: > >> >>> The main issue to sort out is stability of the API. There is a > >> security concern around the fact that clients can freely reconfigure the > >> ensemble. If we follow the plan that Pat proposed some time ago: > >> >>> > >> >>> > >> > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > >> < > >> > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > >> > > >> >>> > >> >>> Locking the API is the main step to move it to beta. Sorting out > bugs > >> is definitely necessary, but it isn't the main thing that is keeping > 3.5 in > >> alpha. > >> >>> > >> >>> About making it experimental, I was raising the option of having a > >> switch that disables the API calls, not the code. The reason why that > could > >> work is that anyone using 3.5 who uses the "experimental" API must > explicit > >> turn on the switch and enable the calls. If they do it, they need to be > >> aware that the API can change. > >> >>> > >> >>> I must say that I haven't really looked closely into doing it, and > I'm > >> not even entirely convinced that this is a good idea, but since Jason > >> raised the point, I'm exploring options. > >> >>> > >> >>> -Flavio > >> >>> > >> >>> > >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <[email protected]> > wrote: > >> >>>> > >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, > >> only 3-4 > >> >>>> are related to reconfig. Given this, and the fact that it is run in > >> >>>> production since 2012 in multiple companies, I don't think its more > >> >>>> unstable than any other part of ZooKeeper. > >> >>>> > >> >>>> There are multiple reconfig-related bugs that turned out really > >> difficult > >> >>>> to debug without access to the actual system or at least to the > Hudson > >> >>>> machines where some tests are failing. In the past, Michi and I > >> physically > >> >>>> went to Hortonworks to debug one such issue, but this is of course > >> not a > >> >>>> good method, and we weren't able to arrange such a visit again. > >> >>>> > >> >>>> Regarding making it optional - the reconfig logic has several > >> different > >> >>>> parts, some would be really difficult to disable using a > configuration > >> >>>> parameter. But the actual dynamic expansion of the cluster is > >> triggered by > >> >>>> the reconfig command, so it should not affect users who don't > invoke > >> it. > >> >>>> > >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA < > [email protected]> > >> wrote: > >> >>>> > >> >>>>> I suppose we could give it a try. How do other folks feel about > it? > >> >>>>> > >> >>>>> -Flavio > >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <[email protected]> wrote: > >> >>>>> > >> >>>>>> So, you could enable the dynamic reconfiguration feature behind a > >> config > >> >>>>>> option, and document that it should only be enabled > experimentally, > >> use > >> >>>>> at > >> >>>>>> your own risk. Keep it off by default. Allow only static > config by > >> >>>>>> default, until it's stable? > >> >>>>>> > >> >>>>>> Jason > >> >>>>>> > >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira < > [email protected]> > >> >>>>> wrote: > >> >>>>>> > >> >>>>>>> Hi Jason, > >> >>>>>>> > >> >>>>>>> The consumer in Kafka is pretty independent from the core > >> (brokers), > >> >>>>>>> that's how that project manages to make such a separation. We > don't > >> >>>>> have > >> >>>>>>> the same with ZooKeeper as the feature we are talking about is > >> part of > >> >>>>>> the > >> >>>>>>> server and the only way I see of doing what you say is to turn > off > >> >>>>>>> features. More specifically, we'd need to disable the reconfig > API > >> and > >> >>>>> do > >> >>>>>>> not allow any change to the configuration, even though the code > is > >> >>>>> there. > >> >>>>>>> > >> >>>>>>> Reconfig here refers to the ability of changing the > configuration > >> of an > >> >>>>>>> ensemble (e.g., changing the set of servers). > >> >>>>>>> > >> >>>>>>> -Flavio > >> >>>>>>> > >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <[email protected]> > >> wrote: > >> >>>>>>>> > >> >>>>>>>> So, it would seem sensible to me to have a release where all > >> features > >> >>>>>> are > >> >>>>>>>> stable, except where noted. E.g. mark certain features as only > >> >>>>> 'alpha > >> >>>>>>>> quality', e.g. the 're-config feature'. (I assume it's > possible > >> to > >> >>>>>>> happily > >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?). > >> >>>>>>>> > >> >>>>>>>> There's precedent for doing this sort of thing in other > projects, > >> >>>>> e.g. > >> >>>>>> in > >> >>>>>>>> Kafka, they've had several release where a new "Consumer API" > is > >> >>>>>> shipped > >> >>>>>>>> that is available for beta-testing, but you can still just use > the > >> >>>>>> older > >> >>>>>>>> stable consumer api, etc. > >> >>>>>>>> > >> >>>>>>>> Jason > >> >>>>>>>> > >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti > >> >>>>>>> <[email protected] > >> >>>>>>>>> wrote: > >> >>>>>>>> > >> >>>>>>>>> Hi Doug, > >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. > Or > >> have > >> >>>>>> you > >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being > >> labeled as > >> >>>>>>> alpha > >> >>>>>>>>> might not be fair, since it is far more stable then what most > >> people > >> >>>>>>>>> associate an alpha release to be. > >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will > just > >> work > >> >>>>>> for > >> >>>>>>>>> you?. > >> >>>>>>>>> There are many examples of 3.5.X being used in productions > from > >> my > >> >>>>>>> limited > >> >>>>>>>>> knowledge. > >> >>>>>>>>> ThanksPowell. > >> >>>>>>>>> > >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira < > >> >>>>>>> [email protected]> > >> >>>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> None of us expected the reconfig changes to take this long to > >> >>>>>> stabilize. > >> >>>>>>>>> Until we get there, I don't think we can do anything else with > >> 3.5. > >> >>>>>> The > >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable > >> rather > >> >>>>>>> than > >> >>>>>>>>> trying to work around it. > >> >>>>>>>>> > >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, and > if > >> we > >> >>>>>> get > >> >>>>>>>>> everyone to contribute more patches and code reviews, we > should > >> be > >> >>>>>> able > >> >>>>>>> to > >> >>>>>>>>> do it sooner. After all, it is a community based effort, so > the > >> >>>>>>> community > >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work. > >> >>>>>>>>> > >> >>>>>>>>> -Flavio > >> >>>>>>>>> > >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth < > >> [email protected]> > >> >>>>>>>>> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4. Since > >> 3.4 is > >> >>>>>> the > >> >>>>>>>>>> stable maintenance line, we are very conservative about > >> >>>>> back-porting > >> >>>>>> to > >> >>>>>>>>>> it. Our policy is to limit back-ports to critical bug fixes > and > >> >>>>> not > >> >>>>>>>>>> introduce any new features in the 3.4 line. This is a > matter of > >> >>>>>>> managing > >> >>>>>>>>>> risk. > >> >>>>>>>>>> > >> >>>>>>>>>> Jason, your question about release cadence is a fair one. If > >> it's > >> >>>>>> any > >> >>>>>>>>>> consolation, we are now taking the approach of trying to > limit > >> the > >> >>>>>>> scope > >> >>>>>>>>>> of anything new introduced in 3.5 too. That would allow us > to > >> >>>>> focus > >> >>>>>> on > >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public > >> APIs. I > >> >>>>>>> think > >> >>>>>>>>>> this will help us accelerate the releases into beta and > eventual > >> >>>>> GA. > >> >>>>>>>>>> > >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to hear > >> >>>>>> thoughts > >> >>>>>>>>>> from more experienced committers and PMC members about your > >> >>>>> proposal > >> >>>>>> to > >> >>>>>>>>>> try to cut a stable release for a limited subset of what is > in > >> >>>>>>> branch-3.5 > >> >>>>>>>>>> now. My instinct is that it would be challenging to > cherry-pick > >> >>>>> out > >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point. This would > become > >> >>>>>>> another > >> >>>>>>>>>> release line for an already resource-constrained volunteer > >> staff to > >> >>>>>>>>>> manage. I'd prefer to dedicate those limited resources to > >> overall > >> >>>>>> 3.5 > >> >>>>>>>>>> stabilization. Also, a 3.5 release in which certain features > >> >>>>>>> "vanished" > >> >>>>>>>>>> because of not meeting some stability criteria would be > >> >>>>> undesirable. > >> >>>>>>>>>> > >> >>>>>>>>>> --Chris Nauroth > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <[email protected]> > >> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>> Chris, > >> >>>>>>>>>>> > >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable than > >> >>>>> others > >> >>>>>>>>> (e.g. > >> >>>>>>>>>>> if we don't care about certain new features, is it > relatively > >> >>>>>> stable)? > >> >>>>>>>>>>> Would it be possible to cut out a version that only has the > >> bits > >> >>>>> we > >> >>>>>>>>> think > >> >>>>>>>>>>> are stable (and release that)? > >> >>>>>>>>>>> > >> >>>>>>>>>>> From that timeline, and the historic release cadence, it > would > >> >>>>> seem > >> >>>>>> to > >> >>>>>>>>> be > >> >>>>>>>>>>> a > >> >>>>>>>>>>> years away before we get to the stable release? > >> >>>>>>>>>>> > >> >>>>>>>>>>> Thanks, > >> >>>>>>>>>>> > >> >>>>>>>>>>> Jason > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth < > >> >>>>>>>>> [email protected]> > >> >>>>>>>>>>> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> Hello Doug, > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Thanks for your interest in the SSL feature! > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> At this point, I think we're still pretty far away from > >> >>>>> declaring a > >> >>>>>>>>>>>> stable > >> >>>>>>>>>>>> release in the 3.5 line. I don't think we're close enough > >> that > >> >>>>>>> anyone > >> >>>>>>>>>>>> can > >> >>>>>>>>>>>> offer a reliable ETA. This is an earlier thread that > >> describes > >> >>>>> the > >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> https://s.apache.org/ADK1 > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release. We're working on > >> >>>>>> resolving a > >> >>>>>>>>>>>> few > >> >>>>>>>>>>>> more blockers before we produce a release candidate. > >> Hopefully > >> >>>>>> that > >> >>>>>>>>>>>> will > >> >>>>>>>>>>>> get done in the next few weeks. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> --Chris Nauroth > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <[email protected]> wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering if > >> there > >> >>>>>>> was a > >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or is > >> there > >> >>>>>> any > >> >>>>>>>>>>>>> chance > >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another > person > >> >>>>>> looking > >> >>>>>>>>> to > >> >>>>>>>>>>>>> have > >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! > :) > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> -- > >> >>>>>>>>>>>>> View this message in context: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> > http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat > >> >>>>>>>>>>>> e > >> >>>>>>>>>>>>> -tp7581744p7582136.html > >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at > >> Nabble.com. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> > > >> >
