On Mon, Mar 21, 2016 at 9:16 PM, Alexander Shraer <[email protected]> wrote: > another thing - shouldn't things like setting quotas also be part of the > admin API ? how does that > work now ? >
Hm, good point. There is no code level API afaik - the quotas are in /zookeeper znodes iirc. So if you care about controlling access to the quota settings you need to setup ACLs. I don't think we thought about that back in the day. Patrick > Alex > > On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <[email protected]> wrote: > >> I don't think that getConfig should be an admin functionality. It is >> essential for client-side re-balancing >> that we implemented (all clients shoudl be able to detect configuration >> changes via getConfig). It could >> be hidden somewhat by defining higher-level re-balancing >> policies (ZOOKEEPER-2016) >> but there hasn't been enough progress on that. Perhaps instead getConfig >> should be controlled >> by a separate flag ? >> >> Alex >> >> On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <[email protected]> wrote: >> >>> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <[email protected]> >>> wrote: >>> > Hi Patrick, Flavio, >>> > >>> > Since there seems to be consensus on this, I can add this flag, unless >>> > someone else wants to. I assume that getConfig should still work >>> regardless >>> > of the flag ? is there a security concern with clients knowing the list >>> of >>> > servers? >>> > >>> >>> We've always hidden that detail from users. We don't even expose which >>> server you're connected to today. I remember Ben (and perhaps Flavio?) >>> highlighting this as important to maintain although I'm not super >>> familiar with the specifics on why. It made sense to me though from >>> the perspective that we don't want clients to make assumptions that >>> probably shouldn't. >>> >>> My thinking is that we should 1) add a config option to enable >>> reconfig (off by default), 2) move reconfig specific functionality >>> from ZooKeeper.java (including getconfig) into an "admin" package, >>> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for >>> when folks do want to enable reconfig and are also worried about auth. >>> (e.g. turn on kerb) >>> >>> Again, I don't see any of this as a quality issue personally. As such >>> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we >>> were interested in doing such a release. Adjusting the API should be >>> done before we move to "beta" though. Although that seems like a >>> pretty mechanical (eclipse/idea) type refactoring? >>> >>> Patrick >>> >>> > Cheers, >>> > Alex >>> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <[email protected]> wrote: >>> > >>> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <[email protected]> >>> wrote: >>> >> > I gotta say that I'm not super excited about this option, but for >>> some >>> >> reason I ended up carrying the flag. To recap, I just raised this >>> option >>> >> because it seems that there are folks interested in features in 3.5 >>> like >>> >> SSL and not necessarily in reconfiguration. SSL is important and to >>> take >>> >> Kafka as an example, it sucks that we can't have a whole set up using >>> SSL. >>> >> For ZK, the real questions are: >>> >> > >>> >> > 1- how fast can we make 3.5 stable? >>> >> > 2- would it be faster if we have a way of disabling reconfiguration? >>> >> > 3- would enough users care about a stable 3.5 that has >>> reconfiguration >>> >> disabled? >>> >> > >>> >> > It is taking a long time to get 3.5 to beta. There has been some good >>> >> activity around 3.5.2 release, which is a great step, but it is unclear >>> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta >>> then. >>> >> Frankly, disabling reconfiguration sounds undesirable because it is an >>> >> important feature, but I currently don't use it in production, so from >>> a >>> >> practical point of view, I can go both ways. Whether we go through the >>> >> trouble of doing 2 depends on users interested in that option and folks >>> >> willing to implement it. >>> >> > >>> >> > To answer your question, Powell, my pseudo-proposal is kind of a >>> funny >>> >> option because once the feature is stable, then we wouldn't need a >>> switch >>> >> any longer, so there is not need of a deprecation path, we just start >>> >> ignoring it from the first beta release. Until it is beta, I'd say that >>> >> default is disabled. >>> >> >>> >> I would argue that we need this even when it does become stable. To me >>> >> this is not a quality issue so much as it is an auth issue. We want to >>> >> make it simple for folks to run a vanilla/old ZK cluster and not worry >>> >> about the security implications of having reconfig enabled. >>> >> >>> >> Patrick >>> >> >>> >> > >>> >> > -Flavio >>> >> > >>> >> >> On 17 Mar 2016, at 17:44, powell molleti >>> <[email protected]> >>> >> wrote: >>> >> >> >>> >> >> Hi Flavio, >>> >> >> >>> >> >> Generally do config options and command line args come under the >>> same >>> >> SLA as API?. I was assuming as such hence my question. Perhaps if the >>> >> expectation is that this config option is temporary from get go then >>> may be >>> >> it is ok. The default for re-config support will be enabled or >>> disabled?. >>> >> >> >>> >> >> I am just thinking from provisioning point of view when people >>> generate >>> >> config options etc. >>> >> >> >>> >> >> Thanks >>> >> >> Powell. >>> >> >> >>> >> >> >>> >> >> >>> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira < >>> [email protected]> >>> >> wrote: >>> >> >> Hi Powell, >>> >> >> >>> >> >> I was thinking config file and system property server side. What's >>> your >>> >> concern with compatibility? The API itself wouldn't change, but the >>> config >>> >> option wouldn't exist in previous versions and would not exist either >>> in >>> >> later stable versions of 3.5. >>> >> >> >>> >> >> -Flavio >>> >> >> >>> >> >> >>> >> >> >>> >> >>> On 16 Mar 2016, at 22:08, powell molleti >>> <[email protected]> >>> >> wrote: >>> >> >>> >>> >> >>> Will this option be supplied via config file/args/API?. Will this >>> >> option be a temporary thing i.e what about its compatibility?. >>> >> >>> >>> >> >>> thanks >>> >> >>> Powell. >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira < >>> [email protected]> >>> >> wrote: >>> >> >>> The main issue to sort out is stability of the API. There is a >>> >> security concern around the fact that clients can freely reconfigure >>> the >>> >> ensemble. If we follow the plan that Pat proposed some time ago: >>> >> >>> >>> >> >>> >>> >> >>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E >>> >> < >>> >> >>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E >>> >> > >>> >> >>> >>> >> >>> Locking the API is the main step to move it to beta. Sorting out >>> bugs >>> >> is definitely necessary, but it isn't the main thing that is keeping >>> 3.5 in >>> >> alpha. >>> >> >>> >>> >> >>> About making it experimental, I was raising the option of having a >>> >> switch that disables the API calls, not the code. The reason why that >>> could >>> >> work is that anyone using 3.5 who uses the "experimental" API must >>> explicit >>> >> turn on the switch and enable the calls. If they do it, they need to be >>> >> aware that the API can change. >>> >> >>> >>> >> >>> I must say that I haven't really looked closely into doing it, and >>> I'm >>> >> not even entirely convinced that this is a good idea, but since Jason >>> >> raised the point, I'm exploring options. >>> >> >>> >>> >> >>> -Flavio >>> >> >>> >>> >> >>> >>> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <[email protected]> >>> wrote: >>> >> >>>> >>> >> >>>> Looking at the list of ~50 blocker and critical bugs in ZooKeeper, >>> >> only 3-4 >>> >> >>>> are related to reconfig. Given this, and the fact that it is run >>> in >>> >> >>>> production since 2012 in multiple companies, I don't think its >>> more >>> >> >>>> unstable than any other part of ZooKeeper. >>> >> >>>> >>> >> >>>> There are multiple reconfig-related bugs that turned out really >>> >> difficult >>> >> >>>> to debug without access to the actual system or at least to the >>> Hudson >>> >> >>>> machines where some tests are failing. In the past, Michi and I >>> >> physically >>> >> >>>> went to Hortonworks to debug one such issue, but this is of course >>> >> not a >>> >> >>>> good method, and we weren't able to arrange such a visit again. >>> >> >>>> >>> >> >>>> Regarding making it optional - the reconfig logic has several >>> >> different >>> >> >>>> parts, some would be really difficult to disable using a >>> configuration >>> >> >>>> parameter. But the actual dynamic expansion of the cluster is >>> >> triggered by >>> >> >>>> the reconfig command, so it should not affect users who don't >>> invoke >>> >> it. >>> >> >>>> >>> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA < >>> [email protected]> >>> >> wrote: >>> >> >>>> >>> >> >>>>> I suppose we could give it a try. How do other folks feel about >>> it? >>> >> >>>>> >>> >> >>>>> -Flavio >>> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <[email protected]> >>> wrote: >>> >> >>>>> >>> >> >>>>>> So, you could enable the dynamic reconfiguration feature behind >>> a >>> >> config >>> >> >>>>>> option, and document that it should only be enabled >>> experimentally, >>> >> use >>> >> >>>>> at >>> >> >>>>>> your own risk. Keep it off by default. Allow only static >>> config by >>> >> >>>>>> default, until it's stable? >>> >> >>>>>> >>> >> >>>>>> Jason >>> >> >>>>>> >>> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira < >>> [email protected]> >>> >> >>>>> wrote: >>> >> >>>>>> >>> >> >>>>>>> Hi Jason, >>> >> >>>>>>> >>> >> >>>>>>> The consumer in Kafka is pretty independent from the core >>> >> (brokers), >>> >> >>>>>>> that's how that project manages to make such a separation. We >>> don't >>> >> >>>>> have >>> >> >>>>>>> the same with ZooKeeper as the feature we are talking about is >>> >> part of >>> >> >>>>>> the >>> >> >>>>>>> server and the only way I see of doing what you say is to turn >>> off >>> >> >>>>>>> features. More specifically, we'd need to disable the reconfig >>> API >>> >> and >>> >> >>>>> do >>> >> >>>>>>> not allow any change to the configuration, even though the >>> code is >>> >> >>>>> there. >>> >> >>>>>>> >>> >> >>>>>>> Reconfig here refers to the ability of changing the >>> configuration >>> >> of an >>> >> >>>>>>> ensemble (e.g., changing the set of servers). >>> >> >>>>>>> >>> >> >>>>>>> -Flavio >>> >> >>>>>>> >>> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <[email protected]> >>> >> wrote: >>> >> >>>>>>>> >>> >> >>>>>>>> So, it would seem sensible to me to have a release where all >>> >> features >>> >> >>>>>> are >>> >> >>>>>>>> stable, except where noted. E.g. mark certain features as >>> only >>> >> >>>>> 'alpha >>> >> >>>>>>>> quality', e.g. the 're-config feature'. (I assume it's >>> possible >>> >> to >>> >> >>>>>>> happily >>> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?). >>> >> >>>>>>>> >>> >> >>>>>>>> There's precedent for doing this sort of thing in other >>> projects, >>> >> >>>>> e.g. >>> >> >>>>>> in >>> >> >>>>>>>> Kafka, they've had several release where a new "Consumer API" >>> is >>> >> >>>>>> shipped >>> >> >>>>>>>> that is available for beta-testing, but you can still just >>> use the >>> >> >>>>>> older >>> >> >>>>>>>> stable consumer api, etc. >>> >> >>>>>>>> >>> >> >>>>>>>> Jason >>> >> >>>>>>>> >>> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti >>> >> >>>>>>> <[email protected] >>> >> >>>>>>>>> wrote: >>> >> >>>>>>>> >>> >> >>>>>>>>> Hi Doug, >>> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using it?. >>> Or >>> >> have >>> >> >>>>>> you >>> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being >>> >> labeled as >>> >> >>>>>>> alpha >>> >> >>>>>>>>> might not be fair, since it is far more stable then what most >>> >> people >>> >> >>>>>>>>> associate an alpha release to be. >>> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will >>> just >>> >> work >>> >> >>>>>> for >>> >> >>>>>>>>> you?. >>> >> >>>>>>>>> There are many examples of 3.5.X being used in productions >>> from >>> >> my >>> >> >>>>>>> limited >>> >> >>>>>>>>> knowledge. >>> >> >>>>>>>>> ThanksPowell. >>> >> >>>>>>>>> >>> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira < >>> >> >>>>>>> [email protected]> >>> >> >>>>>>>>> wrote: >>> >> >>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>>>> None of us expected the reconfig changes to take this long to >>> >> >>>>>> stabilize. >>> >> >>>>>>>>> Until we get there, I don't think we can do anything else >>> with >>> >> 3.5. >>> >> >>>>>> The >>> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a stable >>> >> rather >>> >> >>>>>>> than >>> >> >>>>>>>>> trying to work around it. >>> >> >>>>>>>>> >>> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, >>> and if >>> >> we >>> >> >>>>>> get >>> >> >>>>>>>>> everyone to contribute more patches and code reviews, we >>> should >>> >> be >>> >> >>>>>> able >>> >> >>>>>>> to >>> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so >>> the >>> >> >>>>>>> community >>> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work. >>> >> >>>>>>>>> >>> >> >>>>>>>>> -Flavio >>> >> >>>>>>>>> >>> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth < >>> >> [email protected]> >>> >> >>>>>>>>> wrote: >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4. Since >>> >> 3.4 is >>> >> >>>>>> the >>> >> >>>>>>>>>> stable maintenance line, we are very conservative about >>> >> >>>>> back-porting >>> >> >>>>>> to >>> >> >>>>>>>>>> it. Our policy is to limit back-ports to critical bug >>> fixes and >>> >> >>>>> not >>> >> >>>>>>>>>> introduce any new features in the 3.4 line. This is a >>> matter of >>> >> >>>>>>> managing >>> >> >>>>>>>>>> risk. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> Jason, your question about release cadence is a fair one. >>> If >>> >> it's >>> >> >>>>>> any >>> >> >>>>>>>>>> consolation, we are now taking the approach of trying to >>> limit >>> >> the >>> >> >>>>>>> scope >>> >> >>>>>>>>>> of anything new introduced in 3.5 too. That would allow us >>> to >>> >> >>>>> focus >>> >> >>>>>> on >>> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public >>> >> APIs. I >>> >> >>>>>>> think >>> >> >>>>>>>>>> this will help us accelerate the releases into beta and >>> eventual >>> >> >>>>> GA. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to >>> hear >>> >> >>>>>> thoughts >>> >> >>>>>>>>>> from more experienced committers and PMC members about your >>> >> >>>>> proposal >>> >> >>>>>> to >>> >> >>>>>>>>>> try to cut a stable release for a limited subset of what is >>> in >>> >> >>>>>>> branch-3.5 >>> >> >>>>>>>>>> now. My instinct is that it would be challenging to >>> cherry-pick >>> >> >>>>> out >>> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point. This would >>> become >>> >> >>>>>>> another >>> >> >>>>>>>>>> release line for an already resource-constrained volunteer >>> >> staff to >>> >> >>>>>>>>>> manage. I'd prefer to dedicate those limited resources to >>> >> overall >>> >> >>>>>> 3.5 >>> >> >>>>>>>>>> stabilization. Also, a 3.5 release in which certain >>> features >>> >> >>>>>>> "vanished" >>> >> >>>>>>>>>> because of not meeting some stability criteria would be >>> >> >>>>> undesirable. >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> --Chris Nauroth >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> >>> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <[email protected]> >>> >> wrote: >>> >> >>>>>>>>>> >>> >> >>>>>>>>>>> Chris, >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable >>> than >>> >> >>>>> others >>> >> >>>>>>>>> (e.g. >>> >> >>>>>>>>>>> if we don't care about certain new features, is it >>> relatively >>> >> >>>>>> stable)? >>> >> >>>>>>>>>>> Would it be possible to cut out a version that only has the >>> >> bits >>> >> >>>>> we >>> >> >>>>>>>>> think >>> >> >>>>>>>>>>> are stable (and release that)? >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it >>> would >>> >> >>>>> seem >>> >> >>>>>> to >>> >> >>>>>>>>> be >>> >> >>>>>>>>>>> a >>> >> >>>>>>>>>>> years away before we get to the stable release? >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> Thanks, >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> Jason >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth < >>> >> >>>>>>>>> [email protected]> >>> >> >>>>>>>>>>> wrote: >>> >> >>>>>>>>>>> >>> >> >>>>>>>>>>>> Hello Doug, >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature! >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from >>> >> >>>>> declaring a >>> >> >>>>>>>>>>>> stable >>> >> >>>>>>>>>>>> release in the 3.5 line. I don't think we're close enough >>> >> that >>> >> >>>>>>> anyone >>> >> >>>>>>>>>>>> can >>> >> >>>>>>>>>>>> offer a reliable ETA. This is an earlier thread that >>> >> describes >>> >> >>>>> the >>> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 line: >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> https://s.apache.org/ADK1 >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release. We're working on >>> >> >>>>>> resolving a >>> >> >>>>>>>>>>>> few >>> >> >>>>>>>>>>>> more blockers before we produce a release candidate. >>> >> Hopefully >>> >> >>>>>> that >>> >> >>>>>>>>>>>> will >>> >> >>>>>>>>>>>> get done in the next few weeks. >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> --Chris Nauroth >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <[email protected]> wrote: >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>>> I know it's only been a few months, but I was wondering >>> if >>> >> there >>> >> >>>>>>> was a >>> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. Or >>> is >>> >> there >>> >> >>>>>> any >>> >> >>>>>>>>>>>>> chance >>> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another >>> person >>> >> >>>>>> looking >>> >> >>>>>>>>> to >>> >> >>>>>>>>>>>>> have >>> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you do! >>> :) >>> >> >>>>>>>>>>>>> >>> >> >>>>>>>>>>>>> >>> >> >>>>>>>>>>>>> >>> >> >>>>>>>>>>>>> -- >>> >> >>>>>>>>>>>>> View this message in context: >>> >> >>>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>> >>> >> >>>>>> >>> >> >>>>> >>> >> >>> http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat >>> >> >>>>>>>>>>>> e >>> >> >>>>>>>>>>>>> -tp7581744p7582136.html >>> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at >>> >> Nabble.com. >>> >> >>>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>>>> >>> >> >>>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>>>> >>> >> >>>>>>> >>> >> >>>>>>> >>> >> >>>>>> >>> >> >>>>> >>> >> > >>> >> >>> >> >>
