ok, thanks for the suggestion, I'll look into it. For reconfig I think its pretty clear that its an admin functionality. I just always imagined that its controlled via acls, but I understand the concerns now.
getConfig returns the dynamic config (list of all servers with all ports and quorum system if defined) and has an option to filter that info and just return the server connection string (server and client port only). On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <[email protected]> wrote: > On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <[email protected]> > wrote: > > I don't think that getConfig should be an admin functionality. It is > > essential for client-side re-balancing > > that we implemented (all clients shoudl be able to detect configuration > > changes via getConfig). It could > > be hidden somewhat by defining higher-level re-balancing > > policies (ZOOKEEPER-2016) > > but there hasn't been enough progress on that. Perhaps instead getConfig > > should be controlled > > by a separate flag ? > > > > I believe that the Hadoop community has something we could use: > > https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/InterfaceClassification.html > (whether through annotations or just documenting it in the API javadoc) > > e.g. we could list getConfig as public/unstable for example and still > ship it as GA. That would mark it as something that could change re > API policy. > > Is the entire config exposed through getConfig? If so then we might > want to enable/disable it with a flag similar to reconfig. Might be > safer to just do that if we're not sure. > > > Re classification - we could do the same thing with reconfig, but I > think that would be a mistake. If we feel strongly where it should > live long term we should just move it now. > > Patrick > > > > > On Mon, Mar 21, 2016 at 9:04 PM, Patrick Hunt <[email protected]> wrote: > > > >> On Mon, Mar 21, 2016 at 8:52 PM, Alexander Shraer <[email protected]> > >> wrote: > >> > Hi Patrick, Flavio, > >> > > >> > Since there seems to be consensus on this, I can add this flag, unless > >> > someone else wants to. I assume that getConfig should still work > >> regardless > >> > of the flag ? is there a security concern with clients knowing the > list > >> of > >> > servers? > >> > > >> > >> We've always hidden that detail from users. We don't even expose which > >> server you're connected to today. I remember Ben (and perhaps Flavio?) > >> highlighting this as important to maintain although I'm not super > >> familiar with the specifics on why. It made sense to me though from > >> the perspective that we don't want clients to make assumptions that > >> probably shouldn't. > >> > >> My thinking is that we should 1) add a config option to enable > >> reconfig (off by default), 2) move reconfig specific functionality > >> from ZooKeeper.java (including getconfig) into an "admin" package, > >> within say a class ZooKeeperAdmin, 3) document/test use of ACLs for > >> when folks do want to enable reconfig and are also worried about auth. > >> (e.g. turn on kerb) > >> > >> Again, I don't see any of this as a quality issue personally. As such > >> I don't see why any of this (1-3) should hold up a 3.5.2-alpha if we > >> were interested in doing such a release. Adjusting the API should be > >> done before we move to "beta" though. Although that seems like a > >> pretty mechanical (eclipse/idea) type refactoring? > >> > >> Patrick > >> > >> > Cheers, > >> > Alex > >> > On Mar 21, 2016 8:34 PM, "Patrick Hunt" <[email protected]> wrote: > >> > > >> >> On Thu, Mar 17, 2016 at 4:08 PM, Flavio Junqueira <[email protected]> > >> wrote: > >> >> > I gotta say that I'm not super excited about this option, but for > some > >> >> reason I ended up carrying the flag. To recap, I just raised this > option > >> >> because it seems that there are folks interested in features in 3.5 > like > >> >> SSL and not necessarily in reconfiguration. SSL is important and to > take > >> >> Kafka as an example, it sucks that we can't have a whole set up using > >> SSL. > >> >> For ZK, the real questions are: > >> >> > > >> >> > 1- how fast can we make 3.5 stable? > >> >> > 2- would it be faster if we have a way of disabling > reconfiguration? > >> >> > 3- would enough users care about a stable 3.5 that has > reconfiguration > >> >> disabled? > >> >> > > >> >> > It is taking a long time to get 3.5 to beta. There has been some > good > >> >> activity around 3.5.2 release, which is a great step, but it is > unclear > >> >> when 3.5.3 is going to come and if we will be able to make 3.5 beta > >> then. > >> >> Frankly, disabling reconfiguration sounds undesirable because it is > an > >> >> important feature, but I currently don't use it in production, so > from a > >> >> practical point of view, I can go both ways. Whether we go through > the > >> >> trouble of doing 2 depends on users interested in that option and > folks > >> >> willing to implement it. > >> >> > > >> >> > To answer your question, Powell, my pseudo-proposal is kind of a > funny > >> >> option because once the feature is stable, then we wouldn't need a > >> switch > >> >> any longer, so there is not need of a deprecation path, we just start > >> >> ignoring it from the first beta release. Until it is beta, I'd say > that > >> >> default is disabled. > >> >> > >> >> I would argue that we need this even when it does become stable. To > me > >> >> this is not a quality issue so much as it is an auth issue. We want > to > >> >> make it simple for folks to run a vanilla/old ZK cluster and not > worry > >> >> about the security implications of having reconfig enabled. > >> >> > >> >> Patrick > >> >> > >> >> > > >> >> > -Flavio > >> >> > > >> >> >> On 17 Mar 2016, at 17:44, powell molleti > <[email protected] > >> > > >> >> wrote: > >> >> >> > >> >> >> Hi Flavio, > >> >> >> > >> >> >> Generally do config options and command line args come under the > same > >> >> SLA as API?. I was assuming as such hence my question. Perhaps if the > >> >> expectation is that this config option is temporary from get go then > >> may be > >> >> it is ok. The default for re-config support will be enabled or > >> disabled?. > >> >> >> > >> >> >> I am just thinking from provisioning point of view when people > >> generate > >> >> config options etc. > >> >> >> > >> >> >> Thanks > >> >> >> Powell. > >> >> >> > >> >> >> > >> >> >> > >> >> >> On Thursday, March 17, 2016 12:12 AM, Flavio Junqueira < > >> [email protected]> > >> >> wrote: > >> >> >> Hi Powell, > >> >> >> > >> >> >> I was thinking config file and system property server side. What's > >> your > >> >> concern with compatibility? The API itself wouldn't change, but the > >> config > >> >> option wouldn't exist in previous versions and would not exist > either in > >> >> later stable versions of 3.5. > >> >> >> > >> >> >> -Flavio > >> >> >> > >> >> >> > >> >> >> > >> >> >>> On 16 Mar 2016, at 22:08, powell molleti > >> <[email protected]> > >> >> wrote: > >> >> >>> > >> >> >>> Will this option be supplied via config file/args/API?. Will this > >> >> option be a temporary thing i.e what about its compatibility?. > >> >> >>> > >> >> >>> thanks > >> >> >>> Powell. > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> On Wednesday, March 16, 2016 2:46 PM, Flavio Junqueira < > >> [email protected]> > >> >> wrote: > >> >> >>> The main issue to sort out is stability of the API. There is a > >> >> security concern around the fact that clients can freely reconfigure > the > >> >> ensemble. If we follow the plan that Pat proposed some time ago: > >> >> >>> > >> >> >>> > >> >> > >> > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > >> >> < > >> >> > >> > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/201407.mbox/%3CCANLc_9KG6-Dhm=wwfuwzniogk70pg+ihmhpigyfjdslf9-e...@mail.gmail.com%3E > >> >> > > >> >> >>> > >> >> >>> Locking the API is the main step to move it to beta. Sorting out > >> bugs > >> >> is definitely necessary, but it isn't the main thing that is keeping > >> 3.5 in > >> >> alpha. > >> >> >>> > >> >> >>> About making it experimental, I was raising the option of having > a > >> >> switch that disables the API calls, not the code. The reason why that > >> could > >> >> work is that anyone using 3.5 who uses the "experimental" API must > >> explicit > >> >> turn on the switch and enable the calls. If they do it, they need to > be > >> >> aware that the API can change. > >> >> >>> > >> >> >>> I must say that I haven't really looked closely into doing it, > and > >> I'm > >> >> not even entirely convinced that this is a good idea, but since Jason > >> >> raised the point, I'm exploring options. > >> >> >>> > >> >> >>> -Flavio > >> >> >>> > >> >> >>> > >> >> >>>> On 16 Mar 2016, at 20:59, Alexander Shraer <[email protected]> > >> wrote: > >> >> >>>> > >> >> >>>> Looking at the list of ~50 blocker and critical bugs in > ZooKeeper, > >> >> only 3-4 > >> >> >>>> are related to reconfig. Given this, and the fact that it is > run in > >> >> >>>> production since 2012 in multiple companies, I don't think its > more > >> >> >>>> unstable than any other part of ZooKeeper. > >> >> >>>> > >> >> >>>> There are multiple reconfig-related bugs that turned out really > >> >> difficult > >> >> >>>> to debug without access to the actual system or at least to the > >> Hudson > >> >> >>>> machines where some tests are failing. In the past, Michi and I > >> >> physically > >> >> >>>> went to Hortonworks to debug one such issue, but this is of > course > >> >> not a > >> >> >>>> good method, and we weren't able to arrange such a visit again. > >> >> >>>> > >> >> >>>> Regarding making it optional - the reconfig logic has several > >> >> different > >> >> >>>> parts, some would be really difficult to disable using a > >> configuration > >> >> >>>> parameter. But the actual dynamic expansion of the cluster is > >> >> triggered by > >> >> >>>> the reconfig command, so it should not affect users who don't > >> invoke > >> >> it. > >> >> >>>> > >> >> >>>> On Wed, Mar 16, 2016 at 1:09 PM, Flavio P JUNQUEIRA < > >> [email protected]> > >> >> wrote: > >> >> >>>> > >> >> >>>>> I suppose we could give it a try. How do other folks feel about > >> it? > >> >> >>>>> > >> >> >>>>> -Flavio > >> >> >>>>> On 16 Mar 2016 19:52, "Jason Rosenberg" <[email protected]> > wrote: > >> >> >>>>> > >> >> >>>>>> So, you could enable the dynamic reconfiguration feature > behind a > >> >> config > >> >> >>>>>> option, and document that it should only be enabled > >> experimentally, > >> >> use > >> >> >>>>> at > >> >> >>>>>> your own risk. Keep it off by default. Allow only static > >> config by > >> >> >>>>>> default, until it's stable? > >> >> >>>>>> > >> >> >>>>>> Jason > >> >> >>>>>> > >> >> >>>>>> On Wed, Mar 16, 2016 at 3:34 PM, Flavio Junqueira < > >> [email protected]> > >> >> >>>>> wrote: > >> >> >>>>>> > >> >> >>>>>>> Hi Jason, > >> >> >>>>>>> > >> >> >>>>>>> The consumer in Kafka is pretty independent from the core > >> >> (brokers), > >> >> >>>>>>> that's how that project manages to make such a separation. We > >> don't > >> >> >>>>> have > >> >> >>>>>>> the same with ZooKeeper as the feature we are talking about > is > >> >> part of > >> >> >>>>>> the > >> >> >>>>>>> server and the only way I see of doing what you say is to > turn > >> off > >> >> >>>>>>> features. More specifically, we'd need to disable the > reconfig > >> API > >> >> and > >> >> >>>>> do > >> >> >>>>>>> not allow any change to the configuration, even though the > code > >> is > >> >> >>>>> there. > >> >> >>>>>>> > >> >> >>>>>>> Reconfig here refers to the ability of changing the > >> configuration > >> >> of an > >> >> >>>>>>> ensemble (e.g., changing the set of servers). > >> >> >>>>>>> > >> >> >>>>>>> -Flavio > >> >> >>>>>>> > >> >> >>>>>>>> On 16 Mar 2016, at 19:14, Jason Rosenberg <[email protected] > > > >> >> wrote: > >> >> >>>>>>>> > >> >> >>>>>>>> So, it would seem sensible to me to have a release where all > >> >> features > >> >> >>>>>> are > >> >> >>>>>>>> stable, except where noted. E.g. mark certain features as > only > >> >> >>>>> 'alpha > >> >> >>>>>>>> quality', e.g. the 're-config feature'. (I assume it's > >> possible > >> >> to > >> >> >>>>>>> happily > >> >> >>>>>>>> use 3.5.X without exercising the unstable re-config bits?). > >> >> >>>>>>>> > >> >> >>>>>>>> There's precedent for doing this sort of thing in other > >> projects, > >> >> >>>>> e.g. > >> >> >>>>>> in > >> >> >>>>>>>> Kafka, they've had several release where a new "Consumer > API" > >> is > >> >> >>>>>> shipped > >> >> >>>>>>>> that is available for beta-testing, but you can still just > use > >> the > >> >> >>>>>> older > >> >> >>>>>>>> stable consumer api, etc. > >> >> >>>>>>>> > >> >> >>>>>>>> Jason > >> >> >>>>>>>> > >> >> >>>>>>>> On Wed, Mar 16, 2016 at 2:01 PM, powell molleti > >> >> >>>>>>> <[email protected] > >> >> >>>>>>>>> wrote: > >> >> >>>>>>>> > >> >> >>>>>>>>> Hi Doug, > >> >> >>>>>>>>> Is 3.5 being an alpha release preventing you from using > it?. > >> Or > >> >> have > >> >> >>>>>> you > >> >> >>>>>>>>> run into issues with it?. In general perhaps ZK 3.5 being > >> >> labeled as > >> >> >>>>>>> alpha > >> >> >>>>>>>>> might not be fair, since it is far more stable then what > most > >> >> people > >> >> >>>>>>>>> associate an alpha release to be. > >> >> >>>>>>>>> Perhaps if you do not use re-config feature may be it will > >> just > >> >> work > >> >> >>>>>> for > >> >> >>>>>>>>> you?. > >> >> >>>>>>>>> There are many examples of 3.5.X being used in productions > >> from > >> >> my > >> >> >>>>>>> limited > >> >> >>>>>>>>> knowledge. > >> >> >>>>>>>>> ThanksPowell. > >> >> >>>>>>>>> > >> >> >>>>>>>>> On Wednesday, March 16, 2016 2:44 AM, Flavio Junqueira < > >> >> >>>>>>> [email protected]> > >> >> >>>>>>>>> wrote: > >> >> >>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>>>> None of us expected the reconfig changes to take this long > to > >> >> >>>>>> stabilize. > >> >> >>>>>>>>> Until we get there, I don't think we can do anything else > with > >> >> 3.5. > >> >> >>>>>> The > >> >> >>>>>>>>> best bet we have is to work harder to bring 3.5 into a > stable > >> >> rather > >> >> >>>>>>> than > >> >> >>>>>>>>> trying to work around it. > >> >> >>>>>>>>> > >> >> >>>>>>>>> There are lots of people interested in seeing 3.5 stable, > and > >> if > >> >> we > >> >> >>>>>> get > >> >> >>>>>>>>> everyone to contribute more patches and code reviews, we > >> should > >> >> be > >> >> >>>>>> able > >> >> >>>>>>> to > >> >> >>>>>>>>> do it sooner. After all, it is a community based effort, so > >> the > >> >> >>>>>>> community > >> >> >>>>>>>>> shouldn't rely on just 2-3 people doing the work. > >> >> >>>>>>>>> > >> >> >>>>>>>>> -Flavio > >> >> >>>>>>>>> > >> >> >>>>>>>>>> On 15 Mar 2016, at 17:28, Chris Nauroth < > >> >> [email protected]> > >> >> >>>>>>>>> wrote: > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> Doug, I forgot to respond to your question about 3.4. > Since > >> >> 3.4 is > >> >> >>>>>> the > >> >> >>>>>>>>>> stable maintenance line, we are very conservative about > >> >> >>>>> back-porting > >> >> >>>>>> to > >> >> >>>>>>>>>> it. Our policy is to limit back-ports to critical bug > fixes > >> and > >> >> >>>>> not > >> >> >>>>>>>>>> introduce any new features in the 3.4 line. This is a > >> matter of > >> >> >>>>>>> managing > >> >> >>>>>>>>>> risk. > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> Jason, your question about release cadence is a fair > one. If > >> >> it's > >> >> >>>>>> any > >> >> >>>>>>>>>> consolation, we are now taking the approach of trying to > >> limit > >> >> the > >> >> >>>>>>> scope > >> >> >>>>>>>>>> of anything new introduced in 3.5 too. That would allow > us > >> to > >> >> >>>>> focus > >> >> >>>>>> on > >> >> >>>>>>>>>> stabilization: resolving blocker bugs and freezing public > >> >> APIs. I > >> >> >>>>>>> think > >> >> >>>>>>>>>> this will help us accelerate the releases into beta and > >> eventual > >> >> >>>>> GA. > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> I am new to ZooKeeper release management, so I'd like to > hear > >> >> >>>>>> thoughts > >> >> >>>>>>>>>> from more experienced committers and PMC members about > your > >> >> >>>>> proposal > >> >> >>>>>> to > >> >> >>>>>>>>>> try to cut a stable release for a limited subset of what > is > >> in > >> >> >>>>>>> branch-3.5 > >> >> >>>>>>>>>> now. My instinct is that it would be challenging to > >> cherry-pick > >> >> >>>>> out > >> >> >>>>>>>>>> pieces of branch-3.5 piecemeal at this point. This would > >> become > >> >> >>>>>>> another > >> >> >>>>>>>>>> release line for an already resource-constrained volunteer > >> >> staff to > >> >> >>>>>>>>>> manage. I'd prefer to dedicate those limited resources to > >> >> overall > >> >> >>>>>> 3.5 > >> >> >>>>>>>>>> stabilization. Also, a 3.5 release in which certain > features > >> >> >>>>>>> "vanished" > >> >> >>>>>>>>>> because of not meeting some stability criteria would be > >> >> >>>>> undesirable. > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> --Chris Nauroth > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> > >> >> >>>>>>>>>> On 3/15/16, 10:12 AM, "Jason Rosenberg" <[email protected] > > > >> >> wrote: > >> >> >>>>>>>>>> > >> >> >>>>>>>>>>> Chris, > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>> Can you say whether some parts of 3.5.X are more stable > than > >> >> >>>>> others > >> >> >>>>>>>>> (e.g. > >> >> >>>>>>>>>>> if we don't care about certain new features, is it > >> relatively > >> >> >>>>>> stable)? > >> >> >>>>>>>>>>> Would it be possible to cut out a version that only has > the > >> >> bits > >> >> >>>>> we > >> >> >>>>>>>>> think > >> >> >>>>>>>>>>> are stable (and release that)? > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>> From that timeline, and the historic release cadence, it > >> would > >> >> >>>>> seem > >> >> >>>>>> to > >> >> >>>>>>>>> be > >> >> >>>>>>>>>>> a > >> >> >>>>>>>>>>> years away before we get to the stable release? > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>> Thanks, > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>> Jason > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>> On Tue, Mar 15, 2016 at 1:06 PM, Chris Nauroth < > >> >> >>>>>>>>> [email protected]> > >> >> >>>>>>>>>>> wrote: > >> >> >>>>>>>>>>> > >> >> >>>>>>>>>>>> Hello Doug, > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> Thanks for your interest in the SSL feature! > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> At this point, I think we're still pretty far away from > >> >> >>>>> declaring a > >> >> >>>>>>>>>>>> stable > >> >> >>>>>>>>>>>> release in the 3.5 line. I don't think we're close > enough > >> >> that > >> >> >>>>>>> anyone > >> >> >>>>>>>>>>>> can > >> >> >>>>>>>>>>>> offer a reliable ETA. This is an earlier thread that > >> >> describes > >> >> >>>>> the > >> >> >>>>>>>>>>>> high-level strategy for release planning in the 3.5 > line: > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> https://s.apache.org/ADK1 > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> The next step is a 3.5.2-alpha release. We're working > on > >> >> >>>>>> resolving a > >> >> >>>>>>>>>>>> few > >> >> >>>>>>>>>>>> more blockers before we produce a release candidate. > >> >> Hopefully > >> >> >>>>>> that > >> >> >>>>>>>>>>>> will > >> >> >>>>>>>>>>>> get done in the next few weeks. > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> --Chris Nauroth > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> On 3/15/16, 9:39 AM, "Doug" <[email protected]> > wrote: > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>>> I know it's only been a few months, but I was > wondering if > >> >> there > >> >> >>>>>>> was a > >> >> >>>>>>>>>>>>> ballpark release date for a stable version of 3.5.1. > Or is > >> >> there > >> >> >>>>>> any > >> >> >>>>>>>>>>>>> chance > >> >> >>>>>>>>>>>>> the SSL feature would be added to 3.4.8? Just another > >> person > >> >> >>>>>> looking > >> >> >>>>>>>>> to > >> >> >>>>>>>>>>>>> have > >> >> >>>>>>>>>>>>> that feature in a stable version. Thanks for all you > do! > >> :) > >> >> >>>>>>>>>>>>> > >> >> >>>>>>>>>>>>> > >> >> >>>>>>>>>>>>> > >> >> >>>>>>>>>>>>> -- > >> >> >>>>>>>>>>>>> View this message in context: > >> >> >>>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>> > >> >> >>>>>> > >> >> >>>>> > >> >> > >> > http://zookeeper-user.578899.n2.nabble.com/Zookeeper-with-SSL-release-dat > >> >> >>>>>>>>>>>> e > >> >> >>>>>>>>>>>>> -tp7581744p7582136.html > >> >> >>>>>>>>>>>>> Sent from the zookeeper-user mailing list archive at > >> >> Nabble.com. > >> >> >>>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>>>> > >> >> >>>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>> > >> >> >>>>> > >> >> > > >> >> > >> >
