Regarding ZK deployment automation, I am building an automation that is very similar to the requirement here - the motivation is to save me sometime on validation / regression test of ZOOKEEPER-1045 and I could see same automation being used to validate a release as well. It might not be ready for this release given limited time I have but I'll share it once it's done. It's based on Ansible [1] and what it could do: - Provision a cluster on AWS with specified topology given subscription credential. Support on provisioning on GCP or Azure should be possible as well. This step is optional, as the cluster can be provisioned separately, or using different approach (e.g. a docker cluster instead of VMs), and if the cluster is already provisioned, the list of IPs will be passed to the tool. - Install a pre-configured ZK cluster on all nodes of the cluster, including all dependencies. - Support easy update / change state of the cluster both at cluster level (e.g. kill nodes) and at ZK level (e.g. update zoo.cfg and restart ZK).
We do have some tools internally to provision and deploy ZK cluster but those tools have dependencies that can't be made public and also they tied to specific version of ZK and is not exactly what I want (for 1045 validation at least). [1] http://docs.ansible.com/ansible/index.html On Fri, Jul 8, 2016 at 6:58 AM, Flavio Junqueira <f...@apache.org> wrote: > We could consider using vagrant and something like ducktape (for Kafka): > > > https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafka+system+tests+with+ducktape > < > https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafka+system+tests+with+ducktape > > > > As for resources, ASF committers can have an msn subscription and we could > use some Azure resources for tests. I have actually recently shared one > Azure VM to show Michael a test failure in that setting. The subscription > doesn't give access to a lot of resources as the monthly credit is small, > but should be good enough for some distributed tests: > > > http://mail-archives.apache.org/mod_mbox/www-community/201402.mbox/%3ccab56zcwvv5wmh99xaruldi2fvxmuj6r1axe66ul-afpo3cv...@mail.gmail.com%3E > < > http://mail-archives.apache.org/mod_mbox/www-community/201402.mbox/%3ccab56zcwvv5wmh99xaruldi2fvxmuj6r1axe66ul-afpo3cv...@mail.gmail.com%3E > > > > -Flavio > > > On 08 Jul 2016, at 14:41, Camille Fournier <cami...@apache.org> wrote: > > > > These are all good questions. > > > > My ultimate hope here is that we can actually get something set up so > that > > for each release we want to do, we can easily run a standard smoke test > on > > this cluster, and potentially leave the cluster available for a few days > > for additional testing by the community, to help those of us who do not > > have an easy way to spin up a ZK cluster at our companies for testing to > > feel good about vetting a release. > > > > I think that a good first step would be to create something that can > > actually deploy a configured ZK on top of this cluster, and then deploy > > some sort of basic test script on machines that executes, eg, pat's smoke > > test as Flavio pointed out. I will admit to being a bit perplexed as to > > what the state of the nodes of this cluster look like when you get access > > to them and what needs to be installed on top of them to actually do this > > sort of automation. It looks like you can provision the nodes with an OS > on > > them, but beyond that it looks like we may need to create the automation > to > > install java first, then ZK. There's some very bare bones details here: > > > > https://github.com/cncf/cluster > > > > *For those of you who have internal processes for vetting ZK, if any of > you > > have automation for spinning up new ZK servers in a cloud env hopefully > we > > can use that to start. Does anyone have that? *I think that's where we > need > > to begin. We can then move on to what a good smoke test might look like. > > > > Thanks, > > C > > > > > > On Thu, Jul 7, 2016 at 4:48 PM, Michael Han <h...@cloudera.com> wrote: > > > >> I'll help validating the release on our internal cluster using zk-smoke > and > >> manual testing. Do we have a standard protocol on what should be > validated > >> and how? Also do we perform integration tests (e.g. with HBase / Kafka) > as > >> part of release validation? > >> > >> > >> On Thu, Jul 7, 2016 at 12:25 PM, Flavio P JUNQUEIRA <f...@apache.org> > >> wrote: > >> > >>> And thanks for doing this, Camile, this is great. > >>> > >>> -Flavio > >>> On 7 Jul 2016 8:25 p.m., "Flavio P JUNQUEIRA" <f...@apache.org> wrote: > >>> > >>>> Perhaps start from phunt's smoke test? > >>>> > >>>> https://github.com/phunt/zk-smoketest > >>>> > >>>> -Flavio > >>>> On 7 Jul 2016 7:04 p.m., "Camille Fournier" <cami...@apache.org> > >> wrote: > >>>> > >>>>> So I'm working with the CNCF to see about getting cluster access to > >> spin > >>>>> up > >>>>> ZK clusters for release validation. I was wondering if anyone has > >>> scripts > >>>>> for deploying and configuring ZK clusters quickly for stress testing > >>> that > >>>>> we could use if we get this access? I'm sure some of you in the > >>> community > >>>>> must do some of this internally. > >>>>> > >>>>> I would also very much appreciate any volunteers to contribute to > >> making > >>>>> better public release validation work, if possible, since it's > unclear > >>> how > >>>>> much time I personally will be able to dedicate to this effort beyond > >>>>> getting access to the cluster. > >>>>> > >>>>> Thanks, > >>>>> C > >>>>> > >>>> > >>> > >> > >> > >> > >> -- > >> Cheers > >> Michael. > >> > > -- Cheers Michael.
