Re: [ANNOUNCE] new ZooKeeper PMC member: Mate Szalay-Beko

[Norbert Kalmar]

Congratulations Máté! A great addition to the ZooKeeper PMC ;)

- Norbert

On Mon, Mar 28, 2022 at 5:09 PM Szalay-Bekő Máté 
wrote:

> Thank you all! :)
>
> On Mon, Mar 28, 2022 at 3:38 PM Jordan Zimmerman <
> jor...@jordanzimmerman.com>
> wrote:
>
> > Congrats!!!
> >
> > > On Mar 28, 2022, at 7:42 AM, Enrico Olivelli 
> > wrote:
> > >
> > > I am happy to announce that Mate Szalay-Beko has been invited to join
> > > the Apache ZooKeeper PMC and he accepted.
> > >
> > > Mate is doing great work for our community.
> > >
> > > Please join me in congratulating with him
> > >
> > > Congrats Mate !
> > >
> > >
> > > If you want to know more about the ASF works and what is a PMC you can
> > > read more here
> > > https://www.apache.org/foundation/how-it-works.html#pmc
> > >
> > > Enrico
> >
> >
>

Re: Need help on setting up the zookeeper on my local

[Norbert Kalmar]

I don't see the FileChangeWatcherTest.testCallbackWorksOnFileDeleted on
latest flaky build:
https://ci-hadoop.apache.org/blue/organizations/jenkins/ZooKeeper-Find-Flaky-Tests/detail/ZooKeeper-Find-Flaky-Tests/339/pipeline
It was actually flaky, but should have been already fixed: ZOOKEEPER-3219

However, the other ones, on branch 3.5, looks flaky (so probably also are
on master):
ZooKeeperServerMainTest.testReadOnlySnapshotDir
ZooKeeperServerMainTest.testReadOnlyTxnLogDir

You can try to re-run the failed tests individually:
mvn -Dtest=FileChangeWatcherTest -DfailIfNoTests=false test

If they pass you can try to re-run the whole tests again. We do have some
flaky tests, and fixes are always welcomed. Please check first if there's
already a jira ticket for it:
https://issues.apache.org/jira/browse/ZOOKEEPER-3042?jql=text%20~%20%22flaky%22

>From the logs it looks like it's an issue of the multi thread test runs,
sometimes ZK instances get stuck (it's a valid question why we have ZK
instances fired up in unit tests).

- Norbert

On Mon, Aug 2, 2021 at 11:20 AM dinesh singla 
wrote:

> Hi Norbert,
>
> I am currently at master branch and when i run maven clean and install here
> I am getting an error on a test case and the error is
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *[ERROR] Failures: [ERROR]
> FileChangeWatcherTest.testCallbackWorksOnFileDeleted:210 expected:
>  but was: [ERROR]
> NettyServerCnxnFactoryTest.testOutstandingHandshakeLimit:155 Only 0 out of
> 9 connections created![ERROR]
> ZooKeeperServerMainTest.testReadOnlySnapshotDir:254 waiting for server
> being up ==> expected:  but was: [ERROR]
> ZooKeeperServerMainTest.testReadOnlyTxnLogDir:293 waiting for server being
> up ==> expected:  but was: [ERROR]
>
> LearnerMetricsTest.testLearnerMetricsTest:101->waitForMetric:118->ZKTestCase.waitFor:110
> unable to match on metric: learner_proposal_received_count[ERROR]
> QuorumPeerMainTest.testFailedTxnAsPartOfQuorumLoss:792 create /zk2 should
> have failed[ERROR]   QuorumPeerMainTest.testLeaderOutOfView:916 expected:
>  but was: [ERROR]
> QuorumSSLTest.testCertificateRevocationList:758 expected:  but was:
> [ERROR]
>
> QuorumSSLTest.testHostnameVerificationWithInvalidIpAddressAndValidHostname:657->testHostnameVerification:722
> expected:  but was: [ERROR]   QuorumSSLTest.testOCSP:829
> expected:  but was: [ERROR]
> ReconfigBackupTest.testVersionOfDynamicFilename:319 expected:  but
> was: [ERROR]   StatResetCommandTest.testStatResetWithFollower:82
> expected:  but was: 
>
>
> Can you please tell me if I am doing something wrong here or if this needs
> to be fixed? If this needs to be fixed, I would like to try. Thanks for the
> help.
>
> Regards,
> Dinesh Singla
>
> On Mon, Aug 2, 2021 at 1:49 PM Norbert Kalmar  wrote:
>
> > Hi Dinesh,
> >
> > Welcome to the ZooKeeper community, we are always happy to see interest
> in
> > the project.
> > Please attach logs so we can help pinpointing the problem.
> >
> > You can find a lot of useful howto posts on
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/
> > Mainly:
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
> >
> > As for the mentor programme, do you mean this one:
> > https://community.apache.org/mentoringprogramme.html
> > That is more of a general programme geared toward apache contributions in
> > general, and there's a form to apply.
> >
> > We are happy to help if you have questions, I'm not sure there will be an
> > assigned mentor though.
> >
> > Regards,
> > Norbert
> >
> > On Mon, Aug 2, 2021 at 10:07 AM dinesh singla 
> > wrote:
> >
> > > Hi Dev team,
> > >
> > > I am new to open source and i want to contribute to apache zookeeper. I
> > > need some help as some test cases are failing on my local. Can someone
> > help
> > > me here?
> > > I also know about the mentorship programme of Apache. Please consider
> me
> > > for a mentorship programme. Thanks in advance.
> > >
> > > Regards,
> > > Dinesh
> > >
> >
>

Re: Need help on setting up the zookeeper on my local

[Norbert Kalmar]

Hi Dinesh,

Welcome to the ZooKeeper community, we are always happy to see interest in
the project.
Please attach logs so we can help pinpointing the problem.

You can find a lot of useful howto posts on
https://cwiki.apache.org/confluence/display/ZOOKEEPER/
Mainly:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute

As for the mentor programme, do you mean this one:
https://community.apache.org/mentoringprogramme.html
That is more of a general programme geared toward apache contributions in
general, and there's a form to apply.

We are happy to help if you have questions, I'm not sure there will be an
assigned mentor though.

Regards,
Norbert

On Mon, Aug 2, 2021 at 10:07 AM dinesh singla 
wrote:

> Hi Dev team,
>
> I am new to open source and i want to contribute to apache zookeeper. I
> need some help as some test cases are failing on my local. Can someone help
> me here?
> I also know about the mentorship programme of Apache. Please consider me
> for a mentorship programme. Thanks in advance.
>
> Regards,
> Dinesh
>

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 2

[Norbert Kalmar]

+1 (non-binding)

Checked on macOS: licenses, build and test passed, ran ZK and few standard
commands, signature.
Also built and run unit tests on ubuntu.

Thanks Arshad!

- Norbert

On Sat, Apr 10, 2021 at 10:34 AM Damien Diederen 
wrote:

>
> Hi Arshad, all,
>
> LGTM!  +1 (advisory):
>
>   * Tarball contents match repository tag;
>
>   * Verified checksums and signatures;
>
>   * Built and smoke-tested on NixOS with a slightly adapted version of
> the Nix recipe and test case;
>
>   * Smoke-tested a standalone server with the (corresponding) Java, C
> and Perl clients, as well as the zkfuse contrib;
>
>   * Smoke-tested a 3-ensemble with the (corresponding) Java client and
> SASL/GSSAPI.
>
> Sorry if I made you feel you had to go around with RC 0!  Oh well, at
> least we got some additional CVE fixes bundled in.
>
> Best, -D
>
> P.-S. — As with my previous review, I have *not* tested under Ubuntu nor
>   used the staging repo.  These are important points, but I figured most
>   other testers would focus on them.
>
>
>
> Mohammad Arshad  writes:
> > This is a bug fix release candidate for 3.6.3. It fixes 52 issues,
> > including multiple CVE fixes.
> >
> > The full release notes is available at:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348703
> >
> >  Please download, test and vote by Sunday, April 11th 2021, 23:59
> > UTC+0. 
> >
> > Source and binary files:
> > https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-2/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1071
> >
> > The release candidate tag in git to be voted upon: release-3.6.3-2
> > https://github.com/apache/zookeeper/tree/release-3.6.3-2
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-2/website/
> >
> > Should we release this candidate?
> >
> > Thanks & Regards
> > Arshad
>

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 1

[Norbert Kalmar]

dependency check is green now with
https://github.com/apache/zookeeper/pull/1675
We do backport security fixes to 3.5 branch, correct? I will create a
separate PR for that due to ant support.

- Norbert

On Wed, Apr 7, 2021 at 2:19 PM Norbert Kalmar  wrote:

> Please don't forget to update the license files also in zookeeper-server
> resources folder!
> But better yet I can create the jira and have a PR up soon.
>
> - Norbert
>
> On Wed, Apr 7, 2021 at 1:50 PM Andor Molnar  wrote:
>
>> Good catch Mate!
>>
>> Jetty has to be upgraded.
>>
>> Andor
>>
>>
>>
>>
>> > On 2021. Apr 7., at 13:43, Szalay-Bekő Máté 
>> wrote:
>> >
>> > -1 (non-binding)
>> >
>> > Hello Mohammad!
>> >
>> > Thanks for the great work! Sorry for torpedoing it :(
>> >
>> > I voted with -1, as the CVE check failed for me on the release
>> candidate:
>> >
>> > mvn clean package -DskipTests dependency-check:check
>> > (...)
>> > [ERROR] Failed to execute goal
>> org.owasp:dependency-check-maven:5.3.0:check
>> > (default-cli) on project zookeeper:
>> > [ERROR]
>> > [ERROR] One or more dependencies were identified with vulnerabilities
>> that
>> > have a CVSS score greater than or equal to '0.0':
>> > [ERROR]
>> > [ERROR] jetty-server-9.4.38.v20210224.jar: CVE-2021-28165
>> > [ERROR] jetty-http-9.4.38.v20210224.jar: CVE-2021-28165
>> > [ERROR]
>> > [ERROR] See the dependency-check report for more details.
>> >
>> >
>> > It seems we have a relatively recent (about three weeks old) CVE error
>> in
>> > Jetty: https://nvd.nist.gov/vuln/detail/CVE-2021-28165
>> > " In Eclipse Jetty 7.2.2 to 9.4.38, 10.0.0.alpha0 to 10.0.1, and
>> > 11.0.0.alpha0 to 11.0.1, CPU usage can reach 100% upon receiving a large
>> > invalid TLS frame."
>> >
>> > Looks like we will have to upgrade to jetty-server-9.4.39.
>> >
>> > Kind regards,
>> > Mate
>> >
>> > On Tue, Apr 6, 2021 at 10:17 AM Mohammad arshad <
>> mohammad.ars...@huawei.com>
>> > wrote:
>> >
>> >> +1 (non-binding)
>> >>
>> >> -Verified signature and checksum of release artifacts. all ok
>> >> -Run Junit test cases with jdk1.8.0_232 on Ubuntu 20.04, total 3137
>> test
>> >> cases, 3 skipped, rest all passed
>> >> -Done basic quality checks. run rat, checkstyle, spotbugs
>> >> -Built tarball from source code, Verified it is same as the downloaded
>> >> tarball
>> >> -Installed 3 node cluster and verified basic functionalities from API,
>> >> executed few cli commands. No issues observed
>> >> -Connected HBase, HDFS and Yarn clusters (all using zk 3.5.6) to
>> ZooKeeper
>> >> 3.6.3 cluster, no issues observed.
>> >>
>> >> Though as a release manager my +1 vote is implicit, voting again to
>> share
>> >> few commands  I used to verify the release.
>> >>
>> >> Here are some of the commands I executed while verifying the release.
>> >>
>> >> Download all the required artifacts
>> >> 
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.asc
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.sha512
>> >>
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.asc
>> >> wget
>> >>
>> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.sha512
>> >>
>> >> wget https://www.apache.org/dist/zookeeper/KEYS
>> >>
>> >> Verify Signature
>> >> 
>> >> gpg --import KEYS
>> >> gpg --verify apache-zookeeper-3.6.3-bin.tar.gz.asc
>> >> apache-zookeeper-3.6.3-bin.tar.gz
>> >> gpg --verify apache-zookeeper-3.6.3.tar.gz.asc
>> >> apache-zookeeper-3.6.3.ta

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 1

[Norbert Kalmar]

Please don't forget to update the license files also in zookeeper-server
resources folder!
But better yet I can create the jira and have a PR up soon.

- Norbert

On Wed, Apr 7, 2021 at 1:50 PM Andor Molnar  wrote:

> Good catch Mate!
>
> Jetty has to be upgraded.
>
> Andor
>
>
>
>
> > On 2021. Apr 7., at 13:43, Szalay-Bekő Máté 
> wrote:
> >
> > -1 (non-binding)
> >
> > Hello Mohammad!
> >
> > Thanks for the great work! Sorry for torpedoing it :(
> >
> > I voted with -1, as the CVE check failed for me on the release candidate:
> >
> > mvn clean package -DskipTests dependency-check:check
> > (...)
> > [ERROR] Failed to execute goal
> org.owasp:dependency-check-maven:5.3.0:check
> > (default-cli) on project zookeeper:
> > [ERROR]
> > [ERROR] One or more dependencies were identified with vulnerabilities
> that
> > have a CVSS score greater than or equal to '0.0':
> > [ERROR]
> > [ERROR] jetty-server-9.4.38.v20210224.jar: CVE-2021-28165
> > [ERROR] jetty-http-9.4.38.v20210224.jar: CVE-2021-28165
> > [ERROR]
> > [ERROR] See the dependency-check report for more details.
> >
> >
> > It seems we have a relatively recent (about three weeks old) CVE error in
> > Jetty: https://nvd.nist.gov/vuln/detail/CVE-2021-28165
> > " In Eclipse Jetty 7.2.2 to 9.4.38, 10.0.0.alpha0 to 10.0.1, and
> > 11.0.0.alpha0 to 11.0.1, CPU usage can reach 100% upon receiving a large
> > invalid TLS frame."
> >
> > Looks like we will have to upgrade to jetty-server-9.4.39.
> >
> > Kind regards,
> > Mate
> >
> > On Tue, Apr 6, 2021 at 10:17 AM Mohammad arshad <
> mohammad.ars...@huawei.com>
> > wrote:
> >
> >> +1 (non-binding)
> >>
> >> -Verified signature and checksum of release artifacts. all ok
> >> -Run Junit test cases with jdk1.8.0_232 on Ubuntu 20.04, total 3137 test
> >> cases, 3 skipped, rest all passed
> >> -Done basic quality checks. run rat, checkstyle, spotbugs
> >> -Built tarball from source code, Verified it is same as the downloaded
> >> tarball
> >> -Installed 3 node cluster and verified basic functionalities from API,
> >> executed few cli commands. No issues observed
> >> -Connected HBase, HDFS and Yarn clusters (all using zk 3.5.6) to
> ZooKeeper
> >> 3.6.3 cluster, no issues observed.
> >>
> >> Though as a release manager my +1 vote is implicit, voting again to
> share
> >> few commands  I used to verify the release.
> >>
> >> Here are some of the commands I executed while verifying the release.
> >>
> >> Download all the required artifacts
> >> 
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.asc
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.sha512
> >>
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.asc
> >> wget
> >>
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.sha512
> >>
> >> wget https://www.apache.org/dist/zookeeper/KEYS
> >>
> >> Verify Signature
> >> 
> >> gpg --import KEYS
> >> gpg --verify apache-zookeeper-3.6.3-bin.tar.gz.asc
> >> apache-zookeeper-3.6.3-bin.tar.gz
> >> gpg --verify apache-zookeeper-3.6.3.tar.gz.asc
> >> apache-zookeeper-3.6.3.tar.gz
> >> gpg --fingerprint 68E327C1
> >>
> >> Verify Checksum
> >> 
> >> sha512sum --check apache-zookeeper-3.6.3-bin.tar.gz.sha512
> >> sha512sum --check apache-zookeeper-3.6.3-bin.tar.gz.sha512
> >>
> >>
> >> Verify license header by executing Apache RAT
> >> 
> >> tar -xvf apache-zookeeper-3.6.3.tar.gz
> >> cd apache-zookeeper-3.6.3
> >> mvn clean apache-rat:check -DskipTests
> >>
> >> Perform quality checks, run checkstyle, spotbugs and unit tests
> >> 
> >> mvn clean install checkstyle:check spotbugs:check -DskipTests
> >> mvn clean test -Dsurefire.rerunFailingTestsCount=2
> >> -DtestFailureIgnore=true -Dmaven.test.failure.ignore=true
> >> -Dmaven.test.error.ignore=true
> >> NOTE: use -Pfull-build to include ci tests as well
> >>
> >> Build and Cluster Install
> >> 
> >> Built the tarball from source code and compare that it is same as the
> >> downloaded tarball. Apart from timestamp changes, no other changes are
> >> observed
> >> mvn clean install -DskipTests
> >> Installed the downloaded bin tarball and do some feature sanity tests
> >>
> >> Thanks & Regards
> >> Arshad
> >>
> >> -Original Message-
> >> From: Mohammad Arshad [mailto:ars...@apache.org]
> >> Sent: 

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

[Norbert Kalmar]

+1 (non-binding)

- Built source (including running tests)
- Run and tested with basic commands from built source and also from bin
tarball
- Checked files included in tarballs

Minor side note: I had 2 failed tests on MacOS (ReadOnlyModeTest
and RequestPathMetricsCollectorTest), after re-running them they passed.

Thanks Damien, great work with the release!

- Norbert



On Fri, Mar 19, 2021 at 12:06 PM Szalay-Bekő Máté <
szalay.beko.m...@gmail.com> wrote:

> +1 (non-binding)
>
> Congrats Damien, nice job!
> And also the tests got definitely much more stable!
> (I had a single test that failed on my dockerized
> linux: RequestPathMetricsCollectorTest - but this one was successfully
> executed on my mac host, so possible it is only related to my docker
> environment)
>
> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
> 11.0.10 and maven 3.6.0.
> - all the unit tests passed (both Java and C-client).
> - I also built and executed unit tests for zkpython
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - fatjar built
> - I also executed some quick rolling-upgrade tests from 3.5.9 to 3.7.0 and
> also from 3.6.2 to 3.7.0 (using
> https://github.com/symat/zk-rolling-upgrade-test)
>
> One minor website related issue: the tab on the top of the website page (
>
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/index.html
> )
> still says "ZooKeeper 3.6 Documentation"
>
> Kind regards,
> Mate
>
> On Wed, Mar 17, 2021 at 12:06 PM Damien Diederen 
> wrote:
>
> >
> > Greetings, all!
> >
> > After a long delay, here is a third release candidate for ZooKeeper
> 3.7.0.
> >
> > Compared to RC1, it contains... quite a few changes.  It notably fixes
> > the quota feature for multi transactions, repairs the test suite on
> > macOS (Catalina), makes a few tests less flaky, and avoids a CVE.
> >
> > The complete set of changes can be obtained with the Git range
> > expression 'release-3.7.0-1..release-3.7.0-2', or on GitHub at:
> >
> >
> >
> https://github.com/apache/zookeeper/compare/release-3.7.0-1...release-3.7.0-2
> >
> > I cannot say that I find the state of the test suite satisfactory, but
> > the failures which are often observed are due to timing and/or TCP/IP
> > port assignment issues, and repeated runs are "sufficient" to clear
> > them.
> >
> > I was hoping to contribute more on that front, but have been unable so
> > far, and don't want to keep the 3.7 branch hostage—so here is a timid
> > RC2.
> >
> >
> > ZooKeeper 3.7.0 introduces a number of new features, notably:
> >
> >   * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874);
> >
> >   * Quota enforcement (ZOOKEEPER-3301);
> >
> >   * Host name canonicalization in quorum SASL authentication
> > (ZOOKEEPER-4030);
> >
> >   * Support for BCFKS key/trust store format (ZOOKEEPER-3950);
> >
> >   * A choice of mandatory authentication scheme(s) (ZOOKEEPER-3561);
> >
> >   * A "whoami" API and CLI command (ZOOKEEPER-3969);
> >
> >   * The possibility of disabling digest authentication (ZOOKEEPER-3979);
> >
> >   * Multiple SASL "superUsers" (ZOOKEEPER-3959);
> >
> >   * Fast-tracking of throttled requests (ZOOKEEPER-3683);
> >
> >   * Additional security metrics (ZOOKEEPER-3978);
> >
> >   * SASL support in the C and Perl clients (ZOOKEEPER-1112,
> > ZOOKEEPER-3714);
> >
> >   * A new zkSnapshotComparer.sh tool (ZOOKEEPER-3427);
> >
> >   * Notes on how to benchmark ZooKeeper with the YCSB tool
> > (ZOOKEEPER-3264).
> >
> >
> > The release notes are available here:
> >
> >
> >
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/releasenotes.html
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346617
> >
> > *** Please download, test and vote by March 21st 2021, 23:59 UTC+0. ***
> >
> > Source files:
> >
> >   https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/
> >
> > Maven staging repo:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1067/
> >
> > The release candidate tag in git to be voted upon: release-3.7.0-2
> >
> >   https://github.com/apache/zookeeper/tree/release-3.7.0-2
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> >
> >   https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> >
> >
> >
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/
> >
> >
> > Should we release this candidate?
> >
> >
> > Damien Diederen
> >
>

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1

[Norbert Kalmar]

Hi all,

My problem isn't that it's flaky (although that is a problem, a long
standing one), but that it looks like it fails consistently on macOS, with
the same error, on 2 different environments. (Anyone had success running
tests on macOS? That would be a push for me to also +1 it)
I'm not strongly -1 because of this, as tests run fine on linux systems,
and it's our primary platform. Also, it builds and runs fine. So this is
something we can perhaps fix for the upcoming 3.7.1, and just roll out
3.7.0 as is.

Sorry for the many emails, here is a summary so far:

Patrick Hunt +1 (binding)
Máté Szalay-Bekő +0 (non-binding)
Norbert Kalmar -1 (non-binding, and tentative)

Thanks for putting together the release Damien, everything looks flawless
on that part :)

- Norbert

On Tue, Jan 26, 2021 at 10:17 PM Damien Diederen 
wrote:

>
> Greetings, all,
>
> I have seen your reports, and am planning to dig into them ASAP.
>
> As Máté remarked, I have found tests with -Dsurefire-forkcount > 1 to be
> quite unreliable as of late.  This is something I had been planning to
> look into—but perhaps not at the same time as preparing this release :)
>
> FWIW, branch-3.7.0 "passed" on Jenkins:
>
>
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/
>
> That was just luck, however; one can see that branch-3.7 has failed
> despite holding the same contents.
>
> Best, -D
>
>
>
> Norbert Kalmar  writes:
> > Sorry, the linked test is just an assertion error, no exception there.
> > I had another test failure which had an error thrown. I will investigate
> > further, as this might be some macOD address issue/collission yet again:
> >
> > [ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 26.229 s <<< FAILURE! - in
> org.apache.zookeeper.server.quorum.CnxManagerTest
> > [ERROR] testCnxManagerListenerThreadConfigurableRetry  Time elapsed:
> 0.178
> > s  <<< ERROR!
> > java.net.BindException: Address already in use
> > at sun.nio.ch.Net.bind0(Native Method)
> > at
> >
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:662)
> > at
> >
> org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:109)
> > at
> >
> org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:191)
> > at
> >
> org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:186)
> > at
> >
> org.apache.zookeeper.server.quorum.QuorumPeer.(QuorumPeer.java:1253)
> > at
> >
> org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxManagerListenerThreadConfigurableRetry(CnxManagerTest.java:309)
> >
> >
> >
> > On Mon, Jan 25, 2021 at 5:20 PM Norbert Kalmar 
> wrote:
> >
> >> -1 (non-binding!)
> >>
> >> - built the source, run locally, gave a few basic commands
> >> - run tests on macOS 10.13.6, openjdk 1.8.0_242 - issues found, see
> below
> >> - checked files in tarball
> >> - checked signatures
> >>
> >> I had the same problem as Máté. unit tests did not pass on MacOS.
> Nothing
> >> has changed in my environment since I did the 3.5.9 release, and it had
> no
> >> issue back then.
> >>
> >> I gave a -1 because this error doesn't only occur to me.
> >>
> >> The test that failed:
> >>
> >> [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 1, Time elapsed:
> >> 62.332 s <<< FAILURE! - in
> >> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest
> >>
> >> [ERROR] testMultiThreadPerf  Time elapsed: 4.816 s  <<< FAILURE!
> >>
> >> org.opentest4j.AssertionFailedError: expected:  but was: 
> >>
> >> at
> >>
> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest.testMultiThreadPerf(RequestPathMetricsCollectorTest.java:448)
> >>
> >>
> >>
> >> - Norbert
> >>
> >>
> >>
> >> On Mon, Jan 25, 2021 at 1:39 PM Szalay-Bekő Máté <
> >> szalay.beko.m...@gmail.com> wrote:
> >>
> >>> +0 (and not even binding :) )
> >>>
> >>> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using
> OpenJDK
> >>> 8u265 and maven 3.6.3.
> >>> - I also built and executed unit tests for zkpython
> >>> - the unit tests passed for the C-client and for python client
> >>> - checkstyle and spotbugs passed
> >>> - apache-rat passed
> >>> - owasp (CVE check) passed
> >>> -

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1

[Norbert Kalmar]

Please disregard my last email, I had ZooKeeper running in the background,
killing it solved that test, the fork issue is not caused by that.

On Mon, Jan 25, 2021 at 5:26 PM Norbert Kalmar  wrote:

> Sorry, the linked test is just an assertion error, no exception there.
> I had another test failure which had an error thrown. I will investigate
> further, as this might be some macOD address issue/collission yet again:
>
> [ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 26.229 s <<< FAILURE! - in org.apache.zookeeper.server.quorum.CnxManagerTest
> [ERROR] testCnxManagerListenerThreadConfigurableRetry  Time elapsed: 0.178
> s  <<< ERROR!
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:662)
> at
> org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:109)
> at
> org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:191)
> at
> org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:186)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.(QuorumPeer.java:1253)
> at
> org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxManagerListenerThreadConfigurableRetry(CnxManagerTest.java:309)
>
>
>
> On Mon, Jan 25, 2021 at 5:20 PM Norbert Kalmar 
> wrote:
>
>> -1 (non-binding!)
>>
>> - built the source, run locally, gave a few basic commands
>> - run tests on macOS 10.13.6, openjdk 1.8.0_242 - issues found, see below
>> - checked files in tarball
>> - checked signatures
>>
>> I had the same problem as Máté. unit tests did not pass on MacOS. Nothing
>> has changed in my environment since I did the 3.5.9 release, and it had no
>> issue back then.
>>
>> I gave a -1 because this error doesn't only occur to me.
>>
>> The test that failed:
>>
>> [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 1, Time elapsed:
>> 62.332 s <<< FAILURE! - in
>> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest
>>
>> [ERROR] testMultiThreadPerf  Time elapsed: 4.816 s  <<< FAILURE!
>>
>> org.opentest4j.AssertionFailedError: expected:  but was: 
>>
>> at
>> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest.testMultiThreadPerf(RequestPathMetricsCollectorTest.java:448)
>>
>>
>>
>> - Norbert
>>
>>
>>
>> On Mon, Jan 25, 2021 at 1:39 PM Szalay-Bekő Máté <
>> szalay.beko.m...@gmail.com> wrote:
>>
>>> +0 (and not even binding :) )
>>>
>>> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
>>> 8u265 and maven 3.6.3.
>>> - I also built and executed unit tests for zkpython
>>> - the unit tests passed for the C-client and for python client
>>> - checkstyle and spotbugs passed
>>> - apache-rat passed
>>> - owasp (CVE check) passed
>>> - fatjar built (-Pfatjar)
>>> - I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2.
>>> (using
>>> https://github.com/symat/zk-rolling-upgrade-test)
>>>
>>> for some reason the java unit tests failed for me.
>>>
>>> On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests executed
>>> successfully, but then the maven job still failed for hbase-server test
>>> with error message (with -DforkCount=4 and even with -DforkCount=1) like:
>>> -
>>> [ERROR] ExecutionException There was an error in the forked process
>>> [ERROR] unable to create new native thread
>>> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
>>> ExecutionException There was an error in the forked process
>>> [ERROR] unable to create new native thread
>>> [ERROR] at
>>>
>>> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
>>> -
>>>
>>>
>>> Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 and
>>> maven 3.6.3) and I got other kinds of strange maven errors:
>>> ---
>>> [ERROR] Caused by:
>>> org.apache.maven.surefire.booter.SurefireBooterForkException: The forked
>>> VM
>>> terminated without properly saying goodbye. VM crash or System.exit
>>> called?
>>> [ERROR] Command was /bin/sh -c cd
>>> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server &&
>>> /home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xm

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1

[Norbert Kalmar]

Sorry, the linked test is just an assertion error, no exception there.
I had another test failure which had an error thrown. I will investigate
further, as this might be some macOD address issue/collission yet again:

[ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
26.229 s <<< FAILURE! - in org.apache.zookeeper.server.quorum.CnxManagerTest
[ERROR] testCnxManagerListenerThreadConfigurableRetry  Time elapsed: 0.178
s  <<< ERROR!
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:662)
at
org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:109)
at
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:191)
at
org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:186)
at
org.apache.zookeeper.server.quorum.QuorumPeer.(QuorumPeer.java:1253)
at
org.apache.zookeeper.server.quorum.CnxManagerTest.testCnxManagerListenerThreadConfigurableRetry(CnxManagerTest.java:309)



On Mon, Jan 25, 2021 at 5:20 PM Norbert Kalmar  wrote:

> -1 (non-binding!)
>
> - built the source, run locally, gave a few basic commands
> - run tests on macOS 10.13.6, openjdk 1.8.0_242 - issues found, see below
> - checked files in tarball
> - checked signatures
>
> I had the same problem as Máté. unit tests did not pass on MacOS. Nothing
> has changed in my environment since I did the 3.5.9 release, and it had no
> issue back then.
>
> I gave a -1 because this error doesn't only occur to me.
>
> The test that failed:
>
> [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 1, Time elapsed:
> 62.332 s <<< FAILURE! - in
> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest
>
> [ERROR] testMultiThreadPerf  Time elapsed: 4.816 s  <<< FAILURE!
>
> org.opentest4j.AssertionFailedError: expected:  but was: 
>
> at
> org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest.testMultiThreadPerf(RequestPathMetricsCollectorTest.java:448)
>
>
>
> - Norbert
>
>
>
> On Mon, Jan 25, 2021 at 1:39 PM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com> wrote:
>
>> +0 (and not even binding :) )
>>
>> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
>> 8u265 and maven 3.6.3.
>> - I also built and executed unit tests for zkpython
>> - the unit tests passed for the C-client and for python client
>> - checkstyle and spotbugs passed
>> - apache-rat passed
>> - owasp (CVE check) passed
>> - fatjar built (-Pfatjar)
>> - I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2.
>> (using
>> https://github.com/symat/zk-rolling-upgrade-test)
>>
>> for some reason the java unit tests failed for me.
>>
>> On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests executed
>> successfully, but then the maven job still failed for hbase-server test
>> with error message (with -DforkCount=4 and even with -DforkCount=1) like:
>> -
>> [ERROR] ExecutionException There was an error in the forked process
>> [ERROR] unable to create new native thread
>> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
>> ExecutionException There was an error in the forked process
>> [ERROR] unable to create new native thread
>> [ERROR] at
>>
>> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
>> -
>>
>>
>> Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 and
>> maven 3.6.3) and I got other kinds of strange maven errors:
>> ---
>> [ERROR] Caused by:
>> org.apache.maven.surefire.booter.SurefireBooterForkException: The forked
>> VM
>> terminated without properly saying goodbye. VM crash or System.exit
>> called?
>> [ERROR] Command was /bin/sh -c cd
>> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server &&
>> /home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xmx512m
>> -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=3
>>
>> -javaagent:/home/symat/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
>> -jar
>>
>> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire/surefirebooter8828313385463488429.jar
>> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire
>> 2021-01-25T11-54-03_621-jvmRun3 surefire4024538135165099286tmp
>> surefire_37800399112966511000tmp
>> [ERROR] Process Exit Code: 0
>> [ERROR] at
>>
>> org.apache.maven.plugin.surefire.booterc

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1

[Norbert Kalmar]

-1 (non-binding!)

- built the source, run locally, gave a few basic commands
- run tests on macOS 10.13.6, openjdk 1.8.0_242 - issues found, see below
- checked files in tarball
- checked signatures

I had the same problem as Máté. unit tests did not pass on MacOS. Nothing
has changed in my environment since I did the 3.5.9 release, and it had no
issue back then.

I gave a -1 because this error doesn't only occur to me.

The test that failed:

[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 1, Time elapsed:
62.332 s <<< FAILURE! - in
org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest

[ERROR] testMultiThreadPerf  Time elapsed: 4.816 s  <<< FAILURE!

org.opentest4j.AssertionFailedError: expected:  but was: 

at
org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest.testMultiThreadPerf(RequestPathMetricsCollectorTest.java:448)



- Norbert



On Mon, Jan 25, 2021 at 1:39 PM Szalay-Bekő Máté 
wrote:

> +0 (and not even binding :) )
>
> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
> 8u265 and maven 3.6.3.
> - I also built and executed unit tests for zkpython
> - the unit tests passed for the C-client and for python client
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - fatjar built (-Pfatjar)
> - I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2. (using
> https://github.com/symat/zk-rolling-upgrade-test)
>
> for some reason the java unit tests failed for me.
>
> On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests executed
> successfully, but then the maven job still failed for hbase-server test
> with error message (with -DforkCount=4 and even with -DforkCount=1) like:
> -
> [ERROR] ExecutionException There was an error in the forked process
> [ERROR] unable to create new native thread
> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
> ExecutionException There was an error in the forked process
> [ERROR] unable to create new native thread
> [ERROR] at
>
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
> -
>
>
> Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 and
> maven 3.6.3) and I got other kinds of strange maven errors:
> ---
> [ERROR] Caused by:
> org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM
> terminated without properly saying goodbye. VM crash or System.exit called?
> [ERROR] Command was /bin/sh -c cd
> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server &&
> /home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xmx512m
> -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=3
>
> -javaagent:/home/symat/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
> -jar
>
> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire/surefirebooter8828313385463488429.jar
> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire
> 2021-01-25T11-54-03_621-jvmRun3 surefire4024538135165099286tmp
> surefire_37800399112966511000tmp
> [ERROR] Process Exit Code: 0
> [ERROR] at
>
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
> [ERROR] at
>
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115)
> [ERROR] at
>
> org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444)
> [ERROR] at
>
> org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420)
> [ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [ERROR] at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [ERROR] at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [ERROR] at java.lang.Thread.run(Thread.java:748)
> [ERROR]
> --
>
>
> These issues might be very well specific to my local (mac or docker on mac)
> environments. This is why I didn't vote with -1
> Can someone else run the java unit tests successfully locally?
>
> I also tried to check if the CI was green for the last PR on 3.7.0 (
> https://github.com/apache/zookeeper/pull/1586/checks), but it looks the CI
> haven't even started to execute the tests, due to errors in the "install C
> dependencies" step.
>
> Regards,
> Mate
>
> On Sun, Jan 24, 2021 at 11:39 PM Patrick Hunt  wrote:
>
> > +1. xsum/sig verified. rat ran clean. built and dependency checks are
> fine.
> > Tried running some manual clusters and it was successful.
> >
> > Regards,
> >
> > Patrick
> >
> >
> > On Sun, Jan 24, 2021 at 12:11 PM Damien Diederen 
> > wrote:
> >
> > >
> > > Dear all,
> > >
> > > This is a second release candidate for ZooKeeper 3.7.0.  Compared to
> > > RC0, it fixes a tarball generation issue, includes a description of the
> > > 'whoami' CLI command, and incorporates a contribution to ZooInspector.
> 

Re: Rate Quota

[Norbert Kalmar]

Sorry, I missed the point there.
Unfortunately I don't know the timeline for this feature. Our friends at
Facebook weren't active lately, but hope you'll get an answer (I see you
also asked on the jira).

- Norbert

On Tue, Jan 19, 2021 at 8:33 PM Huizhi Lu  wrote:

> Hi Norbert,
>
> Thanks for the reply. I understand request throttling was added in the
> ticket ZOOKEEPER-3243. My understanding is that it's a global request
> throttling: total requests for all clients. But it is not a per client
> level throttling.
>
> Just realized that the facebook presentation listed the zk ticket:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3467. I think this is
> what I'm looking for: reads/s and writes/s for per client. The feature
> is not yet available. I wonder if there is a timeline for it. Thanks!
>
>
> -Huizhi
>
> On Tue, Jan 19, 2021 at 2:20 AM Norbert Kalmar
>  wrote:
> >
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3243
> >
> > On Tue, Jan 19, 2021 at 11:19 AM Norbert Kalmar 
> > wrote:
> >
> > > Hi Huizhi,
> > >
> > > You might be looking for ZOOKEEPER-3243, it is available from 3.6.0.
> > >
> > > Regards,
> > > Norbert
> > >
> > > On Tue, Jan 19, 2021 at 9:49 AM Huizhi Lu  wrote:
> > >
> > >> Hey ZK Experts,
> > >>
> > >> I am actively looking into the request throttling/quota feature. I
> > >> recalled that at the zookeeper meetup, Facebook presented a rate quota
> > >> per client (see attached pic) feature. But I can't find it in any zk
> > >> doc or code. Is the feature available? If not, is there any reason?
> > >>
> > >>
> > >> Thanks!
> > >> -Huizhi
> > >>
> > >
>

Re: Rate Quota

[Norbert Kalmar]

https://issues.apache.org/jira/browse/ZOOKEEPER-3243

On Tue, Jan 19, 2021 at 11:19 AM Norbert Kalmar 
wrote:

> Hi Huizhi,
>
> You might be looking for ZOOKEEPER-3243, it is available from 3.6.0.
>
> Regards,
> Norbert
>
> On Tue, Jan 19, 2021 at 9:49 AM Huizhi Lu  wrote:
>
>> Hey ZK Experts,
>>
>> I am actively looking into the request throttling/quota feature. I
>> recalled that at the zookeeper meetup, Facebook presented a rate quota
>> per client (see attached pic) feature. But I can't find it in any zk
>> doc or code. Is the feature available? If not, is there any reason?
>>
>>
>> Thanks!
>> -Huizhi
>>
>

Re: Rate Quota

[Norbert Kalmar]

Hi Huizhi,

You might be looking for ZOOKEEPER-3243, it is available from 3.6.0.

Regards,
Norbert

On Tue, Jan 19, 2021 at 9:49 AM Huizhi Lu  wrote:

> Hey ZK Experts,
>
> I am actively looking into the request throttling/quota feature. I
> recalled that at the zookeeper meetup, Facebook presented a rate quota
> per client (see attached pic) feature. But I can't find it in any zk
> doc or code. Is the feature available? If not, is there any reason?
>
>
> Thanks!
> -Huizhi
>

Re: New committer: Justin Mao Ling

[Norbert Kalmar]

Congratulations Maoling! Well-deserved!

- Norbert

On Mon, Jan 18, 2021 at 11:43 AM Andor Molnar  wrote:

> Congrats Maoling!
>
>
>
>
> > On 2021. Jan 18., at 11:09, Enrico Olivelli  wrote:
> >
> > The Project Management Committee (PMC) for Apache ZooKeeper
> >
> > has invited Justin Mao Long to become a committer and we are pleased
> >
> > to announce that he has accepted.
> >
> >
> > Justin has been following the Project for a long time,
> >
> > He is very active in the community with discussions and code reviews
> > and he contributed
> > many patches.
> >
> >
> > Being a committer enables easier contribution to the
> >
> > project since there is no need to go via the patch
> >
> > submission process. This should enable better productivity.
> >
> > Being a PMC member enables assistance with the management
> >
> > and to guide the direction of the project.
> >
> >
> >
> > Congratulations Justin !
> >
> >
> > Enrico
>
>

[ANNOUNCE] Apache ZooKeeper 3.5.9

[Norbert Kalmar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.5.9

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads,
visit:https://zookeeper.apache.org/releases.html

ZooKeeper 3.5.9 Release Notes are
at:https://zookeeper.apache.org/doc/r3.5.9/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,
The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Norbert Kalmar]

Thank you all for the vote and tests. We have the 3 binding votes, I will
continue with the release and announce once completed and propagated to the
mirror sites and maven repo for download.

- Norbert

On Thu, Jan 14, 2021 at 4:15 PM Andor Molnar  wrote:

> +1
>
> - verified sigs/xsums,
> - run full unit tests suite on Ubuntu 20.10 (Java+C),
> - run rat check
> - run checkstyle
> - run smoketest/latencies test against non-SSL cluster,
> - verified SSL quorum + simple commands with SSL CLI
>
> Andor
>
>
>
> > On 2021. Jan 6., at 21:09, Norbert Kalmar  wrote:
> >
> > This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
> > including CVE fixes.
> > (Note: rc1 had a third party CVE which was only noticed during the last
> > check of the release, so it never made it for vote)
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> >
> > *** Please download, test and vote by January 11th 2020, 23:59 UTC+0. ***
> >
> > Source files:
> > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> >
> > The release candidate tag in git to be voted upon: release-3.5.9-rc2
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Should we release this candidate?
> >
> > - Norbert
>
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Norbert Kalmar]

Thanks for the votes so far. We're still missing two binding +1. Could any
PMC member please take a look?
Thank you in advance!

- Norbert

On Thu, Jan 7, 2021 at 3:59 PM Enrico Olivelli  wrote:

> +1 (binding)
> - verified checksums and sigs
> - build sources and run tests on MacOs + JDK8, no issue on Java tests
> - run C client tests on MacOs + docker, only one known failure
> - run C client on Fedora + JDK8, no test failure
> - verified license files on binary distribution, no issue (we have
> different conventions, some files has no LICENSE filebtw this is how we
> packaged previous releases so, not a blocker, and I am not sure it is worth
> an issue)
> - performed smoke tests on single machine + jdk8
>
> Thank you Norbert for driving this release
>
> Enrico
>
>
> Il giorno gio 7 gen 2021 alle ore 15:33 Damien Diederen <
> ddiede...@apache.org> ha scritto:
>
> >
> > Hi Norbert,
> >
> > Looks good.  +1 (advisory):
> >
> >   * Verified signatures and checksums;
> >
> >   * Built and smoke-tested on NixOS with a slightly adapted version of
> > the Nix recipe:
> >
> >   https://github.com/NixOS/nixpkgs/pull/104889
> >
> >   * Smoke-tested a single instance with Java, C and Perl clients,
> > including a quick run of YCSB:
> >
> >   https://github.com/apache/zookeeper/pull/1558
> >
> >   * Smoke-tested a 3-ensemble with the Java client and SASL/GSSAPI,
> > including a quick run of YCSB;
> >
> > I did not spot anything amiss.
> >
> > Cheers, -D
> >
> >
> > Norbert Kalmar  writes:
> > > This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
> > > including CVE fixes.
> > > (Note: rc1 had a third party CVE which was only noticed during the last
> > > check of the release, so it never made it for vote)
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> > >
> > > *** Please download, test and vote by January 11th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/
> > >
> > > Maven staging repo:
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> > >
> > > The release candidate tag in git to be voted upon: release-3.5.9-rc2
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Should we release this candidate?
> > >
> > > - Norbert
> >
>

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Norbert Kalmar]

I'd say there are quite a few tasks aimed at 4.0. I just answered a thread
about jute.maxbuffer error, which could be improved for example. Or better
yet, throw jute out and use a standardized serialization library.
But there's also the issue of separating client and server code. And I'm
sure there's many more I can't recall right now.

But this is worth a separate vote thread, that's for sure.

- Norbert

On Fri, Jan 8, 2021 at 1:33 PM Andor Molnar  wrote:

> Okay, let’s stay on JDK 8 with 3.7.0 release and do the transition in 4.0.
> Not sure if we want 3.8 release or make the master 4.0 from now on.
>
> Andor
>
>
>
> > On 2021. Jan 6., at 22:35, Christopher  wrote:
> >
> > I agree with Enrico on this point. If the ZK PMC is considering a 3.7
> > release, now would not be the right time to make this change, and it
> would
> > be better to make a change at the beginning of the next iteration.
> >
> > That said, I think switching to builds with JDK 11 and supporting JDK 8
> > "passively" is the right thing to do for 3.8, and switching to JDK 11
> fully
> > would be the right thing to do if the PMC decides to do a major version
> > bump to 4.0 instead of a 3.8.
> >
> > On Wed, Jan 6, 2021 at 1:20 PM Enrico Olivelli 
> wrote:
> >
> >> In my opinion we can stay on 8, we should change this kind of stuff at
> the
> >> beginning of a new iteration and not before the release.
> >> IIRC We are not in a hurry, having a stable build process is a value.
> >>
> >> Enrico
> >>
> >> Il Mer 6 Gen 2021, 17:51 Damien Diederen  ha
> >> scritto:
> >>
> >>>
> >>> Dear ZooKeeper team,
> >>>
> >>> Andor just reminded me of this JDK 11 vs. 8 discussion, for which we
> did
> >>> not reach a resolution.  Do we want to make a move for the 3.7.0
> release?
> >>>
> >>> The original proposal, by Christopher, can be found here:
> >>>
> >>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3ccal5zq9z2tnfawjz7etxpf91qmrovuwv+khnxgpsd_msekfp...@mail.gmail.com%3e
> >>>
> >>> I am explicitly replying to Christopher's nice and useful summary,
> which
> >>> you will find directly below.  An archived copy can be found here:
> >>>
> >>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=aolxidz-0zohunvnayqei0...@mail.gmail.com%3e
> >>>
> >>> It contains three proposals, to which we can add the trivial one:
> >>>
> >>> 1. Compile/test with JDK 11, support JDK 8 "passively";
> >>>
> >>> 2. Compile/test with JDK 11, test client only with JDK 8 (requires test
> >>>suite adaptations);
> >>>
> >>> 3. Move to JDK 11 fully;
> >>>
> >>> 4. Postpone any change to 3.8 (or 4.0).
> >>>
> >>> Flavio wrote:
> >>>
>  [Christopher] mentioned a PMC vote, and I don't think this should be a
>  closed vote, independent of how the conversation goes.
> >>>
> >>> Do we have a better idea of what we want?  Or should we organize a
> vote?
> >>>
> >>> Cheers, -D
> >>>
> >>>
> >>> --8<--- original message ->8---
> >>>
> >>> Christopher  writes:
>  I'm happy that this discussion has been so lively! I just want to
>  emphasize a few things:
> 
>  I really do understand the desire to continue to support Java 8... I
>  get it. But all the conversations around this seem based on what
>  people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
>  *future* release... so it should be based more on reasonable
>  expectations for users in the future, and less based on what is
>  happening today. I suspect *most* people today are still using 3.4
>  anyway (it was just so stable for so long...), but that shouldn't mean
>  the developers should hold back development on 3.5 and 3.6, any more
>  than today's users of 3.5/3.6 should hold back 3.7.
> 
>  Some of the opinions expressed in this discussion seem to propose a
>  scenario where users are going to be updating to "bleeding edge"
>  versions of ZooKeeper, but are going to insist on using Java 8.
>  Personally, I find this to be implausible. In my experience, people
>  either upgrade everything as soon as they are able to, or they upgrade
>  each thing individually, only when they are forced to. The first group
>  will be happy to move to Java 11 and ZK 3.7. The second group will
>  probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
>  had to update to 3.7, they'd also be fine updating to Java 11 if they
>  had to in order to use 3.7. I can't imagine the scenario where people
>  are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
>  using Java 8. Perhaps that scenario exists, but it's hard for me to
>  imagine. Even so, my proposal would still support even that group of
>  people.
> 
>  I think there are now effectively three proposals being discussed in
>  this thread:
> 
>  1. (Christopher's original proposal) passively support Java 

Re: ConnectionLoss Upon Length Error - Failed to Write > 1 MB

[Norbert Kalmar]

We see a lot of issues (even on prod systems) around jute.maxbuffer. I
agree it is not the "cleanest" of errors. If ZK is involved in some issue,
usually we always check first for signs of requests being too big (a.k.a.
jute.maxbuffer issue).

But if we wan't to improve on this, we have to make sure it is backward
compatible. Lot of monitoring is placed around this IMHO. That's why I
think there wasn't any refactoring around it.

Just my two cents.

- Norbert



On Fri, Jan 8, 2021 at 10:37 AM Huizhi Lu  wrote:

> OK, I think I get it. The rough sanity check is applied only when
> deserializing, the len of incoming buffer is read. There is no check
> for outgoing data when serializing. And there are 10s of bytes in the
> serialization metadata, so if a client is writing just below 1 MB
> (1024 * 1024 - 1 bytes), the final incoming buffer data would exceed 1
> MB and the write would fail. So it's kind of inaccurate that by
> default zk could store a znode just below 1 MB (1024 * 1024 - 1). To
> make it accurate, maybe we could just check the bytes length before
> serializing, and the server could add some extra bytes based on 1 MB.
> I guess this is minor as it is just a rough sanity check. ZK just does
> not expect a client would write that large data :)
>
> On Fri, Jan 8, 2021 at 12:49 AM Huizhi Lu  wrote:
> >
> > From what I've learned and also the doc:
> >
> > "jute.maxbuffer : (Java system property:jute.maxbuffer).
> >
> > When jute.maxbuffer in the client side is greater than the server
> > side, the client wants to write the data exceeds jute.maxbuffer in the
> > server side, the server side will get java.io.IOException: Len error
> > When jute.maxbuffer in the client side is less than the server side,
> > the client wants to read the data exceeds jute.maxbuffer in the client
> > side, the client side will get java.io.IOException: Unreasonable
> > length or Packet len is out of range!"
> >
> > So I assume: the client only honors jute.maxbuffer when reading. If
> > the client tries to read the data > jute.maxbuffer, it fails. For
> > writing, jute.maxbuffer is honored on server side, the client does not
> > do the sanity check.
> > Correct me if I am wrong. I would really expect the client can also
> > fail the request if it's writing data > jute.maxbuffer.
> >
> > On Fri, Jan 8, 2021 at 12:40 AM Huizhi Lu  wrote:
> > >
> > > Hi Ted,
> > >
> > > Really appreciate your prompt response and detailed explanation!
> > >
> > > For some reason, ZK could be abused for writing large data objects.
> > > I understand we should correctly use ZK for coordination that ZK is
> best at.
> > > It's definitely something we could improve how we use ZK. But maybe
> > > it'd be a long run to arrive.
> > > Thanks for the clarification :)
> > >
> > > Back to the jute maxbuffer setting. With the consistent values 1 MB on
> > > both client and server,
> > > I am still able to produce it: request is sent to server as it throws
> > > IOException "Len error" and closes
> > > the connection. The client log is below, which does not give
> > > descriptive enough info like "Len error".
> > > [main-SendThread(localhost:2181)] WARN
> > > org.apache.zookeeper.ClientCnxn - Session 0x1003e7613ab0005 for sever
> > > localhost:2181, Closing socket connection. Attempting reconnect except
> > > it is a SessionExpiredException.
> > > java.io.IOException: Connection reset by peer
> > >
> > > With this, can I assume the zk client does not fail the request?
> > > I also dig into the code, it seems the request reaches the server and
> > > the server fails the request.
> > > I am actually expecting the request can be failed earlier on the
> > > client side and then get descriptive info "the packet size is too
> > > large".
> > > Is this (when writing, client jute.maxbuffer is not honored) expected?
> > > I think if the client side fails the request and gives more
> > > descriptive info/specific exception, that'd be great and it's what I
> > > would expect.
> > >
> > > -Huizhi
> > >
> > > On Fri, Jan 8, 2021 at 12:01 AM Ted Dunning 
> wrote:
> > > >
> > > > Let's be clear from the start, storing large data objects in
> Zookeeper is
> > > > strongly discouraged. If you want to store large objects with good
> > > > consistency models, store the data in something else (like a
> distributed
> > > > file system or key value store), commit the data and then use ZK to
> provide
> > > > a reference to that data. Zookeeper is intended for coordination,
> not data
> > > > storage. It is not a reasonable alternative to a noSQL database.
> > > >
> > > > That said, good and informative error messages are always useful and
> are
> > > > better than anonymous errors. Even if the connection is closed
> related to
> > > > the error, it would be nice to give some decent feedback.
> > > >
> > > > On the other hand, the error that you are seeing sounds like your
> client
> > > > and your server have inconsistent settings for the maximum jute
> buffer
> > > > size. If 

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Norbert Kalmar]

I also verified C tests and build on an Ubuntu machine, and didn't notice
any test issues.
As this was a known issue before, I would say this is not a blocker, but I
might be biased. Definitely worth a jira ticket though.
Here: https://issues.apache.org/jira/browse/ZOOKEEPER-4049

Thanks for the tests!

- Norbert

On Thu, Jan 7, 2021 at 1:42 PM Szalay-Bekő Máté 
wrote:

> > [exec]
> >
>
> /Users/enrico.olivelli/Downloads/zk359/apache-zookeeper-3.5.9/zookeeper-client/zookeeper-client-c/tests/TestClient.cc:789:
> > Assertion: equality assertion failed [Expected: 0, Actual  : -4]
>
> This test failed for me too with the exact same error message on docker
> many times before (when I was checking RCs for other releases). I usually
> ignore this one, as it does run successfully on my real ubuntu. It would be
> good to check why it fails though on docker... Maybe it's worth to create a
> Jira ticket.
>
> For this RC I compiled and executed the C-client tests on a real native
> ubuntu server (18.04) and all the C client tests passed for me now.
>
> Best regards,
> Mate
>
> On Thu, Jan 7, 2021 at 12:56 PM Enrico Olivelli 
> wrote:
>
> > I am validating the release
> > I have this test that is consistently failing on the c-client
> >
> >  [exec]
> >
> >
> /Users/enrico.olivelli/Downloads/zk359/apache-zookeeper-3.5.9/zookeeper-client/zookeeper-client-c/tests/TestClient.cc:789:
> > Assertion: equality assertion failed [Expected: 0, Actual  : -4]
> >
> > Do you think it is a blocker for the release ?
> > I am on MacOs + docker env (dev/docker/run.sh)
> >
> > Enrico
> >
> > Il giorno gio 7 gen 2021 alle ore 10:46 Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com> ha scritto:
> >
> > > +1 (non-binding)
> > >
> > > - I built the source code (-Pfull-build) on Ubuntu 18.04 using OpenJDK
> > > 8u265 and maven 3.6.3.
> > > - all the unit tests passed eventually (both Java and C-client).
> > > - I also built zkpython
> > > - checkstyle and spotbugs passed
> > > - apache-rat passed
> > > - owasp (CVE check) passed
> > > - I executed a quick rolling-upgrade test from 3.5.9 to 3.6.2. (using
> > > https://github.com/symat/zk-rolling-upgrade-test)
> > >
> > > The only thing I found was 4 unit tests, failed first (when I executed
> > all
> > > tests in docker) but succeeded second time running them on my mac:
> > > - QuorumPeerMainTest  ->  testLeaderOutOfView
> > > - ReconfigExceptionTest  ->  testReconfigDisabled
> > > - NIOServerCnxnFactoryTest  ->
> > testStartupWithoutStart_SocketAlreadyBound
> > > - NIOServerCnxnFactoryTest  ->  testStartupWithStart_SocketAlreadyBound
> > >
> > > Thanks,
> > > Mate
> > >
> > > On Wed, Jan 6, 2021 at 9:10 PM Norbert Kalmar 
> > wrote:
> > >
> > > > This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
> > > > including CVE fixes.
> > > > (Note: rc1 had a third party CVE which was only noticed during the
> last
> > > > check of the release, so it never made it for vote)
> > > >
> > > > The full release notes is available at:
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> > > >
> > > > *** Please download, test and vote by January 11th 2020, 23:59 UTC+0.
> > ***
> > > >
> > > > Source files:
> > > > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/
> > > >
> > > > Maven staging repo:
> > > >
> > > >
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.5.9-rc2
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > Should we release this candidate?
> > > >
> > > > - Norbert
> > > >
> > >
> >
>

[VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Norbert Kalmar]

This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
including CVE fixes.
(Note: rc1 had a third party CVE which was only noticed during the last
check of the release, so it never made it for vote)

The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201

*** Please download, test and vote by January 11th 2020, 23:59 UTC+0. ***

Source files:
https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/

The release candidate tag in git to be voted upon: release-3.5.9-rc2

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

Should we release this candidate?

- Norbert

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 0

[Norbert Kalmar]

Sorry for missing the notification. I did an rc1 but totally missed one CVE
and only realized when doing my final checks. So I will have an rc2 instead
shortly.

- Norbert

On Tue, Jan 5, 2021 at 5:23 PM Enrico Olivelli  wrote:

> Il giorno mar 5 gen 2021 alle ore 15:48 Norbert Kalmar
>  ha scritto:
>
> > It failed due to the CVE, and the fix was not a clean cherry-pick to 3.5.
> >
>
> Thank you Norbert,
> I didn't find any official "CANCELLED" response.
> no hurry
>
> Enrico
>
>
> > Then Holidays hit, and I didn't do RC2. Picking it up now, and checking
> > what needs to be backported and doing an RC2.
> >
> > - Norbert
> >
> > On Tue, Jan 5, 2021 at 12:26 PM Enrico Olivelli 
> > wrote:
> >
> > > What's the status of this VOTE ?
> > >
> > > Enrico
> > >
> > > Il giorno mar 8 dic 2020 alle ore 21:28 Damien Diederen <
> > > ddiede...@sinenomine.net> ha scritto:
> > >
> > > >
> > > > Hi Andor,
> > > >
> > > > > Is this not the same Jar that I’ve upgraded recently, because of a
> > CVE?
> > > >
> > > > It is.  You updated it for CVE-2020-27216, and this is now for
> > > > CVE-2020-27218!
> > > >
> > > > Cheers, -D
> > > >
> > > >
> > > >
> > > >
> > > > >> On 2020. Dec 5., at 22:03, Patrick Hunt  wrote:
> > > > >>
> > > > >> Thanks Damien! I reviewed and it looks good except for one small
> > > > comment I
> > > > >> hope we can also address (commented on PR).
> > > > >>
> > > > >> Regards,
> > > > >>
> > > > >> Patrick
> > > > >>
> > > > >> On Sat, Dec 5, 2020 at 12:05 PM Damien Diederen <
> > > > ddiede...@sinenomine.net>
> > > > >> wrote:
> > > > >>
> > > > >>>
> > > > >>> Hi Patrick, all,
> > > > >>>
> > > > >>>> -1 - the dependency check is failing with a known CVE
> > > > >>>>
> > > > >>>> $ mvn clean package -DskipTests dependency-check:check
> > > > >>>> ...
> > > > >>>> [ERROR] One or more dependencies were identified with
> > > vulnerabilities
> > > > >>> that
> > > > >>>> have a CVSS score greater than or equal to '0.0':
> > > > >>>> [ERROR]
> > > > >>>> [ERROR] jetty-server-9.4.34.v20201102.jar: CVE-2020-27218
> > > > >>>> [ERROR] jetty-http-9.4.34.v20201102.jar: CVE-2020-27218
> > > > >>>
> > > > >>> For the (mailing list) record, I have created:
> > > > >>>
> > > > >>> https://issues.apache.org/jira/browse/ZOOKEEPER-4023
> > > > >>> https://github.com/apache/zookeeper/pull/1552
> > > > >>>
> > > > >>> Best, -D
> > > > >>>
> > > >
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 0

[Norbert Kalmar]

It failed due to the CVE, and the fix was not a clean cherry-pick to 3.5.
Then Holidays hit, and I didn't do RC2. Picking it up now, and checking
what needs to be backported and doing an RC2.

- Norbert

On Tue, Jan 5, 2021 at 12:26 PM Enrico Olivelli  wrote:

> What's the status of this VOTE ?
>
> Enrico
>
> Il giorno mar 8 dic 2020 alle ore 21:28 Damien Diederen <
> ddiede...@sinenomine.net> ha scritto:
>
> >
> > Hi Andor,
> >
> > > Is this not the same Jar that I’ve upgraded recently, because of a CVE?
> >
> > It is.  You updated it for CVE-2020-27216, and this is now for
> > CVE-2020-27218!
> >
> > Cheers, -D
> >
> >
> >
> >
> > >> On 2020. Dec 5., at 22:03, Patrick Hunt  wrote:
> > >>
> > >> Thanks Damien! I reviewed and it looks good except for one small
> > comment I
> > >> hope we can also address (commented on PR).
> > >>
> > >> Regards,
> > >>
> > >> Patrick
> > >>
> > >> On Sat, Dec 5, 2020 at 12:05 PM Damien Diederen <
> > ddiede...@sinenomine.net>
> > >> wrote:
> > >>
> > >>>
> > >>> Hi Patrick, all,
> > >>>
> >  -1 - the dependency check is failing with a known CVE
> > 
> >  $ mvn clean package -DskipTests dependency-check:check
> >  ...
> >  [ERROR] One or more dependencies were identified with
> vulnerabilities
> > >>> that
> >  have a CVSS score greater than or equal to '0.0':
> >  [ERROR]
> >  [ERROR] jetty-server-9.4.34.v20201102.jar: CVE-2020-27218
> >  [ERROR] jetty-http-9.4.34.v20201102.jar: CVE-2020-27218
> > >>>
> > >>> For the (mailing list) record, I have created:
> > >>>
> > >>> https://issues.apache.org/jira/browse/ZOOKEEPER-4023
> > >>> https://github.com/apache/zookeeper/pull/1552
> > >>>
> > >>> Best, -D
> > >>>
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 0

[Norbert Kalmar]

Thank you all for the review.

Damien: I don't think jenkins jira's are even worth noting in release
notes, but the other 2 is of a bigger interest.
ZOOKEEPER-1634 - the jira is missing any 3.5 fix tag. I can fix it in the
jira, but I wouldn't do a new rc to have it in releasenotes.

Now the missing commits, again, what is more interesting is ZOOKEEPER-3933.
Looking at the jira it was a false positive alert, so no change were made.
Same as ZOOKEEPER-3934, false positive, no change.

So thankfully we are not actually missing any commits, but rather have
false positive alert jiras closed with fix versions in them.

Thanks for the thorough review, after looking at these cases I agree it is
not a deal breaker for rc0. Let's wait for more PMC to vote.

I will also try to look into Máté's findings with Python.

-Norbert


On Fri, Dec 4, 2020 at 9:18 AM Szalay-Bekő Máté 
wrote:

> +1 (non-binding)
>
> - I built the source code (-Pfull-build) in docker on Ubuntu 16.04.6 using
> OpenJDK 8u275 and maven  3.3.9.
> - all the unit tests passed (Java and C-client).
> - I also built zkpython
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
>
> The only issue I found was that I was unable to make the python unit tests
> to start. In 3.5.8 I was able to execute the unit tests (although I had to
> do some manual hack before, which didn't help this time). I don't know what
> changed here exactly, maybe just my environment. We might want to create a
> jira ticket to migrate the zkpython build / test to maven properly.
>
> Best regards,
> Mate
>
> On Thu, Dec 3, 2020 at 9:01 PM Damien Diederen 
> wrote:
>
> >
> > Thank you, Norbert!
> >
> > I went through the motions a bit more carefully than usual in
> > preparation for the upcoming 3.7.0 job, which I am planning to start
> > soon, but probably after you finalize this release.
> >
> >
> > +1 (advisory)
> >
> >   * Verified signatures and checksums;
> >
> >   * Built and tested on Ubuntu 20.04.1 LTS with OpenJDK Runtime
> > Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04) using:
> >
> > mvn -B apache-rat:check verify spotbugs:check checkstyle:check \
> > -Pfull-build -Dsurefire-forkcount=1
> >
> >   * Built and smoke-tested on NixOS with a slightly adapted version of
> > this WIP PR:
> >
> > https://github.com/NixOS/nixpkgs/pull/104889
> >
> >   * Smoke-tested a single instance with Java, C and Perl client;
> >
> >   * Smoke-tested a 3-ensemble with Java client, including Kerberos auth;
> >
> >
> > I don't believe these points are blockers, but I noticed that the
> > following commits which are present in the release are not mentioned in
> > the release notes:
> >
> >   * commit 0838c6c1613d7902d6c3419dcad2205682223175
> > Author: Michael Han 
> > Date:   Mon Jul 6 16:25:38 2020 +0200
> >
> > ZOOKEEPER-1634: hardening security by teaching server to enforce
> > client authentication
> >
> >   * commit 54ffaad1b94d72e735fd8fb750117b6ee1550b1b
> > Author: Andor Molnar 
> > Date:   Tue Oct 6 17:51:15 2020 +0200
> >
> > ZOOKEEPER-3957: Created initial version of owasp-check
> Jenkinsfile
> >
> >   * commit db9fed4c95e4828389b30c0f6e94182db26ff99b
> > Author: Enrico Olivelli 
> > Date:   Tue Oct 20 16:21:30 2020 +0200
> >
> > ZOOKEEPER-3980: Fix Jenkinsfiles with new tool names
> >
> >
> > On the other hand, and just FYI, the following tickets mentioned in the
> > release notes do not have corresponding commits:
> >
> >   * ZOOKEEPER-3933: owasp failing with json-simple-1.1.1.jar:
> > CVE-2020-10663, CVE-2020-7712.
> >
> > This was a false positive.  Ticket was closed, but no commit was
> > produced.
> >
> >   * ZOOKEEPER-3934: upgrade dependency-check to version 6.0.0
> >
> > Same as ZOOKEEPER-3933.
> >
> > Cheers, -D
> >
> >
> >
> >
> > Norbert Kalmar  writes:
> > > This is a bugfix release candidate for 3.5.9. It contains 24 fixes,
> > > including 2 CVE fix.
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> > >
> > > *** Please download, test and vote by December 4th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-0/
> > >
> > > Maven staging repo:
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> > >
> > > The release candidate tag in git to be voted upon: release-3.5.9-rc0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Should we release this candidate?
> > >
> > > - Norbert
> >
>

[VOTE] Apache ZooKeeper release 3.5.9 candidate 0

[Norbert Kalmar]

This is a bugfix release candidate for 3.5.9. It contains 24 fixes,
including 2 CVE fix.

The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201

*** Please download, test and vote by December 4th 2020, 23:59 UTC+0. ***

Source files:
https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-0/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/

The release candidate tag in git to be voted upon: release-3.5.9-rc0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

Should we release this candidate?

- Norbert

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

Starting the branching process for 3.5.9

On Thu, Nov 26, 2020 at 7:13 AM Enrico Olivelli  wrote:

> Il Mer 25 Nov 2020, 20:26 Norbert Kalmar  ha
> scritto:
>
> > Owasp CVE has been backported to 3.5.
> > There is still no backport for ZOOKEEPER-3911 (possible inconsistency due
> > to uncommitted DIFF), and is troublesome for 3.6 as well. In addition to
> > 3.6 test fail on 3.5. I had no time to take a closer look.
> >
> > Should we wait for this patch to land on 3.5? Or move along with
> releasing
> > 3.5.9?
> >
>
> If no one has time to do it let's cut 3.5.9, there is already enough stuff
> to make sense to a release.
>
>
> Enrico
>
>
> > Regards,
> > Norbert
> >
> > On Fri, Nov 13, 2020 at 6:43 PM Norbert Kalmar 
> > wrote:
> >
> > > I reverted the patch from 3.5, it seems even after fixing up the code
> > > issues the test on inconsistency fails due to the leader not seeing the
> > > same znodes.
> > >
> > >
> > > On Fri, Nov 13, 2020 at 11:51 AM Norbert Kalmar 
> > > wrote:
> > >
> > >> The patch broke 3.5 branch btw, I'm already looking into it. Sorry.
> > >>
> > >> On Fri, Nov 13, 2020 at 11:48 AM Norbert Kalmar  >
> > >> wrote:
> > >>
> > >>> Sure, here is the 3.5.9 labeled jiras:
> > >>> https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12348201
> > >>>
> > >>> Or tweak this search as you see fit:
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%203.5.9%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
> > >>>
> > >>> -Norbert
> > >>>
> > >>> On Thu, Nov 12, 2020 at 4:51 PM Andor Molnar 
> wrote:
> > >>>
> > >>>> Hi Norbert,
> > >>>>
> > >>>> Thanks for volunteering as RM.
> > >>>> Would you please provide a list of pending tickets either directly
> > here
> > >>>> or a link with the right search in Jira?
> > >>>>
> > >>>> Andor
> > >>>>
> > >>>>
> > >>>>
> > >>>> > On 2020. Nov 12., at 16:12, Norbert Kalmar
> > >>>>  wrote:
> > >>>> >
> > >>>> > ZOOKEEPER-3911 merged to all 3 active branches.
> > >>>> > 3.5 did not pick clean, I will do further testing to verify the
> > patch.
> > >>>> > After that I will start the release process for 3.5.9, as no
> request
> > >>>> has
> > >>>> > arrived for anything else to be included.
> > >>>> >
> > >>>> > -Norbert
> > >>>> >
> > >>>> > On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar <
> nkal...@cloudera.com
> > >
> > >>>> wrote:
> > >>>> >
> > >>>> >> Correction, Michael's patch for ZOOKEEPER-3911 would be also good
> > to
> > >>>> >> backport to 3.5 IMHO. Any thoughts on this?
> > >>>> >> PR: https://github.com/apache/zookeeper/pull/1445
> > >>>> >>
> > >>>> >> -Norbert
> > >>>> >>
> > >>>> >> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar <
> nkal...@apache.org
> > >
> > >>>> wrote:
> > >>>> >>
> > >>>> >>> Hello fellow ZK contributors and enthusiasts!
> > >>>> >>>
> > >>>> >>> Looks like the time has come for a new release on the 3.5
> branch.
> > >>>> >>> Currently it has 4 blocker (1 which is a CVE) and 2 critical
> fixes
> > >>>> >>> completed.
> > >>>> >>>
> > >>>> >>> There are 31 major patch in progress. I checked them, looks like
> > >>>> all of
> > >>>> >>> them are pushed to the next release from several versions
> prior. I
> > >>>> will
> > >>>> >>> check the PRs on them, but I don't see any that is a must have
> to
> > >>>> include
> > >>>> >>> in 3.5.9.
> > >>>> >>>
> > >>>> >>> So please let me know if there is a ticket that you think is
> > >>>> essential in
> > >>>> >>> 3.5.9!
> > >>>> >>>
> > >>>> >>> Thank you all.
> > >>>> >>>
> > >>>> >>> -Norbert
> > >>>> >>>
> > >>>> >>
> > >>>>
> > >>>>
> >
>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

Owasp CVE has been backported to 3.5.
There is still no backport for ZOOKEEPER-3911 (possible inconsistency due
to uncommitted DIFF), and is troublesome for 3.6 as well. In addition to
3.6 test fail on 3.5. I had no time to take a closer look.

Should we wait for this patch to land on 3.5? Or move along with releasing
3.5.9?

Regards,
Norbert

On Fri, Nov 13, 2020 at 6:43 PM Norbert Kalmar  wrote:

> I reverted the patch from 3.5, it seems even after fixing up the code
> issues the test on inconsistency fails due to the leader not seeing the
> same znodes.
>
>
> On Fri, Nov 13, 2020 at 11:51 AM Norbert Kalmar 
> wrote:
>
>> The patch broke 3.5 branch btw, I'm already looking into it. Sorry.
>>
>> On Fri, Nov 13, 2020 at 11:48 AM Norbert Kalmar 
>> wrote:
>>
>>> Sure, here is the 3.5.9 labeled jiras:
>>> https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12348201
>>>
>>> Or tweak this search as you see fit:
>>>
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%203.5.9%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>>>
>>> -Norbert
>>>
>>> On Thu, Nov 12, 2020 at 4:51 PM Andor Molnar  wrote:
>>>
>>>> Hi Norbert,
>>>>
>>>> Thanks for volunteering as RM.
>>>> Would you please provide a list of pending tickets either directly here
>>>> or a link with the right search in Jira?
>>>>
>>>> Andor
>>>>
>>>>
>>>>
>>>> > On 2020. Nov 12., at 16:12, Norbert Kalmar
>>>>  wrote:
>>>> >
>>>> > ZOOKEEPER-3911 merged to all 3 active branches.
>>>> > 3.5 did not pick clean, I will do further testing to verify the patch.
>>>> > After that I will start the release process for 3.5.9, as no request
>>>> has
>>>> > arrived for anything else to be included.
>>>> >
>>>> > -Norbert
>>>> >
>>>> > On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar 
>>>> wrote:
>>>> >
>>>> >> Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
>>>> >> backport to 3.5 IMHO. Any thoughts on this?
>>>> >> PR: https://github.com/apache/zookeeper/pull/1445
>>>> >>
>>>> >> -Norbert
>>>> >>
>>>> >> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar 
>>>> wrote:
>>>> >>
>>>> >>> Hello fellow ZK contributors and enthusiasts!
>>>> >>>
>>>> >>> Looks like the time has come for a new release on the 3.5 branch.
>>>> >>> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
>>>> >>> completed.
>>>> >>>
>>>> >>> There are 31 major patch in progress. I checked them, looks like
>>>> all of
>>>> >>> them are pushed to the next release from several versions prior. I
>>>> will
>>>> >>> check the PRs on them, but I don't see any that is a must have to
>>>> include
>>>> >>> in 3.5.9.
>>>> >>>
>>>> >>> So please let me know if there is a ticket that you think is
>>>> essential in
>>>> >>> 3.5.9!
>>>> >>>
>>>> >>> Thank you all.
>>>> >>>
>>>> >>> -Norbert
>>>> >>>
>>>> >>
>>>>
>>>>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

I reverted the patch from 3.5, it seems even after fixing up the code
issues the test on inconsistency fails due to the leader not seeing the
same znodes.


On Fri, Nov 13, 2020 at 11:51 AM Norbert Kalmar 
wrote:

> The patch broke 3.5 branch btw, I'm already looking into it. Sorry.
>
> On Fri, Nov 13, 2020 at 11:48 AM Norbert Kalmar 
> wrote:
>
>> Sure, here is the 3.5.9 labeled jiras:
>> https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12348201
>>
>> Or tweak this search as you see fit:
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%203.5.9%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>>
>> -Norbert
>>
>> On Thu, Nov 12, 2020 at 4:51 PM Andor Molnar  wrote:
>>
>>> Hi Norbert,
>>>
>>> Thanks for volunteering as RM.
>>> Would you please provide a list of pending tickets either directly here
>>> or a link with the right search in Jira?
>>>
>>> Andor
>>>
>>>
>>>
>>> > On 2020. Nov 12., at 16:12, Norbert Kalmar
>>>  wrote:
>>> >
>>> > ZOOKEEPER-3911 merged to all 3 active branches.
>>> > 3.5 did not pick clean, I will do further testing to verify the patch.
>>> > After that I will start the release process for 3.5.9, as no request
>>> has
>>> > arrived for anything else to be included.
>>> >
>>> > -Norbert
>>> >
>>> > On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar 
>>> wrote:
>>> >
>>> >> Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
>>> >> backport to 3.5 IMHO. Any thoughts on this?
>>> >> PR: https://github.com/apache/zookeeper/pull/1445
>>> >>
>>> >> -Norbert
>>> >>
>>> >> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar 
>>> wrote:
>>> >>
>>> >>> Hello fellow ZK contributors and enthusiasts!
>>> >>>
>>> >>> Looks like the time has come for a new release on the 3.5 branch.
>>> >>> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
>>> >>> completed.
>>> >>>
>>> >>> There are 31 major patch in progress. I checked them, looks like all
>>> of
>>> >>> them are pushed to the next release from several versions prior. I
>>> will
>>> >>> check the PRs on them, but I don't see any that is a must have to
>>> include
>>> >>> in 3.5.9.
>>> >>>
>>> >>> So please let me know if there is a ticket that you think is
>>> essential in
>>> >>> 3.5.9!
>>> >>>
>>> >>> Thank you all.
>>> >>>
>>> >>> -Norbert
>>> >>>
>>> >>
>>>
>>>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

The patch broke 3.5 branch btw, I'm already looking into it. Sorry.

On Fri, Nov 13, 2020 at 11:48 AM Norbert Kalmar 
wrote:

> Sure, here is the 3.5.9 labeled jiras:
> https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12348201
>
> Or tweak this search as you see fit:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%203.5.9%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>
> -Norbert
>
> On Thu, Nov 12, 2020 at 4:51 PM Andor Molnar  wrote:
>
>> Hi Norbert,
>>
>> Thanks for volunteering as RM.
>> Would you please provide a list of pending tickets either directly here
>> or a link with the right search in Jira?
>>
>> Andor
>>
>>
>>
>> > On 2020. Nov 12., at 16:12, Norbert Kalmar 
>> wrote:
>> >
>> > ZOOKEEPER-3911 merged to all 3 active branches.
>> > 3.5 did not pick clean, I will do further testing to verify the patch.
>> > After that I will start the release process for 3.5.9, as no request has
>> > arrived for anything else to be included.
>> >
>> > -Norbert
>> >
>> > On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar 
>> wrote:
>> >
>> >> Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
>> >> backport to 3.5 IMHO. Any thoughts on this?
>> >> PR: https://github.com/apache/zookeeper/pull/1445
>> >>
>> >> -Norbert
>> >>
>> >> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar 
>> wrote:
>> >>
>> >>> Hello fellow ZK contributors and enthusiasts!
>> >>>
>> >>> Looks like the time has come for a new release on the 3.5 branch.
>> >>> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
>> >>> completed.
>> >>>
>> >>> There are 31 major patch in progress. I checked them, looks like all
>> of
>> >>> them are pushed to the next release from several versions prior. I
>> will
>> >>> check the PRs on them, but I don't see any that is a must have to
>> include
>> >>> in 3.5.9.
>> >>>
>> >>> So please let me know if there is a ticket that you think is
>> essential in
>> >>> 3.5.9!
>> >>>
>> >>> Thank you all.
>> >>>
>> >>> -Norbert
>> >>>
>> >>
>>
>>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

Sure, here is the 3.5.9 labeled jiras:
https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12348201

Or tweak this search as you see fit:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%203.5.9%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

-Norbert

On Thu, Nov 12, 2020 at 4:51 PM Andor Molnar  wrote:

> Hi Norbert,
>
> Thanks for volunteering as RM.
> Would you please provide a list of pending tickets either directly here or
> a link with the right search in Jira?
>
> Andor
>
>
>
> > On 2020. Nov 12., at 16:12, Norbert Kalmar 
> wrote:
> >
> > ZOOKEEPER-3911 merged to all 3 active branches.
> > 3.5 did not pick clean, I will do further testing to verify the patch.
> > After that I will start the release process for 3.5.9, as no request has
> > arrived for anything else to be included.
> >
> > -Norbert
> >
> > On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar 
> wrote:
> >
> >> Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
> >> backport to 3.5 IMHO. Any thoughts on this?
> >> PR: https://github.com/apache/zookeeper/pull/1445
> >>
> >> -Norbert
> >>
> >> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar 
> wrote:
> >>
> >>> Hello fellow ZK contributors and enthusiasts!
> >>>
> >>> Looks like the time has come for a new release on the 3.5 branch.
> >>> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
> >>> completed.
> >>>
> >>> There are 31 major patch in progress. I checked them, looks like all of
> >>> them are pushed to the next release from several versions prior. I will
> >>> check the PRs on them, but I don't see any that is a must have to
> include
> >>> in 3.5.9.
> >>>
> >>> So please let me know if there is a ticket that you think is essential
> in
> >>> 3.5.9!
> >>>
> >>> Thank you all.
> >>>
> >>> -Norbert
> >>>
> >>
>
>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

ZOOKEEPER-3911 merged to all 3 active branches.
3.5 did not pick clean, I will do further testing to verify the patch.
After that I will start the release process for 3.5.9, as no request has
arrived for anything else to be included.

-Norbert

On Mon, Nov 9, 2020 at 5:15 PM Norbert Kalmar  wrote:

> Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
> backport to 3.5 IMHO. Any thoughts on this?
> PR: https://github.com/apache/zookeeper/pull/1445
>
> -Norbert
>
> On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar  wrote:
>
>> Hello fellow ZK contributors and enthusiasts!
>>
>> Looks like the time has come for a new release on the 3.5 branch.
>> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
>> completed.
>>
>> There are 31 major patch in progress. I checked them, looks like all of
>> them are pushed to the next release from several versions prior. I will
>> check the PRs on them, but I don't see any that is a must have to include
>> in 3.5.9.
>>
>> So please let me know if there is a ticket that you think is essential in
>> 3.5.9!
>>
>> Thank you all.
>>
>> -Norbert
>>
>

Re: Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

Correction, Michael's patch for ZOOKEEPER-3911 would be also good to
backport to 3.5 IMHO. Any thoughts on this?
PR: https://github.com/apache/zookeeper/pull/1445

-Norbert

On Mon, Nov 9, 2020 at 12:02 PM Norbert Kalmar  wrote:

> Hello fellow ZK contributors and enthusiasts!
>
> Looks like the time has come for a new release on the 3.5 branch.
> Currently it has 4 blocker (1 which is a CVE) and 2 critical fixes
> completed.
>
> There are 31 major patch in progress. I checked them, looks like all of
> them are pushed to the next release from several versions prior. I will
> check the PRs on them, but I don't see any that is a must have to include
> in 3.5.9.
>
> So please let me know if there is a ticket that you think is essential in
> 3.5.9!
>
> Thank you all.
>
> -Norbert
>

Releasing 3.5.9 - Pending Patches

[Norbert Kalmar]

Hello fellow ZK contributors and enthusiasts!

Looks like the time has come for a new release on the 3.5 branch. Currently
it has 4 blocker (1 which is a CVE) and 2 critical fixes completed.

There are 31 major patch in progress. I checked them, looks like all of
them are pushed to the next release from several versions prior. I will
check the PRs on them, but I don't see any that is a must have to include
in 3.5.9.

So please let me know if there is a ticket that you think is essential in
3.5.9!

Thank you all.

-Norbert

Re: ZooKeeper 3.5.9 release

[Norbert Kalmar]

Hi Damien,

I don't think we have a particular JQL, I just use the specific release
link, for 3.7.0:
https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12346617
(I think you also linked this one, but I just noticed in the guide as well
the link get invalid maybe?)

Anyway, as you mentioned, any open, blocker issue should be checked. Then
we ask on this thread if anyone has a ticket that is wanted in the target
release.

I will take up 3.5 release then.

Regards,
Norbert

On Thu, Nov 5, 2020 at 10:29 AM Damien Diederen 
wrote:

>
> Hi Enrico, Norbert, all,
>
> Okay; I'll take care of the big-round-number release, then :)
>
> I don't think we have a target date or specific objectives for 3.7, do
> we?  I am going to prepare an email to dev asking for comments and/or
> concerns.  In the meantime, looking at this result set:
>
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-1?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20AND%20affectedVersion%20in%20(EMPTY%2C%20unreleasedVersions()%2C%203.7.0)%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>
> I only see two remaining blockers, and they have not been assigned a
> specific version.  (I have resolved ZOOKEEPER-3980, as the merged PR
> seems to be satisfactory.)
>
> Would you have a specific "JQL" query you use for preparing releases,
> btw.?  I have noted Norbert's pointers, but they're about the release
> process per se; so far, I haven't found anything obviously relevant to
> release preparation in Confluence.
>
> Cheers, -D
>
>
>
>
> Enrico Olivelli  writes:
> > 3.7.0 is simpler :-) it is just a matter of using the
> maven-release-plugin
> >
> > 3.5 branch was not totally migrated to Maven style :-)
> >
> > If I were you I would choose 3.7.0
> >
> > Enrico
> >
> > Il giorno mer 4 nov 2020 alle ore 21:58 Damien Diederen <
> > ddiede...@sinenomine.net> ha scritto:
> >
> >>
> >> Hi Norbert,
> >>
> >> I'm torn!
> >>
> >> I mentioned 3.5.9 because Enrico was looking for a volunteer.  If you're
> >> really, really up for it, it may be more efficient for you to take care
> >> of this one as I suppose you still have the 3.5.7 "runbook" in mind.
> >>
> >> But on the other hand, I don't want to prevent you from cutting a
> >> release using the new flow, for a change.  Moreover, cutting a 3.7.0
> >> seems a bit overly ambitious for somebody who just dropped in…
> >>
> >> Which one do you think is the least risky?  Or perhaps we should just
> >> flip a coin?  If you see tails, I take care of 3.5.9 :)
> >>
> >> Cheers, -D
> >>
> >>
> >>
> >> Norbert Kalmar  writes:
> >> > Just saw the other thread on 3.7.0 release.
> >> > Damien: Whichever you would like to do, we're happy to help.
> >> > For 3.5.9 see
> >> >
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=109451613
> >> > For 3.7.0 see
> >> >
> >>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToRelease+using+maven+release+plugin
> >> >
> >> > And if you have any questions let us know, it is also a good feedback
> if
> >> > this 2 guide is well written or there are areas of improvement :)
> >> >
> >> > Let us know which release you would like to drive. I can drive the
> other.
> >> >
> >> > - Norbert
> >> >
> >> > On Wed, Nov 4, 2020 at 2:52 PM Norbert Kalmar 
> >> wrote:
> >> >
> >> >> Hi all,
> >> >>
> >> >> I can drive the 3.5.9 release, it's been a while since I did one :)
> >> >>
> >> >> - Norbert
> >> >>
> >> >> On Sun, Nov 1, 2020 at 10:06 AM Enrico Olivelli  >
> >> >> wrote:
> >> >>
> >> >>> Rajini,
> >> >>> Sorry for so late reply.
> >> >>>
> >> >>> Do you have a way to update to 3.6.2? If you want to upgrade to
> 3.5.9
> >> the
> >> >>> effort would be very like updating to 3.6.2.
> >> >>>
> >> >>> Btw I think it would be worth to cut a new release, I am adding dev@
> >> to
> >> >>> my
> >> >>> answer in order to start a discussion.
> >> >>>
> >> >>> We should just find a volounteer for driving the release.
> >> >>>
> >> >>> Enrico
> >> >>>
> >> >>> Il Ven 30 Ott 2020, 18:31 Rajini Sivaram 
> ha
> >> >>> scritto:
> >> >>>
> >> >>> > Hi,
> >> >>> >
> >> >>> > Is there a timeline available for the 3.5.9 bug fix release which
> >> would
> >> >>> > contain the fix for
> >> >>> https://issues.apache.org/jira/browse/ZOOKEEPER-3829?
> >> >>> >
> >> >>> > Thank you,
> >> >>> >
> >> >>> > Rajini
> >> >>> >
> >> >>>
> >> >>
> >>
>

Re: ZooKeeper 3.5.9 release

[Norbert Kalmar]

Just saw the other thread on 3.7.0 release.
Damien: Whichever you would like to do, we're happy to help.
For 3.5.9 see
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=109451613
For 3.7.0 see
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToRelease+using+maven+release+plugin

And if you have any questions let us know, it is also a good feedback if
this 2 guide is well written or there are areas of improvement :)

Let us know which release you would like to drive. I can drive the other.

- Norbert

On Wed, Nov 4, 2020 at 2:52 PM Norbert Kalmar  wrote:

> Hi all,
>
> I can drive the 3.5.9 release, it's been a while since I did one :)
>
> - Norbert
>
> On Sun, Nov 1, 2020 at 10:06 AM Enrico Olivelli 
> wrote:
>
>> Rajini,
>> Sorry for so late reply.
>>
>> Do you have a way to update to 3.6.2? If you want to upgrade to 3.5.9 the
>> effort would be very like updating to 3.6.2.
>>
>> Btw I think it would be worth to cut a new release, I am adding dev@ to
>> my
>> answer in order to start a discussion.
>>
>> We should just find a volounteer for driving the release.
>>
>> Enrico
>>
>> Il Ven 30 Ott 2020, 18:31 Rajini Sivaram  ha
>> scritto:
>>
>> > Hi,
>> >
>> > Is there a timeline available for the 3.5.9 bug fix release which would
>> > contain the fix for
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3829?
>> >
>> > Thank you,
>> >
>> > Rajini
>> >
>>
>

Re: ZooKeeper 3.5.9 release

[Norbert Kalmar]

Hi all,

I can drive the 3.5.9 release, it's been a while since I did one :)

- Norbert

On Sun, Nov 1, 2020 at 10:06 AM Enrico Olivelli  wrote:

> Rajini,
> Sorry for so late reply.
>
> Do you have a way to update to 3.6.2? If you want to upgrade to 3.5.9 the
> effort would be very like updating to 3.6.2.
>
> Btw I think it would be worth to cut a new release, I am adding dev@ to my
> answer in order to start a discussion.
>
> We should just find a volounteer for driving the release.
>
> Enrico
>
> Il Ven 30 Ott 2020, 18:31 Rajini Sivaram  ha
> scritto:
>
> > Hi,
> >
> > Is there a timeline available for the 3.5.9 bug fix release which would
> > contain the fix for https://issues.apache.org/jira/browse/ZOOKEEPER-3829
> ?
> >
> > Thank you,
> >
> > Rajini
> >
>

Re: [ANNOUNCE] New Commiter: Damien Diederen

[Norbert Kalmar]

Congratulations Damien!

- Norbert

On Wed, Oct 28, 2020 at 4:48 PM Damien Diederen 
wrote:

>
> Thank you, everybody!
>
> I hope to make good use of this newly gained trust to contribute useful
> efforts and patches to this very hospitable community.  One thing is for
> sure: it is not lacking in role models to look up to!
>
> Thank you, Enrico, for the nice "onboarding" and introduction.
>
> A small addition: while I contributed the C SASL client in its current
> form, I must say that I relied very heavily on Tom Klonikowski's earlier
> efforts.  So thank you, Tom!
>
> (And yes, I understand and agree that it is the whole point :)
>
> Best regards,
> Damien Diederen
>
>
>
> Enrico Olivelli  writes:
> > The Project Management Committee (PMC) for Apache ZooKeeper
> > has invited Damien Diederen to become a committer and we are pleased
> > to announce that he has accepted.
> >
> > Damien contributed lots of improvements and bug fixes on Zookeeper C
> client
> > and he also participating in the community with good code reviews, and
> > discussions on our mailing lists.
> > For instance he is the author of the SASL support in the C client
> > https://github.com/apache/zookeeper/pull/1134
> >
> > Being a committer enables easier contribution to the
> > project since there is no need to go via the patch
> > submission process. This should enable better productivity.
> > Being a PMC member enables assistance with the management
> > and to guide the direction of the project.
> >
> > Congratulations Damien !
> >
> > Enrico Olivelli
>

Re: ApacheCon Bug Bash

[Norbert Kalmar]

I created one: ZOOKEEPER-3968
When running commit script we can add this to the commit message.

- Norbert

On Fri, Oct 9, 2020 at 2:03 PM Enrico Olivelli  wrote:

> Il Ven 9 Ott 2020, 13:51 Norbert Kalmar  ha
> scritto:
>
> > We could create an umbrella jira for the ones that have not been updated,
> > something like "BugBash fixes" and just commit them under it.
> >
>
> Works for me for trivial patches.
> Good idea Norbert
>
>
> Enrico
>
>
> > On Sat, Oct 3, 2020 at 10:01 AM Enrico Olivelli 
> > wrote:
> >
> > > Tom
> > > Generally the only requirement is that you create a JIRA and report the
> > > JIRA od in the commit message.
> > > Creating a JIRA also helps the contributor to understand/classify the
> > > problem and the proposed solution.
> > >
> > > I forgot to add that there should be some statement in the PR comments
> > that
> > > tells that the contributor agrees to the ASLv2 terms, at least for the
> > > first pull requests.
> > >
> > > I really appreciate this work of MuseDev
> > > Let's try to make it work the best as possible
> > >
> > > Enrico
> > >
> > > Il Sab 3 Ott 2020, 07:18 Tom DuBuisson  ha scritto:
> > >
> > > > > Perhaps Muse.dev can work with us to automate the process of
> creating
> > > > tickets for the issues that were raised?
> > > >
> > > > We don't have any way to automatically open matching JIRA issues but
> > can
> > > > certainly guide the new contributors to the right steps.  I'll check
> in
> > > > with them on their respective PRs.
> > > >
> > > > On Fri, Oct 2, 2020 at 8:34 AM Rich Bowen 
> wrote:
> > > >
> > > >> I know you're not asking me, but with my Community Development hat
> > on, I
> > > >> strenuously encourage you to view this as an opportunity to bring on
> > new
> > > >> contributors, and couch your response accordingly. Anything that
> comes
> > > >> across as scolding them for Doing It Wrong is going to leave a bad
> > taste
> > > >> and possibly lose new contributors, particularly when we invited
> them
> > to
> > > >> participate in this process. We did invite them, and we did point
> them
> > > >> to the issues, via Muse.dev. Perhaps Muse.dev can work with us to
> > > >> automate the process of creating tickets for the issues that were
> > > raised?
> > > >>
> > > >> On 10/2/20 11:26 AM, Enrico Olivelli wrote:
> > > >> > Hey !
> > > >> > it looks like the Bug bash has brought a few Pull Requests
> > > >> > https://github.com/apache/zookeeper/pulls
> > > >> >
> > > >> > Unfortunately they are not following the contribution guidelines
> > (for
> > > >> > instance there is no associated JIRA)
> > > >> >
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
> > > -
> > > >> >
> > > >> > Most of the PR are about trivial fixes, I am not sure if a JIRA is
> > > >> deserved.
> > > >> >
> > > >> > What should we do?
> > > >> > My proposal is to ping the contributor in order to obey the guide
> > and
> > > >> > then finally accept the patches, as Micheal Han did in this patch
> > > >> > https://github.com/apache/zookeeper/pull/1470
> > > >> >
> > > >> > I don't want to see that patches remaining on github as low
> hanging
> > > >> > fruit, so it is better that we decide how to work on them,
> > > >> > another option is to close them as invalid (It would be a pity
> IMHO)
> > > >> >
> > > >> > Enrico
> > > >> >
> > > >> >
> > > >> >
> > > >> > Il giorno lun 28 set 2020 alle ore 15:03 Tom DuBuisson <
> > > to...@muse.dev
> > > >> > <mailto:to...@muse.dev>> ha scritto:
> > > >> >
> > > >> > Enrico,
> > > >> > That sounds great.  We'll get the repo activated.
> > > >> >
> > > >> > Tom
> > > >> >
> > > >> >
> > > >> > On Sun, Sep 27, 2020, 11:11 PM Enrico Olivelli <
> > > eolive...@gmail.com
> > > >> &

Re: ApacheCon Bug Bash

[Norbert Kalmar]

We could create an umbrella jira for the ones that have not been updated,
something like "BugBash fixes" and just commit them under it.

On Sat, Oct 3, 2020 at 10:01 AM Enrico Olivelli  wrote:

> Tom
> Generally the only requirement is that you create a JIRA and report the
> JIRA od in the commit message.
> Creating a JIRA also helps the contributor to understand/classify the
> problem and the proposed solution.
>
> I forgot to add that there should be some statement in the PR comments that
> tells that the contributor agrees to the ASLv2 terms, at least for the
> first pull requests.
>
> I really appreciate this work of MuseDev
> Let's try to make it work the best as possible
>
> Enrico
>
> Il Sab 3 Ott 2020, 07:18 Tom DuBuisson  ha scritto:
>
> > > Perhaps Muse.dev can work with us to automate the process of creating
> > tickets for the issues that were raised?
> >
> > We don't have any way to automatically open matching JIRA issues but can
> > certainly guide the new contributors to the right steps.  I'll check in
> > with them on their respective PRs.
> >
> > On Fri, Oct 2, 2020 at 8:34 AM Rich Bowen  wrote:
> >
> >> I know you're not asking me, but with my Community Development hat on, I
> >> strenuously encourage you to view this as an opportunity to bring on new
> >> contributors, and couch your response accordingly. Anything that comes
> >> across as scolding them for Doing It Wrong is going to leave a bad taste
> >> and possibly lose new contributors, particularly when we invited them to
> >> participate in this process. We did invite them, and we did point them
> >> to the issues, via Muse.dev. Perhaps Muse.dev can work with us to
> >> automate the process of creating tickets for the issues that were
> raised?
> >>
> >> On 10/2/20 11:26 AM, Enrico Olivelli wrote:
> >> > Hey !
> >> > it looks like the Bug bash has brought a few Pull Requests
> >> > https://github.com/apache/zookeeper/pulls
> >> >
> >> > Unfortunately they are not following the contribution guidelines (for
> >> > instance there is no associated JIRA)
> >> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
> -
> >> >
> >> > Most of the PR are about trivial fixes, I am not sure if a JIRA is
> >> deserved.
> >> >
> >> > What should we do?
> >> > My proposal is to ping the contributor in order to obey the guide and
> >> > then finally accept the patches, as Micheal Han did in this patch
> >> > https://github.com/apache/zookeeper/pull/1470
> >> >
> >> > I don't want to see that patches remaining on github as low hanging
> >> > fruit, so it is better that we decide how to work on them,
> >> > another option is to close them as invalid (It would be a pity IMHO)
> >> >
> >> > Enrico
> >> >
> >> >
> >> >
> >> > Il giorno lun 28 set 2020 alle ore 15:03 Tom DuBuisson <
> to...@muse.dev
> >> > <mailto:to...@muse.dev>> ha scritto:
> >> >
> >> > Enrico,
> >> > That sounds great.  We'll get the repo activated.
> >> >
> >> > Tom
> >> >
> >> >
> >> > On Sun, Sep 27, 2020, 11:11 PM Enrico Olivelli <
> eolive...@gmail.com
> >> > <mailto:eolive...@gmail.com>> wrote:
> >> >
> >> >  > Tom
> >> >  > Overall I think that we can move forward.
> >> >  >
> >> >  > This thread has been around for a while, there are no
> objections,
> >> > every
> >> >  > question has been answered.
> >> >  >
> >> >  > Thank you very much
> >> >  >
> >> >  > I hope this activity will help in growing Zookeeper project
> both
> >> > in code
> >> >  > quality and with more contributions, that is to help the
> >> > community to grow.
> >> >  >
> >> >  > Best regards
> >> >  >
> >> >  > Enrico
> >> >  >
> >> >  > Il Lun 28 Set 2020, 01:27 Tom DuBuisson  >> > <mailto:to...@muse.dev>> ha scritto:
> >> >  >
> >> >  > > Norbert,
> >> >  > >
> >> >  > > Yes, you understand that correctly.  And those analyzers are
>

Re: ZooKeeper 3.6.2 release procedure finished - time to drop 3.4 from dist ?

[Norbert Kalmar]

Thanks Enrico, it's merged.

- Norbert

On Mon, Sep 28, 2020 at 7:25 PM Enrico Olivelli  wrote:

> This is the PR for the website
> https://github.com/apache/zookeeper/pull/1468
>
> please anyone merge
> IMO there is no need for a JIRA
>
> Enrico
>
> Il giorno lun 21 set 2020 alle ore 16:56 Enrico Olivelli <
> eolive...@gmail.com> ha scritto:
>
> > Andor,
> > sorry for the late reply.
> >
> > I am hesitating because we should also update the website and we have to
> > coordinate a bit
> > https://zookeeper.apache.org/releases.html
> >
> > I am going to fix the website as first step, then when I see that the
> > website is updated (no more links in the download page)
> > I will remove the files from "dist".
> >
> > Then we can make the announcement.
> >
> > If anyone objects please chime in immediately
> >
> > Enrico
> >
> >
> > Il giorno ven 11 set 2020 alle ore 21:44 Andor Molnar 
> > ha scritto:
> >
> >> Hi Enrico,
> >>
> >> Sounds like a good plan for me.
> >> Go ahead please and drop 3.4 from the dist folder.
> >>
> >> Do we want to make an official announcement (again) about the EOL of
> 3.4?
> >> We already passed the official date and there was an announcement
> already
> >> in the past, so I’m not keen on doing it.
> >>
> >> Andor
> >>
> >>
> >>
> >> > On 2020. Sep 11., at 8:12, Enrico Olivelli 
> wrote:
> >> >
> >> > Tamaas
> >> >
> >> > Il giorno gio 10 set 2020 alle ore 12:59 Tamas Penzes
> >> >  ha scritto:
> >> >
> >> >> Congrats team.
> >> >>
> >> >> May I ask you when do you plan to move the stable pointer from 3.5.x
> to
> >> >> 3.6.x?
> >> >>
> >> >
> >> > As far as I can remember there is no written policy.
> >> > We had users running with 3.5 "BETA" for years, now we see that users
> >> start
> >> > using 3.6 (and we see bug reports and we already cut 2 point releases)
> >> >
> >> > I am switching to 3.6 in all of the projects I am contributing to.
> >> >
> >> > Probably we will switch the 'stable' label to 3.6.x when we release
> >> 3.7.0
> >> > then 3.5 will be far away from the current development branch (it will
> >> be
> >> > 3.8 or even 4.0...)
> >> >
> >> >
> >> > Enrico
> >> >
> >> >
> >> >>
> >> >> Thanks, Tamaas
> >> >>
> >> >> On Thu, Sep 10, 2020 at 12:50 PM Enrico Olivelli <
> eolive...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi,
> >> >>> I have completed the release procedure for 3.6.2.
> >> >>> We have to drop stuff from non active branches from the dist area
> >> >>> according to ASF rules.
> >> >>>
> >> >>> Do we have to remove 3.4.x from the "dist" area ?
> >> >>> https://dist.apache.org/repos/dist/release/zookeeper/
> >> >>>
> >> >>> IIRC we decided to sent it to EOL On June 2020
> >> >>>
> >> >>> My proposal is to drop it
> >> >>>
> >> >>> Thoughts ?
> >> >>>
> >> >>> Enrico
> >> >>>
> >> >>
> >>
> >>
>

Re: ApacheCon Bug Bash

[Norbert Kalmar]

Hello Tom,

+1 on the initiative, thanks for bringing this to our attention.

If I understand correctly, there will be no disclosed security issues which
cannot be found with open source static analyzers.

Regards,
Norbert


On Sun, Sep 27, 2020 at 8:23 AM Szalay-Bekő Máté 
wrote:

> Hello Guys,
>
> In general I like the idea, but unfortunately I can not really participate
> (either in the coding or in the review) as I have a few important projects
> close to deadline at the moment.
>
> My only concern is with the security bugs, which I don't like to be openly
> reported before publishing a release with the fix. But for any other kind
> of bugfixes / improvements, I am very positive with the initiative.
>
>
> Best regards,
> Mate
>
> On Sun, Sep 27, 2020, 07:06 Tom DuBuisson  wrote:
>
> > Enrico et al,
> >
> > Are there other thoughts on this?  It would be great to get setup before
> > the bash actually begins.  Enrico, lacking other voices would you like to
> > make a final call?
> >
> > -Tom
> >
> > On Thu, Sep 24, 2020 at 3:30 AM Enrico Olivelli 
> > wrote:
> >
> > > Tom,
> > > Personally I am +1 with this proposal. Thanks for your clarifications.
> > >
> > > But we should ear opinions from other people in this list
> > >
> > >
> > > Enrico
> > >
> > > Il giorno mer 23 set 2020 alle ore 23:51 Tom DuBuisson  >
> > ha
> > > scritto:
> > >
> > > > Enrico,
> > > >
> > > > On the topic security issues and reporting:  Muse's default
> > configuration
> > > > is open source tools and here it is run on open source projects.  The
> > > > results are thus already available publicly (in this case from FSB,
> > > Infer,
> > > > and Error Prone).  Muse doesn't post anything to GitHub except in the
> > > case
> > > > of pull requests and then only if the bug is deemed to have been
> > > > "introduced" as part of the PR - meaning it shouldn't be a
> > vulnerability
> > > in
> > > > currently shipped software.
> > > >
> > > > If there are desires or proposals about more control over bug reports
> > in
> > > a
> > > > convenient, configurable, manner then we'd really like to dig in and
> > hear
> > > > how to help.  In case there is more discussion on this point I'm
> CCing
> > > > Andrew who leads Muse's product design.
> > > >
> > > > -Tom
> > > >
> > > > On Wed, Sep 23, 2020 at 1:09 PM Enrico Olivelli  >
> > > > wrote:
> > > >
> > > > > Il Mer 23 Set 2020, 19:02 Tom DuBuisson  ha
> scritto:
> > > > >
> > > > > > Enrico,
> > > > > >
> > > > > > The Muse App requires two main abilities.  First is events, such
> as
> > > > > > notification when pull requests are opened or updated.  Second is
> > > > > > permission to post comments (which is always possible for humans
> > but
> > > > more
> > > > > > tightly controlled when the poster authenticates as a github
> > > > > application).
> > > > > > The repository being public has allowed us to run the app and
> > observe
> > > > > > ErrorProne, Infer, and FindSecBugs all run out of the box and
> > without
> > > > > > custom configuration.
> > > > > >
> > > > >
> > > > > Makes sense.
> > > > >
> > > > > One last question from my side
> > > > > What about security issues?
> > > > > Our policy is to have them reported to
> secur...@zookeeper.apache.org
> > > > > before
> > > > > public disclosure
> > > > >
> > > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > > > Cheers,
> > > > > > Tom
> > > > > >
> > > > > > On Wed, Sep 23, 2020 at 6:35 AM Enrico Olivelli <
> > eolive...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Il Mer 23 Set 2020, 00:44 Tom DuBuisson  ha
> > > scritto:
> > > > > > >
> > > > > > > > Zookeeper Developers,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > As part of our sponsorship of ApacheCon, our company MuseDev
> is
> > > > > doing a
> > > > > > > Bug
> > > > > > > > Bash for select Apache projects. We'll bring members of the
> > > > ApacheCon
> > > > > > > > community together to find and fix a range of security and
> > > > > performance
> > > > > > > bugs
> > > > > > > > during the conference, and gameify the experience with
> teams, a
> > > > > > > > leaderboard, and prizes. The bash is open to everyone whether
> > > > > attending
> > > > > > > the
> > > > > > > > conference or not, and our whole dev team will also be
> > > > participating
> > > > > to
> > > > > > > > help fix as many bugs as we can.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > We're seeding the bug list with results from Muse, our code
> > > > analysis
> > > > > > > > platform, which runs as a Github App and comments on possible
> > > bugs
> > > > as
> > > > > > > part
> > > > > > > > of the pull request workflow.  Here's an example of what it
> > looks
> > > > > like:
> > > > > > > >
> > > > > > > >
> > https://github.com/curl/curl/pull/5971#discussion_r490252196
> > > > > > > > 
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > We explored a number of 

Re: [VOTE] Apache ZooKeeper 3.6.2 candidate 1

[Norbert Kalmar]

Thanks Enrico, +1 (non-binding)from me.

- Built the source, run tests
- Run ZK and tried basic commands using zkCli
- files check in tarball


About website issue: generareted website can be fixed manually, we did this
earlier, I don't think it's a blocker.
Changelog, same, it should be fixed manually for the website

@Tison: we released these modules last time, we can have a vote to not
include it in the next release, but definitely not a blocker for this RC.
But good point, we talked about excluding some (mainly contrib) modules.

Regards,
Norbert

On Sun, Sep 6, 2020 at 5:10 AM tison  wrote:

> Besides, shall we release the following modules?
>
> - zookeeper-contrib-rest
> - zookeeper-it
>
> Best,
> tison.
>
>
> tison  于2020年9月6日周日 上午10:57写道：
>
> > jump to
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-1/website/apidocs/zookeeper-server/index.html
> >
> > from sidebar "Developer>API Docs" returns 404, it is a testing
> environment
> > issue?
> >
> > Best,
> > tison.
> >
> >
> > Michael Han  于2020年9月5日周六 上午5:55写道：
> >
> >> +1, with two minor issues:
> >>
> >> There is one unit test deterministically failing for me locally on mac:
> >> MultipleAddressesTest.testGetValidAddressWithNotValid Expected
> exception:
> >> java.net.NoRouteToHostException
> >>
> >> Also missing a couple of items in release notes: ZOOKEEPER-3794,
> >> ZOOKEEPER-3797, ZOOKEEPER-3813
> >>
> >> Neither is blocker IMO but would be nice to update the release note
> >> considering one of the missing items is CVE related fix.
> >>
> >> On Fri, Sep 4, 2020 at 1:04 PM Patrick Hunt  wrote:
> >>
> >> > +1. xsum/sig validate. RAT ran clean. Was able to build and do manual
> >> > testing with various ensemble sizes successfully. lgtm.
> >> >
> >> > Patrick
> >> >
> >> > On Fri, Sep 4, 2020 at 6:01 AM Enrico Olivelli 
> >> > wrote:
> >> >
> >> > > This is a release candidate for 3.6.2.
> >> > >
> >> > > It is a minor release and it fixes a few critical issues and brings
> a
> >> few
> >> > > dependencies upgrades.
> >> > >
> >> > > The full release notes is available at:
> >> > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12347809
> >> > >
> >> > > *** Please download, test and vote by September 7th 2020, 23:59
> UTC+0.
> >> > ***
> >> > >
> >> > > Source files:
> >> > > https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-1/
> >> > >
> >> > > Maven staging repo:
> >> > >
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachezookeeper-1061/
> >> > >
> >> > > The release candidate tag in git to be voted upon: release-3.6.2-1
> >> > > https://github.com/apache/zookeeper/tree/release-3.6.2-1
> >> > >
> >> > > ZooKeeper's KEYS file containing PGP keys we use to sign the
> release:
> >> > > https://www.apache.org/dist/zookeeper/KEYS
> >> > >
> >> > > The staging version of the website is:
> >> > >
> >> >
> >>
> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-1/website/
> >> > >
> >> > > Should we release this candidate?
> >> > > Enrico Olivelli
> >> > >
> >> >
> >>
> >
>

Re: Code coverage

[Norbert Kalmar]

Thanks Enrico, I will check out Coveralls.io, generating reports for
coverage impact on prs definitely sounds good.

Norbert

On Mon, Sep 7, 2020 at 10:37 PM Enrico Olivelli  wrote:

> Norbert
> I like this idea
>
> I have some experience with JaCoCo.
> For an OpenSource project like Zookeeper we should integrate it with
> services like CodeCov or Coveralls.io (I am using Coveralls for instance)
>
> Such tool will be useful for requiring a quality gate on pull requests and
> in order to increase code coverage in general.
>
> Zookeeper codebase is quite simple and it will work well.
>
> My proposal is to enable it on pull requests and link with Coveralls.io (or
> other service)
> This way we will easily have a report about the impact of each patch to
> code coverage.
> Default reports are useful out of the box.
> No need to set Coveralls.io report as blocker for merging a patch.
>
>
> Enrico
>
> Il Lun 7 Set 2020, 22:29 Norbert Kalmar  ha
> scritto:
>
> > Hi all,
> >
> > Just a quick email to raise awareness to code coverage in ZK.
> > We've seen demand for it, also found an old ticket about it:
> ZOOKEEPER-2266
> > I did a quick patch to integrate jacoco:
> > https://github.com/apache/zookeeper/pull/1451
> >
> > My question is, do you think this is useful, something we should have in
> > ZK?
> > It's by default off (the code coverage report generation that is).
> >
> > I know it has it's... perils. (Like when we launch a whole zk server for
> a
> > test and that covers a lot of code). But it's still something I see
> demand
> > for, to have some kind of report about this.
> >
> > Regards,
> > Norbert
> >
>

Code coverage

[Norbert Kalmar]

Hi all,

Just a quick email to raise awareness to code coverage in ZK.
We've seen demand for it, also found an old ticket about it: ZOOKEEPER-2266
I did a quick patch to integrate jacoco:
https://github.com/apache/zookeeper/pull/1451

My question is, do you think this is useful, something we should have in ZK?
It's by default off (the code coverage report generation that is).

I know it has it's... perils. (Like when we launch a whole zk server for a
test and that covers a lot of code). But it's still something I see demand
for, to have some kind of report about this.

Regards,
Norbert

Re: 4 letter words on secureClientPort

[Norbert Kalmar]

Hi Sankalp,

How are you trying to access zk to run 4lw commands? AFAIK nc(telnet) does
not support ssl.
You need something that supports ssl, like openssl s_clien [1] (but I
haven't actually tested this) or by invoking the java client.
But as you use secure communication, I would recommend using adminserver,
if you are on 3.5+ that is.

Regards,
Norbert

[1]
https://serverfault.com/questions/476068/can-netcat-talk-to-an-encrypted-port/476073

On Wed, Jul 22, 2020 at 11:42 PM Sankalp Bhatia 
wrote:

> Hi All,
>
> I am getting an exception while trying to ping the secureClientPort in my
> zookeeper server.
>
> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
> 737461740a
>
> Do the 4 letter words not work with the secureClientPort? Can I tweak some
> system settings to get those to work?
>
> Thanks,
> Sankalp
>

Re: Max data size per znode

[Norbert Kalmar]

Hi Pratyush,

Node size (more precisely jute-maxbuffer size) has quite a debate.
Currently it is 1MB by default. I've seen it used in production as high as
1GB! That is insane, and definitely not recommended, but if you up it to a
few MB it's fine IMHO. But these high numbers are mainly due to
getChildren() getting too big (Kafka and Hive have a tendency to run into
this). They usually still don't store big znodes. And that's a good thing,
since ZK is not an in-memory database.

As for how many znodes, again, depends on a few things (probably a lot
actually) and I've seen surprisingly high numbers here also. But there were
many observers used, and also it was tweaked to the setup they used.
Basically it depends on the hardware ZK is running on. First and foremost
you need to have sufficient memory. And of course disc and write capacity
to write snapshots and txn logs. A few thousands nodes are no biggie, but
if we're talking millions, that's probably a little too much.

So to sum it up, no exact numbers, but it's worth re-thinking your
use-case, if any of these numbers are abused.

Regards,
Norbert

On Fri, May 29, 2020 at 3:11 AM Pratyush Tarar 
wrote:

> Hello
>
> As understand the key-data combination stored in zookeeper nodes should not
> be too large (say < 1 MB).
>
> But is there a limit to the number of key-data combinations (each say < 1
> MB size) that can be stored in zookeeper node?
>
> Response appreciated.
>
> Pratyush
>

Re: how to understand lastProcessedZxid in Zookeeper's code

[Norbert Kalmar]

Hi, that't the last transaction applied to the datatree. It is a committed
proposal.

Regards,
Norbert

On Sat, May 9, 2020 at 6:30 AM hack 4096  wrote:

>  hi all, how to understand lastProcessedZxid in Zookeeper's code? is it
> committed proposal or uncommitted proposal?
> can anyone help on it? Thanks in advance.
>

Re: [VOTE] Apache ZooKeeper release 3.5.8 candidate 0

[Norbert Kalmar]

Great, thanks Máté!

+1 (non-binding)
Did my usual build and testing, verified signature (new public key in
KEYS), compared src release with git repository, checked license files -
all looks good to me.
I'm rooting for an RC0 release! :)

Regards,
Norbert

On Mon, May 4, 2020 at 6:22 PM Szalay-Bekő Máté 
wrote:

> This is a release candidate for 3.5.8, a bugfix release
> introducing bugfixes and improvements including:
> - compatibility with applications built against earlier 3.5 client
> libraries (restored a few non public APIs)
> - CVE fixes (update Netty to 4.1.48.Final and Jackson-databind to 2.10.3)
> - Fix several ZooKeeper leader-election bugs
> - Make sources buildable with JDK14
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346950
>
> *** Please download, test and vote by May 10th 2020, 23:59 UTC+0. ***
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.5.8-rc0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1059/
>
> The release candidate tag in git to be voted upon: release-3.5.8-rc0
> https://github.com/apache/zookeeper/tree/release-3.5.8-rc0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.5.8-rc0/webpage/
>
> Should we release this candidate?
>
> Mate Szalay-Beko
>

Re: [VOTE] Apache ZooKeeper release 3.6.1 candidate 1

[Norbert Kalmar]

Sure, I'll do the update today.
+1 (non-binding) from me (see my testing in my previous mail)

Thanks Enrico!

On Thu, Apr 23, 2020 at 11:20 AM Enrico Olivelli 
wrote:

> Il giorno mer 22 apr 2020 alle ore 16:14 Norbert Kalmar
>  ha scritto:
>
> > Only thing I found is that the bin has netty-codec-4.1.49 license file
> > while the jar included is 4.1.48. I think the license version has a typo
> in
> > the bugfix version. Not sure if it's a showstopper.
> >
>
> I don't consider it a showstopper.
>
> Do you have time to send a fix please ?
> This way if we have to roll out a new RC we can pick it up.
>
> We could anyhow update to 4.1.49.Final
> https://netty.io/news/2020/04/22/4-1-49-Final.html
>
> Enrico
>
>
> >
> > Otherwise LGTM:
> > - Signatures OK
> > - Compared to git and 3.6.0
> > - Compiled both on Mac (without C client) and Linux (with C client)
> > - Run tests (from src) and server (from src and bin tarball), connect
> with
> > client and run simple commands
> > - Spotbugs and checkstyle passed
> >
> > Regards,
> > Norbert
> >
> > On Wed, Apr 22, 2020 at 3:50 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using
> OpenJDK
> > > 8u242 and maven 3.6.0.
> > > - all the unit tests passed (both Java and C-client).
> > > - I also built and executed unit tests for zkpython
> > > - checkstyle and spotbugs passed
> > > - apache-rat passed
> > > - fatjar built
> > > - I executed a quick rolling-upgrade test from 3.5.7 to 3.6.1. (using
> > > https://github.com/symat/zk-rolling-upgrade-test)
> > >
> > > On Tue, Apr 21, 2020 at 5:20 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > This is a release candidate for 3.6.1.
> > > >
> > > > It is a bugfix release and it introduces a few bugfixes and new
> > features
> > > in
> > > > these areas:
> > > > - compatibility with applications built against 3.5 client libraries
> > > > (restored a few non public APIs)
> > > > - update Netty to 4.1.48.Final
> > > > - ability to pass configuration as file in zkCli for TLS config
> > > > - Add setKeepAlive support for NIOServerCnxn
> > > > - Fix server side request throttling
> > > >
> > > > The full release notes is available at:
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346764
> > > >
> > > > *** Please download, test and vote by April 14th 2020, 23:59 UTC+0.
> ***
> > > >
> > > > Source files:
> > > > https://people.apache.org/~eolivelli/zookeeper-3.6.1-candidate-1/
> > > >
> > > > Maven staging repo:
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1058/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.6.1-1
> > > > https://github.com/apache/zookeeper/tree/release-3.6.1-1
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > The staging version of the website is:
> > > >
> > >
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.1-candidate-1/website/
> > > >
> > > > Should we release this candidate?
> > > >
> > > > Enrico Olivelli
> > > >
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.6.1 candidate 1

[Norbert Kalmar]

Only thing I found is that the bin has netty-codec-4.1.49 license file
while the jar included is 4.1.48. I think the license version has a typo in
the bugfix version. Not sure if it's a showstopper.

Otherwise LGTM:
- Signatures OK
- Compared to git and 3.6.0
- Compiled both on Mac (without C client) and Linux (with C client)
- Run tests (from src) and server (from src and bin tarball), connect with
client and run simple commands
- Spotbugs and checkstyle passed

Regards,
Norbert

On Wed, Apr 22, 2020 at 3:50 PM Szalay-Bekő Máté 
wrote:

> +1 (non-binding)
>
> - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
> 8u242 and maven 3.6.0.
> - all the unit tests passed (both Java and C-client).
> - I also built and executed unit tests for zkpython
> - checkstyle and spotbugs passed
> - apache-rat passed
> - fatjar built
> - I executed a quick rolling-upgrade test from 3.5.7 to 3.6.1. (using
> https://github.com/symat/zk-rolling-upgrade-test)
>
> On Tue, Apr 21, 2020 at 5:20 PM Enrico Olivelli 
> wrote:
>
> > This is a release candidate for 3.6.1.
> >
> > It is a bugfix release and it introduces a few bugfixes and new features
> in
> > these areas:
> > - compatibility with applications built against 3.5 client libraries
> > (restored a few non public APIs)
> > - update Netty to 4.1.48.Final
> > - ability to pass configuration as file in zkCli for TLS config
> > - Add setKeepAlive support for NIOServerCnxn
> > - Fix server side request throttling
> >
> > The full release notes is available at:
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346764
> >
> > *** Please download, test and vote by April 14th 2020, 23:59 UTC+0. ***
> >
> > Source files:
> > https://people.apache.org/~eolivelli/zookeeper-3.6.1-candidate-1/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1058/
> >
> > The release candidate tag in git to be voted upon: release-3.6.1-1
> > https://github.com/apache/zookeeper/tree/release-3.6.1-1
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.1-candidate-1/website/
> >
> > Should we release this candidate?
> >
> > Enrico Olivelli
> >
>

Re: Cutting 3.6.1 HEADS UP !

[Norbert Kalmar]

Hi Enrico,

Thanks for driving this!

I managed to build HBase with ZooKeeper 3.5.7 having cherry-picked the
getRevision() patch. I know it's not 3.6.x, but I found the problem with
this 3.5.7 and fixed it according to this on 3.6 as well. So it should be
fine now.

- Norbert

On Thu, Apr 9, 2020 at 11:00 AM Enrico Olivelli  wrote:

> Hi,
> I am going to prepare a release candidate for ZooKeeper 3.6.1.
>
> There is no JIRA issue with fixVersion = 3.6.1 that is unresolved.
>
> I have tested a few projects that had compatibility issues and they are
> resolved (like Apache BookKeeper and other non OS projects in my company).
>
> I remember that Norbert pointed a problem with HBase client, that uses
> getRevision() method, it would be super great to have some feedback of
> compatibility of HBase with 3.6.1 client.
>
> I have deployed the snapshots to snapshots.apache.org, this way you can
> easily test your project in CI, even on Travis.
>
> I have created the work branch for release
> https://github.com/apache/zookeeper/tree/release-3.6.1
>
> Please ping me if you have any questions or concerns or you need to add new
> items.
>
> I will start the release procedure once I have self validated the status of
> that branch
>
> Stay tuned
>
> Enrico
>

Re: Contribs as separate git repos

[Norbert Kalmar]

Hi Christopher,

Yes, moving (most, not all actually) contrib projects to it's own repo is a
long standing discussion. As far as I can tell/remember, the community
agrees that we should move it.
Want to take up the task? :)

- Norbert

On Wed, Apr 8, 2020 at 9:36 AM Christopher  wrote:

> Hi ZK Devs,
>
> I am just curious if anybody has thought about, or perhaps discussed,
> the idea that the projects in the zookeeper-contrib folder should be
> in their own separate git repos?
>
> I'm asking because I've been looking a lot at the build, trying to
> find ways to improve it, and I think this might be a nice improvement
> to streamline the core ZooKeeper build. This can help side-projects
> succeed or fail on their own merits, rather than be bound to the core
> project so tightly, and it could make it easier for contributors to
> know where to contribute, by making each independent component smaller
> and easier to navigate through the code.
>
> Similarly, I think it would also be useful to move the code for the
> website into its own git repo, too.
>
> Anyway, it's just a thought. Let me know what you think.
>
> Thanks,
> Christopher
>
> P.S. I'm in the-asf.slack.com fairly often, especially in the
> #accumulo channel. Feel free to say hi in Slack any time.
>

Re: [DISCUSS] Sending 3.4 release line to End-Of-Life status

[Norbert Kalmar]

Well, for a long time we only had 1 line maintained, 3.4 basically. It is
just right now we have 3 lines, which is I think one too many. (Plus we
also have master to maintain additional to the active release lines, that's
4 active branches. Well, more or less, 3.4 is pretty inactive already).

I pretty much agree with Andor's points. My only comment is what testing do
we do to achieve number 3? (different version of client-server
connectability)

- Norbert

On Thu, Apr 2, 2020 at 10:14 PM Christopher  wrote:

> On Thu, Apr 2, 2020 at 12:20 PM Patrick Hunt  wrote:
> >
> > On Thu, Apr 2, 2020 at 8:15 AM Andor Molnar  wrote:
> >
> > > Alright. Not sure how to coordinate this properly, let’s try to discuss
> > > these 3 points individually.
> > >
> > > 1) EOL is 1st of June, 2020 which means from this day forward 3.4 is
> ...
> > >- not supported by the community dev team,
> > >- not accepting patches, no future releases, no security fixes,
> > >- latest version is still accessible at download page for 1(?) year
> > >
> > >
> > Apache archival process is documented on this page:
> > https://www.apache.org/legal/release-policy.html#when-to-archive
> > we do get nudged on it every so often:  e.g.
> > https://issues.apache.org/jira/browse/ZOOKEEPER-1752
>
> The archival process is regarding removal of old versions from the
> mirroring system. It does not apply to "current" releases ("current"
> being defined as still linked from the project's download page).
>
> The question here is regarding the timing for removal from the
> project's download page. I suggest event-based, rather than
> time-based. Since ZK seems to be maintaining 3 release lines
> concurrently, you could remove 3.4 when 3.7 is released, unless
> there's a good reason to drop it sooner (to reduce the number of
> concurrent releases supported to 2, for example).
>
> >
> >
> > > 2) Supported upgrade path is: latest 3.4 -> latest 3.5 -> latest 3.6
> > >
> > > 3) Interoperability guarantees:
> > >- Previous version of ZooKeeper client is able to connect to server
> as
> > > long as there’s no new feature enforced on server side,
> > >- Previous version of ZooKeeper server is able to accept connections
> > > from clients as long as they don’t want to use new features.
> > >
> > >
> > I believe 2/3 are consistent with "Backward Compatibility" here?
> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement
> >
> > Patrick
> >
> >
> > > Thoughts?
> > >
> > > Andor
> > >
> > >
> > >
> > >
> > > > On 2020. Apr 2., at 6:30, Michael Han  wrote:
> > > >
> > > > +1.
> > > >
> > > > For EOL policy statement, just to throw something out here that i can
> > > think
> > > > of:
> > > >
> > > > * Define what EOL means (such as: not supported by community dev team
> > > > anymore, no future 3.4 releases .. still accessible at download page
> for
> > > X
> > > > years..) and a date of EOL.
> > > >
> > > > * Provide guidelines for upgrading paths to 3.5 / 3.6.
> > > >
> > > > * State interoperability guarantees another post pointed out
> previously ^
> > > >
> > > > On Wed, Apr 1, 2020 at 2:04 AM Andor Molnar 
> wrote:
> > > >
> > > >> Hi folks,
> > > >>
> > > >> Based on Enrico’s latest post about a 3.4 client problem I’d like to
> > > push
> > > >> this initiative.
> > > >> Asking more senior members of the community what communicated
> policy is
> > > >> needed exactly to say 3.4 is EoL?
> > > >>
> > > >> In terms of timing I’d like Patrick’s suggestion about 1st of June,
> > > 2020.
> > > >>
> > > >> Any objections?
> > > >>
> > > >> Andor
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>> On 2020. Mar 4., at 18:45, Michael K. Edwards <
> m.k.edwa...@gmail.com>
> > > >> wrote:
> > > >>>
> > > >>> I think it would be useful for an EOL statement about 3.4.x to
> include
> > > a
> > > >>> policy on interoperability of newer ZooKeeper servers with 3.4.x
> client
> > > >>> code.  Stacks that build on top of Kafka and Hadoop (I'm looking at
> > > you,
> > > >>> Spark) often wind up having an indirect dependency on a comically
> stale
> > > >>> ZooKeeper library.  Even if this library isn't really exercised by
> the
> > > >>> client side of the stack, it's there in the mountain of jars; and
> when
> > > >>> application code also wants to use ZooKeeper more directly, using a
> > > newer
> > > >>> client library can get kind of messy.  The approach I've taken has
> been
> > > >> to
> > > >>> rebuild large swathes of the stack around a consistent, recent
> > > ZooKeeper
> > > >>> build; but I think it would be relevant to a lot of people to know
> > > >> whether,
> > > >>> say, a 3.4.14 client will work reliably with a 3.6.x quorum.
> > > >>>
> > > >>> On Wed, Mar 4, 2020 at 9:28 AM Enrico Olivelli <
> eolive...@gmail.com>
> > > >> wrote:
> > > >>>
> > >  Il giorno mer 4 mar 2020 alle ore 17:23 Patrick Hunt
> > >   ha scritto:
> > > >
> > > > It seems like we should have a stated/communicated policy around
> > > >> release
> 

Re: Version.getRevision gone in 3.5.6

[Norbert Kalmar]

I was just working on it, you already reviewed it. I fixed the javadoc
comment you mentioned, please check again :)

Thanks!

- Norbert

On Thu, Apr 2, 2020 at 10:42 AM Enrico Olivelli  wrote:

> I would like to release 3.6.1 soon.
> Can you send a patch?
>
> Enrico
>
> Il Mer 1 Apr 2020, 13:09 Norbert Kalmar  ha
> scritto:
>
> > I'll create a jira and the PR. I guess we should just return -1, that's
> > what 3.5.5 did anyway:
> >
> >
> https://github.com/apache/zookeeper/blob/branch-3.5.5/zookeeper-server/src/main/java/org/apache/zookeeper/version/util/VerGen.java#L87
> >
> > And keep it on 3.6 as well? Fine by me. So put this function back in
> 3.5.8
> > and 3.6.1 only, leave master (3.7 or 4.0? :) ) out of it?
> >
> > On Wed, Apr 1, 2020 at 12:52 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > I agree with you.
> > > Also as far as I can remember on some of my leader election bug jira
> > > tickets people asked when the next 3.5.x release would happen.
> > >
> > > Kind regards,
> > > Mate
> > >
> > > On Wed, Apr 1, 2020 at 12:18 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > We should restore it as we did for ZKUtils#delete
> > > >
> > > > Both in 3.5x and in 3.6
> > > > We just ensure smooth upgrade
> > > >
> > > >
> > > > Enrico
> > > >
> > > > Il Mer 1 Apr 2020, 12:06 Norbert Kalmar  >
> > > ha
> > > > scritto:
> > > >
> > > > > Hi all,
> > > > >
> > > > > We removed the getRevision() [1] function in 3.5.6 [2].
> > > > > I just tried updating to ZooKeeper 3.5.7 in our system, but HBase
> is
> > > not
> > > > > happy with this. It gets ZooKeeper's revision using this method. It
> > is
> > > > true
> > > > > HBase could just move to using getRevisionHash, but they are still
> > > using
> > > > > 3.4.x, and getRevisionHash is not available there. So I can't just
> > move
> > > > the
> > > > > the new function to be both compatible with 3.4 and 3.5.6+,
> > > > unfortunately.
> > > > >
> > > > > Long story short, we removed a public function in a bugfix release
> > (in
> > > > > 3.5.5 it is still there).
> > > > > With 3.5.5 our internal system works fine, but can't build with
> 3.5.7
> > > (or
> > > > > 3.5.6) because of this change.
> > > > >
> > > > > My question: should we reinstate this function, and maybe do a
> 3.5.8
> > > > > release? Who knows who else runs into this problem.
> > > > > Any thoughts?
> > > > >
> > > > > - Norbert
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/zookeeper/commit/41da3c8e3c39f81aa0f667199c5f4eb3d5a28adc#diff-12bf6d03d211704a4f04cb234a20bf2aL23
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/zookeeper/commit/96880c0b2ba39f1841f8bdc4a0119a467b462d03#diff-1d42dde48d42e3baf819e22b56a880a7
> > > > >
> > > >
> > >
> >
>

Re: Version.getRevision gone in 3.5.6

[Norbert Kalmar]

I'll create a jira and the PR. I guess we should just return -1, that's
what 3.5.5 did anyway:
https://github.com/apache/zookeeper/blob/branch-3.5.5/zookeeper-server/src/main/java/org/apache/zookeeper/version/util/VerGen.java#L87

And keep it on 3.6 as well? Fine by me. So put this function back in 3.5.8
and 3.6.1 only, leave master (3.7 or 4.0? :) ) out of it?

On Wed, Apr 1, 2020 at 12:52 PM Szalay-Bekő Máté 
wrote:

> I agree with you.
> Also as far as I can remember on some of my leader election bug jira
> tickets people asked when the next 3.5.x release would happen.
>
> Kind regards,
> Mate
>
> On Wed, Apr 1, 2020 at 12:18 PM Enrico Olivelli 
> wrote:
>
> > We should restore it as we did for ZKUtils#delete
> >
> > Both in 3.5x and in 3.6
> > We just ensure smooth upgrade
> >
> >
> > Enrico
> >
> > Il Mer 1 Apr 2020, 12:06 Norbert Kalmar 
> ha
> > scritto:
> >
> > > Hi all,
> > >
> > > We removed the getRevision() [1] function in 3.5.6 [2].
> > > I just tried updating to ZooKeeper 3.5.7 in our system, but HBase is
> not
> > > happy with this. It gets ZooKeeper's revision using this method. It is
> > true
> > > HBase could just move to using getRevisionHash, but they are still
> using
> > > 3.4.x, and getRevisionHash is not available there. So I can't just move
> > the
> > > the new function to be both compatible with 3.4 and 3.5.6+,
> > unfortunately.
> > >
> > > Long story short, we removed a public function in a bugfix release (in
> > > 3.5.5 it is still there).
> > > With 3.5.5 our internal system works fine, but can't build with 3.5.7
> (or
> > > 3.5.6) because of this change.
> > >
> > > My question: should we reinstate this function, and maybe do a 3.5.8
> > > release? Who knows who else runs into this problem.
> > > Any thoughts?
> > >
> > > - Norbert
> > >
> > > [1]
> > >
> > >
> >
> https://github.com/apache/zookeeper/commit/41da3c8e3c39f81aa0f667199c5f4eb3d5a28adc#diff-12bf6d03d211704a4f04cb234a20bf2aL23
> > > [2]
> > >
> > >
> >
> https://github.com/apache/zookeeper/commit/96880c0b2ba39f1841f8bdc4a0119a467b462d03#diff-1d42dde48d42e3baf819e22b56a880a7
> > >
> >
>

Version.getRevision gone in 3.5.6

[Norbert Kalmar]

Hi all,

We removed the getRevision() [1] function in 3.5.6 [2].
I just tried updating to ZooKeeper 3.5.7 in our system, but HBase is not
happy with this. It gets ZooKeeper's revision using this method. It is true
HBase could just move to using getRevisionHash, but they are still using
3.4.x, and getRevisionHash is not available there. So I can't just move the
the new function to be both compatible with 3.4 and 3.5.6+, unfortunately.

Long story short, we removed a public function in a bugfix release (in
3.5.5 it is still there).
With 3.5.5 our internal system works fine, but can't build with 3.5.7 (or
3.5.6) because of this change.

My question: should we reinstate this function, and maybe do a 3.5.8
release? Who knows who else runs into this problem.
Any thoughts?

- Norbert

[1]
https://github.com/apache/zookeeper/commit/41da3c8e3c39f81aa0f667199c5f4eb3d5a28adc#diff-12bf6d03d211704a4f04cb234a20bf2aL23
[2]
https://github.com/apache/zookeeper/commit/96880c0b2ba39f1841f8bdc4a0119a467b462d03#diff-1d42dde48d42e3baf819e22b56a880a7

Re: [jira] [Created] (ZOOKEEPER-3779) Zookeeper client 3.4.x fails to connect when using Java 14

[Norbert Kalmar]

+1

On Wed, Apr 1, 2020 at 7:35 AM Enrico Olivelli  wrote:

> Maybe this is the death of 3.4...
>
> Enrico
>
> Il Mer 1 Apr 2020, 07:29 Jaikiran Pai (Jira)  ha scritto:
>
> > Jaikiran Pai created ZOOKEEPER-3779:
> > ---
> >
> >  Summary: Zookeeper client 3.4.x fails to connect when using
> > Java 14
> >  Key: ZOOKEEPER-3779
> >  URL:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3779
> >  Project: ZooKeeper
> >   Issue Type: Bug
> > Affects Versions: 3.4.14
> >  Environment: Java 14
> > Reporter: Jaikiran Pai
> >
> >
> > When running on Java 14, the 3.4.x version of
> > `org.apache.zookeeper.Zookeeper` which creates a client to connect to a
> > Zookeeper server, fails to connect against the server.
> >
> > This appears to be due to the use of `InetSocketAddress.toString()` to
> > parse the host name of an unresolved address in StaticHostProvider[1].
> The
> > issue doesn't show up in 3.5.x versions of Zookeeper. But given that
> 3.4.x
> > isn't that old, I thought I'll raise this here and see if it can be fixed
> > and whether a new 3.4.x release is warranted.
> >
> > Furthermore, there was a discussion here[2] which explains the whole
> > details. There's even a discussion in the openjdk net-dev mailing list[3]
> > to see if there are other similar usages in Zookeeper which might be
> > impacted by this change in the JDK 14. I don't have any real experience
> > with this library, so if someone more knowledgable knows whether or not
> > there are other similar usages in this project, please do reply to the
> > net-dev discussion on the openjdk mailing list.
> >
> >
> >
> >  [1] [
> >
> https://github.com/apache/zookeeper/blob/branch-3.4.14/zookeeper-server/src/main/java/org/apache/zookeeper/client/StaticHostProvider.java#L135
> > ]
> >
> > [2] [
> > https://github.com/quarkusio/quarkus/issues/8212#issuecomment-605403656]
> >
> > [3]
> https://mail.openjdk.java.net/pipermail/net-dev/2020-March/013725.html
> >
> >
> >
> > --
> > This message was sent by Atlassian Jira
> > (v8.3.4#803005)
> >
>

Re: Keep Python 2 scripts...?

[Norbert Kalmar]

I know it's been only a day since the vote, but we already have multiple
+1, three of them from PMC members and no -1, so I'll commit the patch.
Thanks everyone, especially Tison for making the patch!

- Norbert

On Tue, Mar 31, 2020 at 12:14 AM Brian Nixon 
wrote:

> I agree with the consensus, maintaining a single version is preferable to
> maintaining two.
>
> On Mon, Mar 30, 2020 at 2:28 PM tison  wrote:
>
> > Never mind. Updated pr based on the consensus.
> >
> > Best,
> > tison.
> >
> >
> > Enrico Olivelli  于2020年3月31日周二 上午2:10写道：
> >
> > > Tison,
> > > can you please update your PR and drop the PY2 script ?
> > >
> > > I apologize, I asked you to keep it, we could have done it simpler :-)
> > >
> > > Enrico
> > >
> > > Il giorno lun 30 mar 2020 alle ore 18:43 Patrick Hunt
> > >  ha scritto:
> > > >
> > > > p3 is fine with me. Trying to maintain multiple can be a hassle and
> p2
> > is
> > > > EOL already.
> > > >
> > > > Patrick
> > > >
> > > > On Mon, Mar 30, 2020 at 5:19 AM Norbert Kalmar
> > > 
> > > > wrote:
> > > >
> > > > > +1 for v3 only (I have both v2/v3 interpreters, but as you
> mentioned,
> > > we
> > > > > should only need to maintain 1 script).
> > > > >
> > > > > - Norbert
> > > > >
> > > > > On Mon, Mar 30, 2020 at 1:35 PM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Il giorno lun 30 mar 2020 alle ore 13:33 Andor Molnar
> > > > > >  ha scritto:
> > > > > > >
> > > > > > > Hi folks,
> > > > > > >
> > > > > > > We have a nice PR about upgrading the commit script to Python3:
> > > > > > >
> > > > > > > https://github.com/apache/zookeeper/pull/1295
> > > > > > >
> > > > > > > I think we could just drop the previous v2 version with no
> > > problem, but
> > > > > > folks on the PR would like to keep it.
> > > > > > > Source control keeps the old version anyway and we wouldn’t
> need
> > to
> > > > > > maintain 2 scripts in the long run. Is there anybody (committer)
> > who
> > > > > > doesn’t have python3 interpreter?
> > > > > > >
> > > > > > > Additionally Py2 is already EoL since this January.
> > > > > >
> > > > > > I am fine with Python3 only.
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Andor
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>

Re: Keep Python 2 scripts...?

[Norbert Kalmar]

+1 for v3 only (I have both v2/v3 interpreters, but as you mentioned, we
should only need to maintain 1 script).

- Norbert

On Mon, Mar 30, 2020 at 1:35 PM Enrico Olivelli  wrote:

> Il giorno lun 30 mar 2020 alle ore 13:33 Andor Molnar
>  ha scritto:
> >
> > Hi folks,
> >
> > We have a nice PR about upgrading the commit script to Python3:
> >
> > https://github.com/apache/zookeeper/pull/1295
> >
> > I think we could just drop the previous v2 version with no problem, but
> folks on the PR would like to keep it.
> > Source control keeps the old version anyway and we wouldn’t need to
> maintain 2 scripts in the long run. Is there anybody (committer) who
> doesn’t have python3 interpreter?
> >
> > Additionally Py2 is already EoL since this January.
>
> I am fine with Python3 only.
>
> Enrico
>
>
> >
> > Andor
> >
> >
>

Re: ZooKeeper 3.6.1...is it time ?

[Norbert Kalmar]

I just saw the discussion here on ZOOKEEPER-3739, which I committed about a
minute ago. But please do check if anyone is interested in it (we can
always revert).
My very short take on the issue: Removed an unsupported api class that use
was *supposedly* to speed up file modification watch event on OSX systems.

- Norbert

On Fri, Mar 20, 2020 at 8:07 AM Christopher  wrote:

> Ted, please see the JIRA issue and linked PR for detailed answers to those
> questions. My side issue has already hijacked this thread enough :) We can
> continue in the forked thread that Enrico created, if you wish.
>
> On Fri, Mar 20, 2020 at 1:32 AM Ted Dunning  wrote:
>
> > What internal class?
> >
> > Where is it used?
> >
> > Why are others using JDK 13 not seeing the problem that you described?
> >
> >
> >
> > On Thu, Mar 19, 2020 at 8:58 PM Christopher  wrote:
> >
> > > Apologies, but it's a bit hard for me to explain briefly. Here's an
> > attempt
> > > to be as brief as I can:
> > >
> > > Newer JDKs add a `-release` option to cross-compile with strict
> > > compatibility enforced. When this flag is enabled, as my PR does
> > > automatically with the profiles (when it detects building with a newer
> > JDK
> > > or in m2e Eclipse), the compiler identifies the use of a class that is
> > not
> > > strictly compatible with Java 8. My PR removes this not strictly
> > compatible
> > > class. The use of this class should be considered a bug because it will
> > > cause a failure in some JDKs (those without the com.sun class)... a bug
> > > that is caught by the strict compatibility checks of the `-release`
> flag
> > in
> > > newer JDKs.
> > >
> > > The problem with the class is that it only is available on some
> versions
> > of
> > > Java (those with com.sun internals). It is not only not strictly
> > compatible
> > > with Java 8 ("not guaranteed to work"), it is specifically "guaranteed
> to
> > > not work" in future Java versions, because it is planned for explicit
> > > removal. It already causes problems in current versions if you have an
> > > application that uses modules and depends on ZooKeeper, because the
> > > unsupported class is hidden away in an internal module.
> > >
> > > What you say about ensuring that releases are built with JDK8 is
> actually
> > > not necessary any longer with the use of the `-release` flag. Newer
> JDKs
> > > enforce cross-compilation compatibility better. This release flag
> option
> > > eliminates the need to hold yourself back, doing release builds on an
> > older
> > > end-of-life JDK to retain compatibility. Now, you can build with a
> newer
> > > JDK (taking advantage of newer Maven plugins, etc.), avoid complex uses
> > of
> > > Maven toolchains, the annoying `-bootclasspath`, or the
> > > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation
> strict
> > > compatibility is now enforced as a built in feature of newer JDKs.
> > >
> > > Sorry I couldn't explain more briefly... it's a very tiny change, but
> > > there's a lot of backstory to communicate why it matters.
> > >
> > > Incidentally, you can probably get rid of a few build jobs in Jenkins
> if
> > > you were to use the maven-enforcer-plugin to *only* support builds on
> > newer
> > > JDKs (but still using the release flag to target Java 8, of course).
> This
> > > is what Apache Accumulo does for the stuff that we want to be
> compatible
> > > with Java 8: we require the build to use at least 11, but cross-compile
> > to
> > > 8, because some of the Maven plugins we use for quality control checks
> > are
> > > now starting to require Java 11.
> > >
> > >
> > > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> > scritto:
> > > >
> > > > > I would very much like my pull request in
> > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be merged
> in
> > > > 3.6.1
> > > > > and newer, because it helps ensure newer java APIs don't leak into
> > pull
> > > > > requests/patches when developing stuff to contribute using a newer
> > JDK,
> > > > by
> > > > > leveraging the "release" flag of newer JDKs. However, in order to
> do
> > > so,
> > > > it
> > > > > needs to remove the use of an internal com.sun API which is not
> > > supported
> > > > > in newer JDKs. I would like it included, because it makes it easier
> > for
> > > > me
> > > > > (and others using newer JDKs) to contribute.
> > > > >
> > > >
> > > > I usually use jdk13 for my applications and while I work on ZK code.
> > > > What problems are you facing?
> > > >
> > > > It is important to remember to build with jdk8 while making releases.
> > But
> > > > for dev it is not needed.
> > > >
> > > > Maybe we can merge that patch on master but I don't feel it is so
> > > important
> > > > at the moment. (With jdk13)
> > > >
> > > > Enrico
> > > >
> > > >
> > > > > The patch is currently pending additional reviewers.
> > > > >
> > > > > On Thu, Mar 19, 2020 at 11:17 AM Patrick Hunt 
> > 

Re: JIRA permissions request

[Norbert Kalmar]

Hi Sankalp,

I added you to the contributors, and assigned the jira ticket to you to see
if you have access now. From now on you can assign tickets to yourself.

- Norbert

On Fri, Mar 13, 2020 at 1:11 PM Sankalp Bhatia 
wrote:

> Hi,
>
> I would like to raise a PR for
> https://issues.apache.org/jira/browse/ZOOKEEPER-3689#. I don't seem to
> have
> permissions to assign the JIRA to myself.
>
> Request the admins to please grant access.
>
> username: *sankalpbhatia*
>
> Thanks,
> Sankalp
>

Re: ZooKeeper release 3.5.6 document is not available on website

[Norbert Kalmar]

I think it was not committed, there is no commit for it:
https://github.com/apache/zookeeper/commits/website

release.md was also missing 3.5.6, I added a very generic one when doing
3.5.7.

I can build 3.5.6 and add the missing documentation, probably tomorrow. If
anyone wants to do it before that let me know :)

- Norbert

On Thu, Mar 12, 2020 at 3:46 PM Patrick Hunt  wrote:

> Must be mistake - I see 3.5.5 is there still.
>
> Patrick
>
> On Thu, Mar 12, 2020 at 4:25 AM Mohammad arshad <
> mohammad.ars...@huawei.com>
> wrote:
>
> > Hello folks
> > ZooKeeper release 3.5.6 document
> > http://zookeeper.apache.org/doc/r3.5.6/index.html is not available.
> > Does anyone have any idea why it is not available. Is it removed
> > intentionally or it got removed by mistake.
> >
> > Thanks
> > -Arshad
> >
>

Re: WebSite updates needed

[Norbert Kalmar]

Also, the apidocs are no longer linked correctly, I noticed after the 3.5.7
release.
Before the fix it pointed to:
http://zookeeper.apache.org/doc/r3.6.0/api/index.html

The new current format:
http://zookeeper.apache.org/doc/r3.6.0/apidocs/zookeeper-server/index.html

And, if we want to include jute apidocs (which we said is not needed, as no
point really), it is also available at:
http://zookeeper.apache.org/doc/r3.6.0/apidocs/zookeeper-jute/index.html

I just fixed up manually 3.5.7 and 3.6.0 to point to the correct
zookeeper-server apilink.

I also did a quickfix for the generation of the html to include the new
link:
https://github.com/apache/zookeeper/pull/1277

I did not create a jira, I think this 2 liner doesn't need one (we agreed
trivial changes don't need one, right? )

Question is if we want to keep this structure. (Don't link jute, but then
why upload to website? Get rid of it? Change back the link to /api not
/apidocs/zookeeper-server? )

- Norbert

On Thu, Mar 5, 2020 at 11:53 AM tison  wrote:

> Hi Enrico,
>
> I volunteer for it. For the process it would be
>
> 1. open two JIRA tickets for them, respectively.
> 2. update NOTICEs for fixup1
> 3. move from .../dist to downloads.apache.org
>
> Best,
> tison.
>
>
> Enrico Olivelli  于2020年3月5日周四 下午3:41写道：
>
> > Hi,
> > We have to fix up the website:
> > - update copyright notices from 2019 to 2020
> > - move to downloads.apache.org the download page
> >
> > Any volunteer ?
> >
> >
> > Enrico
> >
>

Re: [VOTE] Apache ZooKeeper 3.6.0 candidate 4

[Norbert Kalmar]

+1 (non-binding)

- unit tests pass (PurgeTxnTest as well)
- source tarball: compiled and started ZK + run few commands from source
tarball
- bin tarball: license files checked, started ZK + run few commands
- signatures OK.
- compared source tarball with git repository checked out at RC tag using
meld. Found no divergence.

Tested on MacOS and Ubuntu 16, using openJDK 1.8.242.

- Norbert

On Thu, Feb 27, 2020 at 11:17 AM Szalay-Bekő Máté <
szalay.beko.m...@gmail.com> wrote:

> +1 (non-binding)
>
> - I built the code and executed the java/C unit tests using 8u242
> (everything passed, except PurgeTxnTest.testPurgeWhenLogRollingInProgress
> what seems to never work on my machine.. I saw it before to be flaky also
> on the apache jenkins, I created a Jira iticket for fixing it:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3740)
> - Using https://github.com/symat/zk-rolling-upgrade-test
>   - I tested rolling upgrade from 3.5.7 to 3.6.0
>   - I tested rolling restart on 3.6.0 to enable the multi-address feature
> with the new quorum protocol version
> - Using https://github.com/symat/zookeeper-docker-test I also tested the
> multi-address feature (disabling and re-enabling different virtual network
> interfaces to see that the cluster always recovers)
>
> On Tue, Feb 25, 2020 at 4:13 PM Enrico Olivelli 
> wrote:
>
> > This is the fifth release candidate for 3.6.0.
> >
> > It is a major release and it introduces a lot of new features, most
> > notably:
> > - Built-in data consistency check inside ZooKeeper
> > - Allow Followers to host Observers
> > - A new feature proposal to ZooKeeper: authentication enforcement
> > - Pluggable metrics system for ZooKeeper (and Prometheus.io integration)
> > - TLS Port unification
> > - Audit logging in ZooKeeper servers
> > - Improve resilience to network (advertise multiple addresses for
> > members of a Zookeeper cluster)
> > - Persistent Recursive Watch
> > - add an API and the corresponding CLI to get total count of recursive
> > sub nodes under a specific path
> >
> > The full release notes is available at:
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346617
> >
> > *** Please download, test and vote by February 28th 2020, 23:59 UTC+0.
> ***
> >
> > Source files:
> > https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-4/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1053/
> >
> > The release candidate tag in git to be voted upon: release-3.6.0-4
> > https://github.com/apache/zookeeper/tree/release-3.6.0-4
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> >
> > Please note that this new major release introduces a new JAR for
> > zookeeper client users: zookeeper-metrics-providers
> >
> > The staging version of the website is:
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-4/website/
> >
> >
> > Should we release this candidate?
> >
> > Enrico Olivelli
> >
>

[ANNOUNCE] Apache ZooKeeper 3.5.7

[Norbert Kalmar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.5.7

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.5.7 Release Notes are at:
https://zookeeper.apache.org/doc/r3.5.7/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 2

[Norbert Kalmar]

Thanks Enrico for the check. I created a jira to fix the mentioned
differences between source tarball and git: ZOOKEEPER-3727 (fix version
3.5.8)

Thanks everyone for the vote and tests.

We have 3 binding +1.
I will move forward to publish 3.5.7-rc2 as the 3.5.7 release.

- Norbert

On Fri, Feb 14, 2020 at 12:45 PM Enrico Olivelli 
wrote:

> +1 (binding)
>
> Verified checksums and signatures
> Built from sources on MacOS using docker environment that emulates
> linux (dev/docker directory).
>
> Run a few smoke tests on MacOs using JDK13
>
> Tested the upgrade scenario from 3.4.14 (see
> https://issues.apache.org/jira/browse/ZOOKEEPER-3644) using
> snapshot.trust.empty=true (fix in 3.5.6 was partial and lead to data
> loss)
>
> I have noticed that mvn checkstyle:check does not work due to the lack
> of checkstyle.xml file.
> the source tarball does not match perfectly the tag due to the
> presence of some generated file, but it is the same as for 3.5.6, so
> this is not blocker to me.
>
> Thank you Norbert for driving this release. Well done
>
> Enrico
>
> Il giorno mer 12 feb 2020 alle ore 21:29 Patrick Hunt
>  ha scritto:
> >
> > +1 xsum/sig verify, rat ran clean, compiled from source and ran some
> manual
> > testing.
> >
> > Patrick
> >
> > On Mon, Feb 10, 2020 at 7:23 AM Andor Molnar  wrote:
> >
> > > +1 (binding)
> > >
> > > - release notes are OK,
> > > - documentation looks good,
> > > - verified signatures, checksum,
> > > - Java & C unit tests passed,
> > > - verified 3-node cluster with zk-latencies.py (create, get, delete,
> > > setAcl, getAcl, watchers)
> > >
> > > Andor
> > >
> > >
> > >
> > > > On 2020. Feb 10., at 12:52, Norbert Kalmar 
> wrote:
> > > >
> > > > This is the third bugfix release candidate for 3.5.7. It fixes 25
> issues,
> > > > including third party CVE fixes, potential data loss and potential
> split
> > > > brain if some rare conditions exists.
> > > >
> > > > There are 4 additional patches compared to rc0 and rc1:
> > > > - ZOOKEEPER-3453: missing 'SET' in zkCli on windows
> > > > - ZOOKEEPER-3716: upgrade netty 4.1.42 to address CVE-2019-20444
> CVE-20…
> > > > - ZOOKEEPER-3718: The tarball generated by assembly is missing some
> files
> > > > - ZOOKEEPER-3719: Fix C Client compilation issues
> > > >
> > > > The full release notes are available at:
> > > >
> > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
> > > >
> > > > *** Please download, test and vote by February 13th 2020, 23:59
> UTC+0.
> > > ***
> > > >
> > > > Source files:
> > > > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-2/
> > > >
> > > > Maven staging repo:
> > > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.5.7-rc2
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > Should we release this candidate?
> > >
> > >
>

[VOTE] Apache ZooKeeper release 3.5.7 candidate 2

[Norbert Kalmar]

This is the third bugfix release candidate for 3.5.7. It fixes 25 issues,
including third party CVE fixes, potential data loss and potential split
brain if some rare conditions exists.

There are 4 additional patches compared to rc0 and rc1:
- ZOOKEEPER-3453: missing 'SET' in zkCli on windows
- ZOOKEEPER-3716: upgrade netty 4.1.42 to address CVE-2019-20444 CVE-20…
- ZOOKEEPER-3718: The tarball generated by assembly is missing some files
- ZOOKEEPER-3719: Fix C Client compilation issues

The full release notes are available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098

*** Please download, test and vote by February 13th 2020, 23:59 UTC+0. ***

Source files:
https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-2/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/

The release candidate tag in git to be voted upon: release-3.5.7-rc2

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

Should we release this candidate?

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

Hi Jordan,

It is available again. Rc1 got downvoted, so I created rc2 which is now
available in the staging repo. I'll also write the email about it but
before I'll just run a few more tests.

- Norbert

On Sun, Feb 9, 2020 at 4:43 PM Jordan Zimmerman 
wrote:

> 3.5.7 is not in the staging repo. I'd like to test with Curator.
>
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/
> <
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/
> >
>
> -Jordan
>
> > On Feb 7, 2020, at 7:29 AM, Norbert Kalmar  wrote:
> >
> > This is the second bugfix release candidate for 3.5.7. It fixes 21
> issues,
> > including third party CVE fixes, potential data loss and potential split
> > brain if some rare conditions exists.
> >
> > (I have signed rc0 with the wrong key - sorry for that). Everything else
> is
> > unchanged from rc0.
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
> >
> > *** Please download, test and vote by February 11th 2020, 23:59 UTC+0.
> ***
> >
> > Source files:
> > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-1/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
> >
> > The release candidate tag in git to be voted upon: release-3.5.7-rc1
> > (points to the same commit as rc0)
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Should we release this candidate?
>
>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

Done, thank you Patrick!

- Norbert

On Fri, Feb 7, 2020 at 11:29 PM Patrick Hunt  wrote:

> fwiw I would also be -1 due to the known CVE addressed in 3.6 but not here:
> https://github.com/apache/zookeeper/pull/1246
>
> Norbert as RM can you pull this in to branch-3.5 and the next RC?
>
> Patrick
>
>
> On Fri, Feb 7, 2020 at 10:14 AM Norbert Kalmar
> 
> wrote:
>
> > Thanks Damien!
> >
> > -1 from me for this rc1.
> >
> > I'll start working on rc2! Damien already fixed the mentioned issue, I'll
> > review and test.
> > I tested my modification for the missing files (ZOOKEEPER-3718), my main
> > issue is to use apache-release profile just like in master, or fix up the
> > assembly for source tarball. I lean toward adding apache-release profile
> to
> > 3.5 as well. Only difference I've noticed is NOTICE and LICENSE file
> > getting duplicated (both with file extension and without in filename).
> > Anyone has any preference for this? Fix assembly descriptor for 3.5
> source
> > tarball or add apache-release profile (which will generate the source
> > tarball to parent's target folder).
> > I'll go with adding apache-release unless someone disagrees.
> >
> > Regards,
> > Norbert
> >
> >
> > On Fri, Feb 7, 2020 at 5:43 PM Damien Diederen  >
> > wrote:
> >
> > >
> > > Hi Norbert, all,
> > >
> > > Norbert wrote:
> > > > p.s.: These are pretty easy fixes to include/exclude the files Eniroc
> > > > mentioned, so on second thought probably worth fixing and do an rc2.
> > > > I'll create a jira and start to work on it. We'll see if anything
> else
> > > > comes up or how the vote goes.
> > >
> > > I'm afraid something else came up!  (Sorry about that…)
> > >
> > > I don't know if the C client is supposed to block 3.5.7, but I have
> > > noticed (and, hopefully, fixed) a few compilation issues:
> > >
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-3719
> > > https://github.com/apache/zookeeper/pull/1249
> > >
> > > With these in, the C client builds and passes tests.
> > >
> > > Cheers, -D
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

Thanks Damien!

-1 from me for this rc1.

I'll start working on rc2! Damien already fixed the mentioned issue, I'll
review and test.
I tested my modification for the missing files (ZOOKEEPER-3718), my main
issue is to use apache-release profile just like in master, or fix up the
assembly for source tarball. I lean toward adding apache-release profile to
3.5 as well. Only difference I've noticed is NOTICE and LICENSE file
getting duplicated (both with file extension and without in filename).
Anyone has any preference for this? Fix assembly descriptor for 3.5 source
tarball or add apache-release profile (which will generate the source
tarball to parent's target folder).
I'll go with adding apache-release unless someone disagrees.

Regards,
Norbert


On Fri, Feb 7, 2020 at 5:43 PM Damien Diederen 
wrote:

>
> Hi Norbert, all,
>
> Norbert wrote:
> > p.s.: These are pretty easy fixes to include/exclude the files Eniroc
> > mentioned, so on second thought probably worth fixing and do an rc2.
> > I'll create a jira and start to work on it. We'll see if anything else
> > comes up or how the vote goes.
>
> I'm afraid something else came up!  (Sorry about that…)
>
> I don't know if the C client is supposed to block 3.5.7, but I have
> noticed (and, hopefully, fixed) a few compilation issues:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-3719
> https://github.com/apache/zookeeper/pull/1249
>
> With these in, the C client builds and passes tests.
>
> Cheers, -D
>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

Sorry for the name typo - *Enrico

On Fri, Feb 7, 2020 at 2:24 PM Norbert Kalmar  wrote:

> p.s.: These are pretty easy fixes to include/exclude the files Eniroc
> mentioned, so on second thought probably worth fixing and do an rc2.
> I'll create a jira and start to work on it. We'll see if anything else
> comes up or how the vote goes.
>
> - Norbert
>
> On Fri, Feb 7, 2020 at 2:20 PM Norbert Kalmar 
> wrote:
>
>> Hi Enrico,
>>
>> I checked the release, you are right. I mainly did the comparison with
>> the 3.5.6 release as there were no changes on file/dependency level really.
>> All of these points stand for that release as well. That's why I didn't
>> notice or thought of it as release breaking flaw.
>>
>> In my opinion, we should definitely create a jira to fix these for the
>> next 3.5 release, and just go forward with 3.5.7. This is a bugfix release,
>> and it does fix a few critical bugs and thirdparty CVEs.
>> None of this was introduced with this bugfix version, so from my point of
>> view these are not breaking content errors.
>>
>> On the other hand I am biased I guess due to the fact I made the release
>> :)
>> So knowing this let me know if anyone thinks this is a -1 for rc1.
>>
>> Thanks,
>> Norbert
>>
>> On Fri, Feb 7, 2020 at 2:01 PM Enrico Olivelli 
>> wrote:
>>
>>> Hi Norbert,
>>> thank you for working on this.
>>>
>>> I see differences between the contents of the source tarball and the
>>> git tag (using Meld, as suggested by Patrick some month ago), namely:
>>> - there is not checkstyleSuppressions.xml file, and mvn
>>> checkstyle:check fails (it is not bound to the default lifecycle, so
>>> mvn clean install still works)
>>> - there are ".c" generated files, they should not be part of the source
>>> release
>>> - there is not "dev" directory
>>> - there is not .travis.yml file
>>>
>>> I am somehow biased because I worked on this stuff for 3.6.0rc0,rc1 and
>>> rc2
>>> AKAIK these problems were already present in 3.5.6 so I am not sure
>>> these are blocker issues for a release.
>>>
>>> I am still continuing my tests
>>> I just wanted to inform you about my findings, this way we can choose
>>> what do to as soon as possible.
>>>
>>>
>>> Enrico
>>>
>>>
>>> Il giorno ven 7 feb 2020 alle ore 13:29 Norbert Kalmar
>>>  ha scritto:
>>> >
>>> > This is the second bugfix release candidate for 3.5.7. It fixes 21
>>> issues,
>>> > including third party CVE fixes, potential data loss and potential
>>> split
>>> > brain if some rare conditions exists.
>>> >
>>> > (I have signed rc0 with the wrong key - sorry for that). Everything
>>> else is
>>> > unchanged from rc0.
>>> >
>>> > The full release notes is available at:
>>> >
>>> >
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
>>> >
>>> > *** Please download, test and vote by February 11th 2020, 23:59 UTC+0.
>>> ***
>>> >
>>> > Source files:
>>> > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-1/
>>> >
>>> > Maven staging repo:
>>> >
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
>>> >
>>> > The release candidate tag in git to be voted upon: release-3.5.7-rc1
>>> > (points to the same commit as rc0)
>>> >
>>> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>>> > https://www.apache.org/dist/zookeeper/KEYS
>>> >
>>> > Should we release this candidate?
>>>
>>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

p.s.: These are pretty easy fixes to include/exclude the files Eniroc
mentioned, so on second thought probably worth fixing and do an rc2.
I'll create a jira and start to work on it. We'll see if anything else
comes up or how the vote goes.

- Norbert

On Fri, Feb 7, 2020 at 2:20 PM Norbert Kalmar  wrote:

> Hi Enrico,
>
> I checked the release, you are right. I mainly did the comparison with the
> 3.5.6 release as there were no changes on file/dependency level really. All
> of these points stand for that release as well. That's why I didn't notice
> or thought of it as release breaking flaw.
>
> In my opinion, we should definitely create a jira to fix these for the
> next 3.5 release, and just go forward with 3.5.7. This is a bugfix release,
> and it does fix a few critical bugs and thirdparty CVEs.
> None of this was introduced with this bugfix version, so from my point of
> view these are not breaking content errors.
>
> On the other hand I am biased I guess due to the fact I made the release :)
> So knowing this let me know if anyone thinks this is a -1 for rc1.
>
> Thanks,
> Norbert
>
> On Fri, Feb 7, 2020 at 2:01 PM Enrico Olivelli 
> wrote:
>
>> Hi Norbert,
>> thank you for working on this.
>>
>> I see differences between the contents of the source tarball and the
>> git tag (using Meld, as suggested by Patrick some month ago), namely:
>> - there is not checkstyleSuppressions.xml file, and mvn
>> checkstyle:check fails (it is not bound to the default lifecycle, so
>> mvn clean install still works)
>> - there are ".c" generated files, they should not be part of the source
>> release
>> - there is not "dev" directory
>> - there is not .travis.yml file
>>
>> I am somehow biased because I worked on this stuff for 3.6.0rc0,rc1 and
>> rc2
>> AKAIK these problems were already present in 3.5.6 so I am not sure
>> these are blocker issues for a release.
>>
>> I am still continuing my tests
>> I just wanted to inform you about my findings, this way we can choose
>> what do to as soon as possible.
>>
>>
>> Enrico
>>
>>
>> Il giorno ven 7 feb 2020 alle ore 13:29 Norbert Kalmar
>>  ha scritto:
>> >
>> > This is the second bugfix release candidate for 3.5.7. It fixes 21
>> issues,
>> > including third party CVE fixes, potential data loss and potential split
>> > brain if some rare conditions exists.
>> >
>> > (I have signed rc0 with the wrong key - sorry for that). Everything
>> else is
>> > unchanged from rc0.
>> >
>> > The full release notes is available at:
>> >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
>> >
>> > *** Please download, test and vote by February 11th 2020, 23:59 UTC+0.
>> ***
>> >
>> > Source files:
>> > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-1/
>> >
>> > Maven staging repo:
>> >
>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
>> >
>> > The release candidate tag in git to be voted upon: release-3.5.7-rc1
>> > (points to the same commit as rc0)
>> >
>> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>> > https://www.apache.org/dist/zookeeper/KEYS
>> >
>> > Should we release this candidate?
>>
>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

Hi Enrico,

I checked the release, you are right. I mainly did the comparison with the
3.5.6 release as there were no changes on file/dependency level really. All
of these points stand for that release as well. That's why I didn't notice
or thought of it as release breaking flaw.

In my opinion, we should definitely create a jira to fix these for the
next 3.5 release, and just go forward with 3.5.7. This is a bugfix release,
and it does fix a few critical bugs and thirdparty CVEs.
None of this was introduced with this bugfix version, so from my point of
view these are not breaking content errors.

On the other hand I am biased I guess due to the fact I made the release :)
So knowing this let me know if anyone thinks this is a -1 for rc1.

Thanks,
Norbert

On Fri, Feb 7, 2020 at 2:01 PM Enrico Olivelli  wrote:

> Hi Norbert,
> thank you for working on this.
>
> I see differences between the contents of the source tarball and the
> git tag (using Meld, as suggested by Patrick some month ago), namely:
> - there is not checkstyleSuppressions.xml file, and mvn
> checkstyle:check fails (it is not bound to the default lifecycle, so
> mvn clean install still works)
> - there are ".c" generated files, they should not be part of the source
> release
> - there is not "dev" directory
> - there is not .travis.yml file
>
> I am somehow biased because I worked on this stuff for 3.6.0rc0,rc1 and rc2
> AKAIK these problems were already present in 3.5.6 so I am not sure
> these are blocker issues for a release.
>
> I am still continuing my tests
> I just wanted to inform you about my findings, this way we can choose
> what do to as soon as possible.
>
>
> Enrico
>
>
> Il giorno ven 7 feb 2020 alle ore 13:29 Norbert Kalmar
>  ha scritto:
> >
> > This is the second bugfix release candidate for 3.5.7. It fixes 21
> issues,
> > including third party CVE fixes, potential data loss and potential split
> > brain if some rare conditions exists.
> >
> > (I have signed rc0 with the wrong key - sorry for that). Everything else
> is
> > unchanged from rc0.
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
> >
> > *** Please download, test and vote by February 11th 2020, 23:59 UTC+0.
> ***
> >
> > Source files:
> > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-1/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
> >
> > The release candidate tag in git to be voted upon: release-3.5.7-rc1
> > (points to the same commit as rc0)
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Should we release this candidate?
>

[VOTE] Apache ZooKeeper release 3.5.7 candidate 1

[Norbert Kalmar]

This is the second bugfix release candidate for 3.5.7. It fixes 21 issues,
including third party CVE fixes, potential data loss and potential split
brain if some rare conditions exists.

(I have signed rc0 with the wrong key - sorry for that). Everything else is
unchanged from rc0.

The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098

*** Please download, test and vote by February 11th 2020, 23:59 UTC+0. ***

Source files:
https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-1/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/

The release candidate tag in git to be voted upon: release-3.5.7-rc1
(points to the same commit as rc0)

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

Should we release this candidate?

Re: [VOTE] Apache ZooKeeper release 3.6.0 candidate 2

[Norbert Kalmar]

+1 (non-binding)

- unit tests pass
- source tarball: compiled and started ZK + run few commands from source
tarball
- bin tarball: license files checked, started ZK + run few commands
- signature OK.

Tested on MacOS and Linux, openJDK 1.8.242.

Thanks Enrico!

- Norbert

On Thu, Feb 6, 2020 at 11:20 AM Szalay-Bekő Máté 
wrote:

> +1 (non-binding)
>
> - I compiled and run all the unit tests using Ubuntu 18.04, using maven
> 3.3.9 and OpenJDK 1.8.242 (the tests that failed with this JDK for the
> previous RC now run without a problem)
> - I compiled and tested the C client and the python client (we added SSL
> feature / tests in this release for the C and python clients)
> - I did some manual tests for the multi-address feature with multiple
> virtual networks (using https://github.com/symat/zookeeper-docker-test)
> and
> the cluster did recover quickly after I disabled / enabled various virtual
> network interfaces
>
>
> On Thu, Feb 6, 2020 at 4:36 AM Patrick Hunt  wrote:
>
> > +1 - sig/xsum verified, rat ran clean, I compiled and ran various tests
> and
> > they passed.
> >
> > Patrick
> >
> > On Wed, Feb 5, 2020 at 11:34 AM Enrico Olivelli 
> > wrote:
> >
> > > This is the third release candidate for Apache ZooKeeper 3.6.0.
> > >
> > > It is a major release and it introduces a lot of new features, most
> > > notably:
> > > - Built-in data consistency check inside ZooKeeper
> > > - Allow Followers to host Observers
> > > - Authentication enforcement
> > > - Pluggable metrics system for ZooKeeper (and Prometheus.io
> integration)
> > > - TLS Port unification
> > > - Audit logging in ZooKeeper servers
> > > - Improve resilience to network (advertise multiple addresses for
> > > members of a Zookeeper cluster)
> > > - Persistent Recursive Watches
> > > - add an API and the corresponding CLI to get total count of recursive
> > > sub nodes under a specific path
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12326518
> > >
> > > *** Please download, test and vote by February 8th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-2/
> > >
> > > Maven staging repo:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1049/
> > >
> > > The staging version of the website is:
> > >
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-2/website/
> > >
> > > The release candidate tag in git to be voted upon: release-3.6.0-2
> > > https://github.com/apache/zookeeper/tree/release-3.6.0-2
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Please note that we are adding a new jar to the dependency set for
> > > clients: zookeeper-metrics-providers.
> > >
> > > Should we release this candidate?
> > >
> > > Enrico Olivelli
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.7 candidate 0

[Norbert Kalmar]

Thanks Patrick and Enrico, sorry about that... Obviously I can verify it on
my machine where I generated the key, so I didn't catch this problem.
I did have some issues with my gpg keys when trying to close the maven
staging repo, but that got solved.

- Norbert

On Thu, Feb 6, 2020 at 11:12 PM Enrico Olivelli  wrote:

> I am sorry.
> I am seeing the same error.
>
> btw I am continuing my tests
>
> Enrico
>
> Il giorno gio 6 feb 2020 alle ore 18:46 Patrick Hunt
>  ha scritto:
> >
> > -1 - looks like there are problems with the sig?
> >
> > $gpg --import KEYS
> > ...
> > gpg: key 792D43153B5B5147: public key "Norbert Kalmar <
> nkal...@apache.org>"
> > imported
> >
> >
> > $ gpg --verify apache-zookeeper-3.5.7-bin.tar.gz.asc
> > gpg: assuming signed data in 'apache-zookeeper-3.5.7-bin.tar.gz'
> > gpg: Signature made Wed Feb  5 06:12:51 2020 PST
> > gpg:using RSA key
> 22CA288910DA204539CAD1638A563F46D7345231
> > gpg: Can't check signature: No public key
> >
> > $ gpg --verify apache-zookeeper-3.5.7.tar.gz.asc
> > gpg: assuming signed data in 'apache-zookeeper-3.5.7.tar.gz'
> > gpg: Signature made Wed Feb  5 06:13:32 2020 PST
> > gpg:        using RSA key
> 22CA288910DA204539CAD1638A563F46D7345231
> > gpg: Can't check signature: No public key
> >
> >
> > On Thu, Feb 6, 2020 at 3:10 AM Norbert Kalmar 
> wrote:
> >
> > > This is a bugfix release candidate for 3.5.7. It fixes 21 issues,
> > > including SASL authentication with SSL, third party CVE fixes, data
> loss
> > > and potential split brain if some rare conditions exists.
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346098
> > >
> > > *** Please download, test and vote by February 10th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~nkalmar/zookeeper-3.5.7-candidate-0/
> > >
> > > Maven staging repo:
> > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.7/
> > >
> > > The release candidate tag in git to be voted upon: release-3.5.7-rc0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Should we release this candidate?
> > >
>

Re: [EXTERNAL] Re: 3.5.7

[Norbert Kalmar]

Hi all,

I have branched 3.5.7, and updated branch-3.5 to 3.5.8-SNAPSHOT.

Please let me know if there is anything you think it should be also
cherry-picked to 3.5.7! Anything (from now) backported to just branch-3.5
will NOT be available with 3.5.7.

Thanks!

- Norbert

On Mon, Feb 3, 2020 at 11:14 AM Norbert Kalmar  wrote:

> So about the SSL tests failing... my hosts file was messed up, turns out
> localhost did not translate to 127.0.0.1 very well (it had a few strange
> entry).
> Probably something in the Docker image also around the hosts file, Máté is
> taking a look.
>
> Regards,
> Norbert
>
> On Mon, Feb 3, 2020 at 10:15 AM Norbert Kalmar 
> wrote:
>
>> Hi Craig,
>>
>> It's already committed:
>> https://github.com/apache/zookeeper/commit/59337e7ec8ab67fecf7cfc1b8c5b76397c02bfd6
>>
>> Sorry I'm running a bit late with the release, I got a flu or some virus
>> last week, wasn't too productive.
>>
>> Anyway, I did some testing, and SSL tests seems a bit flaky, they almost
>> constantly kept failing. Reducing fork count helped a bit, first to 4 then
>> to 1. Still was a bit flaky, the truth to be told the machine had some
>> loads and the problem was server not coming up in time. But after killing
>> the load it still happened.
>> On my MacOS unit tests run without a problem with the default 8 surefire
>> core.
>> Turns out the SSL tests are also failing while building with the docker
>> image, I just talked to Máté about this.
>>
>> Oh, it's both 3.5 and master branch.
>>
>> I'll dig into this a little before continuing.
>>
>> Regards,
>> Norbert
>>
>> On Sat, Feb 1, 2020 at 1:01 PM Enrico Olivelli 
>> wrote:
>>
>>> Il Ven 31 Gen 2020, 16:16 Craig.Condit  ha
>>> scritto:
>>>
>>> > Would it be possible to get ZOOKEEPER-3638 included in 3.5.7 as well?
>>>
>>> I thought it already went in
>>>
>>> we really must include it.
>>>
>>> Enrico
>>>
>>> The version of Jetty included in 3.5.6 breaks the admin server. We have
>>> > been running a backport of 3638 (which just upgrades to a later
>>> version)
>>> > successfully on 3.5.6 here without issue.
>>> >
>>> > Thanks,
>>> >
>>> > Craig Condit
>>> >
>>> >
>>> > 
>>> > From: Norbert Kalmar 
>>> > Sent: Thursday, January 30, 2020 3:06 PM
>>> > To: DevZooKeeper 
>>> > Subject: [EXTERNAL] Re: 3.5.7
>>> >
>>> > Hi all,
>>> >
>>> > Just a heads up.
>>> >
>>> > All patch that we wanted (as far as I'm aware, let me know if you miss
>>> > something) for the 3.5.7 release has been committed to branch 3.5.
>>> Mainly
>>> > this was:
>>> > - ZOOKEEPER-3701 (split brain)
>>> > - ZOOKEEPER-3482 (some SASL stuff)
>>> > - ZOOKEEPER-3699 (fix CVE about Jackson)
>>> >
>>> > And a few other nice to haves (like ZOOKEEPER-1105 C client WARN msg
>>> fix)
>>> > that also made it.
>>> >
>>> > I started testing the 3.5 branch and I will create a release branch
>>> soon
>>> > (probably tomorrow).
>>> >
>>> > Regards,
>>> > Norbert
>>> >
>>> > On Mon, Jan 27, 2020 at 11:30 AM Norbert Kalmar 
>>> > wrote:
>>> >
>>> > > Only blocker left for 3.5.7 is ZOOKEEPER-3701, patch available here:
>>> > > https://github.com/apache/zookeeper/pull/1233
>>> > >
>>> > > I'll wait another 0.5-1 day if anyone wants to take a look at it.
>>> Then
>>> > > I'll commit and start the 3.5.7 release process.
>>> > >
>>> > > Thanks,
>>> > > Norbert
>>> > >
>>> > > On Thu, Jan 23, 2020 at 11:29 AM Norbert Kalmar <
>>> nkal...@cloudera.com>
>>> > > wrote:
>>> > >
>>> > >> The patch fixed the CVE warning
>>> > >> https://builds.apache.org/job/zookeeper-master-maven-owasp/339/
>>> > >>
>>> > >> Norbert
>>> > >>
>>> > >> On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar <
>>> nkal...@cloudera.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Thanks Patrick, I'll review and preferably commit your patch, which
>>> > >>> should negate the CVE warning.
>>> > >>>
>>> > >>> Regards,
>>> > >>> Norbert
>>> > >>>
>>> > >>> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt 
>>> wrote:
>>> > >>>
>>> > >>>> owasp is failing on branch-3.5,
>>> > >>>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>>> > >>>>
>>> > >>>> seems the same as:
>>> > >>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>>> > >>>>
>>> > >>>> Patrick
>>> > >>>>
>>> > >>>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly 
>>> wrote:
>>> > >>>>
>>> > >>>> > > Would you have time for a quick fix ?
>>> > >>>> >
>>> > >>>> > The measures to avoid the problem are listed at the end of the
>>> JIRA
>>> > >>>> > description. I can't submit a PR until I get permission from my
>>> > >>>> > company legal to push to ZK.
>>> > >>>> >
>>> > >>>> > -Ivan
>>> > >>>> >
>>> > >>>>
>>> > >>>
>>> >
>>>
>>

Re: [VOTE] Apache ZooKeeper release 3.6.0 candidate 1

[Norbert Kalmar]

Máté's patch fixed it for me. I don't know if this is a blocker for 3.6.0
rc1, but since 3.5.7 is not even branched yet, I'll wait for this patch to
make it there.

Thanks Máté, good catch!

Regards,
Norbert

On Mon, Feb 3, 2020 at 2:02 PM Szalay-Bekő Máté 
wrote:

> I created https://issues.apache.org/jira/browse/ZOOKEEPER-3715 and started
> to work on it
>
> On Mon, Feb 3, 2020 at 1:12 PM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
>
> > (FYI: I tried a few more versions, the problem seems to appear between
> > OpenJDK 8.232 and 8.242. And there are a lot of kerberos related changes
> > after 8.232: see https://hg.openjdk.java.net/jdk8u/jdk8u/jdk )
> >
> > On Mon, Feb 3, 2020 at 12:54 PM Norbert Kalmar
> >  wrote:
> >
> >> I tested with zulu 1.8.212 on the linux machine, and with zulu 1.8.0_163
> >> on
> >> MacOS (whoops). I use sdkman on both machine. I upgraded to the newest
> 1.8
> >> which is _242, at least with sdkman.
> >> And sadly, the mentioned tests also fail for me after the upgrade.
> >>
> >> So, something in the tests that the new versions of java doesn't like :(
> >>
> >> I'm not sure either if it's a showstopper or not. But possibly this
> could
> >> come out when using kerberized ZK? Unfortunately kind of hard to test
> >> "live".
> >>
> >> Regards,
> >> Norbert
> >>
> >> On Mon, Feb 3, 2020 at 12:38 PM Szalay-Bekő Máté <
> >> szalay.beko.m...@gmail.com>
> >> wrote:
> >>
> >> > - I compiled and run all the unit tests using Ubuntu 18.04 (incl. the
> C
> >> > client), using OpenJDK 1.8.212
> >> > - I also built and unit tested the python client
> >> > - I did some manual tests for the multi-address feature with multiple
> >> > virtual networks (using
> https://github.com/symat/zookeeper-docker-test)
> >> >
> >> > everything seemed to be OK, however...
> >> >
> >> > using OpenJDK 1.8.242 or OpenJDK 11.0.6, I got some kerberos related
> >> > exceptions when running the following tests:
> >> > - QuorumKerberosAuthTest
> >> > - QuorumKerberosHostBasedAuthTest
> >> > - SaslKerberosAuthOverSSLTest
> >> >
> >> > the error:
> >> > 2020-02-03 12:11:07,197 [myid:localhost:11223] - ERROR
> >> > [main-SendThread(localhost:11223):ZooKeeperSaslClient@336] - An
> error:
> >> > (java.security.PrivilegedActionException:
> >> > javax.security.sasl.SaslException: GSS initiate failed [Caused by
> >> > GSSException: No valid credentials provided (Mechanism level: null
> >> > (5001))]) occurred when evaluating Zookeeper Quorum Member's  received
> >> SASL
> >> > token. Zookeeper Client will go to AUTH_FAILED state.
> >> >
> >> > I tried it with Zulu 11.0.3 version and OpenJDK 11.0.2 version and
> both
> >> > were working fine. So it looks there might some incompatibility with
> the
> >> > more recent JDK releases. (between 1.8.212 - 1.8.242, and also between
> >> > 11.0.3 and 11.0.6)
> >> >
> >> > I also tested on OpenJDK 13.ea.30 and that worked.
> >> >
> >> > I am not sure if it is a -1 or not... clearly these are some test and
> >> JDK
> >> > related issues. Also it can be only some strange thing with my
> >> environment.
> >> > Can someone try to reproduce my problem?
> >> >
> >> >
> >> > Cheers,
> >> > Mate
> >> >
> >> > On Mon, Feb 3, 2020 at 4:31 AM Jordan Zimmerman <
> >> > jor...@jordanzimmerman.com>
> >> > wrote:
> >> >
> >> > > No big issues with Curator that I could find
> >> > >
> >> > > +1 (non binding)
> >> > >
> >> > > -Jordan
> >> > >
> >> > > > On Feb 1, 2020, at 10:02 AM, Enrico Olivelli  >
> >> > > wrote:
> >> > > >
> >> > > > This is the second release candidate for Apache ZooKeeper 3.6.0.
> >> > > >
> >> > > > It is a major release and it introduces a lot of new features,
> most
> >> > > notably:
> >> > > > - Built-in data consistency check inside ZooKeeper
> >> > > > - Allow Followers to host Observers
> >> > > > - Authentication enforcement
> >> > > > - Pluggable metrics system for ZooK

Re: [VOTE] Apache ZooKeeper release 3.6.0 candidate 1

[Norbert Kalmar]

I tested with zulu 1.8.212 on the linux machine, and with zulu 1.8.0_163 on
MacOS (whoops). I use sdkman on both machine. I upgraded to the newest 1.8
which is _242, at least with sdkman.
And sadly, the mentioned tests also fail for me after the upgrade.

So, something in the tests that the new versions of java doesn't like :(

I'm not sure either if it's a showstopper or not. But possibly this could
come out when using kerberized ZK? Unfortunately kind of hard to test
"live".

Regards,
Norbert

On Mon, Feb 3, 2020 at 12:38 PM Szalay-Bekő Máté 
wrote:

> - I compiled and run all the unit tests using Ubuntu 18.04 (incl. the C
> client), using OpenJDK 1.8.212
> - I also built and unit tested the python client
> - I did some manual tests for the multi-address feature with multiple
> virtual networks (using https://github.com/symat/zookeeper-docker-test)
>
> everything seemed to be OK, however...
>
> using OpenJDK 1.8.242 or OpenJDK 11.0.6, I got some kerberos related
> exceptions when running the following tests:
> - QuorumKerberosAuthTest
> - QuorumKerberosHostBasedAuthTest
> - SaslKerberosAuthOverSSLTest
>
> the error:
> 2020-02-03 12:11:07,197 [myid:localhost:11223] - ERROR
> [main-SendThread(localhost:11223):ZooKeeperSaslClient@336] - An error:
> (java.security.PrivilegedActionException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: null
> (5001))]) occurred when evaluating Zookeeper Quorum Member's  received SASL
> token. Zookeeper Client will go to AUTH_FAILED state.
>
> I tried it with Zulu 11.0.3 version and OpenJDK 11.0.2 version and both
> were working fine. So it looks there might some incompatibility with the
> more recent JDK releases. (between 1.8.212 - 1.8.242, and also between
> 11.0.3 and 11.0.6)
>
> I also tested on OpenJDK 13.ea.30 and that worked.
>
> I am not sure if it is a -1 or not... clearly these are some test and JDK
> related issues. Also it can be only some strange thing with my environment.
> Can someone try to reproduce my problem?
>
>
> Cheers,
> Mate
>
> On Mon, Feb 3, 2020 at 4:31 AM Jordan Zimmerman <
> jor...@jordanzimmerman.com>
> wrote:
>
> > No big issues with Curator that I could find
> >
> > +1 (non binding)
> >
> > -Jordan
> >
> > > On Feb 1, 2020, at 10:02 AM, Enrico Olivelli 
> > wrote:
> > >
> > > This is the second release candidate for Apache ZooKeeper 3.6.0.
> > >
> > > It is a major release and it introduces a lot of new features, most
> > notably:
> > > - Built-in data consistency check inside ZooKeeper
> > > - Allow Followers to host Observers
> > > - Authentication enforcement
> > > - Pluggable metrics system for ZooKeeper (and Prometheus.io
> integration)
> > > - TLS Port unification
> > > - Audit logging in ZooKeeper servers
> > > - Improve resilience to network (advertise multiple addresses for
> > > members of a Zookeeper cluster)
> > > - Persistent Recursive Watches
> > > - add an API and the corresponding CLI to get total count of recursive
> > > sub nodes under a specific path
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12326518
> > >
> > > *** Please download, test and vote by February 4th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-1/
> > >
> > > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1047/
> > >
> > > The staging version of the website is:
> > >
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-1/website/
> > >
> > > The release candidate tag in git to be voted upon: release-3.6.0-1
> > > https://github.com/apache/zookeeper/tree/release-3.6.0-1
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Please note that we are adding a new jar to the dependency set for
> > > clients: zookeeper-metrics-providers.
> > >
> > > Should we release this candidate?
> > >
> > > Enrico Olivelli
> >
> >
>

Re: [EXTERNAL] Re: 3.5.7

[Norbert Kalmar]

So about the SSL tests failing... my hosts file was messed up, turns out
localhost did not translate to 127.0.0.1 very well (it had a few strange
entry).
Probably something in the Docker image also around the hosts file, Máté is
taking a look.

Regards,
Norbert

On Mon, Feb 3, 2020 at 10:15 AM Norbert Kalmar  wrote:

> Hi Craig,
>
> It's already committed:
> https://github.com/apache/zookeeper/commit/59337e7ec8ab67fecf7cfc1b8c5b76397c02bfd6
>
> Sorry I'm running a bit late with the release, I got a flu or some virus
> last week, wasn't too productive.
>
> Anyway, I did some testing, and SSL tests seems a bit flaky, they almost
> constantly kept failing. Reducing fork count helped a bit, first to 4 then
> to 1. Still was a bit flaky, the truth to be told the machine had some
> loads and the problem was server not coming up in time. But after killing
> the load it still happened.
> On my MacOS unit tests run without a problem with the default 8 surefire
> core.
> Turns out the SSL tests are also failing while building with the docker
> image, I just talked to Máté about this.
>
> Oh, it's both 3.5 and master branch.
>
> I'll dig into this a little before continuing.
>
> Regards,
> Norbert
>
> On Sat, Feb 1, 2020 at 1:01 PM Enrico Olivelli 
> wrote:
>
>> Il Ven 31 Gen 2020, 16:16 Craig.Condit  ha
>> scritto:
>>
>> > Would it be possible to get ZOOKEEPER-3638 included in 3.5.7 as well?
>>
>> I thought it already went in
>>
>> we really must include it.
>>
>> Enrico
>>
>> The version of Jetty included in 3.5.6 breaks the admin server. We have
>> > been running a backport of 3638 (which just upgrades to a later version)
>> > successfully on 3.5.6 here without issue.
>> >
>> > Thanks,
>> >
>> > Craig Condit
>> >
>> >
>> > 
>> > From: Norbert Kalmar 
>> > Sent: Thursday, January 30, 2020 3:06 PM
>> > To: DevZooKeeper 
>> > Subject: [EXTERNAL] Re: 3.5.7
>> >
>> > Hi all,
>> >
>> > Just a heads up.
>> >
>> > All patch that we wanted (as far as I'm aware, let me know if you miss
>> > something) for the 3.5.7 release has been committed to branch 3.5.
>> Mainly
>> > this was:
>> > - ZOOKEEPER-3701 (split brain)
>> > - ZOOKEEPER-3482 (some SASL stuff)
>> > - ZOOKEEPER-3699 (fix CVE about Jackson)
>> >
>> > And a few other nice to haves (like ZOOKEEPER-1105 C client WARN msg
>> fix)
>> > that also made it.
>> >
>> > I started testing the 3.5 branch and I will create a release branch soon
>> > (probably tomorrow).
>> >
>> > Regards,
>> > Norbert
>> >
>> > On Mon, Jan 27, 2020 at 11:30 AM Norbert Kalmar 
>> > wrote:
>> >
>> > > Only blocker left for 3.5.7 is ZOOKEEPER-3701, patch available here:
>> > > https://github.com/apache/zookeeper/pull/1233
>> > >
>> > > I'll wait another 0.5-1 day if anyone wants to take a look at it. Then
>> > > I'll commit and start the 3.5.7 release process.
>> > >
>> > > Thanks,
>> > > Norbert
>> > >
>> > > On Thu, Jan 23, 2020 at 11:29 AM Norbert Kalmar > >
>> > > wrote:
>> > >
>> > >> The patch fixed the CVE warning
>> > >> https://builds.apache.org/job/zookeeper-master-maven-owasp/339/
>> > >>
>> > >> Norbert
>> > >>
>> > >> On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar <
>> nkal...@cloudera.com>
>> > >> wrote:
>> > >>
>> > >>> Thanks Patrick, I'll review and preferably commit your patch, which
>> > >>> should negate the CVE warning.
>> > >>>
>> > >>> Regards,
>> > >>> Norbert
>> > >>>
>> > >>> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt 
>> wrote:
>> > >>>
>> > >>>> owasp is failing on branch-3.5,
>> > >>>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>> > >>>>
>> > >>>> seems the same as:
>> > >>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>> > >>>>
>> > >>>> Patrick
>> > >>>>
>> > >>>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly 
>> wrote:
>> > >>>>
>> > >>>> > > Would you have time for a quick fix ?
>> > >>>> >
>> > >>>> > The measures to avoid the problem are listed at the end of the
>> JIRA
>> > >>>> > description. I can't submit a PR until I get permission from my
>> > >>>> > company legal to push to ZK.
>> > >>>> >
>> > >>>> > -Ivan
>> > >>>> >
>> > >>>>
>> > >>>
>> >
>>
>

Re: [VOTE] Apache ZooKeeper release 3.6.0 candidate 1

[Norbert Kalmar]

+1 (non-binding)

- unit tests pass
- built and started ZK + run few commands from source tarball
- checked bin tarball, license files, run ZK + few commands
- signature OK.

Tested on MacOS with openJDK 1.8, no problems

I also run the tests on a Linux machine, found the problem why SSL tests
failed (I just wrote to DEv about this), after fixing my hosts file it
passed!

@Jordan - I reviewed and +1-d the patch you asked, sorry for not making to
3.6.0, hopefully a 3.6.1 comes out quick, people don't like versions ending
in x.x.0 anyway :)

Thanks Enrico!

- Norbert


On Mon, Feb 3, 2020 at 4:31 AM Jordan Zimmerman 
wrote:

> No big issues with Curator that I could find
>
> +1 (non binding)
>
> -Jordan
>
> > On Feb 1, 2020, at 10:02 AM, Enrico Olivelli 
> wrote:
> >
> > This is the second release candidate for Apache ZooKeeper 3.6.0.
> >
> > It is a major release and it introduces a lot of new features, most
> notably:
> > - Built-in data consistency check inside ZooKeeper
> > - Allow Followers to host Observers
> > - Authentication enforcement
> > - Pluggable metrics system for ZooKeeper (and Prometheus.io integration)
> > - TLS Port unification
> > - Audit logging in ZooKeeper servers
> > - Improve resilience to network (advertise multiple addresses for
> > members of a Zookeeper cluster)
> > - Persistent Recursive Watches
> > - add an API and the corresponding CLI to get total count of recursive
> > sub nodes under a specific path
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12326518
> >
> > *** Please download, test and vote by February 4th 2020, 23:59 UTC+0. ***
> >
> > Source files:
> > https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-1/
> >
> > Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1047/
> >
> > The staging version of the website is:
> >
> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-1/website/
> >
> > The release candidate tag in git to be voted upon: release-3.6.0-1
> > https://github.com/apache/zookeeper/tree/release-3.6.0-1
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Please note that we are adding a new jar to the dependency set for
> > clients: zookeeper-metrics-providers.
> >
> > Should we release this candidate?
> >
> > Enrico Olivelli
>
>

Re: My Contributions

[Norbert Kalmar]

Hi David,

Thanks for your contributions!
We consider the patch committed if it should be backported or not. In my
opinion if a contributor finds a bug/improvement to do, we can't expect
him/her to know what branches this should be committed to. Of course if the
author takes the time to dig through the different branches and suggests to
backport or not, than that's great, but just creating a patch against
master works just as well :) And of course sometimes (a great example is
security fixes) we just automatically backport the change to all (or just
some) branch still maintained.

And one more thing: anyone is in a position to drive big changes, ZooKeeper
is open source, contributors are welcomed :)
(Of course depending on how big a change, it might be worth starting on
this dev list to vote / give a heads up / ask for help).

Regards,
Norbert


On Mon, Feb 3, 2020 at 6:04 AM David Mollitor  wrote:

> Hey Team,
>
> I have very much appreciated all of the support and merging of my PRs.  I
> just wanted to clarify / get clarification of the ZK release process.
>
> I'm not in a position to drive any big changes or big big fixes into the
> product.  I'm just trying to keep the code clean, keeping down the lines of
> code. and to hopefully preventing others from making mistakes,... and maybe
> add a percentage point or two on performance.
>
> Unless I suggest otherwise, I intend all of my work to be pushed into
> master.  No need to back port it to existing released versions.  Typically,
> only big fixes are added to such releases.
>
> All the more reason to not back port my work to past releases, several big
> users are consuming changes from trunk anyway.  So my changes are being
> consumed pretty quickly and do not need to go out in previous releases to
> add value.
>
> Thanks,
> David
>

Re: [EXTERNAL] Re: 3.5.7

[Norbert Kalmar]

Hi Craig,

It's already committed:
https://github.com/apache/zookeeper/commit/59337e7ec8ab67fecf7cfc1b8c5b76397c02bfd6

Sorry I'm running a bit late with the release, I got a flu or some virus
last week, wasn't too productive.

Anyway, I did some testing, and SSL tests seems a bit flaky, they almost
constantly kept failing. Reducing fork count helped a bit, first to 4 then
to 1. Still was a bit flaky, the truth to be told the machine had some
loads and the problem was server not coming up in time. But after killing
the load it still happened.
On my MacOS unit tests run without a problem with the default 8 surefire
core.
Turns out the SSL tests are also failing while building with the docker
image, I just talked to Máté about this.

Oh, it's both 3.5 and master branch.

I'll dig into this a little before continuing.

Regards,
Norbert

On Sat, Feb 1, 2020 at 1:01 PM Enrico Olivelli  wrote:

> Il Ven 31 Gen 2020, 16:16 Craig.Condit  ha
> scritto:
>
> > Would it be possible to get ZOOKEEPER-3638 included in 3.5.7 as well?
>
> I thought it already went in
>
> we really must include it.
>
> Enrico
>
> The version of Jetty included in 3.5.6 breaks the admin server. We have
> > been running a backport of 3638 (which just upgrades to a later version)
> > successfully on 3.5.6 here without issue.
> >
> > Thanks,
> >
> > Craig Condit
> >
> >
> > 
> > From: Norbert Kalmar 
> > Sent: Thursday, January 30, 2020 3:06 PM
> > To: DevZooKeeper 
> > Subject: [EXTERNAL] Re: 3.5.7
> >
> > Hi all,
> >
> > Just a heads up.
> >
> > All patch that we wanted (as far as I'm aware, let me know if you miss
> > something) for the 3.5.7 release has been committed to branch 3.5. Mainly
> > this was:
> > - ZOOKEEPER-3701 (split brain)
> > - ZOOKEEPER-3482 (some SASL stuff)
> > - ZOOKEEPER-3699 (fix CVE about Jackson)
> >
> > And a few other nice to haves (like ZOOKEEPER-1105 C client WARN msg fix)
> > that also made it.
> >
> > I started testing the 3.5 branch and I will create a release branch soon
> > (probably tomorrow).
> >
> > Regards,
> > Norbert
> >
> > On Mon, Jan 27, 2020 at 11:30 AM Norbert Kalmar 
> > wrote:
> >
> > > Only blocker left for 3.5.7 is ZOOKEEPER-3701, patch available here:
> > > https://github.com/apache/zookeeper/pull/1233
> > >
> > > I'll wait another 0.5-1 day if anyone wants to take a look at it. Then
> > > I'll commit and start the 3.5.7 release process.
> > >
> > > Thanks,
> > > Norbert
> > >
> > > On Thu, Jan 23, 2020 at 11:29 AM Norbert Kalmar 
> > > wrote:
> > >
> > >> The patch fixed the CVE warning
> > >> https://builds.apache.org/job/zookeeper-master-maven-owasp/339/
> > >>
> > >> Norbert
> > >>
> > >> On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar  >
> > >> wrote:
> > >>
> > >>> Thanks Patrick, I'll review and preferably commit your patch, which
> > >>> should negate the CVE warning.
> > >>>
> > >>> Regards,
> > >>> Norbert
> > >>>
> > >>> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt 
> wrote:
> > >>>
> > >>>> owasp is failing on branch-3.5,
> > >>>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
> > >>>>
> > >>>> seems the same as:
> > >>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
> > >>>>
> > >>>> Patrick
> > >>>>
> > >>>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly 
> wrote:
> > >>>>
> > >>>> > > Would you have time for a quick fix ?
> > >>>> >
> > >>>> > The measures to avoid the problem are listed at the end of the
> JIRA
> > >>>> > description. I can't submit a PR until I get permission from my
> > >>>> > company legal to push to ZK.
> > >>>> >
> > >>>> > -Ivan
> > >>>> >
> > >>>>
> > >>>
> >
>

Re: 3.5.7

[Norbert Kalmar]

Hi all,

Just a heads up.

All patch that we wanted (as far as I'm aware, let me know if you miss
something) for the 3.5.7 release has been committed to branch 3.5. Mainly
this was:
- ZOOKEEPER-3701 (split brain)
- ZOOKEEPER-3482 (some SASL stuff)
- ZOOKEEPER-3699 (fix CVE about Jackson)

And a few other nice to haves (like ZOOKEEPER-1105 C client WARN msg fix)
that also made it.

I started testing the 3.5 branch and I will create a release branch soon
(probably tomorrow).

Regards,
Norbert

On Mon, Jan 27, 2020 at 11:30 AM Norbert Kalmar 
wrote:

> Only blocker left for 3.5.7 is ZOOKEEPER-3701, patch available here:
> https://github.com/apache/zookeeper/pull/1233
>
> I'll wait another 0.5-1 day if anyone wants to take a look at it. Then
> I'll commit and start the 3.5.7 release process.
>
> Thanks,
> Norbert
>
> On Thu, Jan 23, 2020 at 11:29 AM Norbert Kalmar 
> wrote:
>
>> The patch fixed the CVE warning
>> https://builds.apache.org/job/zookeeper-master-maven-owasp/339/
>>
>> Norbert
>>
>> On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar 
>> wrote:
>>
>>> Thanks Patrick, I'll review and preferably commit your patch, which
>>> should negate the CVE warning.
>>>
>>> Regards,
>>> Norbert
>>>
>>> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt  wrote:
>>>
>>>> owasp is failing on branch-3.5,
>>>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>>>>
>>>> seems the same as:
>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>>>>
>>>> Patrick
>>>>
>>>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly  wrote:
>>>>
>>>> > > Would you have time for a quick fix ?
>>>> >
>>>> > The measures to avoid the problem are listed at the end of the JIRA
>>>> > description. I can't submit a PR until I get permission from my
>>>> > company legal to push to ZK.
>>>> >
>>>> > -Ivan
>>>> >
>>>>
>>>

Re: 3.5.7

[Norbert Kalmar]

Only blocker left for 3.5.7 is ZOOKEEPER-3701, patch available here:
https://github.com/apache/zookeeper/pull/1233

I'll wait another 0.5-1 day if anyone wants to take a look at it. Then I'll
commit and start the 3.5.7 release process.

Thanks,
Norbert

On Thu, Jan 23, 2020 at 11:29 AM Norbert Kalmar 
wrote:

> The patch fixed the CVE warning
> https://builds.apache.org/job/zookeeper-master-maven-owasp/339/
>
> Norbert
>
> On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar 
> wrote:
>
>> Thanks Patrick, I'll review and preferably commit your patch, which
>> should negate the CVE warning.
>>
>> Regards,
>> Norbert
>>
>> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt  wrote:
>>
>>> owasp is failing on branch-3.5,
>>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>>>
>>> seems the same as:
>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>>>
>>> Patrick
>>>
>>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly  wrote:
>>>
>>> > > Would you have time for a quick fix ?
>>> >
>>> > The measures to avoid the problem are listed at the end of the JIRA
>>> > description. I can't submit a PR until I get permission from my
>>> > company legal to push to ZK.
>>> >
>>> > -Ivan
>>> >
>>>
>>

Re: 3.5.7

[Norbert Kalmar]

The patch fixed the CVE warning
https://builds.apache.org/job/zookeeper-master-maven-owasp/339/

Norbert

On Thu, Jan 23, 2020 at 11:07 AM Norbert Kalmar 
wrote:

> Thanks Patrick, I'll review and preferably commit your patch, which should
> negate the CVE warning.
>
> Regards,
> Norbert
>
> On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt  wrote:
>
>> owasp is failing on branch-3.5,
>> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>>
>> seems the same as:
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>>
>> Patrick
>>
>> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly  wrote:
>>
>> > > Would you have time for a quick fix ?
>> >
>> > The measures to avoid the problem are listed at the end of the JIRA
>> > description. I can't submit a PR until I get permission from my
>> > company legal to push to ZK.
>> >
>> > -Ivan
>> >
>>
>

Re: 3.5.7

[Norbert Kalmar]

Thanks Patrick, I'll review and preferably commit your patch, which should
negate the CVE warning.

Regards,
Norbert

On Wed, Jan 22, 2020 at 5:31 PM Patrick Hunt  wrote:

> owasp is failing on branch-3.5,
> [ERROR] jackson-databind-2.9.10.1.jar: CVE-2019-20330
>
> seems the same as:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3699
>
> Patrick
>
> On Wed, Jan 22, 2020 at 5:12 AM Ivan Kelly  wrote:
>
> > > Would you have time for a quick fix ?
> >
> > The measures to avoid the problem are listed at the end of the JIRA
> > description. I can't submit a PR until I get permission from my
> > company legal to push to ZK.
> >
> > -Ivan
> >
>

Re: Release procedure for 3.6.0 - new ideas with maven release plugin

[Norbert Kalmar]

I also agree that branching for a release makes the process easier.
Especially if we are talking about the active branch. For example, I
wouldn't do a branch for a new 3.4 release, as it's pretty much EOL and
hardly anything makes it there.
My opinion, if it makes sense to branch out for a release, then why not. A
branch "cost" is pretty much nothing. So probably this can be the decision
of the RM? I know committers should always be aware of the branching
strategy used in order to know where to commit, but I don't think this
should be a problem to follow. We don't have releases that often.
Although it is getting more and more frequent fortunately :)

TLDR: let the RM decide for every release to branch or not? Seems this was
the case recently anyway.

Regards,
Norbert

On Wed, Jan 22, 2020 at 5:22 PM Patrick Hunt  wrote:

> On Wed, Jan 22, 2020 at 2:59 AM Andor Molnar  wrote:
>
> > Hi,
> >
> > We started a discussion on Slack about branching/not branching for new
> > releases like branch-3.5.6, branch-3.6.0, etc. I noticed that Enrico has
> > not created branch for 3.6.0 and I’d like to talk about the motivations.
> >
> > Personally I’ve found the separate branch useful in the release process,
> > because I didn’t have to ask people not to submit new patches on
> branch-3.5
> > due to ongoing release process. On the flipside, fixes which have come up
> > during release validation had to be submitted for 2 branches: branch-3.5
> > and branch-3.5.X.
> >
> >
> We didn't used to have branches for releases and added them specifically
> for this reason - to allow commits while release in progress. Given the RM
> can (should) make the decision whether or not to pull things into a release
> candidate this seems fine with me (3.5 vs 3.5.x)
>
> Patrick
>
>
> > Either way I would keep our current branching strategy, but also would
> > like to hear everybody’s opinion.
> >
> > Regards,
> > Andor
> >
> >
> >
> >
> > > On 2019. Nov 11., at 22:19, Enrico Olivelli 
> wrote:
> > >
> > > Hello,
> > > in 3.5 we did a great work (in particular Andor and Norbert) in order
> to
> > > mavenize our repository and current we are performing releases from 3.5
> > > branch with Maven.
> > >
> > > For 3.6.0 I would like to try to enhance the procedure a bit and make
> it
> > > simpler, by using the Maven Release Plugin [1].
> > >
> > > I am drafting a new procedure [2], but it is still not ready,
> > > Once I am done with it the procedure will look like this [3] or [4]
> > >
> > > The major problem with the Maven release plugin is to update the
> versions
> > > on the C client, but I have found some trick so I am doing tests.
> > >
> > > I am just waiting for pending PRs that have been said to be nice to
> have
> > on
> > > 3.6.0 to land to master then I am confident we are ready to cut a
> release
> > >
> > > Any comments and help are welcome
> > >
> > > Enrico
> > >
> > > [1] https://maven.apache.org/maven-release/maven-release-plugin/
> > > [2]
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=135860428
> > > [3]
> > https://www.apache.org/dev/publishing-maven-artifacts.html#staging-maven
> > > [4] https://github.com/diennea/herddb/wiki/Release-guide
> >
> >
>

Re: [ANNOUNCE] Enrico Olivelli new ZooKeeper PMC Member

[Norbert Kalmar]

Congratulations Enrico, well earned! :)

Regards,
Norbert

On Tue, Jan 21, 2020 at 11:15 PM rammohan ganapavarapu <
rammohanga...@gmail.com> wrote:

> Congratulations Enrico!!
>
> On Tue, Jan 21, 2020 at 1:41 PM Flavio Junqueira  wrote:
>
> > I'm pleased to announce that Enrico Olivelli recently became the newest
> > ZooKeeper PMC member. Enrico has contributed immensely to this community;
> > he became a ZooKeeper committer in May 2019 and now he joins the PMC.
> >
> > Join me in congratulating him on the achievement. Congrats, Enrico!
> >
> > -Flavio on behalf of the Apache ZooKeeper PMC
>

Re: Messy releasenotes.md file on master and on website

[Norbert Kalmar]

+1 on option A.
But!
I think we should have a page containing all the changes in all supported
line. So like option C.

Why? - I agree a specific release should only contain changes to that
version. If someone wants to see the changes that went into various
releases, he/she should check the website (page that is basically option
C). And a link to this page from releasenotes.
At least that's what I usually see and expect from projects.
But this is based on my personal preference, I don't know what was the
original intention on ZooKeeper, so let's wait for the PMCs to chip in :)

Thanks for bringing this up Enrico!

Regards,
Norbert

On Thu, Dec 19, 2019 at 11:57 PM Enrico Olivelli 
wrote:

> Hi,
> I am preparing release notes for 3.6.0,
> as branch-3.6 has been cut from master branch I found that
> releasenotes.mdtalk about ZooKeeper 3.0.0 !!
>
>
> https://github.com/apache/zookeeper/blob/master/zookeeper-docs/src/main/resources/markdown/releasenotes.md
>
> I found also that when I wrote the release notes of 3.5.6 I had only
> committed them to branch-3.5.6 and not to branch-3.5.
> I have fixed branch-3.5 by adding the release notes for 3.5.6.
>
> Then I found that in branch-3.5 we only have the release notes from 3.5.0
> to 3.5.6 so we are missing the release notes for 3.4, 3.3.
>
> here is it the public website:
> http://zookeeper.apache.org/doc/r3.5.6/releasenotes.html
>
> If you see release notes of 3.4.14 you will see ONLY notes about 3.4
> release line
> http://zookeeper.apache.org/doc/r3.4.14/releasenotes.html
>
> is this intentional?
>
>
> If this is not intentional I suggest these ways:
> A) let every release hold only the specific changes for that release (so in
> 3.4.14 I would see ONLY 3.1.14 news)
> B) let every release hold all of the changes from the beginning of the
> project up to that version
> C) like B),but keep only latest supported release line, so keep the history
> from 3.4 up to the current version
>
> I think that the best option is A):
> - the list won't grow without bounds
> - in the "relases notes" page you see only news about that version
>
>
> Enrico
>

Re: ZK makes apache 2019 "top 5" projects

[Norbert Kalmar]

Kudos to everyone!
Also, nice to see so many contributions. And not just veterans, but plenty
of new community members! Thank you all!

Regards,
Norbert

On Thu, Dec 12, 2019 at 2:00 PM Jordan Zimmerman 
wrote:

> Fantastic
>
> 
> Jordan Zimmerman
>
> > On Dec 12, 2019, at 3:49 AM, Flavio Junqueira  wrote:
> >
> > +1, thank you all for the hard work.
> >
> > -Flavio
> >
> >> On 12 Dec 2019, at 08:36, Enrico Olivelli  wrote:
> >>
> >> Yes, great.
> >>
> >> Please also note that Kafka and Lucene/Solr that are still listed in
> that
> >> list  are using Zookeeper :)
> >>
> >>
> >> Enrico
> >>
> >> Il gio 12 dic 2019, 05:46 tison  ha scritto:
> >>
> >>> Kudos!
> >>>
> >>> Best,
> >>> tison.
> >>>
> >>>
> >>> Patrick Hunt  于2019年12月12日周四 上午11:32写道：
> >>>
>  This is really awesome, check it out:
>  https://twitter.com/phunt/status/1204966326118141952
> 
>  Kudos ZooKeeper community on all the hard work and efforts!
> 
>  Patrick
> 
> >>>
> >
>

Re: Hadoop logo

[Norbert Kalmar]

Oh, wow, I didn't even notice that until now.
Makes sense, knowing a lot of the time ZK is used "standalone" (I mean
outside of any hadoop ecosystem).

Regards,
Norbert

On Thu, Dec 12, 2019 at 2:52 PM Flavio Junqueira  wrote:

> Should we remove that Hadoop logo from the documentation? It has been a
> while that we aren't a subproject of Hadoop any longer.
>
> -Flavio

Re: Creating branch-3.6

[Norbert Kalmar]

Oh, and shouldn't we decommission branch-3.4? :)

On Tue, Dec 10, 2019 at 12:45 PM Andor Molnar 
wrote:

> Okay Enrico, go ahead please.
>
> Andor
>
>
> On Tue, Dec 10, 2019 at 10:42 AM Norbert Kalmar
>  wrote:
>
> > +1 to create the 3.6 branch now. I don't expect a whole lot of patches
> that
> > needs to be added to 3.6 and master as well. Only the few remaining
> tickets
> > that is planned and occasional bugfixes. But, imho, better to push out a
> > 3.6.0 that no one will probably use because they'll just wait for a 3.6.1
> > "stable" :) And as Enrico pointed out, branching speeds things up, as the
> > work for releasing can start with things like testing and stabilization.
> >
> > Regards,
> > Norbert
> >
> > On Mon, Dec 9, 2019 at 4:28 PM Enrico Olivelli 
> > wrote:
> >
> > > Il giorno lun 9 dic 2019 alle ore 14:00 Andor Molnar  >
> > ha
> > > scritto:
> > >
> > > > Thanks for driving this Enrico.
> > > >
> > > > a.) Version number: I think we discussed this already and agreed on
> > 3.7.0
> > > > for now. As Enrico pointed out we can change it at any time anyway.
> > > >
> > > > b.) Why do you want to cut now instead of the point when we do the
> > first
> > > > release? To me it will just add one more branch when committing. What
> > are
> > > > the pros?
> > > >
> > >
> > >
> > > Because we are close to the release (hopefully).
> > > We have a few patches that the community (Andro, Fangmin) wanted to
> > add
> > > on 3.6.0, all of the other patches can go to 3.6.0
> > >
> > > Once we have a branch-3.6 we can focus on consolidation, manual
> > testing...
> > > I we continue to add patches we can't be sure the branch is very good
> > shape
> > > before the release
> > >
> > > I really would like to have the first RC before Christmas
> > >
> > > Enrico
> > >
> > >
> > >
> > > >
> > > > Andor
> > > >
> > > >
> > > >
> > > > > On 2019. Dec 9., at 13:37, Enrico Olivelli 
> > > wrote:
> > > > >
> > > > > Tamas,
> > > > >
> > > > > Il lun 9 dic 2019, 13:33 Tamas Penzes  >
> > ha
> > > > > scritto:
> > > > >
> > > > >> Hi All,
> > > > >>
> > > > >> Just a question to the community, should we set the version number
> > to
> > > > >> 4.0.0-SNAPSHOT in the master branch?
> > > > >>
> > > > >
> > > > > We will use 4 when we decide to break compatibility in a very hard
> > way.
> > > > >
> > > > > Currently there is no plan, so I would go with 3.7.0.
> > > > >
> > > > > Btw We can change it at any time.
> > > > > Let's make one step at a time
> > > > >
> > > > > Thanks for bringing up the topic
> > > > >
> > > > > Enrico
> > > > >
> > > > > I think it's time to decide whether we want to do it or not.
> > > > >>
> > > > >> Regards, Tamaas
> > > > >>
> > > > >> On Mon, Dec 9, 2019 at 8:53 AM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > >> wrote:
> > > > >>
> > > > >>> Hi folks,
> > > > >>> I think it is time to create branch-3.6 and bump master to
> > > > 3.7.0-SNAPSHOT
> > > > >>>
> > > > >>> This does not mean to release 3.6.0, we can still cherry pick
> > patches
> > > > >> from
> > > > >>> master brach.
> > > > >>> But it is time to start consolidating the branch and pick only
> > > > important
> > > > >>> changes
> > > > >>>
> > > > >>> If no one objects I will proceed with this step next weekend
> (14-15
> > > > >>> December)
> > > > >>>
> > > > >>> Best regards
> > > > >>>
> > > > >>> Enrico
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: Creating branch-3.6

[Norbert Kalmar]

+1 to create the 3.6 branch now. I don't expect a whole lot of patches that
needs to be added to 3.6 and master as well. Only the few remaining tickets
that is planned and occasional bugfixes. But, imho, better to push out a
3.6.0 that no one will probably use because they'll just wait for a 3.6.1
"stable" :) And as Enrico pointed out, branching speeds things up, as the
work for releasing can start with things like testing and stabilization.

Regards,
Norbert

On Mon, Dec 9, 2019 at 4:28 PM Enrico Olivelli  wrote:

> Il giorno lun 9 dic 2019 alle ore 14:00 Andor Molnar  ha
> scritto:
>
> > Thanks for driving this Enrico.
> >
> > a.) Version number: I think we discussed this already and agreed on 3.7.0
> > for now. As Enrico pointed out we can change it at any time anyway.
> >
> > b.) Why do you want to cut now instead of the point when we do the first
> > release? To me it will just add one more branch when committing. What are
> > the pros?
> >
>
>
> Because we are close to the release (hopefully).
> We have a few patches that the community (Andro, Fangmin) wanted to add
> on 3.6.0, all of the other patches can go to 3.6.0
>
> Once we have a branch-3.6 we can focus on consolidation, manual testing...
> I we continue to add patches we can't be sure the branch is very good shape
> before the release
>
> I really would like to have the first RC before Christmas
>
> Enrico
>
>
>
> >
> > Andor
> >
> >
> >
> > > On 2019. Dec 9., at 13:37, Enrico Olivelli 
> wrote:
> > >
> > > Tamas,
> > >
> > > Il lun 9 dic 2019, 13:33 Tamas Penzes  ha
> > > scritto:
> > >
> > >> Hi All,
> > >>
> > >> Just a question to the community, should we set the version number to
> > >> 4.0.0-SNAPSHOT in the master branch?
> > >>
> > >
> > > We will use 4 when we decide to break compatibility in a very hard way.
> > >
> > > Currently there is no plan, so I would go with 3.7.0.
> > >
> > > Btw We can change it at any time.
> > > Let's make one step at a time
> > >
> > > Thanks for bringing up the topic
> > >
> > > Enrico
> > >
> > > I think it's time to decide whether we want to do it or not.
> > >>
> > >> Regards, Tamaas
> > >>
> > >> On Mon, Dec 9, 2019 at 8:53 AM Enrico Olivelli 
> > >> wrote:
> > >>
> > >>> Hi folks,
> > >>> I think it is time to create branch-3.6 and bump master to
> > 3.7.0-SNAPSHOT
> > >>>
> > >>> This does not mean to release 3.6.0, we can still cherry pick patches
> > >> from
> > >>> master brach.
> > >>> But it is time to start consolidating the branch and pick only
> > important
> > >>> changes
> > >>>
> > >>> If no one objects I will proceed with this step next weekend (14-15
> > >>> December)
> > >>>
> > >>> Best regards
> > >>>
> > >>> Enrico
> > >>>
> > >>
> >
> >
>

Re: Releasing 3.6.0 - Pending Patches

[Norbert Kalmar]

Thanks Mate, committed to master, this fix will now be in 3.6.0!
(AdminServer NPE)

Regards,
Norbert

On Fri, Nov 29, 2019 at 4:43 PM Mate Szalay-Beko
 wrote:

> Sorry to extend the list :p
> here is a low risk / quite trivial bugfix I created based on the NPE
> problem reported yesterday on the user mail list:
> https://github.com/apache/zookeeper/pull/1161
> I think we could add this to 3.6.0 as well...
>
> Cheers,
> Mate
>
> On Fri, Nov 29, 2019 at 3:50 PM Enrico Olivelli 
> wrote:
>
> > Okay, seems that we are only waiting for:
> > https://github.com/apache/zookeeper/pull/934 Enforce the quota limit
> > https://github.com/apache/zookeeper/pull/1059  Real time data
> consistency
> > check during broadcast time
> > https://github.com/apache/zookeeper/pull/1159 Autodetection of openssl
> > during ZooKeeper C client build
> >
> > I am also preparing a new release procedure that uses the Maven Release
> > Plugin.
> >
> > Hopefully we will have a first RC within XMas !
> >
> >
> > Enrico
> >
> > Il giorno ven 15 nov 2019 alle ore 20:30 Fangmin Lv  >
> > ha scritto:
> >
> > > Would like to see the following feature being included as part of the
> > 3.6.0
> > > release:
> > >
> > > * ZOOKEEPER-3512: Real time data consistency check during broadcast
> time
> > (
> > > https://github.com/apache/zookeeper/pull/1059)
> > >
> > > We've merged the "ZOOKEEPER-3150: Add tree digest check and verify data
> > > integrity when loading from disk",
> > > which is partial of this consistency checking feature. It's nice to
> > > announce that we'll have real time consistency
> > > check in 3.6.0.
> > >
> > > Was busy on some internal features work, will commit time next week to
> > get
> > > that up to date.
> > >
> > > Thanks,
> > > Fangmin
> > >
> > > On Fri, Oct 4, 2019 at 7:16 AM Mate Szalay-Beko
> > >  wrote:
> > >
> > > > > ZOOKEEPER-2122: Impplement SSL support in the Zookeeper C client
> > > library
> > > >
> > > > I just pushed a new PR for this one yesterday (I was taking over the
> > > ticket
> > > > from Suhas Dantkale and extended his last PR).
> > > > Please review this PR for ZOOKEEPER-2122:
> > > > https://github.com/apache/zookeeper/pull/1107
> > > >
> > > > Cheers,
> > > > Mate
> > > >
> > > > On Thu, Oct 3, 2019 at 12:22 AM Enrico Olivelli  >
> > > > wrote:
> > > >
> > > > > I am taking a look to those patches.
> > > > >
> > > > > I feel we are not in hurry in adding other stuff, 3.6.0 is already
> a
> > > > greate
> > > > > feature release.
> > > > > I hope we will start to cut release more often so if a feature
> can't
> > be
> > > > > delivered it only has to wait for the next train
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il mer 2 ott 2019, 16:00 David Mollitor  ha
> > > scritto:
> > > > >
> > > > > > Can I also throw in ZOOKEEPER-3342 and ZOOKEEPER-3340 ? :)
> > > > > >
> > > > > > On Wed, Oct 2, 2019, 9:36 AM Andor Molnar 
> > wrote:
> > > > > >
> > > > > > > + ZOOKEEPER-1260 Audit logging in ZooKeeper servers (
> > > > > > > https://github.com/apache/zookeeper/pull/338)
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > On 2019. Oct 2., at 11:05, Andor Molnar 
> > > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > (opening new thread for clarity)
> > > > > > > >
> > > > > > > > Imho the next thing that we should agree on is the list of
> > > pending
> > > > > > > patches we’d like to submit before the first beta. Here’re some
> > of
> > > > the
> > > > > > > pending major new features off the top of my head:
> > > > > > > >
> > > > > > > > - ZOOKEEPER-3188: Improve resilience to network (
> > > > > > > https://github.com/apache/zookeeper/pull/1048)
> > > > > > > > - ZOOKEEPER-2122: Impplement SSL support in the Zookeeper C
> > > client
> > > > > > > library (https://github.com/apache/zookeeper/pull/990)
> > > > > > > > - ZOOKEEPER-1416: Persistent Recursive Watch (
> > > > > > > https://github.com/apache/zookeeper/pull/136)
> > > > > > > > - ZOOKEEPER-3301: Enforce the quota limit (
> > > > > > > https://github.com/apache/zookeeper/pull/934)
> > > > > > > >
> > > > > > > > These’re all in Patch Available state, so in _theory_ we only
> > > need
> > > > > > > reviewers to submit them.
> > > > > > > >
> > > > > > > > Andor
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

[Norbert Kalmar]

Hi Pierre,

As this is a critical bugfix IMO, I don't see why it shouldn't be
backported to 3.4. It hasn't been stated AFAIK that 3.4 is no longer
supported.
As for when there could be a new 3.4 release, that's a different question
though. Let's see what the PMCs think about it.

(I also added 3.5.5 to the jira's fix version, az is contains the fix but
was missing.)

Regards,
Norbert

On Tue, Oct 22, 2019 at 2:33 PM 殷芳玺  wrote:

> Hi, Folks:
>
>
> ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also
> exists in 3.4 branch.
>
> In one of our 3.4.13 clusters, the data inconsistency happens for many
> times.
>
> After digging some transaction logs and snapshot, we believe that
> ZOOKEEPER-3104 is
> the main risk to contributes to our data inconsistency.
>
> The risk probability maybe higher than we can consider in real product
> environment.  The serialization of big DataTree may leads to a big risk
> time window in the high writing traffic situation. Any failure during the
> risk time window would cause the data inconsistency.
>
> The data inconsistency is almost unacceptable in zookeeper semantic.
>
> This issue is already fixed in 3.6. But I think it is very necessary to
> backport ZOOKEEPER-3104<
> https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4,
> especially in the situation that the migration from 3.4 to 3.5 actually
> take more effort to evaluate the compatibility risk in real product
> environment.
>
> I have already opened an issue [ZOOKEEPER-3589](
> https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a github
> pull request https://github.com/apache/zookeeper/pull/1123 to fix it.
>
> So I suggest to accept the pull request and release it in 3.4.16. This fix
> would make branch-3.4 more robust and fully-fledged.
>
>
> Thanks!
>
>
> Best regards
>
> Pierre Yin
>
>
>

Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 4

[Norbert Kalmar]

+1 (non-binding)

- unit tests pass
- built and started ZK + run few commands from source tarball
- checked bin tarball, license files, run ZK + few commands
- signature OK.
- git tag OK :)

Thanks Enrico!

Norbert

On Wed, Oct 9, 2019 at 10:45 PM Andor Molnar  wrote:

> +1
>
> Verified…
>
> - checksums, signature,
> - unit tests
> - 3-node cluster smoke tests.
>
> Andor
>
>
> > On 2019. Oct 9., at 22:40, Enrico Olivelli  wrote:
> >
> > Il mer 9 ott 2019, 21:14 Patrick Hunt  ha scritto:
> >
> >> +1 checksums/sig validated. rat ran clean and I was able to build and
> >> exercise the code just fine with java 8.
> >>
> >> Note dep check is failing again however:
> >>
> >> jackson-databind-2.9.10.jar
> >> (pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.9.10,
> >> cpe:2.3:a:fasterxml:jackson:2.9.10:*:*:*:*:*:*:*,
> >> cpe:2.3:a:fasterxml:jackson-databind:2.9.10:*:*:*:*:*:*:*) :
> >> CVE-2019-16942, CVE-2019-16943
> >>
> >> I looked at the issue and they seem very specific, given that and the
> >> status of databind these days I think we should get this one next time
> >> around vs re-re... spinning the rc. What do you think?
> >>
> >
> > Agreed.
> > And as we are doing a very limited use of Jackson we can look for a
> > replacement
> >
> > Enrico
> >
> >>
> >> Patrick
> >>
> >>
> >> On Tue, Oct 8, 2019 at 1:46 PM Enrico Olivelli 
> >> wrote:
> >>
> >>> This is a bugfix release candidate for 3.5.6.
> >>>
> >>> It fixes 29 issues, including upgrade of third party libraries,
> >>> TTL Node APIs for C API, support for PCKS12 Keystores, upgrade of
> Netty 4
> >>> and better procedure for the upgrade of servers from 3.4 to 3.5.
> >>>
> >>> The full release notes is available at:
> >>>
> >>>
> >>>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12345243
> >>>
> >>> *** Please download, test and vote by October 11th 2019, 23:59 UTC+0.
> ***
> >>>
> >>> Source files:
> >>> https://people.apache.org/~eolivelli/zookeeper-3.5.6-candidate-4
> >>>
> >>> Maven staging repo:
> >>>
> >>
> https://repository.apache.org/content/repositories/orgapachezookeeper-1044
> >>>
> >>> The release candidate tag in git to be voted upon: release-3.5.6-rc4
> >>> https://github.com/apache/zookeeper/tree/release-3.5.6-rc4
> >>>
> >>> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> >>> https://www.apache.org/dist/zookeeper/KEYS
> >>>
> >>> Should we release this candidate?
> >>>
> >>> Enrico Olivelli
>
>

Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 3

[Norbert Kalmar]

Oh, bummer, I totally overlooked the version in the configs.

We should maybe add some automated config checks to the build... :) This is
not the first time an RC is cancelled because of the License files.

Norbert


On Mon, Oct 7, 2019 at 3:13 PM Enrico Olivelli  wrote:

> Andor, Norbert,
> I have fixed the tag
> https://github.com/apache/zookeeper/commits/release-3.5.6-rc3
>
> Explanation of what happeed:
> I had created the original tag as 'release-3.5.6rc3' (without the dash
> sign) and when I re-created it I was on the master branch.
> I apologize for the mistake.
>
> Binaries and source code were takes from this commit
>
> https://github.com/apache/zookeeper/commit/3433841cb8e8ed1e1a3ef65e6e986aae3de55677
>
> But I am sorry as I have to CANCEL again this vote, as inside the "lib"
> directory we still have Netty 4.1.29 license files instead of 4.1.42.
>
> so -1 from my side.
>
> I am sorry this is taking so long.
>
> Fortunately ZK codebase is in good shape, we are only cancelling votes for
> third party dependencies and license files
>
>
> Enrico
>
>
>
>
> Il giorno lun 7 ott 2019 alle ore 14:27 Norbert Kalmar
>  ha scritto:
>
> > +1 (non-binding) - once tag is fixed, which has the extra commit Andor
> > mentioned. I verified that the RC does not contain the commit, so it's
> just
> > the tag.
> > But the RC looks good to me!
> >
> > - unit tests pass
> > - built and started ZK + run few commands from source tarball
> > - checked bin tarball, license files, run ZK + few commands
> > - signature OK.
> >
> > Thanks Enrico!
> >
> >
> > On Mon, Oct 7, 2019 at 2:01 PM Andor Molnar  wrote:
> >
> > > +0
> > >
> > > Release artifacts looks good to me:
> > > - verified Release Notes,
> > > - verified signatures, checksum,
> > > - code built on Mac, Centos7
> > > - tests were successful, including C++ tests,
> > > - verified 3-node cluster non-ssl, ssl mode with some basic cli
> commands.
> > >
> > > However the git tag “release-3.5.6-rc3” does not seem to be right: it
> > > points to commit: 768ac0fe which is a master-only commit with Jira id:
> > > ZOOKEEPER-3532 (docker stuff).
> > >
> > > It should point to the latest commit on branch-3.5.6 instead: 3433841c
> > >
> > > (I’ll change my vote to +1 once the tag is fixed.)
> > >
> > > Andor
> > >
> > >
> > >
> > > > On 2019. Oct 5., at 18:34, Enrico Olivelli 
> > wrote:
> > > >
> > > > This is a bugfix release candidate for 3.5.6.
> > > >
> > > > It fixes 29 issues, including upgrade of third party libraries,
> > > > TTL Node APIs for C API, support for PCKS12 Keystores, upgrade of
> > Netty 4
> > > > and better procedure for the upgrade of servers from 3.4 to 3.5.
> > > >
> > > > The full release notes is available at:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12345243
> > > >
> > > > *** Please download, test and vote by October 8nd 2019, 23:59 UTC+0.
> > ***
> > > >
> > > > Source files:
> > > > https://people.apache.org/~eolivelli/zookeeper-3.5.6-candidate-3
> > > >
> > > > Maven staging repo:
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1043/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.5.6-rc3
> > > > https://github.com/apache/zookeeper/tree/release-3.5.6-rc3
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > Should we release this candidate?
> > > >
> > > > Enrico Olivelli
> > >
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 3

[Norbert Kalmar]

+1 (non-binding) - once tag is fixed, which has the extra commit Andor
mentioned. I verified that the RC does not contain the commit, so it's just
the tag.
But the RC looks good to me!

- unit tests pass
- built and started ZK + run few commands from source tarball
- checked bin tarball, license files, run ZK + few commands
- signature OK.

Thanks Enrico!


On Mon, Oct 7, 2019 at 2:01 PM Andor Molnar  wrote:

> +0
>
> Release artifacts looks good to me:
> - verified Release Notes,
> - verified signatures, checksum,
> - code built on Mac, Centos7
> - tests were successful, including C++ tests,
> - verified 3-node cluster non-ssl, ssl mode with some basic cli commands.
>
> However the git tag “release-3.5.6-rc3” does not seem to be right: it
> points to commit: 768ac0fe which is a master-only commit with Jira id:
> ZOOKEEPER-3532 (docker stuff).
>
> It should point to the latest commit on branch-3.5.6 instead: 3433841c
>
> (I’ll change my vote to +1 once the tag is fixed.)
>
> Andor
>
>
>
> > On 2019. Oct 5., at 18:34, Enrico Olivelli  wrote:
> >
> > This is a bugfix release candidate for 3.5.6.
> >
> > It fixes 29 issues, including upgrade of third party libraries,
> > TTL Node APIs for C API, support for PCKS12 Keystores, upgrade of Netty 4
> > and better procedure for the upgrade of servers from 3.4 to 3.5.
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12345243
> >
> > *** Please download, test and vote by October 8nd 2019, 23:59 UTC+0. ***
> >
> > Source files:
> > https://people.apache.org/~eolivelli/zookeeper-3.5.6-candidate-3
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1043/
> >
> > The release candidate tag in git to be voted upon: release-3.5.6-rc3
> > https://github.com/apache/zookeeper/tree/release-3.5.6-rc3
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Should we release this candidate?
> >
> > Enrico Olivelli
>
>