Re: [ANNOUNCE] New ZooKeeper PMC member: Damien Diederen

[Szalay-Bekő Máté]

Congrats Damien!!

Máté

On Wed, Apr 17, 2024, 2:01 AM ZhangJian He  wrote:

> Congratulations Damien!
>
> Thanks
> ZhangJian He
>
>
> On Wed, Apr 17, 2024 at 7:00 AM Li Wang  wrote:
>
> > Congrats Damien! Well deserved.
> >
> > Best,
> > Li
> >
> >
> > On Tue, Apr 16, 2024 at 11:11 AM Andor Molnar  wrote:
> >
> > > I am happy to announce that Damien Diederen has been invited to join
> > > the Apache ZooKeeper PMC and he accepted.
> > >
> > > Damien is doing great work for our community.
> > >
> > > Please join me in congratulating with him
> > >
> > > Congrats Damien !
> > >
> > >
> > > If you want to know more about the ASF works and what is a PMC you can
> > > read more here
> > > https://www.apache.org/foundation/how-it-works.html#pmc
> > >
> > >
> > >
> > >
> >
>

Re: [ANNOUNCE] New ZooKeeper PMC member: Zili Chen

[Szalay-Bekő Máté]

Congrats Zili!!

Máté

On Wed, Apr 17, 2024, 2:01 AM ZhangJian He  wrote:

> Congratulations Zili!
>
> Thanks
> ZhangJian He
>
>
> On Wed, Apr 17, 2024 at 7:01 AM Li Wang  wrote:
>
> > Congrats Mate!
> >
> > Li
> >
> > On Tue, Apr 16, 2024 at 11:09 AM Andor Molnar  wrote:
> >
> > > I am happy to announce that Zili Chen (tison) has been invited to join
> > > the Apache ZooKeeper PMC and he accepted.
> > >
> > > Zili is doing great work for our community.
> > >
> > > Please join me in congratulating with him
> > >
> > > Congrats Mate !
> > >
> > >
> > > If you want to know more about the ASF works and what is a PMC you can
> > > read more here
> > > https://www.apache.org/foundation/how-it-works.html#pmc
> > >
> > >
> > >
> >
>

Re: [ANNOUNCE] Apache ZooKeeper 3.9.2

[Szalay-Bekő Máté]

Thank you Damien for driving these releases!!

Cheers,
Máté

On Tue, Mar 12, 2024 at 12:14 PM Damien Diederen 
wrote:

>
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.9.2
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.9.2 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.9.2/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>

Re: [VOTE] Apache ZooKeeper release 3.8.4 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 22.04.3 using OpenJDK 8u402, maven 3.9.6 and GCC version 11.4.0
- all the unit tests passed eventually (both Java and C-client), although I
saw a few flakies again
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.22, 17.0.10, 21.0.1 (but didn't run the tests this time,
just used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests without SSL (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.6.4 to 3.8.4 RC0
  - rolling upgrade from 3.7.2 to 3.8.4 RC0
  - rolling upgrade from 3.8.3 to 3.8.4 RC0
  - rolling upgrade from 3.8.4 RC0 to 3.9.1
  - rolling upgrade from 3.8.4 RC0 to 3.9.2 RC0
- checked the uploaded documentation (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/website/index.html
)
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12353693=12310801
)

Best regards,
Máté

On Mon, Feb 12, 2024 at 11:35 PM Damien Diederen 
wrote:

>
> Greetings, all!
>
>
> This is a release candidate for 3.8.4.
>
> This is a bugfix release for the 3.8 release line. Includes important
> dependency upgrades to address CVEs.
>
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353693
>
> *** Please download, test and vote by February 16th 2024, 23:59 UTC+0. ***
>
> Source files:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.4/
>
> The release candidate tag in git to be voted upon: release-3.8.4-0
> https://github.com/apache/zookeeper/releases/tag/release-3.8.4-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/website/index.html
>
>
> Should we release this candidate?
>
>
> Regards,
> Damien Diederen
>

Re: [VOTE] Apache ZooKeeper release 3.9.2 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 22.04.3 using OpenJDK 8u402, maven 3.9.6 and GCC version 11.4.0
- all the unit tests passed eventually (both Java and C-client), although I
saw a few flakies again
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.22, 17.0.10, 21.0.1 (but didn't run the tests this time,
just used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests without SSL (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.6.4 to 3.9.2 RC0
  - rolling upgrade from 3.7.2 to 3.9.2 RC0
  - rolling upgrade from 3.8.3 to 3.9.2 RC0
  - rolling upgrade from 3.8.4 RC0 to 3.9.2 RC0
  - rolling upgrade from 3.9.1 to 3.9.1 RC0
- checked the uploaded documentation (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/website/index.html
)
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12353694=12310801
)

Best regards,
Máté

On Mon, Feb 12, 2024 at 10:37 PM Damien Diederen 
wrote:

>
> Greetings, all!
>
>
> This is a release candidate for 3.9.2.
>
> This is a bugfix release for the 3.9 release line. Includes important
> dependency upgrades to address CVEs.
>
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353694
>
> *** Please download, test and vote by February 16th 2024, 23:59 UTC+0. ***
>
> Source files:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.2/
>
> The release candidate tag in git to be voted upon: release-3.9.2-0
> https://github.com/apache/zookeeper/releases/tag/release-3.9.2-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/website/index.html
>
>
> Should we release this candidate?
>
>
> Regards,
> Damien Diederen
>

Re: [VOTE] Apache ZooKeeper release 3.7.2 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed eventually (both Java and C-client), although I
saw a few flakies again
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.7.2
  - rolling upgrade from 3.6.4 to 3.7.2
  - rolling upgrade from 3.7.1 to 3.7.2
  - rolling upgrade from 3.7.2 to 3.8.2
  - rolling upgrade from 3.7.2 to 3.8.3 RC0
  - rolling upgrade from 3.7.2 to 3.9.0
  - rolling upgrade from 3.7.2 to 3.9.1 RC0
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351732
)

Best regards,
Máté

On Fri, Oct 6, 2023 at 12:06 PM Andor Molnar  wrote:

> Hi ZK folks,
>
> This is a release candidate for 3.7.2.
>
> This is a bugfix release for the 3.7 release line. Includes important
> bugfixes and dependency upgrades to address CVEs.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351732
>
>
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
>
>
> Source files:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1098/
>
> The release candidate tag in git to be voted upon: release-3.7.2-0
> https://github.com/apache/zookeeper/releases/tag/release-3.7.2-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/website/index.html
>
>
> Should we release this candidate?
>
>
> Regards,
> Andor
>
>
>
>

Re: [VOTE] Apache ZooKeeper release 3.9.1 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.9.1
  - rolling upgrade from 3.6.4 to 3.9.1
  - rolling upgrade from 3.7.1 to 3.9.1
  - rolling upgrade from 3.8.2 to 3.9.1
  - rolling upgrade from 3.8.3 RC0 to 3.9.1
  - rolling upgrade from 3.9.0 to 3.9.1
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480
)

Best regards,
Máté

On Thu, Oct 5, 2023 at 10:49 AM Enrico Olivelli  wrote:

> +1 (binding)
>
> - Built from sources, run all the tests
> - Validated checksums and signatures
> - Run a single node cluster from the binaries
>
> Enrico
>
> Il giorno mer 4 ott 2023 alle ore 19:12 Damien Diederen
>  ha scritto:
> >
> >
> > Greetings, all,
> >
> > +1 (non-binding)
> >
> >   * Verified checksum/signature;
> >   * Upgraded a 5-node ensemble to apache-zookeeper-3.9.1-bin.tar.gz,
> > ran a few smoke tests;
> >   * Reviewed the Release Notes;
> >   * Ran dependency-check:check on the source tree.
> >
> > Cheers, -D
> >
> >
> > Andor Molnar  writes:
> > > Hi team,
> > >
> > > This is a release candidate for 3.9.1.
> > >
> > > This is a bugfix release for the 3.9 release line. Includes important
> > > dependency upgrades to address CVEs.
> > >
> > >
> > > The full release notes is available at:
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480
> > >
> > > *** Please download, test and vote by October 6th 2023, 23:59 UTC+0.
> > > ***
> > >
> > > Source files:
> > >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/
> > >
> > > Maven staging repo:
> > >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1096/
> > >
> > > The release candidate tag in git to be voted upon: release-3.9.1-0
> > > https://github.com/apache/zookeeper/releases/tag/release-3.9.1-0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Best regards,
> > >
> > > Andor
>

Re: [VOTE] Apache ZooKeeper release 3.8.3 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
   - rolling upgrade from 3.5.10 to 3.8.3
   - rolling upgrade from 3.6.4 to 3.8.3
   - rolling upgrade from 3.7.1 to 3.8.3
   - rolling upgrade from 3.8.2 to 3.8.3
   - rolling upgrade from 3.8.3 to 3.9.0
   - rolling upgrade from 3.8.3 to 3.9.1 RC0
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353400
)

Best regards,
Máté

On Thu, Oct 5, 2023 at 12:50 PM Andor Molnar  wrote:
>
> Hi,
>
> This is a release candidate for 3.8.3.
>
> This is a bugfix release for the 3.8 release line. Includes important
> dependency upgrades to address CVEs.
>
> The full release notes is available at:
>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353400
>
>
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
>
>
> Source files:
>
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/
>
> Maven staging repo:
>
https://repository.apache.org/content/repositories/orgapachezookeeper-1097/
>
> The release candidate tag in git to be voted upon: release-3.8.3-0
> https://github.com/apache/zookeeper/releases/tag/release-3.8.3-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/website/index.html
>
>
> Should we release this candidate?
>
>
> Regards,
> Andor
>
>

Re: [ANNOUNCE] Apache ZooKeeper 3.9.0

[Szalay-Bekő Máté]

Great job indeed, thanks to everyone!! :)
Máté

On Fri, Aug 4, 2023 at 6:24 PM Li Wang  wrote:

> Congrats! Thanks Andor and Enrico for leading this and all the contributors
> that made this possible.
>
> Best,
>
> Li
>
> On Fri, Aug 4, 2023 at 7:28 AM Enrico Olivelli 
> wrote:
>
> > Congratulations !
> >
> > This is great step forward
> >
> > I hope that people will try out soon the Backup/Restore feature and
> > that they provide feedback
> >
> > Enrico
> >
> > Il giorno ven 4 ago 2023 alle ore 13:24 Andor Molnar
> >  ha scritto:
> > >
> > > The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> > > 3.9.0
> > >
> > > ZooKeeper is a high-performance coordination service for distributed
> > > applications. It exposes common services - such as naming,
> > > configuration management, synchronization, and group services - in a
> > > simple interface so you don't have to write them from scratch. You can
> > > use it off-the-shelf to implement consensus, group management, leader
> > > election, and presence protocols. And you can build on it for your
> > > own, specific needs.
> > >
> > > For ZooKeeper release details and downloads, visit:
> > > https://zookeeper.apache.org/releases.html
> > >
> > > ZooKeeper 3.9.0 Release Notes are at:
> > > https://zookeeper.apache.org/doc/r3.9.0/releasenotes.html
> > >
> > > We would like to thank the contributors that made the release possible.
> > >
> > > Regards,
> > >
> > > The ZooKeeper Team
> > >
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 1

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
 - rolling upgrade from 3.5.10 to 3.9.0
 - rolling upgrade from 3.6.4 to 3.9.0
 - rolling upgrade from 3.7.1 to 3.9.0
 - rolling upgrade from 3.8.2 to 3.9.0
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/website/index.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
)

Best regards,
Máté

On Wed, Jul 19, 2023 at 11:19 AM Andor Molnar  wrote:

> This is release candidate for ZooKeeper 3.9.0.
>
> It is a major release and it introduces a lot of new features, most
> notably:
> - Admin server API for taking snapshot and stream out the data
> - Communicate the Zxid that triggered a WatchEvent to fire
> - TLS - dynamic loading for client trust/key store
> - Add Netty-TcNative OpenSSL Support
> - Adding SSL support to Zktreeutil
> - Improve syncRequestProcessor performance
> - Updates to all the third party dependencies to get rid of every known
> CVE.
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
>
> *** Please download, test and vote by July 30th 2023, 23:59 UTC+0. ***
>
> Source files:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
>
> The release candidate tag in git to be voted upon: release-3.9.0-1
> https://github.com/apache/zookeeper/tree/release-3.9.0-1
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/website/index.html
>
>
> Should we release this candidate?
>
> Regards,
> Andor
>
>
>
>

[ANNOUNCE] Apache ZooKeeper 3.8.2

[Szalay-Bekő Máté]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.8.2

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

Release 3.8.2 is a bugfix release, solving 12 issues, including CVE fixes
and
additional test, security and other improvements.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.8.2 Release Notes are at:
https://zookeeper.apache.org/doc/r3.8.2/releasenotes.html

We would like to thank the contributors who made the release possible.

Regards,
The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Szalay-Bekő Máté]

Thanks Andor!

> Since we don't have better idea, I opened a pull request to upgrade OWASP
to the latest (8.3.1) version.

Yes, I tested the PR, upgrading OWASP is solving the issue also on my
machine. I approved the PR.

Regards,
Máté

On Tue, Jul 18, 2023 at 1:45 PM Andor Molnar  wrote:

> Hi Mate,
>
> I take your e-mail as a -1 vote, so this RC VOTE is CANCELLED.
> I'll prepare another rc.
>
> Regards,
> Andor
>
>
> On Mon, 2023-07-17 at 22:50 +0200, Szalay-Bekő Máté wrote:
> > Hello Andor!
> >
> > Thanks for this great release!
> >
> > I found two issues with RC0:
> >
> > 1) OWASP CVE check (mvn dependency-check:check) failed with
> > "netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:
> > CVE-2011-1797(9.3)"
> >
> > This seems to be a false positive to me (looks to be some security
> > issue
> > affecting old safari / chromium web browser versions?). I didn't get
> > deep
> > into this, but I guess we see this since
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> >
> > Interestingly, the CI pipeline doesn't catch this CVE (
> >
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/
> ),
> > maybe this is some bug in OWASP that is triggered only with certain
> > maven
> > versions or during building on certain platforms? I ran OWASP on
> > Ubuntu
> > 18.04.2 with maven 3.9.3.
> >
> > 2) Also I see that the website (
> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
> )
> > is still showing "ZooKeeper 3.8 Documentation" on the top
> >
> >
> > What do you think? We shouldn't pass the RC until we are certain
> > about the
> > CVE issue. (unless this is something happening only on my setup... it
> > is
> > strange that OWAPS is green on CI)
> >
> >
> > Beside these, I ran all my usual RC test steps, and found no other
> > issues
> > with the RC:
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build) on
> > Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built and executed unit tests for zkpython
> > - I also built the java code (without -Pfull-build) using other JDK
> > versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this
> > time, just
> > used 'clean install -DskipTests')
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - fatjar built
> > - I executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> >  - rolling upgrade from 3.5.10 to 3.9.0
> >  - rolling upgrade from 3.6.4 to 3.9.0
> >  - rolling upgrade from 3.7.1 to 3.9.0
> >  - rolling upgrade from 3.8.2 to 3.9.0
> > - compared generated release notes (
> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/releasenotes.html
> > ) with Jira (
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > )
> >
> >
> > Best regards,
> > Máté
> >
> > On Mon, Jul 17, 2023 at 3:11 PM Andor Molnar 
> > wrote:
> >
> > > Hi team,
> > >
> > > This is a release candidate for 3.9.0.
> > >
> > > It is a major release and it introduces a lot of new features, most
> > > notably:
> > > - Admin server API for taking snapshot and stream out the data
> > > - Communicate the Zxid that triggered a WatchEvent to fire
> > > - TLS - dynamic loading for client trust/key store
> > > - Add Netty-TcNative OpenSSL Support
> > > - Adding SSL support to Zktreeutil
> > > - Improve syncRequestProcessor performance
> > > - Updates to all the third party dependencies to get rid of every
> > > known
> > > CVE.
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > >
> > > *** Please download, test and vote by July 30th 2023, 23:59 UTC+0.
> > > ***
> > >
> > > Source files:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/
> > >
> > > Maven staging repo:
> > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > >
> > > The release candidate tag in git to be voted upon: release-3.8.0-1
> > > https://github.com/apache/zookeeper/tree/release-3.9.0-0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > > release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
>
>

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Szalay-Bekő Máté]

Hello Andor!

Thanks for this great release!

I found two issues with RC0:

1) OWASP CVE check (mvn dependency-check:check) failed with
"netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:
CVE-2011-1797(9.3)"

This seems to be a false positive to me (looks to be some security issue
affecting old safari / chromium web browser versions?). I didn't get deep
into this, but I guess we see this since
https://issues.apache.org/jira/browse/ZOOKEEPER-4622

Interestingly, the CI pipeline doesn't catch this CVE (
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/),
maybe this is some bug in OWASP that is triggered only with certain maven
versions or during building on certain platforms? I ran OWASP on Ubuntu
18.04.2 with maven 3.9.3.

2) Also I see that the website (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html)
is still showing "ZooKeeper 3.8 Documentation" on the top


What do you think? We shouldn't pass the RC until we are certain about the
CVE issue. (unless this is something happening only on my setup... it is
strange that OWAPS is green on CI)


Beside these, I ran all my usual RC test steps, and found no other issues
with the RC:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
 - rolling upgrade from 3.5.10 to 3.9.0
 - rolling upgrade from 3.6.4 to 3.9.0
 - rolling upgrade from 3.7.1 to 3.9.0
 - rolling upgrade from 3.8.2 to 3.9.0
- compared generated release notes (
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/releasenotes.html
) with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
)


Best regards,
Máté

On Mon, Jul 17, 2023 at 3:11 PM Andor Molnar  wrote:

> Hi team,
>
> This is a release candidate for 3.9.0.
>
> It is a major release and it introduces a lot of new features, most
> notably:
> - Admin server API for taking snapshot and stream out the data
> - Communicate the Zxid that triggered a WatchEvent to fire
> - TLS - dynamic loading for client trust/key store
> - Add Netty-TcNative OpenSSL Support
> - Adding SSL support to Zktreeutil
> - Improve syncRequestProcessor performance
> - Updates to all the third party dependencies to get rid of every known
> CVE.
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
>
> *** Please download, test and vote by July 30th 2023, 23:59 UTC+0. ***
>
> Source files:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
>
> The release candidate tag in git to be voted upon: release-3.8.0-1
> https://github.com/apache/zookeeper/tree/release-3.9.0-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
>
>
> Should we release this candidate?
>
>
> Regards,
> Andor
>
>
>

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Szalay-Bekő Máté]

Thank you for the reviews!

I'm happy to announce that we have unanimously approved this release.
There were 3 approving votes, all of them binding:

- Enrico Olivelli (binding)
- Andor Molnár (binding)
- Máté Szalay-Bekő (binding)

There were no disapproving votes.

I will promote the artifacts and complete the release procedure.

Thanks to everyone who contributed to this release!

Best Regards,
Máté

On Mon, Jul 17, 2023 at 4:08 PM Andor Molnar  wrote:

> +1 (binding)
>
> - verified checksum and gpg signature of the artifacts
> - I built the source code (incl. the C-client, using -Pfull-build) on
> Ubuntu 20.04 using OpenJDK 8u302, maven 3.6.3 and GCC version 9.4.0
> - all the unit tests passed (both Java and C-client)
> - I also built the code using Oracle JDK 20.0.1
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - checked the generated documentation (zookeeper-docs/target/html)
> - checked release notes
> - created 3-node cluster with TLS enabled and ran some smoke tests
> - run zk-smoketest.py (https://github.com/phunt/zk-smoketest)
> - run zk-latencies.py (https://github.com/phunt/zk-smoketest)
>
> Thanks,
> Andor
>
>
>
>
> On Wed, 2023-07-05 at 23:20 +0200, Szalay-Bekő Máté wrote:
> > This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> > including
> > CVE fixes and additional test, security and other improvements.
> >
> > Please find the full release notes in the following link:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> >
> > *** Please download, test and vote by July 14th 2023, 23:59 UTC+0.
> > ***
> >
> >
> > Source files:
> > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
> >
> > The release candidate tag in git to be voted upon: release-3.8.2-0
> > (please note, branch-3.8.2 will move here only after the vote)
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
> >
> >
> > Should we release this candidate?
> >
> >
> > Best regards,
> > Máté
>
>

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Szalay-Bekő Máté]

Thank you Enrico!
Just a friendly reminder, we still miss a vote, please take a look if you
have the time!

Thanks,
Máté

On Fri, Jul 14, 2023 at 11:35 AM Enrico Olivelli 
wrote:

> +1 (binding)
>
> - verified checksums and signatures
> - built from the sources
> - run all the tests on JDK11 on Mac M1
> - run some application/service (and all the unit/integration tests)
> thats uses ZooKeeper (HerdDB.org + Apache BookKeeper) using the
> binaries that I have built locally
>
>
> Thank you for driving the release
>
> Enrico
>
> Il giorno mer 5 lug 2023 alle ore 23:22 Szalay-Bekő Máté
>  ha scritto:
> >
> > +1 (binding)
> >
> > I did the following tests for the release candidate:
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build) on
> > Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built and executed unit tests for zkpython
> > - I also built the java code (without -Pfull-build) using other JDK
> > versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time,
> just
> > used 'clean install -DskipTests')
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built
> > - I executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> > - rolling upgrade from 3.5.10 to 3.8.2
> > - rolling upgrade from 3.6.4 to 3.8.2
> > - rolling upgrade from 3.7.1 to 3.8.2
> > - rolling upgrade from 3.8.1 to 3.8.1
> > - checked the generated documentation (zookeeper-docs/target/html)
> > - compared generated release notes (
> >
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/releasenotes.html
> )
> > with Jira (
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> > )
> >
> > Best regards,
> > Máté
> >
> > On Wed, Jul 5, 2023 at 11:20 PM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> > > including CVE fixes and additional test, security and other
> improvements.
> > >
> > > Please find the full release notes in the following link:
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> > >
> > > *** Please download, test and vote by July 14th 2023, 23:59 UTC+0. ***
> > >
> > >
> > > Source files:
> > > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
> > >
> > > Maven staging repo:
> > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
> > >
> > > The release candidate tag in git to be voted upon: release-3.8.2-0
> > > (please note, branch-3.8.2 will move here only after the vote)
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Best regards,
> > > Máté
> > >
>

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
- rolling upgrade from 3.5.10 to 3.8.2
- rolling upgrade from 3.6.4 to 3.8.2
- rolling upgrade from 3.7.1 to 3.8.2
- rolling upgrade from 3.8.1 to 3.8.1
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
)

Best regards,
Máté

On Wed, Jul 5, 2023 at 11:20 PM Szalay-Bekő Máté 
wrote:

> This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> including CVE fixes and additional test, security and other improvements.
>
> Please find the full release notes in the following link:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
>
> *** Please download, test and vote by July 14th 2023, 23:59 UTC+0. ***
>
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
>
> The release candidate tag in git to be voted upon: release-3.8.2-0
> (please note, branch-3.8.2 will move here only after the vote)
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
>
>
> Should we release this candidate?
>
>
> Best regards,
> Máté
>

[VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.8.2. It fixes 12 issues, including
CVE fixes and additional test, security and other improvements.

Please find the full release notes in the following link:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866

*** Please download, test and vote by July 14th 2023, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.8.2-rc0/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/

The release candidate tag in git to be voted upon: release-3.8.2-0
(please note, branch-3.8.2 will move here only after the vote)

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/


Should we release this candidate?


Best regards,
Máté

Re: Next release zookeeper

[Szalay-Bekő Máté]

Hello Cristina!

We recently started to work on two releases:
- 3.9.0
- 3.8.2

I don't know about exact ETA, but I would assume they will happen "soon".
(e.g. 3.8.2 I assume in the next few weeks).

Best regards,
Máté

On Tue, Jun 20, 2023 at 5:29 PM Cristina Gonzalez Marrero
 wrote:

> Hi,
>
>
>
> I was wondering when the next release of Zookeeper is likely to be ?
>
>
>
> Kind Regards,
>
>
>
> [image: signature_268875190]
>
> Cristina Gonzalez Marrero
> Software Developer – CP4NA, IBM in Ireland
>
> Follow me on LinkedIn 
>
>
>

planning release 3.8.2

[Szalay-Bekő Máté]

Hello ZooKeepers,

release 3.8.1 happened this January and I volunteered to do 3.8.2 soon,
maybe in June or early July. Let me know if someone would also like to do
it, I am happy to hand it over! :)

Of course we need to do all the 3pp CVE / vulnerability fixes first.
And Andor already started a thread about a FIPS TLS improvement which we
should wait for.

Does anyone know about any other open ticket we should wait for in 3.8.2?

Best regards,
Máté

Re: Volounteers for releases ?

[Szalay-Bekő Máté]

we have all the room we need :)

I won't start anything for a few days. Also I planned to start with looking
through Jira and also with fixing the 3pp vulnerabilities. Also I will
write a separate email, asking if anyone has some tickets open we should
wait.

Máté

On Wed, Jun 14, 2023 at 12:00 PM Andor Molnar  wrote:

> Hi Mate,
>
> Not sure how much room do we have, but TLS patches are going to be
> backported to 3.8 too.
>
> Andor
>
>
>
>
> On Wed, 2023-06-14 at 08:34 +0200, Enrico Olivelli wrote:
> > Il Mer 14 Giu 2023, 08:06 Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com> ha
> > scritto:
> >
> > > Hello!
> > >
> > > I also might be able to manage a release. I let Andor to do the
> > > 3.9.0 ;)
> > > but I can make a 3.7 or 3.8 release.
> > >
> > > Based on our policy, when 3.9.0 will be released, we will deprecate
> > > the 3.7
> > > line. 3.7.1 happened one year ago. Normally I would say it would be
> > > good to
> > > make a last 3.7 release (3.7.2) before we terminate 3.7. But 3.7
> > > and 3.8
> > > are very similar (we cut 3.8 only for log4j vulnerability). Maybe
> > > we can
> > > deprecate 3.7 without  3.7.2 and I should focus on 3.8.2 instead?
> > >
> > > what do you think?
> > >
> >
> > Let's focus on 3.8.2.
> > Users on 3.7 can easily migrate to 3.8
> >
> >
> > Thanks
> >
> > Enrico
> >
> >
> >
> >
> > > Mate
> > >
> > > On Tue, Jun 13, 2023 at 10:58 AM Andor Molnar 
> > > wrote:
> > >
> > > > Awesome. Thanks Enrico!
> > > >
> > > > I owe you an apology: found an important TLS ticket which is
> > > > another
> > > > low hanging fruit:
> > > >
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> > > >
> > > > ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> > > >
> > > > We've already done this for HBase and I always wanted to port
> > > > this back
> > > > to ZooKeeper. It's a very cool performance improvement for Linux-
> > > > based
> > > > installations (or whereever else Netty has OpenSSL support),
> > > > unfortunately for ZooKeeper this is only for the server-client
> > > > stack.
> > > >
> > > > Let me land this for 3.9.0.
> > > >
> > > > Andor
> > > >
> > > >
> > > > On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> > > > > Tests on master branch are passing on JDK20
> > > > >
> > > > > Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> > > > > Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> > > > > Java version: 20, vendor: Eclipse Adoptium, runtime:
> > > > > /usr/local/asfpackages/java/adoptium-jdk-20+36
> > > > > Default locale: en_US, platform encoding: UTF-8
> > > > > OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> > > > > family: "unix"
> > > > >
> > > > >
> https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
> > > > >  ha scritto:
> > > > > > Sure. I've just noticed that the patch has been outstanding
> > > > > > for a
> > > > > > year
> > > > > > now, small and ready to be submitted.
> > > > > >
> > > > > > Andor
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > > > > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > > > > > >  ha scritto:
> > > > > > > > I came across the graceful termination patch yesterday.
> > > > > > > > Sounds
> > > > > > > > like
> > > > > > > > important for K8s environments. Enrico, what do you
> > > > > > > > think?
> > > > > > > > Looks
> > > > > > > > like
> > > > > > > > you're not a fan of that.
> > > > > > > >
> > > > > > > > ZOOKEEPER-440

Re: Volounteers for releases ?

[Szalay-Bekő Máté]

Hello!

I also might be able to manage a release. I let Andor to do the 3.9.0 ;)
but I can make a 3.7 or 3.8 release.

Based on our policy, when 3.9.0 will be released, we will deprecate the 3.7
line. 3.7.1 happened one year ago. Normally I would say it would be good to
make a last 3.7 release (3.7.2) before we terminate 3.7. But 3.7 and 3.8
are very similar (we cut 3.8 only for log4j vulnerability). Maybe we can
deprecate 3.7 without  3.7.2 and I should focus on 3.8.2 instead?

what do you think?

Mate

On Tue, Jun 13, 2023 at 10:58 AM Andor Molnar  wrote:

> Awesome. Thanks Enrico!
>
> I owe you an apology: found an important TLS ticket which is another
> low hanging fruit:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-4622
>
> ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
>
> We've already done this for HBase and I always wanted to port this back
> to ZooKeeper. It's a very cool performance improvement for Linux-based
> installations (or whereever else Netty has OpenSSL support),
> unfortunately for ZooKeeper this is only for the server-client stack.
>
> Let me land this for 3.9.0.
>
> Andor
>
>
> On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> > Tests on master branch are passing on JDK20
> >
> > Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> > Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> > Java version: 20, vendor: Eclipse Adoptium, runtime:
> > /usr/local/asfpackages/java/adoptium-jdk-20+36
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> > family: "unix"
> >
> > https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> >
> > Enrico
> >
> > Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
> >  ha scritto:
> > > Sure. I've just noticed that the patch has been outstanding for a
> > > year
> > > now, small and ready to be submitted.
> > >
> > > Andor
> > >
> > >
> > >
> > > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > > >  ha scritto:
> > > > > I came across the graceful termination patch yesterday. Sounds
> > > > > like
> > > > > important for K8s environments. Enrico, what do you think?
> > > > > Looks
> > > > > like
> > > > > you're not a fan of that.
> > > > >
> > > > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > > > >
> > > > > https://github.com/apache/zookeeper/pull/1898
> > > >
> > > > I have taken a look and I have posted some feedback about
> > > > renaming
> > > > the
> > > > configuration flag.
> > > > I am not sure that we are in a hurry to commit that patch, we can
> > > > release it with 3.9.1.
> > > >
> > > > Enrico
> > > >
> > > >
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > > > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > > > > >  ha scritto:
> > > > > > > Hi Kezhu,
> > > > > > >
> > > > > > > Sure, I'll take a look at the open PRs before cutting 3.9.0
> > > > > > > from
> > > > > > > master. Let's mark these tickets release blockers as you
> > > > > > > suggested.
> > > > > > >
> > > > > > > Any more blockers of 3.9.0 that anyone knows about?
> > > > > >
> > > > > > No, there are no critical issues at the moment.
> > > > > >
> > > > > > I will double check on compatibility with the latest JDKs, it
> > > > > > is
> > > > > > better that when we cut a new major release
> > > > > > it works well with the newer JDKs
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > >
> > > > > > > Andor
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Sorry for the disruption.
> > > > > > > >
> > > > > > > > I want to ask if there is any possibility for us to
> > > > > > > > include
> > > > > > > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > > > > > >
> > > > > > > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent`
> > > > > > > > and
> > > > > > > > `WatcherType.PersistentRecursive` to remove
> > > > > > > > `AddWatchMode.PERSISTENT`
> > > > > > > > and
> > > > > > > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > > > > > > complementary to
> > > > > > > > [ZOOKEEPER-4466][3] which supports multiple different
> > > > > > > > watches
> > > > > > > > on
> > > > > > > > one
> > > > > > > > path
> > > > > > > > and was merged to master one month ago. If we don't get
> > > > > > > > it to
> > > > > > > > 3.9.0,
> > > > > > > > it is
> > > > > > > > probably impossible for us to merge it to patch versions
> > > > > > > > of
> > > > > > > > 3.9
> > > > > > > > series as
> > > > > > > > it touches both server logic and api side. It might be
> > > > > > > > strange to
> > > > > > > > ship
> > > > > > > > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to
> > > > > > > > support
> > > > > > > > different
> > > > > > > > 

Re: FIPS: removing ZKTrustManager

[Szalay-Bekő Máté]

yeah, I remember these tickets, thanks for picking them up!
I agree and like the solution you proposed, in general in the long term it
is good not to use a custom trust manager, but rely on the standard one.

Máté


On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli  wrote:

> Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
>  ha scritto:
> >
> > I'd like to backport this to the 3.8 branch too.
> >
> > Let's say I'll add new "zookeeper.fips-mode" parameter which will be
> > "false" by default in 3.8 and "true" for 3.9.0.
>
> I am +1
> ZK 3.9 will take time to be adopted and this is an important security
> related topic
>
> Enrico
>
> >
> > Thoughts?
> >
> > Andor
> >
> >
> >
> > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > I think that switching to
> > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is a good
> > > option.
> > > The less tweaks we have about Security code the better.
> > >
> > >
> > > It would be great to see this in 3.9.0.
> > >
> > > Enrico
> > >
> > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > >  ha scritto:
> > > > Hi zk folks,
> > > >
> > > > Problem(s)
> > > > ==
> > > >
> > > > One problem that we're having with a custom Trust Manager in ZK is
> > > > that
> > > > FIPS doesn't allow that:
> > > >
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > >
> > > > In FIPS mode the only allowed TrustManager in the JDK is
> > > > X509TrustManagerImpl which is the default implementation. The class
> > > > is
> > > > final, so extending it is not an option unfortunately.
> > > >
> > > > The intention behind implementing a custom trust manager in ZK was,
> > > > I
> > > > believe, the need for server and client-side hostname verification.
> > > > Hostname verification officially is not part of the SSL/TLS
> > > > protocol,
> > > > it's the responsibility of an upper level protocol like HTTPS.
> > > >
> > > > Hacking hostname verification in the SSL handshake is nice and was
> > > > working fine so far, but unfortunately breaks the FIPS standard.
> > > >
> > > > Another annoying issue with ZKTrustManager is the need for reverse
> > > > DNS
> > > > lookup. This is usually needed when the hostname of the certificate
> > > > provider is not known at the time of handshake. For instance, when
> > > > somebody connects the client via IP address, which is generally not
> > > > recommended when TLS is active in the client-server protocol.
> > > >
> > > > The bigger problem I've found is in the leader election: when a
> > > > peer
> > > > connects with a smaller id, the node will close the existing
> > > > connection
> > > > and opens a new one in the other direction, based on the
> > > > information
> > > > received in the InitialMessage from the peer which only contains
> > > > the IP
> > > > address, not the hostname. Therefore TrustManager needs to perform
> > > > reverse DNS lookup.
> > > >
> > > > Tickets about reverse DNS lookup issues:
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3860
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4268
> > > >
> > > > Proposal
> > > > 
> > > >
> > > > I suggest to remove ZKTrustManager entirely from the codebase and
> > > > use
> > > > the built-in, FIPS-Enabled X509TrustManagerImpl instead. It has the
> > > > downside of losing hostname verification, but we have an option to
> > > > re-
> > > > enable it in client-server communication: Netty has built-in
> > > > support
> > > > for it, we just need to do
> > > >
> > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
> > > >
> > > > when creating the SSLEngine and that will result in a behaviour
> > > > very
> > > > similar to what we provide currently. I can show some examples.
> > > >
> > > > What we will truly lose is the hostname verification option in the
> > > > Quorum and Leader Election protocols. Since in these protocols we
> > > > manipulate the sockets directly, we would need to implement the
> > > > verification manually.
> > > >
> > > > What do you think about this trade-off?
> > > >
> > > > Of course, we can put this change behind a feature flag "fips-
> > > > mode",
> > > > which will lead to a new mode in ZooKeeper that is actually less
> > > > strict
> > > > as the original behaviour.
> > > >
> > > > Regards,
> > > > Andor
> > > >
> > > >
> > > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 1

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- built the source code (incl. the C-client, using -Pfull-build) on Ubuntu
18.04.5 using OpenJDK 8u265, maven 3.6.3 and GCC version 7.5.0
- all the unit tests passed eventually (both Java and C-client)
- also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.8.1
  - rolling upgrade from 3.6.4 to 3.8.1
  - rolling upgrade from 3.7.1 to 3.8.1
  - rolling upgrade from 3.8.0 to 3.8.1

Best regards,
Máté

On Fri, Jan 27, 2023 at 6:54 PM Patrick Hunt  wrote:

> +1 - xsums validated, rat ran clean, built/compiled fine and I was able to
> run some manual tests on various cluster sizes.
>
> Regards,
>
> Patrick
>
> On Wed, Jan 25, 2023 at 8:39 AM Enrico Olivelli 
> wrote:
>
> > This is the second release candidate for 3.8.1.
> >
> > This is a bugfix release. The full release notes is available at:
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> >
> > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0.
> ***
> >
> > Source files:
> >
> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1088/
> >
> > The release candidate tag in git to be voted upon: release-3.8.1-1
> > https://github.com/apache/zookeeper/tree/release-3.8.1-1
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> >
> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Enrico Olivelli
> >
>

[ANNOUNCE] Apache ZooKeeper 3.6.4

[Szalay-Bekő Máté]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.6.4

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

3.6.4 is the last bugfix release for branch 3.6, as 3.6 is EoL since 30th
December, 2022.
It fixes 42 issues, including CVE fixes, log4j1 removal (using reload4j
from now)
and various other bug fixes (e.g. snapshotting, SASL and C client related
fixes).

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.6.4 Release Notes are at:
https://zookeeper.apache.org/doc/r3.6.4/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,
The ZooKeeper Team

[ANNOUNCE] Year 2023 :)

[Szalay-Bekő Máté]

The Apache ZooKeeper team is proud to announce Year 2023

You may have noticed that the year 2022 also gets EoL very soon. The new
2023 version was released successfully and - depending on time zones -
people started to use it already.

Based on the feedbacks the upgrades seemed to be smooth so far, at least no
incompatibilities found. Sadly, we were not able to test this version
before the release. Most likely it will introduce some new issues but
hopefully also will solve some of the old ones. Let's enjoy it anyway...

Happy new year! ;)

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 2

[Szalay-Bekő Máté]

Thank you all for testing the release candidate!

I'm happy to announce that we have unanimously approved this release.
There are 3 approving votes, all of them are binding:

- Chris Nauroth (binding)
- Enrico Olivelli (binding)
- Máté Szalay-Bekő (binding)

There are no disapproving votes.

I will promote the artifacts and complete the release procedure.

Thanks to everyone who contributed to this release!

Best Regards,
Máté

On Fri, Dec 23, 2022 at 11:37 AM Enrico Olivelli 
wrote:

> +1 (binding)
>
> Built from sources and run all the tests (on Mac M1 with JDK8
> 1.8.0_352, vendor: Azul Systems)
>
> Enrico Olivelli
>
> Il giorno lun 19 dic 2022 alle ore 18:07 Chris Nauroth
>  ha scritto:
> >
> > +1 (binding)
> >
> > - Verified all checksums.
> > - Verified all signatures.
> > - Built from source, including native code on Linux.
> > - Tests passed.
> > - Ran several small samples successfully.
> >
> > Máté, thank you for your diligence with these release candidates!
> >
> > Chris Nauroth
> >
> >
> > On Sun, Dec 18, 2022 at 12:03 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com> wrote:
> >
> > > +1 (binding)
> > >
> > > I did the following tests for the release candidate:
> > > - verified checksum and gpg signature of the artifacts
> > > - I built the source code (incl. the C-client, using -Pfull-build) on
> > > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
> > > - all the unit tests passed (both Java and C-client)
> > > - I also built and executed unit tests for zkpython
> > > - I also built the java code (without -Pfull-build) using other JDK
> > > versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests
> this
> > > time, just used 'clean install -DskipTests')
> > > - checkstyle and spotbugs passed
> > > - apache-rat passed
> > > - owasp (CVE check) passed
> > > - fatjar built
> > > - I executed quick rolling-upgrade tests (using
> > > https://github.com/symat/zk-rolling-upgrade-test):
> > >   - rolling upgrade from 3.5.10 to 3.6.4
> > >   - rolling upgrade from 3.6.3 to 3.6.4
> > >   - rolling upgrade from 3.6.4 to 3.7.1
> > >   - rolling upgrade from 3.6.4 to 3.8.0
> > > - checked the generated documentation (zookeeper-docs/target/html)
> > > - compared generated release notes (
> > >
> > >
> https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/releasenotes.html
> > > )
> > > with Jira (
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > > )
> > >
> > > Best regards,
> > > Máté
> > >
> > > On Sun, Dec 18, 2022 at 9:01 PM Szalay-Bekő Máté <
> > > szalay.beko.m...@gmail.com>
> > > wrote:
> > >
> > > > This is a bugfix release candidate for 3.6.4. It fixes 42 issues,
> > > > including CVE fixes,
> > > > log4j1 removal (using reload4j from now) and various other bug fixes
> > > > (thread leaks, data
> > > > corruption, snapshotting and SASL related fixes).
> > > >
> > > > Please note, that based on our Release Strategy (
> > > > https://zookeeper.apache.org/releases.html#release-strategy) branch
> 3.6
> > > > should become end-of-life and most likely 3.6.4 will be our last 3.6
> > > > release.
> > > >
> > > > The full release notes is available at:
> > > >
> > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > > >
> > > > *** Please download, test and vote by December 29th 2022, 23:59
> UTC+0.
> > > ***
> > > >
> > > >
> > > > Source files:
> > > > https://people.apache.org/~symat/zookeeper-3.6.4-rc2/
> > > >
> > > > Maven staging repo:
> > > >
> > > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.6.4-2
> > > > (please note, branch-3.6.4 will move here only after the vote)
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > The staging version of the website is:
> > > > https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/
> > > >
> > > >
> > > > Should we release this candidate?
> > > >
> > > >
> > > > Best regards,
> > > > Máté
> > > >
> > >
>

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 2

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
time, just used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.6.4
  - rolling upgrade from 3.6.3 to 3.6.4
  - rolling upgrade from 3.6.4 to 3.7.1
  - rolling upgrade from 3.6.4 to 3.8.0
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
)

Best regards,
Máté

On Sun, Dec 18, 2022 at 9:01 PM Szalay-Bekő Máté 
wrote:

> This is a bugfix release candidate for 3.6.4. It fixes 42 issues,
> including CVE fixes,
> log4j1 removal (using reload4j from now) and various other bug fixes
> (thread leaks, data
> corruption, snapshotting and SASL related fixes).
>
> Please note, that based on our Release Strategy (
> https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
> should become end-of-life and most likely 3.6.4 will be our last 3.6
> release.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
>
> *** Please download, test and vote by December 29th 2022, 23:59 UTC+0. ***
>
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc2/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
>
> The release candidate tag in git to be voted upon: release-3.6.4-2
> (please note, branch-3.6.4 will move here only after the vote)
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/
>
>
> Should we release this candidate?
>
>
> Best regards,
> Máté
>

[VOTE] Apache ZooKeeper release 3.6.4 candidate 2

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.6.4. It fixes 42 issues, including
CVE fixes,
log4j1 removal (using reload4j from now) and various other bug fixes
(thread leaks, data
corruption, snapshotting and SASL related fixes).

Please note, that based on our Release Strategy (
https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
should become end-of-life and most likely 3.6.4 will be our last 3.6
release.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076

*** Please download, test and vote by December 29th 2022, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.6.4-rc2/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/

The release candidate tag in git to be voted upon: release-3.6.4-2
(please note, branch-3.6.4 will move here only after the vote)

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/


Should we release this candidate?


Best regards,
Máté

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 1

[Szalay-Bekő Máté]

I think it is best to fix this issue before doing the release, especially
as Chris already provided the fix. Thank you for working on it!! Also
thanks Enrico for the quick review!
I merged ZOOKEEPER-4654
<https://issues.apache.org/jira/browse/ZOOKEEPER-4654> to all branches and
will include it also to RC 2.

I am cancelling this VOTE and will send a new one soon.

Best regards,
Máté

On Sat, Dec 17, 2022 at 3:03 PM Szalay-Bekő Máté 
wrote:

> Hello Chris,
>
> I tried to reproduce the test build problem locally on docker, but was not
> able to do so. I used ubuntu 22.04 (which uses gcc 11 by default),
> installing gcc-12 (version: 12.1.0) and openssl (version: 3.0.2 15 Mar
> 2022).
>
> My steps were:
>
> docker run --volume ~/git:/git -it --rm ubuntu:22.04 /bin/bash
>
> apt update && apt install -y libcppunit-dev maven default-jdk-headless
> autoconf libtool gcc-12 g++-12 make software-properties-common pkg-config
> wget git python-setuptools openssl libssl-dev ant
> libsasl2-modules-gssapi-mit libsasl2-modules libsasl2-dev
> export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10
> update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 10
> g++ -v
> openssl version
> cd /git/apache-zookeeper-3.6.4/
> mvn clean install -DskipTests -Pfull-build
> cd zookeeper-client/
> mvn clean install -Pfull-build
>
> the C client compiled and all the tests compiled too. A single test
> continuously fails (I see the same test failing in CI), but I don't know if
> this is related to the gcc/library versions or (more likely) related to the
> test being executed in docker (because the test works for me on my host
> machine).
>
>  [exec]   what():  equality assertion failed
>  [exec] - Expected: -101
>  [exec] - Actual  : -4
>  [exec]
>  [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoResetFAIL:
> zktest-mt
>
> So the tests compiled for me, at least with gcc 12.1.0, and I saw no error
> message related to the fix you created in
> https://github.com/apache/zookeeper/pull/1967
> Or maybe I'm missing something.
>
> I'm OK to create a new RC with the proposed fix. But I am a bit
> uncomfortable with not being able to reproduce this problem locally. Also I
> would rather like to have some well defined gcc / library version
> combinations (maybe some widely used LTS linux distro / docker images)
> which we commit to support.
>
> Maybe I should test with your exact gcc version (12.2.0 instead of 12.1.0)
> or openssl version (3.0.7 instead of 3.0.2)? Or maybe this problem can be
> also caused by mismatch of the stdlib we use? or maybe make or libsasl?
>
> What do you think?
>
> Máté
>
>
>
> On Fri, Dec 16, 2022 at 10:15 PM Chris Nauroth 
> wrote:
>
>> Unfortunately, I discovered one more problem: a similar compilation error
>> in C client test code. I filed this issue and pull request:
>>
>> https://issues.apache.org/jira/browse/ZOOKEEPER-4654
>> https://github.com/apache/zookeeper/pull/1967
>>
>> I must have been compiling only the product code and not the test code
>> when
>> I did my confirmation of branch-3.6 yesterday. Sorry about that.
>>
>> Since this is only test code, the argument is weaker for canceling the
>> vote
>> and creating a new RC with the fix. What do others think? I'd be fine
>> either way. (I haven't found any other issues with RC1.)
>>
>> Chris Nauroth
>>
>>
>> On Fri, Dec 16, 2022 at 4:31 AM Szalay-Bekő Máté <
>> szalay.beko.m...@gmail.com>
>> wrote:
>>
>> > +1 (binding)
>> >
>> > I did the following tests for the release candidate:
>> > - verified checksum and gpg signature of the artifacts
>> > - I built the source code (incl. the C-client, using -Pfull-build) on
>> > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
>> > - all the unit tests passed (both Java and C-client)
>> > - I also built and executed unit tests for zkpython
>> > - I also built the java code (without -Pfull-build) using other JDK
>> > versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
>> > time, just used 'clean install -DskipTests')
>> > - checkstyle and spotbugs passed
>> > - apache-rat passed
>> > - owasp (CVE check) passed
>> > - fatjar built
>> > - I executed quick rolling-upgrade tests (using
>> > https://github.com/symat/zk-rolling-upgrade-test):
>> >   - rolling upgrade from 3.5.10 to 3.6.4
>> >   - rolling upgrade from 3.6.3 to 3.6.4
>> >   - rolling upgrade from 3.6.4 to 3.7.

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 1

[Szalay-Bekő Máté]

Hello Chris,

I tried to reproduce the test build problem locally on docker, but was not
able to do so. I used ubuntu 22.04 (which uses gcc 11 by default),
installing gcc-12 (version: 12.1.0) and openssl (version: 3.0.2 15 Mar
2022).

My steps were:

docker run --volume ~/git:/git -it --rm ubuntu:22.04 /bin/bash

apt update && apt install -y libcppunit-dev maven default-jdk-headless
autoconf libtool gcc-12 g++-12 make software-properties-common pkg-config
wget git python-setuptools openssl libssl-dev ant
libsasl2-modules-gssapi-mit libsasl2-modules libsasl2-dev
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10
update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 10
g++ -v
openssl version
cd /git/apache-zookeeper-3.6.4/
mvn clean install -DskipTests -Pfull-build
cd zookeeper-client/
mvn clean install -Pfull-build

the C client compiled and all the tests compiled too. A single test
continuously fails (I see the same test failing in CI), but I don't know if
this is related to the gcc/library versions or (more likely) related to the
test being executed in docker (because the test works for me on my host
machine).

 [exec]   what():  equality assertion failed
 [exec] - Expected: -101
 [exec] - Actual  : -4
 [exec]
 [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoResetFAIL: zktest-mt

So the tests compiled for me, at least with gcc 12.1.0, and I saw no error
message related to the fix you created in
https://github.com/apache/zookeeper/pull/1967
Or maybe I'm missing something.

I'm OK to create a new RC with the proposed fix. But I am a bit
uncomfortable with not being able to reproduce this problem locally. Also I
would rather like to have some well defined gcc / library version
combinations (maybe some widely used LTS linux distro / docker images)
which we commit to support.

Maybe I should test with your exact gcc version (12.2.0 instead of 12.1.0)
or openssl version (3.0.7 instead of 3.0.2)? Or maybe this problem can be
also caused by mismatch of the stdlib we use? or maybe make or libsasl?

What do you think?

Máté



On Fri, Dec 16, 2022 at 10:15 PM Chris Nauroth  wrote:

> Unfortunately, I discovered one more problem: a similar compilation error
> in C client test code. I filed this issue and pull request:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-4654
> https://github.com/apache/zookeeper/pull/1967
>
> I must have been compiling only the product code and not the test code when
> I did my confirmation of branch-3.6 yesterday. Sorry about that.
>
> Since this is only test code, the argument is weaker for canceling the vote
> and creating a new RC with the fix. What do others think? I'd be fine
> either way. (I haven't found any other issues with RC1.)
>
> Chris Nauroth
>
>
> On Fri, Dec 16, 2022 at 4:31 AM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
>
> > +1 (binding)
> >
> > I did the following tests for the release candidate:
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build) on
> > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built and executed unit tests for zkpython
> > - I also built the java code (without -Pfull-build) using other JDK
> > versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
> > time, just used 'clean install -DskipTests')
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built
> > - I executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> >   - rolling upgrade from 3.5.10 to 3.6.4
> >   - rolling upgrade from 3.6.3 to 3.6.4
> >   - rolling upgrade from 3.6.4 to 3.7.1
> >   - rolling upgrade from 3.6.4 to 3.8.0
> > - checked the generated documentation (zookeeper-docs/target/html)
> > - compared generated release notes (
> >
> >
> https://people.apache.org/~symat/zookeeper-3.6.4-rc1/website/releasenotes.html
> > )
> > with Jira (
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > )
> >
> > Best regards,
> > Máté
> >
> > On Fri, Dec 16, 2022 at 1:28 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > This is a bugfix release candidate for 3.6.4. It fixes 41 issues,
> > > including CVE fixes,
> > > log4j1 removal (using reload4j from now) and various other bug fixes
> > > (thread leaks, data
> > > corruption, snapshotting and SASL related fixes).
> &g

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 1

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
time, just used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.6.4
  - rolling upgrade from 3.6.3 to 3.6.4
  - rolling upgrade from 3.6.4 to 3.7.1
  - rolling upgrade from 3.6.4 to 3.8.0
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (
https://people.apache.org/~symat/zookeeper-3.6.4-rc1/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
)

Best regards,
Máté

On Fri, Dec 16, 2022 at 1:28 PM Szalay-Bekő Máté 
wrote:

> This is a bugfix release candidate for 3.6.4. It fixes 41 issues,
> including CVE fixes,
> log4j1 removal (using reload4j from now) and various other bug fixes
> (thread leaks, data
> corruption, snapshotting and SASL related fixes).
>
> Please note, that based on our Release Strategy (
> https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
> should become end-of-life and most likely 3.6.4 will be our last 3.6
> release.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
>
> *** Please download, test and vote by December 28th 2022, 23:59 UTC+0. ***
>
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc1/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
>
> The release candidate tag in git to be voted upon: release-3.6.4-1
> https://github.com/apache/zookeeper/commits/release-3.6.4-1
> (please note, branch-3.6.4 will move here only after the vote)
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc1/website/
>
>
> Should we release this candidate?
>
>
> Best regards,
> Máté
>

[VOTE] Apache ZooKeeper release 3.6.4 candidate 1

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.6.4. It fixes 41 issues, including
CVE fixes,
log4j1 removal (using reload4j from now) and various other bug fixes
(thread leaks, data
corruption, snapshotting and SASL related fixes).

Please note, that based on our Release Strategy (
https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
should become end-of-life and most likely 3.6.4 will be our last 3.6
release.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076

*** Please download, test and vote by December 28th 2022, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.6.4-rc1/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/

The release candidate tag in git to be voted upon: release-3.6.4-1
https://github.com/apache/zookeeper/commits/release-3.6.4-1
(please note, branch-3.6.4 will move here only after the vote)

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.6.4-rc1/website/


Should we release this candidate?


Best regards,
Máté

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 0

[Szalay-Bekő Máté]

> I have confirmed that I can build the current branch-3.6 on my machine,
using GCC 12.2.0.

Thank you for testing it!!


> Does the build/test pipeline need to be updated to verify this? Why make
it a manual/release step.

> Perhaps part of the reason we're seeing this now is lack of CI on
branch-3.6.

Yes, the CI is simpler on branch-3.6. We have some CI, a PR jenkins job is
is building java and C code and runs all the tests (
https://ci-hadoop.apache.org/blue/organizations/jenkins/zookeeper-precommit-github-pr/detail/PR-1965/1/pipeline)
but we definitely miss the new github CI checks on this old branch. On the
other hand, even on newer branches we don't have any explicit checks to
build and test the C code with a matrix of GCC versions and libraries. We
could do something similar to how we test the Java code with multiple JDK
versions. (or at least add a few docker files that one can use locally to
have test environments with more important gcc/library version combinations)

Anyway, I knew about this problem based on other failures on newer
branches. I guess I should have waited for the fix to be merged before
cutting the release.

I am cancelling this VOTE for 3.6.4 RC 0 now.

I will send RC 1 soon.

Thank you for your time and feedback!
Máté

On Fri, Dec 16, 2022 at 1:45 AM Chris Nauroth  wrote:

> I have confirmed that I can build the current branch-3.6 on my machine,
> using GCC 12.2.0. I agree with your point that the GCC version might not be
> root cause, and it's also possible that it's caused by a difference in
> OpenSSL headers. My OpenSSL version is 3.0.7.
>
> Perhaps part of the reason we're seeing this now is lack of CI on
> branch-3.6. If we had CI on that branch, then we would have seen failures
> in recent branch-3.6 pull requests, like this one:
>
> https://github.com/apache/zookeeper/pull/1965
>
> Chris Nauroth
>
>
> On Thu, Dec 15, 2022 at 3:39 PM Patrick Hunt  wrote:
>
> > Does the build/test pipeline need to be updated to verify this? Why make
> it
> > a manual/release step.
> >
> > Patrick
> >
> > On Thu, Dec 15, 2022 at 3:35 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > Thanks for checking!
> > >
> > > I don't have a strong opinion. It would make sense to support newer gcc
> > > versions in a new release. On the other hand, it is not a regression on
> > the
> > > branch-3.6 (the c-client in this rc compiles with the same gcc versions
> > > which were compatible with 3.6 3).
> > >
> > > But I am OK to make a new RC. I don't have new gcc installed though to
> > test
> > > it locally. Also, I wonder if installing a new gcc is enough, or the
> > > openssl or other library versions also matter?
> > >
> > > Could you maybe check if the current branch-3.6 compiles on your
> machine,
> > > with gcc 12.2.0? As you mentioned, the fix for ZOOKEEPER-4641 is
> already
> > > present there (I merged it after RC 0). If it works for you, then I can
> > add
> > > this commit and cut RC 1.
> > >
> > > Unless someone disagree...
> > >
> > > Thanks,
> > > Máté
> > >
> > >
> > > On Thu, Dec 15, 2022, 6:45 PM Chris Nauroth 
> wrote:
> > >
> > > > Unfortunately, I can't compile the C client because of the FIPS_mode
> > bug
> > > > (ZOOKEEPER-4641). I'm on a newer version of GCC: 12.2.0. I see that
> > > > ZOOKEEPER-4641 was committed to branch-3.6 with a fix version of
> 3.6.5.
> > > > However, we're intending that 3.6.4 is the final 3.6 release, so
> there
> > > > never will be a 3.6.5.
> > > >
> > > > Sorry for the churn, but I'd prefer if we could bring that fix into a
> > new
> > > > release candidate. Let me know your thoughts on it.
> > > >
> > > > Chris Nauroth
> > > >
> > > >
> > > > On Wed, Dec 14, 2022 at 4:08 PM Szalay-Bekő Máté <
> > > > szalay.beko.m...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > I did the following tests for the release candidate:
> > > > > - verified checksum and gpg signature of the artifacts
> > > > > - I built the source code (incl. the C-client, using -Pfull-build)
> on
> > > > > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version
> 9.4.0
> > > > > - all the unit tests passed (both Java and C-client)
> > > > > - I also built and executed unit tests for zkpython
> > > > > - I also built th

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 0

[Szalay-Bekő Máté]

Thanks for checking!

I don't have a strong opinion. It would make sense to support newer gcc
versions in a new release. On the other hand, it is not a regression on the
branch-3.6 (the c-client in this rc compiles with the same gcc versions
which were compatible with 3.6 3).

But I am OK to make a new RC. I don't have new gcc installed though to test
it locally. Also, I wonder if installing a new gcc is enough, or the
openssl or other library versions also matter?

Could you maybe check if the current branch-3.6 compiles on your machine,
with gcc 12.2.0? As you mentioned, the fix for ZOOKEEPER-4641 is already
present there (I merged it after RC 0). If it works for you, then I can add
this commit and cut RC 1.

Unless someone disagree...

Thanks,
Máté


On Thu, Dec 15, 2022, 6:45 PM Chris Nauroth  wrote:

> Unfortunately, I can't compile the C client because of the FIPS_mode bug
> (ZOOKEEPER-4641). I'm on a newer version of GCC: 12.2.0. I see that
> ZOOKEEPER-4641 was committed to branch-3.6 with a fix version of 3.6.5.
> However, we're intending that 3.6.4 is the final 3.6 release, so there
> never will be a 3.6.5.
>
> Sorry for the churn, but I'd prefer if we could bring that fix into a new
> release candidate. Let me know your thoughts on it.
>
> Chris Nauroth
>
>
> On Wed, Dec 14, 2022 at 4:08 PM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
>
> > +1 (binding)
> >
> > I did the following tests for the release candidate:
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build) on
> > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built and executed unit tests for zkpython
> > - I also built the java code (without -Pfull-build) using other JDK
> > versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
> > time, just used 'clean install -DskipTests')
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built
> > - I executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> >   - rolling upgrade from 3.5.10 to 3.6.4
> >   - rolling upgrade from 3.6.3 to 3.6.4
> >   - rolling upgrade from 3.6.4 to 3.7.1
> >   - rolling upgrade from 3.6.4 to 3.8.0
> > - checked the generated documentation (zookeeper-docs/target/html)
> > - compared generated release notes (releasenotes.html) with Jira (
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > )
> >
> > Best regards,
> > Máté
> >
> > On Thu, Dec 15, 2022 at 1:05 AM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > This is a bugfix release candidate for 3.6.4. It fixes 40 issues,
> > > including CVE fixes,
> > > log4j1 removal (using reload4j from now) and various other bug fixes
> > > (thread leaks, data
> > > corruption, snapshotting and SASL related fixes).
> > >
> > > Please note, that based on our Release Strategy (
> > > https://zookeeper.apache.org/releases.html#release-strategy) branch
> 3.6
> > > should become end-of-life and most likely 3.6.4 will be our last 3.6
> > > release.
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > >
> > > *** Please download, test and vote by December 23th 2022, 23:59 UTC+0.
> > ***
> > >
> > >
> > > Source files:
> > > https://people.apache.org/~symat/zookeeper-3.6.4-rc0/
> > >
> > > Maven staging repo:
> > >
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
> > >
> > > The release candidate tag in git to be voted upon: release-3.6.4-0
> > > (please note, branch-3.6.4 will move here only after the vote)
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://people.apache.org/~symat/zookeeper-3.6.4-rc0/webpage/
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Best regards,
> > > Máté
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

I did the following tests for the release candidate:
- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- I also built the java code (without -Pfull-build) using other JDK
versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
time, just used 'clean install -DskipTests')
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.10 to 3.6.4
  - rolling upgrade from 3.6.3 to 3.6.4
  - rolling upgrade from 3.6.4 to 3.7.1
  - rolling upgrade from 3.6.4 to 3.8.0
- checked the generated documentation (zookeeper-docs/target/html)
- compared generated release notes (releasenotes.html) with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
)

Best regards,
Máté

On Thu, Dec 15, 2022 at 1:05 AM Szalay-Bekő Máté 
wrote:

> This is a bugfix release candidate for 3.6.4. It fixes 40 issues,
> including CVE fixes,
> log4j1 removal (using reload4j from now) and various other bug fixes
> (thread leaks, data
> corruption, snapshotting and SASL related fixes).
>
> Please note, that based on our Release Strategy (
> https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
> should become end-of-life and most likely 3.6.4 will be our last 3.6
> release.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
>
> *** Please download, test and vote by December 23th 2022, 23:59 UTC+0. ***
>
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
>
> The release candidate tag in git to be voted upon: release-3.6.4-0
> (please note, branch-3.6.4 will move here only after the vote)
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.6.4-rc0/webpage/
>
>
> Should we release this candidate?
>
>
> Best regards,
> Máté
>

[VOTE] Apache ZooKeeper release 3.6.4 candidate 0

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.6.4. It fixes 40 issues, including
CVE fixes,
log4j1 removal (using reload4j from now) and various other bug fixes
(thread leaks, data
corruption, snapshotting and SASL related fixes).

Please note, that based on our Release Strategy (
https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
should become end-of-life and most likely 3.6.4 will be our last 3.6
release.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076

*** Please download, test and vote by December 23th 2022, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.6.4-rc0/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/

The release candidate tag in git to be voted upon: release-3.6.4-0
(please note, branch-3.6.4 will move here only after the vote)

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.6.4-rc0/webpage/


Should we release this candidate?


Best regards,
Máté

Re: ZooKeeper 3.6.4 Release Date - EoL for 3.6 after 3.6.4?

[Szalay-Bekő Máté]

Thank you for taking care of Jetty!!
I'll run CVE checks and also go through all the important dependencies,
bumping them if needed.

Máté

On Tue, Dec 6, 2022 at 7:00 PM Chris Nauroth  wrote:

> I would like to get this Jetty dependency upgrade into 3.6.4:
>
> https://github.com/apache/zookeeper/pull/1941
>
> This may be a good time to look at locking in any other relevant dependency
> upgrades, as we're considering this to be the last 3.6 release.
>
> Chris Nauroth
>
>
> On Mon, Dec 5, 2022 at 9:14 AM Chris Nauroth  wrote:
>
> > +1 for retiring the 3.6 release line after 3.6.4.
> >
> > Máté, thank you for helping with the release!
> >
> > Chris Nauroth
> >
> >
> > On Sat, Dec 3, 2022 at 10:21 AM Enrico Olivelli 
> > wrote:
> >
> >> Mate,
> >>
> >> Il Sab 3 Dic 2022, 16:53 Szalay-Bekő Máté 
> ha
> >> scritto:
> >>
> >> > Hello!
> >> >
> >> > I think I can spare the time for starting the release process sometime
> >> in
> >> > the second part of next week.
> >> >
> >>
> >> Thanks! I will test it and VOTE
> >>
> >>
> >>
> >> > I see the last 3.6 release happened on 13 April, 2021 - pretty long
> ago.
> >> > There are plenty of tickets to ship, including the log4j -> reload4j
> >> > migration: https://issues.apache.org/jira/browse/ZOOKEEPER-4455
> >>
> >>
> >>
> >> This is very important
> >>
> >> >
> >> >
> >> > All tickets with fixVersion=3.6.4:
> >> >
> >> >
> >>
> https://issues.apache.org/jira/browse/ZOOKEEPER-4602?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.6.4%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
> >> >
> >> > Reading our official release strategy (
> >> > https://zookeeper.apache.org/releases.html), I am not exactly sure
> if /
> >> > when we should move branch 3.6 to EoL. Maybe after 3.6.4?
> >> >
> >> > What do you think?
> >> >
> >>
> >>
> >> I agree.
> >> The migration from 3.6.x to 3.8.x is pretty straightforward and anybody
> >> who
> >> is on 3.6 should be able to move to 3.8 pretty easily.
> >>
> >> There is no big reason to maintain the 3.6 branch
> >>
> >> Thanks for bringing up this discussion
> >>
> >> Enrico
> >>
> >>
> >> > Máté
> >> >
> >> >
> >> > On Thu, Dec 1, 2022 at 3:31 PM Enrico Olivelli 
> >> > wrote:
> >> >
> >> > > Mark
> >> > >
> >> > > Il giorno sab 26 nov 2022 alle ore 17:57 Mark Sangster
> >> > >  ha scritto:
> >> > > >
> >> > > > Hi,
> >> > > >
> >> > > > Is there a planned release date for 3.6.4?
> >> > >
> >> > > Currently there is no plan.
> >> > > but if there is someone who needs it I believe that we can bootstrap
> >> > > the process.
> >> > >
> >> > > It is only a matter of finding a volunteer to cut the release.
> >> > >
> >> > > Anyone is willing to help here ?
> >> > >
> >> > > Enrico
> >> > >
> >> > > >
> >> > > > Mark
> >> > >
> >> >
> >>
> >
>

ZooKeeper 3.6.4 Release Date - EoL for 3.6 after 3.6.4?

[Szalay-Bekő Máté]

Hello!

I think I can spare the time for starting the release process sometime in
the second part of next week.

I see the last 3.6 release happened on 13 April, 2021 - pretty long ago.
There are plenty of tickets to ship, including the log4j -> reload4j
migration: https://issues.apache.org/jira/browse/ZOOKEEPER-4455

All tickets with fixVersion=3.6.4:
https://issues.apache.org/jira/browse/ZOOKEEPER-4602?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.6.4%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

Reading our official release strategy (
https://zookeeper.apache.org/releases.html), I am not exactly sure if /
when we should move branch 3.6 to EoL. Maybe after 3.6.4?

What do you think?

Máté


On Thu, Dec 1, 2022 at 3:31 PM Enrico Olivelli  wrote:

> Mark
>
> Il giorno sab 26 nov 2022 alle ore 17:57 Mark Sangster
>  ha scritto:
> >
> > Hi,
> >
> > Is there a planned release date for 3.6.4?
>
> Currently there is no plan.
> but if there is someone who needs it I believe that we can bootstrap
> the process.
>
> It is only a matter of finding a volunteer to cut the release.
>
> Anyone is willing to help here ?
>
> Enrico
>
> >
> > Mark
>

Re: Can the leader of a Zookeeper be specifically selected at startup?

[Szalay-Bekő Máté]

I also don't really know why you would need a single host being "preferred"
as leader. I think the safest (and the best practice) is to make sure all
your ZooKeeper servers are the same in terms of networking / performance /
etc.

Not knowing your goals, maybe the Observer feature is also something you
can take a look into:
https://zookeeper.apache.org/doc/r3.6.3/zookeeperObservers.html

Best regards,
Mate

On Mon, Jun 20, 2022 at 9:57 AM Enrico Olivelli  wrote:

> George,
> really, it should not be a problem which is the leader. it is
> automatically chosen.
> Each node should be ideally as powerful as the other peers.
>
> why do you need this "preferred leader" ?
> I am afraid that you have some flaw in your design
>
> Enrico
>
> Il giorno lun 20 giu 2022 alle ore 05:39 Kezhu Wang 
> ha scritto:
> >
> > Hi,
> >
> > I think this could be achieved with help from `reconfig`[1]:
> > * Configs all nodes with `standaloneEnabled=false`,
> `reconfigEnabled=true`.
> > * Starts node-2 as sole quorum participant.
> > * Now node-2 is the leader. You will see "No server failure will be
> > tolerated. You need at least 3 servers”.
> > * Starts node-1 and node-3 with all quorum.
> > * `zkCli.sh config` shows only node-2 for now.
> > * `zkCli.sh reconfig -add node-1,node-2` will add both node-1, node-3 to
> > quorum.
> > * According to `Leader.tryToCommit`[2], node-2 will be the leader due to
> > old leadership in old quorum and voter in new quorum.
> >
> > node-2 is the leader in whole progress.
> >
> > [1]: https://zookeeper.apache.org/doc/current/zookeeperReconfig.html
> > [2]:
> >
> https://github.com/apache/zookeeper/blob/b4f9aab099880ba8ef08eaff697debe6cdeae057/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Leader.java#L950
> >
> > Best,
> > Kezhu Wang
> >
> > On June 19, 2022 at 23:00:59, Heller, George A III CTR (USA) (
> > george.a.heller2@mail.mil.invalid) wrote:
> >
> > We have 3 Zookeeper nodes and would like node 2 to always be the leader
> > unless node 2 goes down. IF node 2 goes down, then either node 1 or node
> 3
> > would be the leader.
> >
> >
> >
> > Can this be done? If so, how would this be done?
>

Re: Few questions on connection retry on auth failure.

[Szalay-Bekő Máté]

It's been a while since I was checking these parts...

I also think the generic idea is that when you create a ZooKeeper class on
the client side, it will asynchronously try to connect to the server and
publish its state (connecting / connected / session-timeout / etc) through
the watcher.

I remember that ZooKeeper class is using the ClientCnx class to manage the
state of the connection which has a notion of sessionTimeout and
connectTimeout. It is trying to connect to each known server in a
round-robin fashion. Each connection attempt is tried for 'connectTimeout'
time and I think a SessionTimeoutException is thrown when no server was
responding in sessionTimeout time. (I think by default connectTimeout =
SessionTimeout / number_of_servers). But I am not entirely sure what
happens after the SessionTimeoutException. Normally I think ZooKeeper
client doesn't reconnect automatically after a session timeout, as this is
a case that needs to be handled by the client application. (no consistency
can be guaranteed among different sessions; also ephemeral znodes will be
deleted, etc. see:
https://zookeeper.apache.org/doc/r3.6.3/zookeeperOver.html#Guarantees)

But maybe if there was no active session created yet, then maybe there is
an infinite retry logic in the client.

I don't have much time right now to dig deeper into these classes. I would
assume we already have some unit tests around here too, which could be
checked to see the expected behaviour.

Also I doin't know exactly how authentication failure is handled in the
client side... The server might fall-back to an 'unauthenticated session'
in case of authentication failures, or it can refuse the connection attempt
(this can be configured, at least for SASL authentication:
'*zookeeper.sessionRequireClientSASLAuth'
*)

Also I think the best would be to actually test this with your exact setup.
(e.g. on the clusters we use, we still run ZooKeeper 3.5 in production with
SSL encryption + Kerberos authentication... which might behave differently
than what is your setup with 3.6.3... and also you might use x509
authentication?) But it shouldn't be hard to emulate some authentication
failures with your setup.

Best regards,
Mate

On Fri, Jun 17, 2022 at 11:23 PM Rahul Rane 
wrote:

> Bumping up on this one.
>
> Thanks,
> Rahul Rane
>
> From: Rahul Rane 
> Date: Wednesday, May 25, 2022 at 2:57 PM
> To: dev@zookeeper.apache.org 
> Subject: Few questions on connection retry on auth failure.
>
> Hello team,
>
>
>
> We need some help in understanding the zookeeper expected behavior and
> potential solution to the problem.
>
>
>
> Context :
>
> We have extended ServerAuthenticationProvider with x509 scheme based on
> 3.6.3 zookeeper server. We are trying to understand connection retry
> scenario. On auth failure, we see that zookeeper client retries to
> establish connection with server until the timeout or infinitely if no
> timeout is set. We are using
> org.apache.zookeeper.server.NettyServerCnxnFactory as Server connection
> factory.
>
>
>
> Couple of questions :
>
>   1.  Is zookeeper client supposed to retry infinitely on auth failure
> from zookeeper server?
>   2.  Is there a way zookeeper client does not perform infinitely retries
> on auth failure errors and bails out after first auth failure itself?
>   3.  We can’t find anything about auth failure errors in zookeeper client
> logs but just that connection is closed. After looking into Netty Server
> code, we see the auth failure is not communicated to client but got masked
> here<
> https://github.com/linkedin/zookeeper/blob/8bcaf7bb3cfa6470e1660e2b36964ae2284197df/zookeeper-server/src/main/java/org/apache/zookeeper/server/NettyServerCnxn.java#L99>.
> So we were wondering if we are missing something here?
>
>
>
> Thanks for the help and let me know if you need any clarification on any
> of the questions.
>
>
>
> Thanks,
>
> Rahul Rane
>

outdated info on 'credits' page

[Szalay-Bekő Máté]

Hello!

I just added myself to the list on the Credits page:
https://zookeeper.apache.org/credits.html
And I think in general these tables might not be up-to-date (especially the
organization columns).

If someone wants to push an update, there is the info how to do it:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/WebSiteSetup
(or you can just send me what you want to change and I can commit it
quickly)

Best regards,
Mate

[ANNOUNCE] Apache ZooKeeper 3.5.10

[Szalay-Bekő Máté]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.5.10

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

3.5.10 is the last bugfix release for branch 3.5, as 3.5 is EoL since 1st
June, 2022.
It fixes 44 issues, including CVE fixes, log4j1 removal (using reload4j
from now)
and various other bug fixes (thread leaks, data corruption, snapshotting
and SASL related fixes).

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.5.10 Release Notes are at:
https://zookeeper.apache.org/doc/r3.5.10/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,
The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 1

[Szalay-Bekő Máté]

Thank you all for testing the release candidate!

I'm happy to announce that we have unanimously approved this release.
There are 4 approving votes, all of them are binding:

- Andor Molnár (binding)
- Patrick Hunt (binding)
- Chris Nauroth (binding)
- Máté Szalay-Bekő (binding)

There are no disapproving votes.

I will promote the artifacts and complete the release procedure.

Thanks to everyone who contributed to this release!

Best Regards,
Máté

On Thu, Jun 2, 2022 at 9:50 PM Chris Nauroth  wrote:

> +1 (binding)
>
> - Verified all checksums.
> - Verified all signatures.
> - Built from source, including native code on Linux.
> - Tests passed.
> - Ran several small samples successfully.
>
> Chris Nauroth
>
>
> On Wed, Jun 1, 2022 at 1:07 PM Patrick Hunt  wrote:
>
> > +1. xsum/sig are fine. rat ran clean. The build issue I encountered with
> > rc0 is now fixed. I did some manual testing with various cluster sizes
> and
> > it all came clean.
> >
> > Patrick
> >
> > On Sun, May 29, 2022 at 10:09 AM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com> wrote:
> >
> > > This is a bugfix release candidate for 3.5.10. It fixes 44 issues,
> > > including CVE fixes,
> > > log4j1 removal (using reload4j from now) and various other bug fixes
> > > (thread leaks, data
> > > corruption, snapshotting and SASL related fixes).
> > >
> > > Please note, we announced 3.5 to be EOL from June 1st 2022, so most
> > likely
> > > this will be our
> > > last 3.5 release.
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
> > >
> > > *** Please download, test and vote by June 3rd 2022, 23:59 UTC+0. ***
> > >
> > >
> > > Source files:
> > > https://people.apache.org/~symat/zookeeper-3.5.10-rc1/
> > >
> > > Maven staging repo:
> > >
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/
> > >
> > > The release candidate tag in git to be voted upon: release-3.5.10-rc1
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://people.apache.org/~symat/zookeeper-3.5.10-rc1/website/
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Best regards,
> > > Máté
> > >
> >
>

Re: Need information around adding group and weight.

[Szalay-Bekő Máté]

Hello!

I don't think it is possible to change these with dynamic reconfig,
although I might be wrong.
I never really used groups and weights in production yet and I don't think
many people would use them in general.

But maybe others have more experience.

Best regards,
Mate


On Wed, Jun 1, 2022 at 8:04 PM Shivam Kumar 
wrote:

> Hey Team,
> We wanted to add groups and weights to zk with the help of the reconfig
> command but it looks like it is not supported.
> Is there any way we can change or add groups and weights in the running
> cluster??
>
> Regards,
> Shivam kumar
>

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 1

[Szalay-Bekő Máté]

+1 (binding)

- I built the java code (-DskipTests) on Ubuntu 20.04, using:
  - maven 3.6.3 and 3.8.5
  - OpenJDK 8u212, 8u322, 11.0.15, 17.0.3 and 18.0.1

- I built the full source code (-Pfull-build) on Ubuntu 20.04 using OpenJDK
8u212 and maven 3.6.3
- all the unit tests passed (both Java and C-client)
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.9  to 3.5.10
  - rolling upgrade from 3.5.10 to 3.6.3
  - rolling upgrade from 3.5.10 to 3.7.1
  - rolling upgrade from 3.5.10  to 3.8.0
- checked generated documentation
- compared generated release notes (
https://people.apache.org/~symat/zookeeper-3.5.10-rc1/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
)
- checked the signature and checksum of the artifacts
- run some smoke tests using both the binary distribution
(apache-zookeeper-3.5.10-bin.tar.gz) and on a freshly compiled version
(based on apache-zookeeper-3.5.10.tar.gz)


Best regards,
Máté

On Sun, May 29, 2022 at 7:08 PM Szalay-Bekő Máté 
wrote:

> This is a bugfix release candidate for 3.5.10. It fixes 44 issues,
> including CVE fixes,
> log4j1 removal (using reload4j from now) and various other bug fixes
> (thread leaks, data
> corruption, snapshotting and SASL related fixes).
>
> Please note, we announced 3.5 to be EOL from June 1st 2022, so most likely
> this will be our
> last 3.5 release.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
>
> *** Please download, test and vote by June 3rd 2022, 23:59 UTC+0. ***
>
>
> Source files:
> https://people.apache.org/~symat/zookeeper-3.5.10-rc1/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/
>
> The release candidate tag in git to be voted upon: release-3.5.10-rc1
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.5.10-rc1/website/
>
>
> Should we release this candidate?
>
>
> Best regards,
> Máté
>

[VOTE] Apache ZooKeeper release 3.5.10 candidate 1

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.5.10. It fixes 44 issues,
including CVE fixes,
log4j1 removal (using reload4j from now) and various other bug fixes
(thread leaks, data
corruption, snapshotting and SASL related fixes).

Please note, we announced 3.5 to be EOL from June 1st 2022, so most likely
this will be our
last 3.5 release.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434

*** Please download, test and vote by June 3rd 2022, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.5.10-rc1/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/

The release candidate tag in git to be voted upon: release-3.5.10-rc1

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.5.10-rc1/website/


Should we release this candidate?


Best regards,
Máté

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 0

[Szalay-Bekő Máté]

Thank you for checking and reporting the issue!

Maven should be the 'official way' of building the artifacts on 3.5.
The release candidate builds with java 8 and java 11, but I was able to
reproduce the same problem with maven 3.6.3 and JDK 18.

I found a related jira issue:
https://issues.apache.org/jira/browse/ZOOKEEPER-4382 - "Update Maven Bundle
Plugin in order to allow builds on JDK18"
This was not backported originally to branch-3.5 and I also missed it
preparing RC 0.

The strange thing is that I can still build the ZooKeeper 3.5.9 with JDK
18, even without ZOOKEEPER-4382. Something with the dependency upgrades (or
other pod modifications) must have triggered this issue between 3.5.9 and
3.5.10.

If 3.5.9 was built with jdk 18, it should also work with 3.5.10.

I cancel this vote and prepare a new RC.

Thanks again everyone for testing!

Best regards,
Mate

On Thu, May 26, 2022 at 10:29 PM Patrick Hunt  wrote:

> maven build is failing for me - should it? I can't remember if we
> "officially" supported maven in 3.5?
>
> [phunt:apache-zookeeper-3.5.10] $ mvn --version
> Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
> Maven home: /opt/homebrew/Cellar/maven/3.8.5/libexec
> Java version: 18.0.1, vendor: Homebrew, runtime:
> /opt/homebrew/Cellar/openjdk/18.0.1/libexec/openjdk.jdk/Contents/Home
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "12.3.1", arch: "aarch64", family: "mac"
>
> mvn clean install -DskipTests
> 
> [ERROR] Failed to execute goal
> org.apache.felix:maven-bundle-plugin:4.1.0:bundle (build bundle) on project
> zookeeper-jute: Execution build bundle of goal
> org.apache.felix:maven-bundle-plugin:4.1.0:bundle failed.:
> ConcurrentModificationException -> [Help 1]
>
> On Thu, May 26, 2022 at 7:56 AM Enrico Olivelli 
> wrote:
>
> > Sorry for the late reply.
> > I will test the RC tomorrow
> >
> > Enrico
> >
> > Il giorno gio 26 mag 2022 alle ore 16:29 Szalay-Bekő Máté
> >  ha scritto:
> > >
> > > Hello All,
> > >
> > > Thank you Chris for the quick vote!
> > >
> > > Despite my earlier attempt to mislead everyone (I made a copy-paste
> error
> > > and wrote 'non-binding' when I voted), we already have two binding +1
> for
> > > this release.
> > > If some of you have the time, please test the RC and vote.
> > >
> > > Best regards,
> > > Mate
> > >
> > >
> > > On Fri, May 20, 2022 at 9:02 AM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > > wrote:
> > >
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - I built the source code (-Pfull-build) on Ubuntu 20.04 using
> OpenJDK
> > > > 8u212 and maven 3.6.3.
> > > > - all the unit tests passed (both Java and C-client).
> > > > - I also built and executed unit tests for zkpython
> > > > - checkstyle and spotbugs passed
> > > > - apache-rat passed
> > > > - owasp (CVE check) passed
> > > > - I executed quick rolling-upgrade tests (using
> > > > https://github.com/symat/zk-rolling-upgrade-test):
> > > >   - rolling upgrade from 3.5.9  to 3.5.10
> > > >   - rolling upgrade from 3.5.10 to 3.6.3
> > > >   - rolling upgrade from 3.5.10 to 3.7.1
> > > >   - rolling upgrade from 3.7.0  to 3.8.0
> > > > - check generated documentation
> > > > - compared generated release notes (
> > > >
> >
> https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/releasenotes.html
> > )
> > > > with Jira (
> > > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
> > > > )
> > > > - check the signature and checksum of the artifacts
> > > > - run some smoke tests using both the binary distribution
> > > > (apache-zookeeper-3.5.10-bin.tar.gz) and on a freshly compiled
> version
> > > > (based on apache-zookeeper-3.5.10.tar.gz)
> > > >
> > > > The C-client tests can be tricky (and annoying) sometimes. On docker
> > some
> > > > of them fail for me more frequently (also IPV6 tests can be tricky to
> > setup
> > > > on docker+mac). Also sometimes they leave ZooKeeper processes open
> > after a
> > > > test failure, and these are preventing later test runs to pass. Worth
> > to
> > > > take a look and kill these before running the C tests again. In the
> > end I
> > > > got all of th

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 0

[Szalay-Bekő Máté]

Hello All,

Thank you Chris for the quick vote!

Despite my earlier attempt to mislead everyone (I made a copy-paste error
and wrote 'non-binding' when I voted), we already have two binding +1 for
this release.
If some of you have the time, please test the RC and vote.

Best regards,
Mate


On Fri, May 20, 2022 at 9:02 AM Szalay-Bekő Máté 
wrote:

>
> +1 (non-binding)
>
> - I built the source code (-Pfull-build) on Ubuntu 20.04 using OpenJDK
> 8u212 and maven 3.6.3.
> - all the unit tests passed (both Java and C-client).
> - I also built and executed unit tests for zkpython
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - I executed quick rolling-upgrade tests (using
> https://github.com/symat/zk-rolling-upgrade-test):
>   - rolling upgrade from 3.5.9  to 3.5.10
>   - rolling upgrade from 3.5.10 to 3.6.3
>   - rolling upgrade from 3.5.10 to 3.7.1
>   - rolling upgrade from 3.7.0  to 3.8.0
> - check generated documentation
> - compared generated release notes (
> https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/releasenotes.html)
> with Jira (
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
> )
> - check the signature and checksum of the artifacts
> - run some smoke tests using both the binary distribution
> (apache-zookeeper-3.5.10-bin.tar.gz) and on a freshly compiled version
> (based on apache-zookeeper-3.5.10.tar.gz)
>
> The C-client tests can be tricky (and annoying) sometimes. On docker some
> of them fail for me more frequently (also IPV6 tests can be tricky to setup
> on docker+mac). Also sometimes they leave ZooKeeper processes open after a
> test failure, and these are preventing later test runs to pass. Worth to
> take a look and kill these before running the C tests again. In the end I
> got all of them to pass on Ubuntu 20.04 and gcc 9.4.0 (using native ubuntu,
> not mac+docker), having all the recommended ubuntu packages installed (
> https://github.com/apache/zookeeper/blob/master/README_packaging.md). But
> would be nice to improve our test quality here... (unfortunately I don't
> think more recent branches would be in a much better shape)
>
> Best regards,
> Mate
>
> On Fri, May 20, 2022 at 12:10 AM Chris Nauroth 
> wrote:
>
>> +1 (binding)
>>
>> - Verified all checksums.
>> - Verified all signatures.
>> - Built from source, including native code on Linux.
>> - Tests passed.
>> - Ran several small samples successfully.
>>
>> I'm seeing failures in the cppunit tests in zookeeper-client-c. The same
>> failures reproduce on version 3.5.9 though, so it's not related to this
>> release. I assume this is a configuration issue I need to diagnose in my
>> environment.
>>
>> Chris Nauroth
>>
>>
>> On Thu, May 19, 2022 at 2:31 AM Szalay-Bekő Máté <
>> szalay.beko.m...@gmail.com>
>> wrote:
>>
>> > This is a bugfix release candidate for 3.5.10. It fixes 43 issues,
>> > including CVE fixes, log4j1 removal (by default using reload4j from now)
>> > and various other bug fixes (thread leaks, data corruption, snapshotting
>> > and SASL related fixes).
>> >
>> > Please note, we announced 3.5 to be EOL from June 1st 2022, so most
>> likely
>> > this will be our
>> > last 3.5 release.
>> >
>> > The full release notes is available at:
>> >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
>> >
>> > *** Please download, test and vote by May 27th 2022, 23:59 UTC+0. ***
>> >
>> >
>> > Source files:
>> > https://people.apache.org/~symat/zookeeper-3.5.10-rc0/
>> >
>> > Maven staging repo:
>> >
>> >
>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/
>> >
>> > The release candidate tag in git to be voted upon: release-3.5.10-rc0
>> >
>> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>> > https://www.apache.org/dist/zookeeper/KEYS
>> >
>> > The staging version of the website is:
>> > https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/
>> >
>> >
>> > Should we release this candidate?
>> >
>> >
>> > Best regards,
>> > Máté
>> >
>>
>

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 0

[Szalay-Bekő Máté]

+1 (non-binding)

- I built the source code (-Pfull-build) on Ubuntu 20.04 using OpenJDK
8u212 and maven 3.6.3.
- all the unit tests passed (both Java and C-client).
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.9  to 3.5.10
  - rolling upgrade from 3.5.10 to 3.6.3
  - rolling upgrade from 3.5.10 to 3.7.1
  - rolling upgrade from 3.7.0  to 3.8.0
- check generated documentation
- compared generated release notes (
https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/releasenotes.html)
with Jira (
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
)
- check the signature and checksum of the artifacts
- run some smoke tests using both the binary distribution
(apache-zookeeper-3.5.10-bin.tar.gz) and on a freshly compiled version
(based on apache-zookeeper-3.5.10.tar.gz)

The C-client tests can be tricky (and annoying) sometimes. On docker some
of them fail for me more frequently (also IPV6 tests can be tricky to setup
on docker+mac). Also sometimes they leave ZooKeeper processes open after a
test failure, and these are preventing later test runs to pass. Worth to
take a look and kill these before running the C tests again. In the end I
got all of them to pass on Ubuntu 20.04 and gcc 9.4.0 (using native ubuntu,
not mac+docker), having all the recommended ubuntu packages installed (
https://github.com/apache/zookeeper/blob/master/README_packaging.md). But
would be nice to improve our test quality here... (unfortunately I don't
think more recent branches would be in a much better shape)

Best regards,
Mate

On Fri, May 20, 2022 at 12:10 AM Chris Nauroth  wrote:

> +1 (binding)
>
> - Verified all checksums.
> - Verified all signatures.
> - Built from source, including native code on Linux.
> - Tests passed.
> - Ran several small samples successfully.
>
> I'm seeing failures in the cppunit tests in zookeeper-client-c. The same
> failures reproduce on version 3.5.9 though, so it's not related to this
> release. I assume this is a configuration issue I need to diagnose in my
> environment.
>
> Chris Nauroth
>
>
> On Thu, May 19, 2022 at 2:31 AM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
>
> > This is a bugfix release candidate for 3.5.10. It fixes 43 issues,
> > including CVE fixes, log4j1 removal (by default using reload4j from now)
> > and various other bug fixes (thread leaks, data corruption, snapshotting
> > and SASL related fixes).
> >
> > Please note, we announced 3.5 to be EOL from June 1st 2022, so most
> likely
> > this will be our
> > last 3.5 release.
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
> >
> > *** Please download, test and vote by May 27th 2022, 23:59 UTC+0. ***
> >
> >
> > Source files:
> > https://people.apache.org/~symat/zookeeper-3.5.10-rc0/
> >
> > Maven staging repo:
> >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/
> >
> > The release candidate tag in git to be voted upon: release-3.5.10-rc0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/
> >
> >
> > Should we release this candidate?
> >
> >
> > Best regards,
> > Máté
> >
>

[VOTE] Apache ZooKeeper release 3.5.10 candidate 0

[Szalay-Bekő Máté]

This is a bugfix release candidate for 3.5.10. It fixes 43 issues,
including CVE fixes, log4j1 removal (by default using reload4j from now)
and various other bug fixes (thread leaks, data corruption, snapshotting
and SASL related fixes).

Please note, we announced 3.5 to be EOL from June 1st 2022, so most likely
this will be our
last 3.5 release.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434

*** Please download, test and vote by May 27th 2022, 23:59 UTC+0. ***


Source files:
https://people.apache.org/~symat/zookeeper-3.5.10-rc0/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/

The release candidate tag in git to be voted upon: release-3.5.10-rc0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://people.apache.org/~symat/zookeeper-3.5.10-rc0/website/


Should we release this candidate?


Best regards,
Máté

Re: [ANNOUNCE] Apache ZooKeeper 3.7.1

[Szalay-Bekő Máté]

hurray! :)
thank you for coordinating this release!!

On Thu, May 12, 2022 at 6:12 AM Mohammad Arshad  wrote:

> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.7.1
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.7.1 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.7.1/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>

Re: [VOTE] Apache ZooKeeper release 3.7.1 candidate 1

[Szalay-Bekő Máté]

+1 (binding)

- I built the source code (-Pfull-build) on Ubuntu 18.04.6 using OpenJDK
11.0.14.1 and maven 3.6.0.
- all the Java unit tests passed eventually
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed (with some false-positives, see ZOOKEEPER-4510)
- fatjar built
- I executed quick rolling-upgrade smoke tests (using
https://github.com/symat/zk-rolling-upgrade-test):
- rolling upgrade from 3.5.9 to 3.7.1 RC1
- rolling upgrade from 3.6.3 to 3.7.1 RC1
- rolling upgrade from 3.7.0 to 3.7.1 RC1
- rolling upgrade from 3.7.1 RC1 to 3.8.0

Some unit tests failed for me the first time, but succeeded when I run them
one-by-one:
- org.apache.zookeeper.ZKUtilTest
- org.apache.zookeeper.server.ZooKeeperServerMainTest
- org.apache.zookeeper.server.quorum.QuorumPeerMainMultiAddressTest
- org.apache.zookeeper.server.quorum.QuorumPeerMainTest
- org.apache.zookeeper.server.quorum.ReadOnlyModeTest
- org.apache.zookeeper.server.util.RestoreCommittedLogTest
- org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest
(when I tested RC 0, I also had some flaky tests... some of these were
flaky this time too, but not all)

Thanks for preparing the RC!

Best regards,
Máté

On Sat, May 7, 2022 at 5:34 PM Patrick Hunt  wrote:

> +1 xsum/sigs are valid, rat ran clean as did the dep checks. I tested with
> various sized clusters manually and it all ran fine.
>
> Regards,
>
> Patrick
>
> On Sat, May 7, 2022 at 1:01 AM Mohammad Arshad  wrote:
>
> > This is a bug fix release candidate for 3.7.1. It contains 64 fixes.
> >
> > The full release notes is available at:
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350030
> >
> > *** Please download, test and vote by Monday, 09 May, 2022, 23:59 UTC+0.
> > ***
> >
> > Source files:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc1/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1076
> >
> > The release candidate tag in git to be voted upon: release-3.7.1-1
> > https://github.com/apache/zookeeper/tree/release-3.7.1-1
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> >
> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc1/website/index.html
> >
> >
> > Should we release this candidate?
> >
> > Thanks Regards
> > -Arshad
> >
>

last 3.5 release

[Szalay-Bekő Máté]

Hello All,

Our communicated EoL date for branch 3.5 is approaching (1st of June), and
I volunteer to cut the 3.5.10 release soon. Our last release (3.5.9)
happened on 15 January, 2021, and I think it would make sense to have one
last release on 3.5. Let me know if you disagree.

It would include CVE fixes, the log4j1 elimination / reload4j migration (I
need to backport that still) and a couple of other bug fixes.

Please let me know if you think three would be any ticket / PR I should
wait for. I'll also go through the recent list of bug fixes to see if we
missed to backport something security related / really burning fix.

Best regards,
Mate

Re: [VOTE] Apache ZooKeeper release 3.7.1 candidate 0

[Szalay-Bekő Máté]

+1 (binding)

- I built the source code (-Pfull-build) on Ubuntu 18.04.6 using OpenJDK
11.0.14.1 and maven 3.6.0.
- all the unit tests passed eventually (both Java and C-client).
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed (with some false-positives, see ZOOKEEPER-4510)
- fatjar built
- I executed quick rolling-upgrade smoke tests (using
https://github.com/symat/zk-rolling-upgrade-test):
- rolling upgrade from 3.5.9 to 3.7.1
- rolling upgrade from 3.6.3 to 3.7.1
- rolling upgrade from 3.7.0 to 3.7.1
- rolling upgrade from 3.7.1 to 3.8.0

Few minor issues, none of them blocker in my opinion:
- some false positive CVE problems (followed in
https://issues.apache.org/jira/browse/ZOOKEEPER-4510)
- some unit tests failed for me the first time, but succeeded when I run
them one-by-one:
- org.apache.zookeeper.ZKUtilTest
- org.apache.zookeeper.server.ZooKeeperServerMainTest
- org.apache.zookeeper.server.quorum.QuorumPeerMainMultiAddressTest
- org.apache.zookeeper.server.quorum.QuorumPeerMainTest
- org.apache.zookeeper.server.quorum.Zab1_0Test
- org.apache.zookeeper.server.util.JvmPauseMonitorTest
- org.apache.zookeeper.server.util.RequestPathMetricsCollectorTest
- some C unit tests failed also on my docker environment (these run
successfully on CI, so I assume it is only a problem on my docker setup):
- Zookeeper_readOnly::testReadOnly (only on the multi-threaded C-client
test suite)
- Zookeeper_readOnly::testReadOnlyWithSSL (only on the multi-threaded
C-client test suite)

Thanks for your work preparing the RC!

Kind regards,
Máté

On Fri, Apr 29, 2022 at 4:03 PM Christopher  wrote:

> FWIW, this is already being tracked on
> https://issues.apache.org/jira/browse/ZOOKEEPER-4510
> It's a false positive. I don't think it should hold up a vote.
>
> On Fri, Apr 29, 2022 at 7:40 AM Szalay-Bekő Máté
>  wrote:
> >
> > Hello Mohammad,
> >
> > Thanks for the RC! I'm still testing it (so no vote just yet), but I
> found
> > some CVE errors reported. The command "mvn clean package -DskipTests
> > dependency-check:check" failed with:
> >
> > [ERROR] One or more dependencies were identified with vulnerabilities
> that
> > have a CVSS score greater than or equal to '0.0':
> > [ERROR]
> > [ERROR] reload4j-1.2.19.jar: CVE-2020-9493, CVE-2022-23307
> > [ERROR]
> > [ERROR] See the dependency-check report for more details.
> >
> > I think this is a dependency-check plugin error and not an actual
> security
> > problem. At least I don't see Apache Chainsaw in our dependency tree, I
> > don't know why maven dependency-check reports this. Anyway, it would be
> > good if someone else can take a look too.
> >
> > Best regards,
> > Máté
> >
> > On Mon, Apr 25, 2022 at 3:25 AM Mohammad Arshad 
> wrote:
> >
> > > This is a bug fix release candidate for 3.7.1. It contains 61 fixes.
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350030
> > >
> > > *** Please download, test and vote by Sunday, 01 May, 2022, 23:59
> UTC+0.
> > > ***
> > >
> > > Source files:
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc0/
> > >
> > > Maven staging repo:
> > >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1075
> > >
> > > The release candidate tag in git to be voted upon: release-3.7.1-0
> > > https://github.com/apache/zookeeper/tree/release-3.7.1-0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc0/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > -Arshad
> > >
>

Re: [VOTE] Apache ZooKeeper release 3.7.1 candidate 0

[Szalay-Bekő Máté]

Hello Mohammad,

Thanks for the RC! I'm still testing it (so no vote just yet), but I found
some CVE errors reported. The command "mvn clean package -DskipTests
dependency-check:check" failed with:

[ERROR] One or more dependencies were identified with vulnerabilities that
have a CVSS score greater than or equal to '0.0':
[ERROR]
[ERROR] reload4j-1.2.19.jar: CVE-2020-9493, CVE-2022-23307
[ERROR]
[ERROR] See the dependency-check report for more details.

I think this is a dependency-check plugin error and not an actual security
problem. At least I don't see Apache Chainsaw in our dependency tree, I
don't know why maven dependency-check reports this. Anyway, it would be
good if someone else can take a look too.

Best regards,
Máté

On Mon, Apr 25, 2022 at 3:25 AM Mohammad Arshad  wrote:

> This is a bug fix release candidate for 3.7.1. It contains 61 fixes.
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350030
>
> *** Please download, test and vote by Sunday, 01 May, 2022, 23:59 UTC+0.
> ***
>
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1075
>
> The release candidate tag in git to be voted upon: release-3.7.1-0
> https://github.com/apache/zookeeper/tree/release-3.7.1-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.1-rc0/website/index.html
>
>
> Should we release this candidate?
>
>
> -Arshad
>

Re: [ANNOUNCE] new ZooKeeper PMC member: Mate Szalay-Beko

[Szalay-Bekő Máté]

Thank you all! :)

On Mon, Mar 28, 2022 at 3:38 PM Jordan Zimmerman 
wrote:

> Congrats!!!
>
> > On Mar 28, 2022, at 7:42 AM, Enrico Olivelli 
> wrote:
> >
> > I am happy to announce that Mate Szalay-Beko has been invited to join
> > the Apache ZooKeeper PMC and he accepted.
> >
> > Mate is doing great work for our community.
> >
> > Please join me in congratulating with him
> >
> > Congrats Mate !
> >
> >
> > If you want to know more about the ASF works and what is a PMC you can
> > read more here
> > https://www.apache.org/foundation/how-it-works.html#pmc
> >
> > Enrico
>
>

Re: [RESULT] [VOTE] Apache ZooKeeper release 3.8.0 candidate 1

[Szalay-Bekő Máté]

Thank you Enrico for managing this release! Nice job!! :)

Best regards,
Mate

On Thu, Mar 3, 2022 at 1:10 PM Enrico Olivelli  wrote:

> Hello,
> with 4 positive +1 votes (3 bindings):
> - Enrico Olivelli
> - Patrick Hunt
> - Andor Molnar
> - Szalay-Bekő Máté
>
> I am closing this VOTE as successful
>
> I will proceed with the next steps for the release
>
> Thanks to everyone who worked on this milestone !
>
> Enrico
>
>
> Il giorno gio 3 mar 2022 alle ore 12:02 Andor Molnar
>  ha scritto:
> >
> > +1 (binding)
> >
> > - checksum / signatures verified
> > - compiled the full build on Mac,
> > - unit test run on Java 11 - I had a few tests which was constantly
> failing on my Mac, but given that CI already passed all tests and others
> reported the same, I take it as passed,
> >
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.8.0/34/
> > - rat run clean
> > - spotbugs, owasp checks passed
> > - 3-node TLS quorum up and running with some basic manual tests
> >
> > Thanks,
> > Andor
> >
> >
> >
> >
> > > On 2022. Feb 28., at 22:02, Patrick Hunt  wrote:
> > >
> > > +1 - xsum/sig verified. Rat ran clean, compiled fine and I was able to
> run
> > > some manual clusters successfully.
> > >
> > > Regards,
> > >
> > > Patrick
> > >
> > > On Fri, Feb 25, 2022 at 2:32 AM Enrico Olivelli 
> wrote:
> > >
> > >> This is the second release candidate for 3.8.0.
> > >>
> > >> It is a major release and it introduces a lot of new features, most
> > >> notably:
> > >> - Migration of the logging framework from Apache Log4j1 to LogBack
> > >> - Read Key/trust store password from file (and other security related
> > >> improvements)
> > >> - Restored support for OSGI
> > >> - Reduced the performance impact of Prometheus metrics
> > >> - Official support for JDK17 (all tests are passing)
> > >> - Updates to all the third party dependencies to get rid of every
> known
> > >> CVE.
> > >>
> > >> The full release notes is available at:
> > >>
> > >>
> > >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349587
> > >>
> > >> *** Please download, test and vote by February 28th 2022, 23:59
> UTC+0. ***
> > >>
> > >> Source files:
> > >>
> > >>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/
> > >>
> > >> Maven staging repo:
> > >>
> https://repository.apache.org/content/repositories/orgapachezookeeper-1073/
> > >>
> > >> The release candidate tag in git to be voted upon: release-3.8.0-1
> > >> https://github.com/apache/zookeeper/tree/release-3.8.0-1
> > >>
> > >> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > >> https://www.apache.org/dist/zookeeper/KEYS
> > >>
> > >> The staging version of the website is:
> > >>
> > >>
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/website/index.html
> > >>
> > >>
> > >> Should we release this candidate?
> > >> Enrico Olivelli
> > >>
> >
>

Re: [VOTE] Apache ZooKeeper release 3.8.0 candidate 1

[Szalay-Bekő Máté]

+1 (non-binding)

- I built the source code (-Pfull-build) on Ubuntu 18.04.6 using OpenJDK
11.0.13 and maven 3.6.0.
- all the unit tests passed (both Java and C-client), although many (10+)
tests failed for the first time, we should improve our flaky test situation.
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.9 to 3.8.0
  - rolling upgrade from 3.6.3 to 3.8.0
  - rolling upgrade from 3.7.0 to 3.8.0

Best regards,
Máté

On Mon, Feb 28, 2022 at 10:10 AM Enrico Olivelli 
wrote:

> +1 (binding)
> - built and run unit tests on Ubuntu, with JDK11
> - run some tests using Patrick's project
> https://github.com/phunt/zkconf (it allows you to easily start a N
> servers cluster on a single machine)
>
> I am sending a patch for the commons-io LICENSE file
> (https://issues.apache.org/jira/browse/ZOOKEEPER-4482)
>
> Enrico
>
> Il giorno dom 27 feb 2022 alle ore 19:40 Enrico Olivelli
>  ha scritto:
> >
> >
> >
> > Il Dom 27 Feb 2022, 17:51 Patrick Hunt  ha scritto:
> >>
> >> The license file in the binary (which also comes from the src) doesn't
> >> match the jar version:
> >>
> >>   -rw-r--r--   1 phunt  staff   327135 Jan 16 23:54
> commons-io-2.11.0.jar
> >>   -rw-r--r--   1 phunt  staff11359 Feb 25 00:47
> >> commons-io-2.7.LICENSE.txt
> >>
> >> Is this something you want to fix, or not a release blocker, I can't
> >> remember how we treat this...
> >
> >
> > We can fix it, let's see how it goes with the other verifications.
> >
> > I won't consider this a blocker.
> >
> > Thanks
> > For reporting this.
> >
> > We should have a script that does this verification
> >
> > Enrico
> >
> >
> >
> >>
> >> Patrick
> >>
> >> On Fri, Feb 25, 2022 at 2:32 AM Enrico Olivelli 
> wrote:
> >>
> >> > This is the second release candidate for 3.8.0.
> >> >
> >> > It is a major release and it introduces a lot of new features, most
> >> > notably:
> >> > - Migration of the logging framework from Apache Log4j1 to LogBack
> >> > - Read Key/trust store password from file (and other security related
> >> > improvements)
> >> > - Restored support for OSGI
> >> > - Reduced the performance impact of Prometheus metrics
> >> > - Official support for JDK17 (all tests are passing)
> >> > - Updates to all the third party dependencies to get rid of every
> known
> >> > CVE.
> >> >
> >> > The full release notes is available at:
> >> >
> >> >
> >> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349587
> >> >
> >> > *** Please download, test and vote by February 28th 2022, 23:59
> UTC+0. ***
> >> >
> >> > Source files:
> >> >
> >> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/
> >> >
> >> > Maven staging repo:
> >> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1073/
> >> >
> >> > The release candidate tag in git to be voted upon: release-3.8.0-1
> >> > https://github.com/apache/zookeeper/tree/release-3.8.0-1
> >> >
> >> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> >> > https://www.apache.org/dist/zookeeper/KEYS
> >> >
> >> > The staging version of the website is:
> >> >
> >> >
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/website/index.html
> >> >
> >> >
> >> > Should we release this candidate?
> >> > Enrico Olivelli
> >> >
>

Re: Moving 3.5 to EOL

[Szalay-Bekő Máté]

Thanks for the clarification, I like the plan!

> having 2 active versions (stable and current) and when a new minor
version is announced, the least recent will get another 6 months of support

What does this mean exactly? Just to be on the same page, this is what you
propose if we release 3.8.0 until let's say end of February 2022?
- 3.5 EoL 1st of June 2022
- 3.6 EoL 1st of Sept 2022 (~6 months after 3.8.0 release)
- 3.7 will become "stable"
- 3.8 will become "current"

Did anyone in the community test the latest 3.7 (which is still 3.7.0) with
large clusters in production? Are we confident saying 3.7 is stable?
(on the other hand, if we don't do the announcement, most likely people
won't start to migrate to 3.7)

Mate

On Wed, Feb 16, 2022 at 1:33 PM Enrico Olivelli  wrote:

> Andor,
>
> Il Mer 16 Feb 2022, 12:47 Andor Molnar  ha scritto:
>
> > Okay, I agree that keeping 2 active versions rather than tying ourselves
> > to some fixed deadlines makes more sense for ZooKeeper. Let’s go with
> this
> > approach then if there’s no other objections:
> >
> > 1) Add this information to the Releases web page: I’ll describe that
> > ZooKeeper is having 2 active versions (stable and current) and when a new
> > minor version is announced, the least recent will get another 6 months of
> > support (security and bugfixes), but after that it will become EoL. That
> > means no further releases are expected from the community and users
> should
> > follow the supported upgrade path. I’ll send this out for review soon.
> >
>
> +1
>
>
> > 2) Announce 3.5 EoL 1st of June 2022. (sorry Enrico, the end of the long
> > discussion is essentially what you originally proposed)
> >
>
> +1
> Thanks
>
>
> Enrico
>
>
>
> > Please let me know if you have concerns with this path.
> >
> > Andor
> >
> >
> >
> > > On 2022. Feb 14., at 17:07, Patrick Hunt  wrote:
> > >
> > > "Define what EOL means" - whatever we do let's make sure it gets onto
> the
> > > "releases" page so that folks have official information they can
> > reference
> > > from the project.
> > >
> > > I like having a max of 2 versions. Stable and current. I agree that due
> > to
> > > our lack of communication/policy so far we should ensure that people
> have
> > > opportunity to move/support on the release versions (3.x minors) we
> > current
> > > support.
> > >
> > > I like the idea of tying old releases to new ones. I don't think tying
> > > ourselves to a specific, long term is good though. It definitely
> reduces
> > > flexibility. Same with saying that new minors are going to be released
> > > every Y time. Can't we just say that a stable release will be supported
> > for
> > > a minimum of 6 months (other timeframe?) after moving the stable
> > indicator
> > > from 3.x to 3.x+1. We then have the flexibility to keep it around
> longer
> > if
> > > there is a reason why folks want to stick for a longer time (eg major
> > > changes in the more recent versions)
> > >
> > > Patrick
> > >
> > > On Fri, Feb 11, 2022 at 8:08 AM Christopher 
> wrote:
> > >
> > >> Regarding the suggestion: "Maybe we can also communicate that we’re
> > going
> > >> to officially EoL the least recent ZK version every 2 years." If you
> > >> release new versions less frequently than that, the number of
> > maintenance
> > >> versions will go to 0 (though, in practice, you wouldn't EOL your
> > current
> > >> release). If you release more frequently, you'll be stuck maintaining
> an
> > >> increasing number of versions.
> > >>
> > >> To keep the maintenance burden relatively consistent, I suggest tying
> > your
> > >> EOL schedule to your release schedule, so when you release a new
> > version,
> > >> you drop the oldest one. If you release every 2 years, then it works
> out
> > >> the same. But if you release more or less often, your maintenance
> burden
> > >> stays consistent.
> > >>
> > >> I would start by deciding the minimum number of concurrent versions
> you
> > >> want to maintain. I suggest no more than 2, but ZK currently has 3,
> and
> > is
> > >> about to be 4 soon. If you're not marking specific versions as
> long-term
> > >> stable, then the default would be to assume you're maintaining the
> most
> > >> recent versions.
> > >>
> > >> Then, consider churn. If you release frequently, you may want to set a
> > >> minimum age for maintenance, so users aren't forced to upgrade too
> > often.
> > >> So, if you start with 2 concurrent versions and you have a few
> versions
> > >> released rapidly, you may temporarily need to support up to 3 or 4
> > releases
> > >> until the oldest ones reach the minimum age, like 2 years for example,
> > and
> > >> are able to be EOL'd.
> > >>
> > >> Then, consider upgrade overlap. When you release, you could EOL the
> > oldest
> > >> version right away. But, it might be nicer to wait a few months, or
> > maybe
> > >> up to a year, before the oldest one is EOL'd.
> > >>
> > >> I previously mentioned Accumulo's "LTM" strategy. These are the core
> 

Re: [VOTE] Apache ZooKeeper release 3.8.0 candidate 0

[Szalay-Bekő Máté]

Thanks Enrico for working on the release candidate!

The RC looks good to me if we are sure that the OWASP problem is a false
positive and we can skip this netty-tcnative jar check. However, these CVEs
are old... Is it possible that we just added this jar by accident with the
recent netty upgrade? If we don't need it, should we exclude it?

I wouldn't vote with +1 until we clarify the state of these CVEs.

My RC check:

- apache-rat passed
- I built the source code (-Pfull-build) on dockerized Ubuntu 18.04.6 using
OpenJDK 11.0.13 and maven 3.6.0.
- all the java unit tests passed eventually. I had 4-8 tests failing in
each run, but after 4 runs all tests passed at least once. (I used
-Dsurefire-forkcount=1) We should somehow fix these flakies. There are
flakies on the CI, but not this many. I executed in docker, maybe this is
the reason or the CI is using a different java version?
- checkstyle and spotbugs passed
- OWASP (CVE check) failed with the mentioned
netty-tcnative-2.0.48.Final.jar failures.
- I built the fatjar
- I executed C client tests. Two of these failed constantly for me:
Zookeeper_simpleSystem::testIPV6 and
Zookeeper_SASLAuth::testClientSASLOverIPv6. (I think these fail for me
because I execute C unit tests on docker, there might be some issues with
the IPv6 interface) I see these passed on CI running on the branch-3.8.0. (
https://github.com/apache/zookeeper/runs/5048875668?check_suite_focus=true)
- I also built and executed unit tests for zkpython
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
  - rolling upgrade from 3.5.9 to 3.8.0
  - rolling upgrade from 3.6.3 to 3.8.0
  - rolling upgrade from 3.7.0 to 3.8.0
- The web page looks OK

Best regards,
Máté

On Wed, Feb 9, 2022 at 8:04 PM Chris Nauroth  wrote:

> Enrico, thank you for putting together a release candidate.
>
> I briefly looked at the OWASP check failure. It's flagging multiple old
> CVEs against netty-tcnative-2.0.48.Final.jar. I can't imagine how these are
> still applicable. This is the newest version of the dependency, so we don't
> have another upgrade path we can try.
>
> I don't understand it. Unfortunately, I haven't found a solution yet.
>
> Chris Nauroth
>
>
> On Wed, Feb 9, 2022 at 2:05 AM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
>
> > I started to test it. apache-rat passed for me, but owasp first failed
> due
> > to some environment issue:
> >
> > [ERROR] Failed to execute goal
> org.owasp:dependency-check-maven:5.3.0:check
> > (default-cli) on project parent: Fatal exception(s) analyzing Apache
> > ZooKeeper: One or more exceptions occurred during analysis:
> > [ERROR] Unable to download meta file:
> > https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2004.meta
> > [ERROR] No documents exist
> > [ERROR] -> [Help 1]
> >
> > Now I just re-run and this error disappeared, I assume nvd.nist.gov was
> > down for a while.
> > Now the owasp is failing for me with this error:
> >
> > [ERROR] Failed to execute goal
> org.owasp:dependency-check-maven:5.3.0:check
> > (default-cli) on project zookeeper:
> > [ERROR]
> > [ERROR] One or more dependencies were identified with vulnerabilities
> that
> > have a CVSS score greater than or equal to '0.0':
> > [ERROR]
> > [ERROR] netty-tcnative-2.0.48.Final.jar: CVE-2021-43797, CVE-2019-16869,
> > CVE-2015-2156, CVE-2021-37136, CVE-2014-3488, CVE-2021-37137,
> > CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2021-21409,
> > CVE-2021-21290
> > [ERROR]
> > [ERROR] See the dependency-check report for more details.
> >
> >
> > I still continue to test the RC, let me know if it gets cancelled.
> >
> >
> > On Tue, Feb 8, 2022 at 9:52 PM Patrick Hunt  wrote:
> >
> > > On Tue, Feb 8, 2022 at 12:36 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > Any comments?
> > > >
> > >
> > > owasp is still red - as such I assumed this release candidate is on
> hold
> > > until that's fixed. Is that not the case?
> > >
> > > Patrick
> > >
> > >
> > > >
> > > > Il Ven 4 Feb 2022, 12:07 Enrico Olivelli  ha
> > > > scritto:
> > > >
> > > > > This is a release candidate for 3.8.0.
> > > > >
> > > > > It is a major release and it introduces a lot of new features, most
> > > > > notably:
> > > > > - Migration of the logging framework from Apache Log4j1 to LogBack
> > > > > - Read Key/trust store password from file (and other security
> related
> > > > > improvements)
>

Re: [VOTE] Apache ZooKeeper release 3.8.0 candidate 0

[Szalay-Bekő Máté]

I started to test it. apache-rat passed for me, but owasp first failed due
to some environment issue:

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.3.0:check
(default-cli) on project parent: Fatal exception(s) analyzing Apache
ZooKeeper: One or more exceptions occurred during analysis:
[ERROR] Unable to download meta file:
https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2004.meta
[ERROR] No documents exist
[ERROR] -> [Help 1]

Now I just re-run and this error disappeared, I assume nvd.nist.gov was
down for a while.
Now the owasp is failing for me with this error:

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.3.0:check
(default-cli) on project zookeeper:
[ERROR]
[ERROR] One or more dependencies were identified with vulnerabilities that
have a CVSS score greater than or equal to '0.0':
[ERROR]
[ERROR] netty-tcnative-2.0.48.Final.jar: CVE-2021-43797, CVE-2019-16869,
CVE-2015-2156, CVE-2021-37136, CVE-2014-3488, CVE-2021-37137,
CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2021-21409,
CVE-2021-21290
[ERROR]
[ERROR] See the dependency-check report for more details.


I still continue to test the RC, let me know if it gets cancelled.


On Tue, Feb 8, 2022 at 9:52 PM Patrick Hunt  wrote:

> On Tue, Feb 8, 2022 at 12:36 PM Enrico Olivelli 
> wrote:
>
> > Any comments?
> >
>
> owasp is still red - as such I assumed this release candidate is on hold
> until that's fixed. Is that not the case?
>
> Patrick
>
>
> >
> > Il Ven 4 Feb 2022, 12:07 Enrico Olivelli  ha
> > scritto:
> >
> > > This is a release candidate for 3.8.0.
> > >
> > > It is a major release and it introduces a lot of new features, most
> > > notably:
> > > - Migration of the logging framework from Apache Log4j1 to LogBack
> > > - Read Key/trust store password from file (and other security related
> > > improvements)
> > > - Restored support for OSGI
> > > - Reduced the performance impact of Prometheus metrics
> > > - Official support for JDK17 (all tests are passing)
> > > - Updates to all the third party dependencies to get rid of every known
> > > CVE.
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349587
> > >
> > > *** Please download, test and vote by February 7th 2022, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~eolivelli/zookeeper-3.8.0-candidate-0/
> > >
> > > Maven staging repo:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1072/
> > >
> > > The release candidate tag in git to be voted upon: release-3.8.0-0
> > > https://github.com/apache/zookeeper/tree/release-3.8.0-0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> >
> https://people.apache.org/~eolivelli/zookeeper-3.8.0-candidate-0/website/
> > >
> > >
> > > Should we release this candidate?
> > > Enrico Olivelli
> > >
> >
>

Re: ZooInspector updates

[Szalay-Bekő Máté]

Hello Brent,

Thanks for the contribution! this contrib project in ZooKeeper was not
really touched recently, it is great that you were able to refactor / test
/ improve it!
It will be a nice addition to 3.8.
(I left a small code style related comment on the PR. I don't think this is
a blocker, but if you can fix it quickly, then it would be great. Anyway, I
plan to merge your PR tomorrow.)

Best regards,
Mate

On Mon, Jan 31, 2022 at 6:30 PM Enrico Olivelli  wrote:

> Brent,
> I reviewed the patch.
> Great work
>
> I wish someone else could take a look at it so that we can release it
> together with 3.8
>
>
> Enrico
>
> Il giorno lun 31 gen 2022 alle ore 18:25 Brent
>  ha scritto:
> >
> > Hey all,
> >
> > I know that with all the release discussions and logging framework
> updates
> > things have been really busy.  It's been ~3 weeks or so, so I just wanted
> > to follow up since this is a pretty large pull request from me.  Is there
> > anything else I could supply that would make it easier to review (e.g.
> > design documentation, a video of demo usage, etc.)?  I appreciate
> > everyone's time is valuable and limited and I'd like to help make this PR
> > as painless as possible, so I don't mind doing more if it would help you
> > all.
> >
> > Thanks!
> >
> > ~Brent
> >
> > On Mon, Jan 10, 2022 at 4:14 PM Brent  wrote:
> >
> > > Sounds great!  Thank you Enrico and please reach out via email or on
> the
> > > PR if I can answer any questions.
> > >
> > > ~Brent
> > >
> > > On Mon, Jan 10, 2022 at 1:19 PM Enrico Olivelli 
> > > wrote:
> > >
> > >> Brent,
> > >> Thank you very much
> > >>
> > >> I will review your patches tomorrow.
> > >>
> > >> It is great to move forward with this tool
> > >>
> > >> Enrico
> > >>
> > >>
> > >> Il Lun 10 Gen 2022, 19:38 Brent  ha
> scritto:
> > >>
> > >> > Hi everyone,
> > >> >
> > >> > I just submitted a fairly big PR targeted at the ZooInspector
> contrib
> > >> > project: https://github.com/apache/zookeeper/pull/1796
> > >> >
> > >> > I'm not sure how many people use ZooInspector or how well
> maintained it
> > >> has
> > >> > been, but I have been getting some value out of it and had
> submitted a
> > >> > couple fixes previously:
> > >> >
> > >> > https://github.com/apache/zookeeper/pull/1551
> > >> > https://github.com/apache/zookeeper/pull/1601
> > >> >
> > >> > My biggest remaining issue has been performance.  When using it
> over a
> > >> > network to a remote Zookeeper cluster, especially with many ZNodes,
> it
> > >> has
> > >> > been almost unusably slow.  In my testing, the changes I just
> > >> > submitted seem to fix a lot of these issues.
> > >> >
> > >> > Since Exhibitor (https://github.com/soabase/exhibitor) hasn't
> received
> > >> an
> > >> > update since 2018, I was hoping to put some more effort back into
> making
> > >> > ZooInspector a bit more widely usable again.
> > >> >
> > >> > I wanted to send a note to this mailing list in case it warranted
> > >> further
> > >> > discussion since I know it's a substantial change and I haven't
> been a
> > >> very
> > >> > frequent contributor up to this point.
> > >> >
> > >> > Thank you!
> > >> >
> > >> > ~Brent
> > >> >
> > >>
> > >
>

Re: Switch to Reloadj4 on old but active branches 3.5, 3.6 and 3.7

[Szalay-Bekő Máté]

Thank you Enrico!

I like the reload4j idea for all our older, but still active releases. This
is as backward compatible as we can get with solving the security issues.
I think it would be good to do this also on branch-3.5, as this is an
important security fix and I don't think we ever communicated EOL for 3.5
yet. Actually I'm happy to coordinate a new release on 3.5 after 3.8.0 is
finished.

We should also clearly communicate on the webpage the default logging
framework versions we ship with our different releases.

I'm not sure, maybe the PMC members discussed this already, but it would be
also important to come up with some EOL strategy to keep our active
branches low and put it to our webpage.

Best regards,
Mate

On Sun, Jan 30, 2022 at 9:11 AM Enrico Olivelli  wrote:

> Hello folks,
> We are close to cutting some releases, at least I would like to do so,
> but we should remove log41 from our binary tarballs.
>
> For the next upcoming major release, 3.8.0, we decided to switch to
> Logback, good.
> But the older branches, 3.5, 3.6, 3.7 we should remove log41
>
> There is an initiative, https://reload4j.qos.ch/, that basically is a
> fork of log4j1 with the security fixes, so it is 100% compatible.
>
> I sent a PR for the branch-3.7, I will do the same at least for 3.6, I
> can do the same for 3.5 if anyone would like to cut a release some day
> (I am not sure)
>
> https://github.com/apache/zookeeper/pull/1802
>
> If there are any objections please anyone review my patch and we can
> merge it in a few days, just to give time to the community to see this
> message and express their thoughts.
>
> Best regards
> Enrico
>

Re: Cutting 3.8.0 release

[Szalay-Bekő Máté]

Great news, thanks for the work, Enrico!!

I think we should wait for https://github.com/apache/zookeeper/pull/1807 (
https://issues.apache.org/jira/browse/ZOOKEEPER-4461) so that we can
eliminate all references for log4j1 from our pom.xml files. What do
you think?

Regards,
Máté


On Fri, Jan 28, 2022 at 5:24 AM Chris Nauroth  wrote:

> +1
>
> Thanks for driving this, Enrico!
>
> Chris Nauroth
>
>
> On Thu, Jan 27, 2022 at 7:08 AM Enrico Olivelli 
> wrote:
>
> > Hello ZooKeepers,
> > I believe that the master branch is in good shape.
> >
> > I would like to start the release procedure for 3.8.0.
> >
> > This is the list of issues for 3.8.0
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.8.0
> >
> > We recently addressed all of the CVEs by updating some key
> > dependencies, like Netty, and moving away from Log4j1 (we switched to
> > LogBack)
> >
> > If no one has objections I will start the release procedure on Monday
> >
> > Regards
> >
> > Enrico
> >
>

Re: Logback

[Szalay-Bekő Máté]

I think we are slowly converging toward the following conclusion (at least
this is how I see it).

- We want to make either Log4j2 or Logback as the default log engine.
- We would provide some blogpost / documentation / how-to about how to
change the default log engine. (even as simple as described here in this
comment:
https://github.com/apache/zookeeper/pull/1793#pullrequestreview-857545860 ,
extended with some audit logging example)
- Currently we have a good patch for Logback thanks to Andor and to all the
reviewers.

As I would rather have something out sooner than later, for me the main
questions are:
(1) is logback good enough, or do we need log4j2?
(2) if we need log4j2, then is there anyone who could prepare a patch for
it soon?

What do you think?

Máté

On Thu, Jan 20, 2022 at 9:15 AM Andor Molnar  wrote:

> Thanks for the quick review Chris.
>
> I agree with the second part of your e-mail completely. I’m not sure
> either that the community has given a thumbs-up for logback, but I wanted
> to finalize my patch sooner, because I have other duties to take care of.
>
> I feel like logback is generally acceptable for ZK, but log4j2 would be
> more convenient, because most projects will eventually swap for it.
>
> Andor
>
>
>
> > On 2022. Jan 20., at 2:42, Chris Nauroth  wrote:
> >
> > Thank you, Andor. I entered one more round of very minor feedback.
> >
> > I'm not sure about the licensing changes. I responded on the PR with my
> > thoughts, but I'd appreciate a second set of eyes on the licensing in
> > particular.
> >
> > After resolving that feedback, I'll be ready to +1 from a code
> perspective,
> > but it sounds like the discussion of direction is not necessarily settled
> > here. Can others who have raised red flags please clarify the degree of
> > their objections? Is anyone actually -1 on a move to Logback? For my
> part,
> > even though I raised objections, I'm OK proceeding with Logback.  I'll
> > likely swap it for the Log4J 2 SLF4J back-end in my deployments. (I
> > specifically tested this on your branch and confirmed it works.)
> >
> > Chris Nauroth
> >
> >
> > On Wed, Jan 19, 2022 at 1:46 PM Andor Molnar  wrote:
> >
> >> I’m done with all the changes that I wanted to include in the first
> >> logback patch.
> >> Most of Chris’ feedback has also been addressed as well as the licensing
> >> changes.
> >> We have binary distribution which includes the logback jar, so I added
> EPL
> >> v1.0
> >> to LINCENSE.txt and mentioned Logback in the NOTICE.txt file. Hope all
> >> done correctly.
> >>
> >> Documentation has also been updated according to the new logging
> backend.
> >>
> >> Migration of zookeeper-recipes and zookeeper-contrib projects will come
> in
> >> the upcoming patch.
> >>
> >> Andor
> >>
> >>
> >>
> >>> On 2022. Jan 19., at 1:45, Ted Dunning  wrote:
> >>>
> >>> I believe that the primary contributor to logback was highly skeptical
> >> that
> >>> the recent problems could possible affect logback. That isn't a good
> >>> attitude for security problems.
> >>>
> >>> It isn't just a matter of patch rate. There is also the question of
> >>> community size. Is logback effectively a one-man show?
> >>>
> >>>
> >>>
> >>> On Tue, Jan 18, 2022 at 3:25 PM Christopher 
> wrote:
> >>>
>  While it has had recent activity, it is notable that logback only
> >> recently
>  became active again for patches to the stable 1.2 releases. After
> >> several
>  releases in early 2017, it did not have a stable release for over four
>  years between 31-Mar-2017 (v1.2.3) and  19-Jul-2021 (v1.2.4).
> 
>  On Tue, Jan 18, 2022 at 6:20 PM Christopher 
> >> wrote:
> 
> > Yes. It looks like logback is still actively being developed. 1.2
> had a
> > release in December. The 1.3 line is still alpha and has also seen
> >> recent
> > releases (interestingly, it requires at least Java 9 to build, but
> will
>  run
> > on Java 8, which is similar to what I had recommended for ZK in a
>  different
> > thread). 1.2 only requires Java 1.6 or later. Since it's still
> >> receiving
> > patches, and it's not alpha, that's probably the best version to use.
> > Currently, it seems to be at 1.2.9.
> >
> > On Tue, Jan 18, 2022 at 2:25 PM Andor Molnar 
> wrote:
> >
> >> I agree with you completely and this is crucial for logback too, so
> >> correct me if I'm wrong. Logback is current and actively maintained.
> >> Is
> >> that correct?
> >>
> >> Andor
> >>
> >>
> >> On Tue, 2022-01-18 at 12:43 -0500, Christopher wrote:
> >>> I do think these are more good reasons to adopt
> >>> something that is current and actively maintained, though, rather
> >>> than
> >>> something that is old and not active.
> >>
> >>
> >>
> 
> >>
> >>
>
>

Re: [ANNOUNCE] Apache ZooKeeper 3.6.3 Release

[Szalay-Bekő Máté]

This is great!! :)
Thank you Arshad for driving this release and thanks to the whole community
for the contribution!

On Tue, Apr 13, 2021 at 4:29 PM Mohammad Arshad  wrote:

> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.6.3
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.6.3 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.6.3/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 2

[Szalay-Bekő Máté]

+1 (non-binding)

Thank you Arshad, nice job!

My checks:
- I built the java source code and run all the unit tests on MacOS (using
OpenJDK 8u212 and maven 3.6.3)
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I built the the C and zkPython code (also run the zkPython and C-client
tests successfully) in docker using Ubuntu 18.04.3 with OpenJDK 11.0.10 and
maven 3.6.0.
- I executed quick rolling-upgrade tests (using
https://github.com/symat/zk-rolling-upgrade-test):
   - from 3.5.9 to the current RC (3.6.3)
   - from the current RC (3.6.3) to 3.7.0

Kind regards,
Mate

On Mon, Apr 12, 2021 at 1:00 PM Norbert Kalmar 
wrote:

> +1 (non-binding)
>
> Checked on macOS: licenses, build and test passed, ran ZK and few standard
> commands, signature.
> Also built and run unit tests on ubuntu.
>
> Thanks Arshad!
>
> - Norbert
>
> On Sat, Apr 10, 2021 at 10:34 AM Damien Diederen 
> wrote:
>
> >
> > Hi Arshad, all,
> >
> > LGTM!  +1 (advisory):
> >
> >   * Tarball contents match repository tag;
> >
> >   * Verified checksums and signatures;
> >
> >   * Built and smoke-tested on NixOS with a slightly adapted version of
> > the Nix recipe and test case;
> >
> >   * Smoke-tested a standalone server with the (corresponding) Java, C
> > and Perl clients, as well as the zkfuse contrib;
> >
> >   * Smoke-tested a 3-ensemble with the (corresponding) Java client and
> > SASL/GSSAPI.
> >
> > Sorry if I made you feel you had to go around with RC 0!  Oh well, at
> > least we got some additional CVE fixes bundled in.
> >
> > Best, -D
> >
> > P.-S. — As with my previous review, I have *not* tested under Ubuntu nor
> >   used the staging repo.  These are important points, but I figured most
> >   other testers would focus on them.
> >
> >
> >
> > Mohammad Arshad  writes:
> > > This is a bug fix release candidate for 3.6.3. It fixes 52 issues,
> > > including multiple CVE fixes.
> > >
> > > The full release notes is available at:
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348703
> > >
> > >  Please download, test and vote by Sunday, April 11th 2021, 23:59
> > > UTC+0. 
> > >
> > > Source and binary files:
> > > https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-2/
> > >
> > > Maven staging repo:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezookeeper-1071
> > >
> > > The release candidate tag in git to be voted upon: release-3.6.3-2
> > > https://github.com/apache/zookeeper/tree/release-3.6.3-2
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-2/website/
> > >
> > > Should we release this candidate?
> > >
> > > Thanks & Regards
> > > Arshad
> >
>

Re: [VOTE] Apache ZooKeeper release 3.6.3 candidate 1

[Szalay-Bekő Máté]

 -1 (non-binding)

Hello Mohammad!

Thanks for the great work! Sorry for torpedoing it :(

I voted with -1, as the CVE check failed for me on the release candidate:

mvn clean package -DskipTests dependency-check:check
(...)
[ERROR] Failed to execute goal org.owasp:dependency-check-maven:5.3.0:check
(default-cli) on project zookeeper:
[ERROR]
[ERROR] One or more dependencies were identified with vulnerabilities that
have a CVSS score greater than or equal to '0.0':
[ERROR]
[ERROR] jetty-server-9.4.38.v20210224.jar: CVE-2021-28165
[ERROR] jetty-http-9.4.38.v20210224.jar: CVE-2021-28165
[ERROR]
[ERROR] See the dependency-check report for more details.


It seems we have a relatively recent (about three weeks old) CVE error in
Jetty: https://nvd.nist.gov/vuln/detail/CVE-2021-28165
" In Eclipse Jetty 7.2.2 to 9.4.38, 10.0.0.alpha0 to 10.0.1, and
11.0.0.alpha0 to 11.0.1, CPU usage can reach 100% upon receiving a large
invalid TLS frame."

Looks like we will have to upgrade to jetty-server-9.4.39.

Kind regards,
Mate

On Tue, Apr 6, 2021 at 10:17 AM Mohammad arshad 
wrote:

> +1 (non-binding)
>
> -Verified signature and checksum of release artifacts. all ok
> -Run Junit test cases with jdk1.8.0_232 on Ubuntu 20.04, total 3137 test
> cases, 3 skipped, rest all passed
> -Done basic quality checks. run rat, checkstyle, spotbugs
> -Built tarball from source code, Verified it is same as the downloaded
> tarball
> -Installed 3 node cluster and verified basic functionalities from API,
> executed few cli commands. No issues observed
> -Connected HBase, HDFS and Yarn clusters (all using zk 3.5.6) to ZooKeeper
> 3.6.3 cluster, no issues observed.
>
> Though as a release manager my +1 vote is implicit, voting again to share
> few commands  I used to verify the release.
>
> Here are some of the commands I executed while verifying the release.
>
> Download all the required artifacts
> 
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.asc
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3-bin.tar.gz.sha512
>
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.asc
> wget
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/apache-zookeeper-3.6.3.tar.gz.sha512
>
> wget https://www.apache.org/dist/zookeeper/KEYS
>
> Verify Signature
> 
> gpg --import KEYS
> gpg --verify apache-zookeeper-3.6.3-bin.tar.gz.asc
> apache-zookeeper-3.6.3-bin.tar.gz
> gpg --verify apache-zookeeper-3.6.3.tar.gz.asc
> apache-zookeeper-3.6.3.tar.gz
> gpg --fingerprint 68E327C1
>
> Verify Checksum
> 
> sha512sum --check apache-zookeeper-3.6.3-bin.tar.gz.sha512
> sha512sum --check apache-zookeeper-3.6.3-bin.tar.gz.sha512
>
>
> Verify license header by executing Apache RAT
> 
> tar -xvf apache-zookeeper-3.6.3.tar.gz
> cd apache-zookeeper-3.6.3
> mvn clean apache-rat:check -DskipTests
>
> Perform quality checks, run checkstyle, spotbugs and unit tests
> 
> mvn clean install checkstyle:check spotbugs:check -DskipTests
> mvn clean test -Dsurefire.rerunFailingTestsCount=2
> -DtestFailureIgnore=true -Dmaven.test.failure.ignore=true
> -Dmaven.test.error.ignore=true
> NOTE: use -Pfull-build to include ci tests as well
>
> Build and Cluster Install
> 
> Built the tarball from source code and compare that it is same as the
> downloaded tarball. Apart from timestamp changes, no other changes are
> observed
> mvn clean install -DskipTests
> Installed the downloaded bin tarball and do some feature sanity tests
>
> Thanks & Regards
> Arshad
>
> -Original Message-
> From: Mohammad Arshad [mailto:ars...@apache.org]
> Sent: Sunday, April 4, 2021 4:48 PM
> To: dev@zookeeper.apache.org
> Subject: [VOTE] Apache ZooKeeper release 3.6.3 candidate 1
>
> This is a bug fix release candidate for 3.6.3. It contains 50 fixes.
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348703
>
> *** Please download, test and vote by Wednesday, April 7th 2021, 23:59
> UTC+0. ***
>
> Source and binary files:
> https://people.apache.org/~arshad/zookeeper-3.6.3-candidate-1/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1070
>
> The release candidate tag in git to be voted upon: release-3.6.3-1
> https://github.com/apache/zookeeper/tree/release-3.6.3-1
>
> ZooKeeper's KEYS file 

Re: [ANNOUNCE] Apache ZooKeeper 3.7.0 released

[Szalay-Bekő Máté]

Thanks for all your work Damien, and also for all the contributors and for
the whole community.
It's great to see 3.7.0 out! :)

Cheers,
Mate

On Sun, Mar 28, 2021 at 9:39 AM Damien Diederen 
wrote:

>
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.7.0.
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.7.0 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.7.0/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 2

[Szalay-Bekő Máté]

+1 (non-binding)

Congrats Damien, nice job!
And also the tests got definitely much more stable!
(I had a single test that failed on my dockerized
linux: RequestPathMetricsCollectorTest - but this one was successfully
executed on my mac host, so possible it is only related to my docker
environment)

- I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
11.0.10 and maven 3.6.0.
- all the unit tests passed (both Java and C-client).
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built
- I also executed some quick rolling-upgrade tests from 3.5.9 to 3.7.0 and
also from 3.6.2 to 3.7.0 (using
https://github.com/symat/zk-rolling-upgrade-test)

One minor website related issue: the tab on the top of the website page (
https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/index.html)
still says "ZooKeeper 3.6 Documentation"

Kind regards,
Mate

On Wed, Mar 17, 2021 at 12:06 PM Damien Diederen 
wrote:

>
> Greetings, all!
>
> After a long delay, here is a third release candidate for ZooKeeper 3.7.0.
>
> Compared to RC1, it contains... quite a few changes.  It notably fixes
> the quota feature for multi transactions, repairs the test suite on
> macOS (Catalina), makes a few tests less flaky, and avoids a CVE.
>
> The complete set of changes can be obtained with the Git range
> expression 'release-3.7.0-1..release-3.7.0-2', or on GitHub at:
>
>
> https://github.com/apache/zookeeper/compare/release-3.7.0-1...release-3.7.0-2
>
> I cannot say that I find the state of the test suite satisfactory, but
> the failures which are often observed are due to timing and/or TCP/IP
> port assignment issues, and repeated runs are "sufficient" to clear
> them.
>
> I was hoping to contribute more on that front, but have been unable so
> far, and don't want to keep the 3.7 branch hostage—so here is a timid
> RC2.
>
>
> ZooKeeper 3.7.0 introduces a number of new features, notably:
>
>   * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874);
>
>   * Quota enforcement (ZOOKEEPER-3301);
>
>   * Host name canonicalization in quorum SASL authentication
> (ZOOKEEPER-4030);
>
>   * Support for BCFKS key/trust store format (ZOOKEEPER-3950);
>
>   * A choice of mandatory authentication scheme(s) (ZOOKEEPER-3561);
>
>   * A "whoami" API and CLI command (ZOOKEEPER-3969);
>
>   * The possibility of disabling digest authentication (ZOOKEEPER-3979);
>
>   * Multiple SASL "superUsers" (ZOOKEEPER-3959);
>
>   * Fast-tracking of throttled requests (ZOOKEEPER-3683);
>
>   * Additional security metrics (ZOOKEEPER-3978);
>
>   * SASL support in the C and Perl clients (ZOOKEEPER-1112,
> ZOOKEEPER-3714);
>
>   * A new zkSnapshotComparer.sh tool (ZOOKEEPER-3427);
>
>   * Notes on how to benchmark ZooKeeper with the YCSB tool
> (ZOOKEEPER-3264).
>
>
> The release notes are available here:
>
>
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/releasenotes.html
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346617
>
> *** Please download, test and vote by March 21st 2021, 23:59 UTC+0. ***
>
> Source files:
>
>   https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/
>
> Maven staging repo:
>
>
> https://repository.apache.org/content/repositories/orgapachezookeeper-1067/
>
> The release candidate tag in git to be voted upon: release-3.7.0-2
>
>   https://github.com/apache/zookeeper/tree/release-3.7.0-2
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>
>   https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
>
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-2/website/
>
>
> Should we release this candidate?
>
>
> Damien Diederen
>

Re: quota stats data consistency

[Szalay-Bekő Máté]

this is a great news!
Thanks for raising the issue and also for digging into it!

Regards,
Mate

On Mon, Mar 1, 2021 at 10:00 PM Li Wang  wrote:

> Some updates on this.
>
> I looked more into this and found that quota usage is actually
> re-calculated when deserializing the data tree from the snapshot,  so the
> quota data will be consistent even in  the case zk server crashes.
>
> Best,
>
> Li
>
> On Tue, Feb 23, 2021 at 10:13 AM Li Wang  wrote:
>
> >
> > Hi,
> >
> > The Quota stats are updated by calling updateQuotaStat() API in the
> > FinalRequestProcessor. However, the updateQuotaStat() API only updates
> the
> > in-memory data tree.
> >
> > In the scenario that a ZK server crashed right before the updateQuotaStat
> > is called or a snapshot was taken, are we going to have the following
> data
> > inconsistency issue?
> >
> > 1. the quota stats is  inaccurate in the crashed server as the
> > write transaction has been logged by  the SyncRequestProcessor but the
> > quota stats update was lost?
> >
> > 2. the quota stats will be inconsistent across different servers as the
> > quota stats persisted in other servers via snapshot but not the crashed
> one
> >
> >
> > updateQuotaStat() code snippet
> > 
> > synchronized (statNode) {
> > updatedStat = new StatsTrack(statNode.data);
> > updatedStat.setCount(updatedStat.getCount() + countDiff);
> > updatedStat.setBytes(updatedStat.getBytes() + bytesDiff);
> >
> > statNode.data = updatedStat.getStatsBytes();
> > }
> >
> > Any thoughts?
> >
> > Thanks,
> >
> > Li
> >
>

Re: Re: [Commit Accident Case Study] Commit 4faf507 broke the build

[Szalay-Bekő Máté]

> For punishment:
> I will frozen/forbid my committership permission for three months

I think you took this too seriously. Mistakes / accidents happen when
someone is working (I did much more serious ones myself on different
projects). And the community is grateful for the contribution, no one
should expect perfection. At least I hope so, for my sake :p

Independently from this issue we really should focus on making our CI to be
rock-solid. So if the CI is red, then we could assume the PR broke
something. Currently I think flaky tests and independent CI issues are more
frequently causing red builds than actual failures introduced by PRs.

Cheers,
Mate

On Wed, Feb 10, 2021 at 10:26 AM Justin Ling Mao 
wrote:

> Haha, it scared me. Let me go through this accident.
> The root cause is: I'm over-confident, frivolous and hasty. I flatter
> myself that it's just a typo and committing it could not have anything bad
> happens. And I also don't give this PR a buffer time for other people's
> review.
> Accident is bad, but it's much more terrible if we can not reflect on it
> and think about how to avoid it next time.
> For remedy:
> I will add a new section: Commit Accident Case Study in [1] for the
> successor’s learning (Can anyone give me the permission to edit that wiki)?
> I will sum up our commit rules and the checklists before committing one
> patch, and do some works to use the Github CI and commit script to
> protect/check these constraint.
> For punishment:
> I will frozen/forbid my committership permission for three months(02-10 ~
> 05-10). During this period, I must not commit anything. I wish I could
> reflect on my fault and have a better understanding on the wording: "With
> great power comes great responsibility"
>
> Reference:[1]
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
> - Original Message -
> From: Andor Molnar 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Commit 4faf507 broke the build
> Date: 2021-02-10 00:26
>
> I’m sorry Justin. There’s no excuse for a mistake like this. We should not
> show mercy for anybody, otherwise it would erode the trust in our
> community. Your committership is now revoked.
> Just kidding. Don’t worry at all. ;-)
> I reverted the patch, so now please create a new PR with all the required
> changes included.
> Also I second Enrico’s comment: if CI is in bad shape, we should fix it.
> Regards,
> Andor
> > On 2021. Feb 9., at 13:45, Justin Ling Mao 
> wrote:
> >
> > Oops, it's my blame. I'm very sorry for my mistakes. Since these days
> the CI is in disorder and it's a typo, so I'm not waiting for CI check and
> forgot that an UT has covered this change although I wrote these related
> codes. It's all my mistake and I will summarize our submission process and
> this accident. I will write another letter to discuss the commit rules and
> how to improve our code review throughput
> >
> >
> > - Original Message -
> > From: Andor Molnar 
> > To: DevZooKeeper 
> > Subject: Commit 4faf507 broke the build
> > Date: 2021-02-09 19:43
> >
> > Hi,
> > I noticed that the latest commit 4faf507 ZOOKEEPER-4007: A typo in the
> ZKUtil#validateFileInput method broke the build, because the unit test has
> not been amended.
> > I reverted the commit to fix the build. Please create new PR with a
> proper patch.
> > Has the committer verified that the build is green before submitting it?
> > Andor
>

Re: Test failures on 3.7.0 RC1

[Szalay-Bekő Máté]

> It seems forkCount is overriden by the surefire-forkcount property.
> This seems to do what you expect:
>
>mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build
-Dsurefire-forkcount=1

indeed... this seems to be the case on all active branches. my mistake :)

Thanks!
Mate

On Thu, Jan 28, 2021 at 9:38 PM Damien Diederen 
wrote:

>
> Hi Máté, all,
>
> I managed to fight my way though some Apple ID insanity to get a Mac
> running with Catalina, Java, Maven and a C compiler.  This part of the
> puzzle was particularly fun:
>
> https://dd.crosstwine.com/tmp/apple-id.png
>
> Anyway.
>
> > So I prefer to execute the java unit tests using the following command
> when
> > checking RCs:
> >
> > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build
> -DforkCount=1
>
> This command does not work, at least on Catalina—where it forks eight
> runners:
>
> % pstree 7846
> -+= 07846 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09342 administrator /bin/sh -c cd /Users/...
>  | \--- 09343 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09640 administrator /bin/sh -c cd /Users/...
>  | \--- 09641 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09797 administrator /bin/sh -c cd /Users/...
>  | \--- 09798 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09893 administrator /bin/sh -c cd /Users/...
>  | \--- 09898 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09912 administrator /bin/sh -c cd /Users/...
>  | \--- 09913 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09942 administrator /bin/sh -c cd /Users/...
>  | \--- 09944 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  |-+- 09955 administrator /bin/sh -c cd /Users/...
>  | \--- 09956 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>  \-+- 09960 administrator /bin/sh -c cd /Users/...
>\--- 09961 administrator /Library/Java/JavaVir.../adoptopenjdk-8...
>
> It seems forkCount is overriden by the surefire-forkcount property.
> This seems to do what you expect:
>
> mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build
> -Dsurefire-forkcount=1
>
> But independently of that, I can reproduce the issue—even with a "real"
> sequential run.  It does not always result in the stack trace you saw,
> but often hangs with "funny" errors such as:
>
> [WARNING] Corrupted STDOUT by directly writing to native stream in
> forked JVM 1. See FAQ web page and the dump file
> /Users/administrator/zookeeper/zookeeper-server/target/surefire-reports/2021-01-28T17-56-26_094-jvmRun1.dumpstream
>
> # Created at 2021-01-28T18:37:51.381
> Corrupted STDOUT by directly writing to native stream in forked JVM 1.
> Stream '[59.370s][warning][os,thread] Failed to start thread -
> pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize:
> 4k, detached.'.
>
> I'm planning to have a closer look soon.
>
> Cu, -D
>
> P.-S. — Enrico: I am also able to reproduce the C compilation issues you
> reported.
>
>
> --8<---original message->8---
>
> Szalay-Bekő Máté  writes:
> >  this is not specific to 3.7, but in general:
> > from time to time I see different tests failing intermittently. I assume
> > this can be caused by other tests running in parallel. (because
> re-running
> > those tests alone usually makes them succeed)
> > So I prefer to execute the java unit tests using the following command
> when
> > checking RCs:
> >
> > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build
> -DforkCount=1
> >
> > As far as I remember, by default the fork count is set to 8 in maven, and
> > the CI is using (or was using before the migration) -DforkCount=4.
> >
> >
> > Regards,
> > Mate
> >
> > On Mon, Jan 25, 2021 at 1:51 PM Flavio Junqueira  wrote:
> >
> >> I don't want to mess up with the vote thread, so I'm responding to this
> >> separately. I have been trying to build locally too unsuccessfully. I've
> >> been trying on an Ubuntu VM, Java 8 (build 1.8.0_181-b13), Maven 3.6.0.
> The
> >> set of tests failing varies from build to build, if it makes sense, I
> can
> >> try to collect all test failures I have seen and post.
> >>
> >> -Flavio
> >>
> >> > On 25 Jan 2021, at 13:38, Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> >> wrote:
> >> >
> >> > +0 (and not even binding :) )
> >> >
> >> >

Re: Test failures on 3.7.0 RC1 (was: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1)

[Szalay-Bekő Máté]

 this is not specific to 3.7, but in general:
from time to time I see different tests failing intermittently. I assume
this can be caused by other tests running in parallel. (because re-running
those tests alone usually makes them succeed)
So I prefer to execute the java unit tests using the following command when
checking RCs:

mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build -DforkCount=1

As far as I remember, by default the fork count is set to 8 in maven, and
the CI is using (or was using before the migration) -DforkCount=4.


Regards,
Mate

On Mon, Jan 25, 2021 at 1:51 PM Flavio Junqueira  wrote:

> I don't want to mess up with the vote thread, so I'm responding to this
> separately. I have been trying to build locally too unsuccessfully. I've
> been trying on an Ubuntu VM, Java 8 (build 1.8.0_181-b13), Maven 3.6.0. The
> set of tests failing varies from build to build, if it makes sense, I can
> try to collect all test failures I have seen and post.
>
> -Flavio
>
> > On 25 Jan 2021, at 13:38, Szalay-Bekő Máté 
> wrote:
> >
> > +0 (and not even binding :) )
> >
> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
> > 8u265 and maven 3.6.3.
> > - I also built and executed unit tests for zkpython
> > - the unit tests passed for the C-client and for python client
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built (-Pfatjar)
> > - I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2.
> (using
> > https://github.com/symat/zk-rolling-upgrade-test)
> >
> > for some reason the java unit tests failed for me.
> >
> > On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests executed
> > successfully, but then the maven job still failed for hbase-server test
> > with error message (with -DforkCount=4 and even with -DforkCount=1) like:
> > -
> > [ERROR] ExecutionException There was an error in the forked process
> > [ERROR] unable to create new native thread
> > [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
> > ExecutionException There was an error in the forked process
> > [ERROR] unable to create new native thread
> > [ERROR] at
> >
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
> > -
> >
> >
> > Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 and
> > maven 3.6.3) and I got other kinds of strange maven errors:
> > ---
> > [ERROR] Caused by:
> > org.apache.maven.surefire.booter.SurefireBooterForkException: The forked
> VM
> > terminated without properly saying goodbye. VM crash or System.exit
> called?
> > [ERROR] Command was /bin/sh -c cd
> > /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server &&
> > /home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xmx512m
> > -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=3
> >
> -javaagent:/home/symat/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
> > -jar
> >
> /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire/surefirebooter8828313385463488429.jar
> > /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire
> > 2021-01-25T11-54-03_621-jvmRun3 surefire4024538135165099286tmp
> > surefire_37800399112966511000tmp
> > [ERROR] Process Exit Code: 0
> > [ERROR] at
> >
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
> > [ERROR] at
> >
> org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115)
> > [ERROR] at
> >
> org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444)
> > [ERROR] at
> >
> org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420)
> > [ERROR] at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > [ERROR] at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [ERROR] at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [ERROR] at java.lang.Thread.run(Thread.java:748)
> > [ERROR]
> > --
> >
> >
> > These issues might be very well specific to my local (mac or docker on
> mac)
> > environments. This is why I didn't vote with -1
> > Can someone else run the java unit tests successfully locally?
> >
> > I also tried to check if the CI was green for the last PR on 3.7.0 (

Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 1

[Szalay-Bekő Máté]

+0 (and not even binding :) )

- I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
8u265 and maven 3.6.3.
- I also built and executed unit tests for zkpython
- the unit tests passed for the C-client and for python client
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built (-Pfatjar)
- I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2. (using
https://github.com/symat/zk-rolling-upgrade-test)

for some reason the java unit tests failed for me.

On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests executed
successfully, but then the maven job still failed for hbase-server test
with error message (with -DforkCount=4 and even with -DforkCount=1) like:
-
[ERROR] ExecutionException There was an error in the forked process
[ERROR] unable to create new native thread
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
ExecutionException There was an error in the forked process
[ERROR] unable to create new native thread
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
-


Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 and
maven 3.6.3) and I got other kinds of strange maven errors:
---
[ERROR] Caused by:
org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM
terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd
/tmp/zk/apache-zookeeper-3.7.0/zookeeper-server &&
/home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xmx512m
-Dtest.junit.threads=8 -Dzookeeper.junit.threadid=3
-javaagent:/home/symat/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar
-jar
/tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire/surefirebooter8828313385463488429.jar
/tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire
2021-01-25T11-54-03_621-jvmRun3 surefire4024538135165099286tmp
surefire_37800399112966511000tmp
[ERROR] Process Exit Code: 0
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ERROR] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ERROR] at java.lang.Thread.run(Thread.java:748)
[ERROR]
--


These issues might be very well specific to my local (mac or docker on mac)
environments. This is why I didn't vote with -1
Can someone else run the java unit tests successfully locally?

I also tried to check if the CI was green for the last PR on 3.7.0 (
https://github.com/apache/zookeeper/pull/1586/checks), but it looks the CI
haven't even started to execute the tests, due to errors in the "install C
dependencies" step.

Regards,
Mate

On Sun, Jan 24, 2021 at 11:39 PM Patrick Hunt  wrote:

> +1. xsum/sig verified. rat ran clean. built and dependency checks are fine.
> Tried running some manual clusters and it was successful.
>
> Regards,
>
> Patrick
>
>
> On Sun, Jan 24, 2021 at 12:11 PM Damien Diederen 
> wrote:
>
> >
> > Dear all,
> >
> > This is a second release candidate for ZooKeeper 3.7.0.  Compared to
> > RC0, it fixes a tarball generation issue, includes a description of the
> > 'whoami' CLI command, and incorporates a contribution to ZooInspector.
> >
> > ZooKeeper 3.7.0 introduces a number of new features, notably:
> >
> >   * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874);
> >
> >   * Quota enforcement (ZOOKEEPER-3301);
> >
> >   * Host name canonicalization in quorum SASL authentication
> > (ZOOKEEPER-4030);
> >
> >   * Support for BCFKS key/trust store format (ZOOKEEPER-3950);
> >
> >   * A choice of mandatory authentication scheme(s) (ZOOKEEPER-3561);
> >
> >   * A "whoami" API and CLI command (ZOOKEEPER-3969);
> >
> >   * The possibility of disabling digest authentication (ZOOKEEPER-3979);
> >
> >   * Multiple SASL "superUsers" (ZOOKEEPER-3959);
> >
> >   * Fast-tracking of throttled requests (ZOOKEEPER-3683);
> >
> >   * Additional security metrics (ZOOKEEPER-3978);
> >
> >   * SASL support in the C and Perl clients (ZOOKEEPER-1112,
> > ZOOKEEPER-3714);
> >
> >   * A new zkSnapshotComparer.sh tool (ZOOKEEPER-3427);
> >
> >   * Notes on how to benchmark ZooKeeper with the YCSB tool
> > (ZOOKEEPER-3264).
> >
> > The release notes are available here:
> >
> >
> >
> 

Re: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 0

[Szalay-Bekő Máté]

Thanks for the quick feedback!

>  If you build from sources and you pick the resulting binaries tarball and
> unpack it, does it work?

I just did this, and the same files in the resulting
zookeeper-assembly/target/apache-zookeeper-3.7.0-bin.tar.gz does have the
execution flag set correctly.

> Running bin/ex.sh from the sources is probably only a developer feature
and
> it does not impact production usage.

true enough I guess. At least as far as I can remember in our company, we
re-build ZooKeeper from source in our CI each time and we took the sh (and
jar) files from the ...-bin.tar.gz files indeed.

> Is the problem present even on 3.6.2?

I don't know, maybe this is the same for a long time? Although this is a
new minor version, so the 3.6.x behaviour shouldn't necessarily matter.

Anyway, I'm absolutely OK if we agree that it is not a blocker for the
release candidate. This really seems to be a minor issue.
We can still submit a jira to set these permission flags in the source
tar.gz file too (at least for the convenience of the developers)

Regards
Mate



On Sat, Jan 23, 2021 at 6:03 PM Enrico Olivelli  wrote:

> Mate,
> Thanks foe pointing it out
> If you build from sources and you pick the rwsulting binaries tarball and
> unpack it, does it work?
> Running bin/ex.sh from the sources is probably only a developer feature and
> it does not impact production usage.
>
> Is the problem present even on 3.6.2?
>
> Enrico
>
> Il Sab 23 Gen 2021, 17:13 Szalay-Bekő Máté  ha
> scritto:
>
> > I just ran a few small manual upgrade tests, 3.5.9 -> 3.7.0 and 3.6.2 ->
> > 3.7.0) using https://github.com/symat/zk-rolling-upgrade-test
> > Both tests succeeded (which is great! :) ), however I had to do a manual
> > fix: my test scripts were unable to start the 3.7.0 servers,
> > because zkServer.sh didn't have execute permissions.
> >
> > It looks this is only about the source tar.gz release file (which is our
> > main deliverable). When I downloaded the binary tar.gz file, or checked
> out
> > the release tag, then the files had the execution tag. See below.
> >
> > I don't know if this is officially a blocker or not, PMC members please
> > advise here.
> > (my non-binding opinion is that this is a blocker unfortunately...)
> >
> > Regards,
> > Mate
> >
> > More info:
> >
> > $  git clone --depth 1 --branch release-3.7.0-0 g...@github.com:
> > apache/zookeeper.git
> > $  ls -la zookeeper/bin/*.sh
> > -rwxr-xr-x 1 symat symat  2066 Jan 23 16:58 zookeeper/bin/zkCleanup.sh
> > -rwxr-xr-x 1 symat symat  1620 Jan 23 16:58 zookeeper/bin/zkCli.sh
> > -rwxr-xr-x 1 symat symat  3690 Jan 23 16:58 zookeeper/bin/zkEnv.sh
> > -rwxr-xr-x 1 symat symat  4559 Jan 23 16:58
> > zookeeper/bin/zkServer-initialize.sh
> > -rwxr-xr-x 1 symat symat 11561 Jan 23 16:58 zookeeper/bin/zkServer.sh
> > -rwxr-xr-x 1 symat symat  1377 Jan 23 16:58
> > zookeeper/bin/zkSnapShotToolkit.sh
> > -rwxr-xr-x 1 symat symat  1374 Jan 23 16:58
> > zookeeper/bin/zkSnapshotComparer.sh
> > -rwxr-xr-x 1 symat symat  1385 Jan 23 16:58
> > zookeeper/bin/zkTxnLogToolkit.sh
> >
> > $ wget
> >
> >
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/apache-zookeeper-3.7.0-bin.tar.gz
> > $ tar xzvf  apache-zookeeper-3.7.0-bin.tar.gz
> > $ ls -la apache-zookeeper-3.7.0-bin/bin/*.sh
> > -rwxr-xr-x 1 symat symat  2066 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkCleanup.sh
> > -rwxr-xr-x 1 symat symat  1620 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkCli.sh
> > -rwxr-xr-x 1 symat symat  3690 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkEnv.sh
> > -rwxr-xr-x 1 symat symat  4559 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkServer-initialize.sh
> > -rwxr-xr-x 1 symat symat 11561 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkServer.sh
> > -rwxr-xr-x 1 symat symat  1377 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkSnapShotToolkit.sh
> > -rwxr-xr-x 1 symat symat  1374 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkSnapshotComparer.sh
> > -rwxr-xr-x 1 symat symat  1385 Jan 22  2020
> > apache-zookeeper-3.7.0-bin/bin/zkTxnLogToolkit.sh
> >
> > $ wget
> >
> >
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/apache-zookeeper-3.7.0.tar.gz
> > $ tar xzvf  apache-zookeeper-3.7.0.tar.gz
> > $ ls -la apache-zookeeper-3.7.0/bin/*.sh
> > -rw-r--r-- 1 symat symat  2066 Jan 22  2020
> > apache-zookeeper-3.7.0/bin/zkCleanup.sh
> > -rw-r--r-- 1 symat symat  1620 Jan 22  2020
> > apache-zookeeper-3.7.0/bin/zkCli.sh
> > -rw-r--r-- 1 symat symat  3690 Jan 22  202

Re: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 0

[Szalay-Bekő Máté]

I just ran a few small manual upgrade tests, 3.5.9 -> 3.7.0 and 3.6.2 ->
3.7.0) using https://github.com/symat/zk-rolling-upgrade-test
Both tests succeeded (which is great! :) ), however I had to do a manual
fix: my test scripts were unable to start the 3.7.0 servers,
because zkServer.sh didn't have execute permissions.

It looks this is only about the source tar.gz release file (which is our
main deliverable). When I downloaded the binary tar.gz file, or checked out
the release tag, then the files had the execution tag. See below.

I don't know if this is officially a blocker or not, PMC members please
advise here.
(my non-binding opinion is that this is a blocker unfortunately...)

Regards,
Mate

More info:

$  git clone --depth 1 --branch release-3.7.0-0 g...@github.com:
apache/zookeeper.git
$  ls -la zookeeper/bin/*.sh
-rwxr-xr-x 1 symat symat  2066 Jan 23 16:58 zookeeper/bin/zkCleanup.sh
-rwxr-xr-x 1 symat symat  1620 Jan 23 16:58 zookeeper/bin/zkCli.sh
-rwxr-xr-x 1 symat symat  3690 Jan 23 16:58 zookeeper/bin/zkEnv.sh
-rwxr-xr-x 1 symat symat  4559 Jan 23 16:58
zookeeper/bin/zkServer-initialize.sh
-rwxr-xr-x 1 symat symat 11561 Jan 23 16:58 zookeeper/bin/zkServer.sh
-rwxr-xr-x 1 symat symat  1377 Jan 23 16:58
zookeeper/bin/zkSnapShotToolkit.sh
-rwxr-xr-x 1 symat symat  1374 Jan 23 16:58
zookeeper/bin/zkSnapshotComparer.sh
-rwxr-xr-x 1 symat symat  1385 Jan 23 16:58 zookeeper/bin/zkTxnLogToolkit.sh

$ wget
https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/apache-zookeeper-3.7.0-bin.tar.gz
$ tar xzvf  apache-zookeeper-3.7.0-bin.tar.gz
$ ls -la apache-zookeeper-3.7.0-bin/bin/*.sh
-rwxr-xr-x 1 symat symat  2066 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkCleanup.sh
-rwxr-xr-x 1 symat symat  1620 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkCli.sh
-rwxr-xr-x 1 symat symat  3690 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkEnv.sh
-rwxr-xr-x 1 symat symat  4559 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkServer-initialize.sh
-rwxr-xr-x 1 symat symat 11561 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkServer.sh
-rwxr-xr-x 1 symat symat  1377 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkSnapShotToolkit.sh
-rwxr-xr-x 1 symat symat  1374 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkSnapshotComparer.sh
-rwxr-xr-x 1 symat symat  1385 Jan 22  2020
apache-zookeeper-3.7.0-bin/bin/zkTxnLogToolkit.sh

$ wget
https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/apache-zookeeper-3.7.0.tar.gz
$ tar xzvf  apache-zookeeper-3.7.0.tar.gz
$ ls -la apache-zookeeper-3.7.0/bin/*.sh
-rw-r--r-- 1 symat symat  2066 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkCleanup.sh
-rw-r--r-- 1 symat symat  1620 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkCli.sh
-rw-r--r-- 1 symat symat  3690 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkEnv.sh
-rw-r--r-- 1 symat symat  4559 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkServer-initialize.sh
-rw-r--r-- 1 symat symat 11561 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkServer.sh
-rw-r--r-- 1 symat symat  1377 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkSnapShotToolkit.sh
-rw-r--r-- 1 symat symat  1374 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkSnapshotComparer.sh
-rw-r--r-- 1 symat symat  1385 Jan 22  2020
apache-zookeeper-3.7.0/bin/zkTxnLogToolkit.sh


On Sat, Jan 23, 2021 at 4:32 AM Patrick Hunt  wrote:

> On Fri, Jan 22, 2021 at 6:55 PM Justin Ling Mao  >
> wrote:
>
> > I create the ticket: ZOOKEEPER-4188: add a doc about whoami CLI for me to
> > do. It's not a blocker. Let's go ahead:)
> >
> >
> Thanks Justin. Note that a release can't be vetoed (also I did give
> a +1) and as the RM Damien should make the final decision on the
> seriousness of any issues found. Perhaps a good opportunity to review the
> apache release voting guidelines:
> https://www.apache.org/foundation/voting.html#ReleaseVotes
> http://www.apache.org/legal/release-policy.html#release-approval
>
> Regards,
>
> Patrick
>
>
> > - Original Message -
> > From: Patrick Hunt 
> > To: DevZooKeeper 
> > Subject: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 0
> > Date: 2021-01-23 05:09
> >
> > +1 - xsum/sig validated. Compiles/runs fine on macos+jdk11. Verified some
> > larger ensemble sizes manually and it worked ok.
> > I looked at a few of the new features listed - they look great! I did
> > notice some changes without documentation though (whoami eg), would be
> good
> > for committers to ensure that docs get updated along the way...
> > Thanks Damien for acting as RM. Regards,
> > Patrick
> > On Tue, Jan 19, 2021 at 4:40 AM Damien Diederen 
> > wrote:
> > >
> > > Dear all,
> > >
> > > This is a first release candidate for ZooKeeper 3.7.0.
> > >
> > > It introduces a number of new features, notably:
> > >
> > >   * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874);
> > >
> > >   * Quota enforcement (ZOOKEEPER-3301);
> > >
> > >   * Host name canonicalization in quorum SASL authentication
> > > (ZOOKEEPER-4030);
> > >
> > >   * Support for BCFKS key/trust store format (ZOOKEEPER-3950);

Re: New committer: Justin Mao Ling

[Szalay-Bekő Máté]

Congratulations Maoling!!! :))

Regards,
Mate

On Mon, Jan 18, 2021 at 11:48 AM Norbert Kalmar
 wrote:

> Congratulations Maoling! Well-deserved!
>
> - Norbert
>
> On Mon, Jan 18, 2021 at 11:43 AM Andor Molnar  wrote:
>
> > Congrats Maoling!
> >
> >
> >
> >
> > > On 2021. Jan 18., at 11:09, Enrico Olivelli 
> wrote:
> > >
> > > The Project Management Committee (PMC) for Apache ZooKeeper
> > >
> > > has invited Justin Mao Long to become a committer and we are pleased
> > >
> > > to announce that he has accepted.
> > >
> > >
> > > Justin has been following the Project for a long time,
> > >
> > > He is very active in the community with discussions and code reviews
> > > and he contributed
> > > many patches.
> > >
> > >
> > > Being a committer enables easier contribution to the
> > >
> > > project since there is no need to go via the patch
> > >
> > > submission process. This should enable better productivity.
> > >
> > > Being a PMC member enables assistance with the management
> > >
> > > and to guide the direction of the project.
> > >
> > >
> > >
> > > Congratulations Justin !
> > >
> > >
> > > Enrico
> >
> >
>

Re: [ANNOUNCE] Apache ZooKeeper 3.5.9

[Szalay-Bekő Máté]

Thank you Norbert for driving this! :)

Regards,
Mate

On Fri, Jan 15, 2021 at 4:04 PM Norbert Kalmar  wrote:

> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.5.9
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads,
> visit:https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.5.9 Release Notes are
> at:https://zookeeper.apache.org/doc/r3.5.9/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
> The ZooKeeper Team
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Szalay-Bekő Máté]

> [exec]
>
/Users/enrico.olivelli/Downloads/zk359/apache-zookeeper-3.5.9/zookeeper-client/zookeeper-client-c/tests/TestClient.cc:789:
> Assertion: equality assertion failed [Expected: 0, Actual  : -4]

This test failed for me too with the exact same error message on docker
many times before (when I was checking RCs for other releases). I usually
ignore this one, as it does run successfully on my real ubuntu. It would be
good to check why it fails though on docker... Maybe it's worth to create a
Jira ticket.

For this RC I compiled and executed the C-client tests on a real native
ubuntu server (18.04) and all the C client tests passed for me now.

Best regards,
Mate

On Thu, Jan 7, 2021 at 12:56 PM Enrico Olivelli  wrote:

> I am validating the release
> I have this test that is consistently failing on the c-client
>
>  [exec]
>
> /Users/enrico.olivelli/Downloads/zk359/apache-zookeeper-3.5.9/zookeeper-client/zookeeper-client-c/tests/TestClient.cc:789:
> Assertion: equality assertion failed [Expected: 0, Actual  : -4]
>
> Do you think it is a blocker for the release ?
> I am on MacOs + docker env (dev/docker/run.sh)
>
> Enrico
>
> Il giorno gio 7 gen 2021 alle ore 10:46 Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com> ha scritto:
>
> > +1 (non-binding)
> >
> > - I built the source code (-Pfull-build) on Ubuntu 18.04 using OpenJDK
> > 8u265 and maven 3.6.3.
> > - all the unit tests passed eventually (both Java and C-client).
> > - I also built zkpython
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - I executed a quick rolling-upgrade test from 3.5.9 to 3.6.2. (using
> > https://github.com/symat/zk-rolling-upgrade-test)
> >
> > The only thing I found was 4 unit tests, failed first (when I executed
> all
> > tests in docker) but succeeded second time running them on my mac:
> > - QuorumPeerMainTest  ->  testLeaderOutOfView
> > - ReconfigExceptionTest  ->  testReconfigDisabled
> > - NIOServerCnxnFactoryTest  ->
> testStartupWithoutStart_SocketAlreadyBound
> > - NIOServerCnxnFactoryTest  ->  testStartupWithStart_SocketAlreadyBound
> >
> > Thanks,
> > Mate
> >
> > On Wed, Jan 6, 2021 at 9:10 PM Norbert Kalmar 
> wrote:
> >
> > > This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
> > > including CVE fixes.
> > > (Note: rc1 had a third party CVE which was only noticed during the last
> > > check of the release, so it never made it for vote)
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> > >
> > > *** Please download, test and vote by January 11th 2020, 23:59 UTC+0.
> ***
> > >
> > > Source files:
> > > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/
> > >
> > > Maven staging repo:
> > >
> > >
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> > >
> > > The release candidate tag in git to be voted upon: release-3.5.9-rc2
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > Should we release this candidate?
> > >
> > > - Norbert
> > >
> >
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 2

[Szalay-Bekő Máté]

+1 (non-binding)

- I built the source code (-Pfull-build) on Ubuntu 18.04 using OpenJDK
8u265 and maven 3.6.3.
- all the unit tests passed eventually (both Java and C-client).
- I also built zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- I executed a quick rolling-upgrade test from 3.5.9 to 3.6.2. (using
https://github.com/symat/zk-rolling-upgrade-test)

The only thing I found was 4 unit tests, failed first (when I executed all
tests in docker) but succeeded second time running them on my mac:
- QuorumPeerMainTest  ->  testLeaderOutOfView
- ReconfigExceptionTest  ->  testReconfigDisabled
- NIOServerCnxnFactoryTest  ->  testStartupWithoutStart_SocketAlreadyBound
- NIOServerCnxnFactoryTest  ->  testStartupWithStart_SocketAlreadyBound

Thanks,
Mate

On Wed, Jan 6, 2021 at 9:10 PM Norbert Kalmar  wrote:

> This is a bugfix release candidate for 3.5.9. It contains 25 fixes,
> including CVE fixes.
> (Note: rc1 had a third party CVE which was only noticed during the last
> check of the release, so it never made it for vote)
>
> The full release notes is available at:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
>
> *** Please download, test and vote by January 11th 2020, 23:59 UTC+0. ***
>
> Source files:
> https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-2/
>
> Maven staging repo:
>
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
>
> The release candidate tag in git to be voted upon: release-3.5.9-rc2
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> Should we release this candidate?
>
> - Norbert
>

Re: 3.7.0: Last call for tickets (was: Time to Cut 3.7.0 ?)

[Szalay-Bekő Máté]

Thank you Damien!! :)

One thing came to my mind: there was the larger story of the junit upgrade.
I'm not sure how we are with that. Is it finished? (@Tamas can you
summarize the status?)
I see the parent story is still in progress:
https://issues.apache.org/jira/browse/ZOOKEEPER-3732 (although the most of
the work is done already)

Kind regards,
Mate

On Wed, Jan 6, 2021 at 3:06 PM Andor Molnar  wrote:

> Hi Damien,
>
> Thanks for volunteering as release manager. This is gonna be good fun.
>
> I’m not following the tickets closely, but I remember the discussion about
> deco’ing Java 8 in branch-3.7 and unfortunately we didn’t reached an
> agreement.
>
> Andor
>
>
>
>
> > On 2021. Jan 6., at 13:19, Damien Diederen 
> wrote:
> >
> >
> > Greetings, all,
> >
> >
> > Unless someone objects, this is a last call for tickets before I create
> > branches 'branch-3.7' and 'branch-3.7.0' in preparation for a new
> > release, aiming for a first candidate in the coming days.
> >
> > Besides the tickets mentioned below, are there other patches you would
> > like to see included in 3.7.0?  If not, I will soon start pushing the
> > open tickets to 3.8.0 in Jira.
> >
> > (Of course, my ticket-tagging, Git-branching, or even publishing a first
> > release candidate does not preclude "cherry-picking" further patches.)
> >
> >
> > I already went through (and somewhat cleaned up) these two views:
> >
> >  * https://issues.apache.org/jira/projects/ZOOKEEPER/versions/12346617
> >
> >  *
> https://issues.apache.org/jira/browse/ZOOKEEPER-3426?jql=project%20%3D%20ZOOKEEPER%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC
> >
> > We have a single remaining blocker, concerning the C client, and it is
> > not currently marked for 3.7.0:
> >
> >  * ZOOKEEPER-3426: ZK prime_connection(the Handshake) can complete
> >without reading all the payload.
> >
> > (It looks nasty, though, so I will try to have a look ASAP.)
> >
> > I also hope to get a green light for merging this first iteration of
> > quota support:
> >
> >  * ZOOKEEPER-3301: Enforce the quota limit
> >
> >https://github.com/apache/zookeeper/pull/934#issuecomment-754882022
> >
> >
> > What do you think?
> >
> > Cheers,
> > Damien Diederen
> >
> > P.-S. — This is my first time as a ZooKeeper release manager, so don't
> > hesitate to point out any perceived or actual missteps!
> >
> >
> >
> > Damien Diederen  writes:
> >> Hi Enrico, all,
> >>
> >> A Happy New Year!
> >>
> >> Yes, I am still willing to take care of 3.7.0.  I thought I would start
> >> once 3.5.9 was out, but I suppose there is no reason not to make
> >> progress in parallel.
> >>
> >> I still have to dig into the tracker and sort the list of applicable
> >> tickets, but was already wondering about the ones below, for which pull
> >> requests are more or less ready.
> >>
> >> (I really would like to include the first one, which is just stalled;
> >> the latter two, which are mine, are perhaps a bit "fresh.")
> >>
> >>  * ZOOKEEPER-3301:Enforce the quota limit
> >>https://github.com/apache/zookeeper/pull/934
> >>
> >>  * ZOOKEEPER-4026: Complete support for Stat objects (and create2) in
> >>multi requests
> >>https://github.com/apache/zookeeper/pull/1559
> >>
> >>  * ZOOKEEPER-4030: Optionally canonicalize host names in quorum SASL
> >>authentication
> >>https://github.com/apache/zookeeper/pull/1564
> >>
> >> What do you think?
> >>
> >> I'll ping people on the individual PRs.
> >>
> >> Cheers, -D
> >>
> >>
> >>
> >> Enrico Olivelli  writes:
> >>> Hi ZooKeepers,
> >>> I feel it is time to deliver 3.7.0 to the public.
> >>>
> >>> Any volunteer ?
> >>> Probably Demien, you already told you were available for this release
> >>>
> >>> Please also remember that there is a VOTE open for 3.5.9
> >>>
> >>> Best regards and Happy new year
> >>> Enrico
>
>

Re: The YCSB benchmark tool for zookeeper is now available

[Szalay-Bekő Máté]

this is a great contribution!
Thank you for both the implementation of the new YCSB binding and also for
documenting it in the ZooKeeper project!

Best regards,
Mate

On Mon, Dec 14, 2020 at 8:58 AM Enrico Olivelli  wrote:

> Great
>
> Thank you very much Justin
>
>
> Enrico
>
> Il Lun 14 Dic 2020, 05:36 Justin Ling Mao  ha
> scritto:
>
> > Now users can benchmark your zookeeper ensemble with YCSB. Here is the
> > ZK-PR(https://github.com/apache/zookeeper/pull/1558). Have fun for it:)
>

Re: [VOTE] Apache ZooKeeper release 3.5.9 candidate 0

[Szalay-Bekő Máté]

+1 (non-binding)

- I built the source code (-Pfull-build) in docker on Ubuntu 16.04.6 using
OpenJDK 8u275 and maven  3.3.9.
- all the unit tests passed (Java and C-client).
- I also built zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed

The only issue I found was that I was unable to make the python unit tests
to start. In 3.5.8 I was able to execute the unit tests (although I had to
do some manual hack before, which didn't help this time). I don't know what
changed here exactly, maybe just my environment. We might want to create a
jira ticket to migrate the zkpython build / test to maven properly.

Best regards,
Mate

On Thu, Dec 3, 2020 at 9:01 PM Damien Diederen 
wrote:

>
> Thank you, Norbert!
>
> I went through the motions a bit more carefully than usual in
> preparation for the upcoming 3.7.0 job, which I am planning to start
> soon, but probably after you finalize this release.
>
>
> +1 (advisory)
>
>   * Verified signatures and checksums;
>
>   * Built and tested on Ubuntu 20.04.1 LTS with OpenJDK Runtime
> Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04) using:
>
> mvn -B apache-rat:check verify spotbugs:check checkstyle:check \
> -Pfull-build -Dsurefire-forkcount=1
>
>   * Built and smoke-tested on NixOS with a slightly adapted version of
> this WIP PR:
>
> https://github.com/NixOS/nixpkgs/pull/104889
>
>   * Smoke-tested a single instance with Java, C and Perl client;
>
>   * Smoke-tested a 3-ensemble with Java client, including Kerberos auth;
>
>
> I don't believe these points are blockers, but I noticed that the
> following commits which are present in the release are not mentioned in
> the release notes:
>
>   * commit 0838c6c1613d7902d6c3419dcad2205682223175
> Author: Michael Han 
> Date:   Mon Jul 6 16:25:38 2020 +0200
>
> ZOOKEEPER-1634: hardening security by teaching server to enforce
> client authentication
>
>   * commit 54ffaad1b94d72e735fd8fb750117b6ee1550b1b
> Author: Andor Molnar 
> Date:   Tue Oct 6 17:51:15 2020 +0200
>
> ZOOKEEPER-3957: Created initial version of owasp-check Jenkinsfile
>
>   * commit db9fed4c95e4828389b30c0f6e94182db26ff99b
> Author: Enrico Olivelli 
> Date:   Tue Oct 20 16:21:30 2020 +0200
>
> ZOOKEEPER-3980: Fix Jenkinsfiles with new tool names
>
>
> On the other hand, and just FYI, the following tickets mentioned in the
> release notes do not have corresponding commits:
>
>   * ZOOKEEPER-3933: owasp failing with json-simple-1.1.1.jar:
> CVE-2020-10663, CVE-2020-7712.
>
> This was a false positive.  Ticket was closed, but no commit was
> produced.
>
>   * ZOOKEEPER-3934: upgrade dependency-check to version 6.0.0
>
> Same as ZOOKEEPER-3933.
>
> Cheers, -D
>
>
>
>
> Norbert Kalmar  writes:
> > This is a bugfix release candidate for 3.5.9. It contains 24 fixes,
> > including 2 CVE fix.
> >
> > The full release notes is available at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12348201
> >
> > *** Please download, test and vote by December 4th 2020, 23:59 UTC+0. ***
> >
> > Source files:
> > https://people.apache.org/~nkalmar/zookeeper-3.5.9-candidate-0/
> >
> > Maven staging repo:
> >
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.9/
> >
> > The release candidate tag in git to be voted upon: release-3.5.9-rc0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > Should we release this candidate?
> >
> > - Norbert
>

Re: [ANNOUNCE] New Commiter: Damien Diederen

[Szalay-Bekő Máté]

Congratulations Damien! :)

On Wed, Oct 28, 2020 at 12:37 PM Tamas Penzes 
wrote:

> Congrats Damien!
>
> On Wed, Oct 28, 2020 at 11:34 AM Enrico Olivelli 
> wrote:
>
> > The Project Management Committee (PMC) for Apache ZooKeeper
> > has invited Damien Diederen to become a committer and we are pleased
> > to announce that he has accepted.
> >
> > Damien contributed lots of improvements and bug fixes on Zookeeper C
> client
> > and he also participating in the community with good code reviews, and
> > discussions on our mailing lists.
> > For instance he is the author of the SASL support in the C client
> > https://github.com/apache/zookeeper/pull/1134
> >
> > Being a committer enables easier contribution to the
> > project since there is no need to go via the patch
> > submission process. This should enable better productivity.
> > Being a PMC member enables assistance with the management
> > and to guide the direction of the project.
> >
> > Congratulations Damien !
> >
> > Enrico Olivelli
> >
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Szalay-Bekő Máté]

Hello All,

A few reflections:

- I don't think that backporting fixes from a JDK 11 only version to a JDK
8 compatible version would be necessarily a harder thing than any regular
backport. It kind of depends on us (whether we use many JDK 11 only
features or not until we drop JDK 8 from all supported versions). Also if
it gets more painful, then we can decide to limit the number of backported
commits (e.g. only strictly to security fixes / CVEs)
- But even if the above point is true (?), I would not do this cut (e.g.
moving entirely to JDK 11 only) in a minor release. I think this should be
a major change in 4.0. (maybe together with other more "risky" changes,
like the separation of the client and server artifacts/code; or some
incompatible changes in the leader election protocol. Although these are
separate discussions)
- So all-in-all I like the JDK 11 only option the most. But I wouldn't do
it in 3.7 (which might happen very soon), but rather in 4.0. The question
for me is wether to do any in-between step in 3.x. (like the options 1 or 2
above in Christopher's mail). I think it mainly should depend on the timing
of 4.0.

Kind regards,
Mate

On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira  wrote:

> There are three points that stand out for me in this thread:
>
> - How do we determine how such a change affects our user base?
> - How much effort do the different options induce with respect to
> maintenance?
> - What's the right timeline for changes and how do we communicate them so
> that our users have enough time to prepare?
>
> Someone mentioned a PMC vote, and I don't think this should be a closed
> vote, independent of how the conversation goes.
>
> -Flavio
>
> > On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
>  wrote:
> >
> > Hi all,
> > If I might chime in as a zookeeper user (in multiple products) and a
> follower of the project I think the drop of Java8 support (official and/or
> unofficial) could be a big mistake.
> >
> > From my own company point of view we already support Java11 in all our
> applications so we are not directly impacted (and we have upgrade path for
> older versions to provide to our customers).
> > My worries resides in the (high) probability of a userbase
> fragmentation: in the recent past Zookeeper development picked up speed
> thanks to a bunch of new committers and PMCs after a period of mostly
> maintenance focused works, but the number of active committers and PMCs is
> still very low for a project like this.
> >
> > I foresee the risk of spreading thin the resources of the project if we
> force the userbase to stick to an older version and, in turn, we are forced
> to backport many issue to the 3.6 branch.
> >
> > Alessandro Luccaroni
> > Platform Manager @ Diennea - MagNews
> > Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519
> > Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy
> >
> > -Messaggio originale-
> > Da: Christopher 
> > Inviato: giovedì 22 ottobre 2020 05:21
> > A: dev@zookeeper.apache.org
> > Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> >
> > I'm happy that this discussion has been so lively! I just want to
> emphasize a few things:
> >
> > I really do understand the desire to continue to support Java 8... I get
> it. But all the conversations around this seem based on what people are
> doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > *future* release... so it should be based more on reasonable
> expectations for users in the future, and less based on what is happening
> today. I suspect *most* people today are still using 3.4 anyway (it was
> just so stable for so long...), but that shouldn't mean the developers
> should hold back development on 3.5 and 3.6, any more than today's users of
> 3.5/3.6 should hold back 3.7.
> >
> > Some of the opinions expressed in this discussion seem to propose a
> scenario where users are going to be updating to "bleeding edge"
> > versions of ZooKeeper, but are going to insist on using Java 8.
> > Personally, I find this to be implausible. In my experience, people
> either upgrade everything as soon as they are able to, or they upgrade each
> thing individually, only when they are forced to. The first group will be
> happy to move to Java 11 and ZK 3.7. The second group will probably avoid
> 3.7 anyway, and are fine sticking with 3.6, but if they had to update to
> 3.7, they'd also be fine updating to Java 11 if they had to in order to use
> 3.7. I can't imagine the scenario where people are eagerly choosing to
> upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps that
> scenario exists, but it's hard for me to imagine. Even so, my proposal
> would still support even that group of people.
> >
> > I think there are now effectively three proposals being discussed in
> this thread:
> >
> > 1. (Christopher's original proposal) passively support Java 8 at runtime
> by making JDK 11 the minimum requirement to build and test.
> > This scenario 

Re: Custom SSLContext

[Szalay-Bekő Máté]

I like the initiative, as long as it is backward compatible.
Nathan, the patch attached for ZOOKEEPER-2643
 is very old (also
missing documentation / tests). Would you like to contribute a new PR
against our current master branch?
Do we consider both Quorum SSL and Client SSL here?

Best regards,
Mate

On Tue, Oct 13, 2020 at 1:42 AM Nathan Gough  wrote:

> Is there no great way of doing this? Seems like it would solve a lot of
> problems for us: providing the context rather than key/truststore paths and
> properties will work a lot more cleanly.
>
> Cheers!
> Nathan
>
> On Wed, Oct 7, 2020 at 5:22 PM Nathan Gough  wrote:
>
> > Hi Enrico,
> >
> > Yes, the goal is to be strict about what protocols and ciphers to allow.
> > We have an SSLContext factory we use consistently across NiFi to provide
> a
> > better security guarantee.
> >
> > On Wed, Oct 7, 2020 at 5:13 PM Enrico Olivelli 
> > wrote:
> >
> >> Nathan,
> >>
> >> Il Mer 7 Ott 2020, 23:06 Nathan Gough  ha scritto:
> >>
> >> > Hi,
> >> >
> >> > I develop for Apache NiFi and was working on adding TLS to one of our
> >> > clients that use Zookeeper. I was wondering if it's possible to
> inject a
> >> > custom SSLContext similar in concept to this ticket:
> >> >
> >> > https://issues.apache.org/jira/browse/ZOOKEEPER-2643
> >> >
> >> > I can't see a way to provide this through the ZooKeeperAdmin
> interface.
> >> >
> >>
> >> Why do you need to inject a custom SSLContext? Can you explain a little
> >> more? Is it about limiting ciphers or protocols
> >>
> >> Enrico
> >>
> >>
> >> > Thanks!
> >> > Nathan
> >> >
> >>
> >
>

hacktoberfest

[Szalay-Bekő Máté]

Hello Devs!

Are you participating in Hacktoberfest? What about tagging the
ZooKeeper repo with the 'hacktoberfest' label to make it visible for
Hacktoberfest automatically?

Currently I see more than 32.000 open-source repositories in this
topic (https://github.com/topics/hacktoberfest), I think we should add
ZooKeeper too.

see:
- https://hacktoberfest.digitalocean.com/hacktoberfest-update
- 
https://docs.github.com/en/free-pro-team@latest/github/administering-a-repository/classifying-your-repository-with-topics

BR,
Mate

p.s. I'm not sure who has the rights to assign our repo to this topic.
(I don't have.)

Re: CI build issues

[Szalay-Bekő Máté]

This is how I do usually for RC checks:

mvn clean package -DskipTests dependency-check:check




On Mon, Oct 5, 2020, 16:58 Enrico Olivelli  wrote:

> Il giorno lun 5 ott 2020 alle ore 16:35 Andor Molnar  ha
> scritto:
>
> > Does anybody recall the command to run the owasp check?
> > Looks like we lost the old build configs, because old Jenkins has been
> > shut down.
> >
>
> https://jeremylong.github.io/DependencyCheck/dependency-check-maven/
> something like this:
> mvn dependency-check:check
>
> Enrico
>
>
> >
> > Andor
> >
> >
> >
> > > On 2020. Sep 16., at 11:28, Andor Molnar  wrote:
> > >
> > >
> > >
> > >> On 2020. Sep 16., at 1:38, Patrick Hunt  wrote:
> > >>
> > >> On Tue, Sep 15, 2020 at 2:46 PM Andor Molnar 
> wrote:
> > >>
> > >>> "What's the process for making changes now?”
> > >>>
> > >>> Like for any code changes: open Github PR.
> > >>>
> > >>>
> > >> Sure I know how to submit a PR, but what's the process for creating
> one
> > for
> > >> jenkins? I'm familiar with manually editing jobs, but not whatever
> else
> > is
> > >> required.
> > >
> > >
> > > Sorry Pat, perhaps I didn’t completely understand your question (or was
> > too late evening).
> > >
> > > You want to create a new job for the Owasp check, so first start with
> > manually creating a new job in Jenkins under the ZooKeeper View. Select
> > “Multibranch pipeline” and “Copy from” this job:
> > "zookeeper-multi-branch-build”.
> > >
> > > Leave everything in place, but change the Script Path of Jenkinsfile to
> > “Jenkinsfile-owasp”. Apply & Save.
> > >
> > > Job will automatically start scanning the branches for the specified
> > Jenkinsfile, but won’t find any and stop.
> > >
> > > Now start implementing and open PR. You probably just need to copy the
> > existing Jenkinsfile and change the “sh” command for the owasp build and
> > let’s see how it goes.
> > >
> > > For testing the patch, you need another Jenkins job similar to what I
> > mentioned above, but point it to your git repo.
> > >
> > > Hope that helps. Please shout if you’re stuck.
> > >
> > > Andor
> > >
> > >
> > >
> > >>
> > >> Patrick
> > >>
> > >>
> > >>> "How do I verify a job before submitting it via git?”
> > >>>
> > >>> Create a personal job which is pointing to your repo like mine:
> > >>>
> > >>>
> >
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline-andor/
> > >>>
> > >>> Not so nice, but that’s what we have for now.
> > >>>
> > >>> Andor
> > >>>
> > >>>
> > >>>
> >  On 2020. Sep 15., at 22:54, Patrick Hunt  wrote:
> > 
> >  On Tue, Sep 15, 2020 at 12:55 PM Andor Molnar 
> > wrote:
> > 
> > > Hi Michael,
> > >
> > > I was working on the CI migration and there’re still a few things
> > which
> > >>> is
> > > not available in the new system. I haven’t found any solution for
> the
> > > “retest” trigger, but I’ll take another look tomorrow. I need to
> dig
> > the
> > > builds@ list if there’s anything happened since I’ve last checked
> > e.g.
> > > new plugins installed, etc.
> > >
> > > I’m not sure I understand your concern about dead links. Here’s the
> > link
> > > of the pre-commit job for your PR:
> > >
> > >
> > >>>
> >
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-precommit-github-pr/view/change-requests/job/PR-1380/
> > >
> > > From the Github PR page I can see the following link:
> > >
> > >
> > >>>
> >
> https://ci-hadoop.apache.org/blue/organizations/jenkins/zookeeper-precommit-github-pr/detail/PR-1380/4/pipeline
> > >
> > > Which takes me to the Pipeline Report and definitely not dead.
> (This
> > >>> must
> > > be some new thing, but looks quite cool.)
> > >
> > > OWASP Build - Good point Patrick, that’s still missing.
> > > I’ve created the branch and PreCommit jobs as Jenkins pipelines,
> > nicely
> > > committed and tracked in Git. I believe that’s how we should do CI
> in
> > >>> the
> > > future. But I was reluctant do the same with flaky-test job which
> is
> > >>> just a
> > > copy-and-paste Jenkins job atm.
> > >
> > >
> >  What's the process for making changes now? How do I verify a job
> > before
> >  submitting it via git?
> > 
> >  Patrick
> > 
> > 
> > > Feel free to choose your way for the Owasp build, if you’re willing
> > to
> > > migrate it, but I think at the end of the way we should have
> > everything
> > >>> in
> > > source control to be perfect.
> > >
> > > We still don’t have Windows build either, but I’m not sure if
> > > Windows-based nodes are available.
> > >
> > > Andor
> > >
> > >
> > >
> > >> On 2020. Sep 13., at 23:02, Michael Han  wrote:
> > >>
> > >> Folks,
> > >>
> > >> I am seeing some CI build issues. Specifically:
> > >>
> > >> * Comment on github PR with "retest maven build" does not trigger
> a
> > > rebuild
> > >> of 

Re: ApacheCon Bug Bash

[Szalay-Bekő Máté]

Hello Guys,

In general I like the idea, but unfortunately I can not really participate
(either in the coding or in the review) as I have a few important projects
close to deadline at the moment.

My only concern is with the security bugs, which I don't like to be openly
reported before publishing a release with the fix. But for any other kind
of bugfixes / improvements, I am very positive with the initiative.


Best regards,
Mate

On Sun, Sep 27, 2020, 07:06 Tom DuBuisson  wrote:

> Enrico et al,
>
> Are there other thoughts on this?  It would be great to get setup before
> the bash actually begins.  Enrico, lacking other voices would you like to
> make a final call?
>
> -Tom
>
> On Thu, Sep 24, 2020 at 3:30 AM Enrico Olivelli 
> wrote:
>
> > Tom,
> > Personally I am +1 with this proposal. Thanks for your clarifications.
> >
> > But we should ear opinions from other people in this list
> >
> >
> > Enrico
> >
> > Il giorno mer 23 set 2020 alle ore 23:51 Tom DuBuisson 
> ha
> > scritto:
> >
> > > Enrico,
> > >
> > > On the topic security issues and reporting:  Muse's default
> configuration
> > > is open source tools and here it is run on open source projects.  The
> > > results are thus already available publicly (in this case from FSB,
> > Infer,
> > > and Error Prone).  Muse doesn't post anything to GitHub except in the
> > case
> > > of pull requests and then only if the bug is deemed to have been
> > > "introduced" as part of the PR - meaning it shouldn't be a
> vulnerability
> > in
> > > currently shipped software.
> > >
> > > If there are desires or proposals about more control over bug reports
> in
> > a
> > > convenient, configurable, manner then we'd really like to dig in and
> hear
> > > how to help.  In case there is more discussion on this point I'm CCing
> > > Andrew who leads Muse's product design.
> > >
> > > -Tom
> > >
> > > On Wed, Sep 23, 2020 at 1:09 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > Il Mer 23 Set 2020, 19:02 Tom DuBuisson  ha scritto:
> > > >
> > > > > Enrico,
> > > > >
> > > > > The Muse App requires two main abilities.  First is events, such as
> > > > > notification when pull requests are opened or updated.  Second is
> > > > > permission to post comments (which is always possible for humans
> but
> > > more
> > > > > tightly controlled when the poster authenticates as a github
> > > > application).
> > > > > The repository being public has allowed us to run the app and
> observe
> > > > > ErrorProne, Infer, and FindSecBugs all run out of the box and
> without
> > > > > custom configuration.
> > > > >
> > > >
> > > > Makes sense.
> > > >
> > > > One last question from my side
> > > > What about security issues?
> > > > Our policy is to have them reported to secur...@zookeeper.apache.org
> > > > before
> > > > public disclosure
> > > >
> > > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > > > Cheers,
> > > > > Tom
> > > > >
> > > > > On Wed, Sep 23, 2020 at 6:35 AM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Il Mer 23 Set 2020, 00:44 Tom DuBuisson  ha
> > scritto:
> > > > > >
> > > > > > > Zookeeper Developers,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > As part of our sponsorship of ApacheCon, our company MuseDev is
> > > > doing a
> > > > > > Bug
> > > > > > > Bash for select Apache projects. We'll bring members of the
> > > ApacheCon
> > > > > > > community together to find and fix a range of security and
> > > > performance
> > > > > > bugs
> > > > > > > during the conference, and gameify the experience with teams, a
> > > > > > > leaderboard, and prizes. The bash is open to everyone whether
> > > > attending
> > > > > > the
> > > > > > > conference or not, and our whole dev team will also be
> > > participating
> > > > to
> > > > > > > help fix as many bugs as we can.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We're seeding the bug list with results from Muse, our code
> > > analysis
> > > > > > > platform, which runs as a Github App and comments on possible
> > bugs
> > > as
> > > > > > part
> > > > > > > of the pull request workflow.  Here's an example of what it
> looks
> > > > like:
> > > > > > >
> > > > > > >
> https://github.com/curl/curl/pull/5971#discussion_r490252196
> > > > > > > 
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We explored a number of Apache projects and are reaching out
> > > because
> > > > > our
> > > > > > > analysis through Muse found some interesting bugs that could be
> > > fixed
> > > > > > > during the Bash.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We're writing to see if you'd be interested in having your
> > project
> > > > > > included
> > > > > > > in the Bash. Everything is set up on our end, and if you're
> > > > interested,
> > > > > > we
> > > > > > > would need you to say yes on this listserv, and we’ll work with
> > the
> > > > > > Apache
> > > > > > > Infrastructure team to grant Muse 

Re: Test flakiness (was: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0)

[Szalay-Bekő Máté]

as:<1>
>> at
>> org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:84)
>>
>>
>> I can provide more logs if we decide to open a ticket to track this topic.
>>
>>
>>
>>
>> >> Il giorno mer 2 set 2020 alle ore 20:42 Damien Diederen <
>> >> ddiede...@sinenomine.net> ha scritto:
>> >>
>> >>>
>> >>> Hi Enrico, all,
>> >>>
>> >>> I was also able to build and successfully run the tests of this
>> release
>> >>> candidate on Ubuntu 20.04.1, with the provided Java & Maven:
>> >>>
>> >>> $ grep VERSION= /etc/os-release
>> >>> VERSION="20.04.1 LTS (Focal Fossa)"
>> >>> $ java -version
>> >>> openjdk version "11.0.8" 2020-07-14
>> >>> $ mvn -version
>> >>> Apache Maven 3.6.3
>> >>>
>> >>> (It took me a number of tries, because that is a VM and the tests are
>> >>> somewhat flaky in that environment.  But this is, as far as I know, a
>> >>> long-standing issue and completely unrelated to 3.6.2.  Please let me
>> >>> know if you have tips/tricks for avoiding such temporary failures.)
>> >>>
>> >>> Cheers, -D
>> >>>
>> >>>
>> >>>
>> >>> Szalay-Bekő Máté  writes:
>> >>> > +1 (non-binding)
>> >>> >
>> >>> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using
>> OpenJDK
>> >>> > 8u242, OpenJDK 11.0.8 and maven 3.6.0.
>> >>> > - all the unit tests passed (both Java and C-client).
>> >>> > - I also built and executed unit tests for zkpython
>> >>> > - checkstyle and spotbugs passed
>> >>> > - apache-rat passed
>> >>> > - owasp (CVE check) passed
>> >>> > - fatjar built (-Pfatjar)
>> >>> >
>> >>> > On Tue, Sep 1, 2020 at 11:35 AM Enrico Olivelli <
>> eolive...@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> >> This is a release candidate for 3.6.2.
>> >>> >>
>> >>> >> It is a minor release and it fixes a few critical issues and
>> brings a
>> >>> few
>> >>> >> dependencies upgrades.
>> >>> >>
>> >>> >> The full release notes is available at:
>> >>> >>
>> >>> >>
>> >>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12347809
>> >>> >>
>> >>> >> *** Please download, test and vote by September 4th 2020, 23:59
>> UTC+0.
>> >>> ***
>> >>> >>
>> >>> >> Source files:
>> >>> >> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/
>> >>> >>
>> >>> >> Maven staging repo:
>> >>> >>
>> >>>
>> https://repository.apache.org/content/repositories/orgapachezookeeper-1060/
>> >>> >>
>> >>> >> The release candidate tag in git to be voted upon: release-3.6.2-0
>> >>> >> https://github.com/apache/zookeeper/tree/release-3.6.2-0
>> >>> >>
>> >>> >> ZooKeeper's KEYS file containing PGP keys we use to sign the
>> release:
>> >>> >> https://www.apache.org/dist/zookeeper/KEYS
>> >>> >>
>> >>> >> The staging version of the website is:
>> >>> >>
>> >>>
>> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/website/
>> >>> >>
>> >>> >> Should we release this candidate?
>> >>> >>
>> >>> >> Enrico Olivelli
>> >>> >>
>> >>>
>>
>

Re: [VOTE] Apache ZooKeeper release 3.6.2 candidate 0

[Szalay-Bekő Máté]

+1 (non-binding)

- I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
8u242, OpenJDK 11.0.8 and maven 3.6.0.
- all the unit tests passed (both Java and C-client).
- I also built and executed unit tests for zkpython
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- fatjar built (-Pfatjar)

On Tue, Sep 1, 2020 at 11:35 AM Enrico Olivelli  wrote:

> This is a release candidate for 3.6.2.
>
> It is a minor release and it fixes a few critical issues and brings a few
> dependencies upgrades.
>
> The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12347809
>
> *** Please download, test and vote by September 4th 2020, 23:59 UTC+0. ***
>
> Source files:
> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1060/
>
> The release candidate tag in git to be voted upon: release-3.6.2-0
> https://github.com/apache/zookeeper/tree/release-3.6.2-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/website/
>
> Should we release this candidate?
>
> Enrico Olivelli
>

Re: C client build is failing for branch-3.5 on new CI

[Szalay-Bekő Máté]

Thanks for raising the issue!

I checked the logs. The 'configure' step finds the openSSL library before
make:
19:58:34  checking openssl/ssl.h usability... yes
19:58:34  checking openssl/ssl.h presence... yes
19:58:34  checking for openssl/ssl.h... yes
19:58:34  checking for SSL_CTX_new in -lssl... yes
19:58:34  configure: building with SSL support

So some openSSL should be installed on the nodes. (could be a wrong
version?)
The same job on branch 3.5 was succeeding on 22nd of August:
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.5/39/consoleFull

It's either
1) a bug in our pre-build verifications (an unsupported openssl library
version in which case the build scripts should build the C client without
SSL support), or
2) maybe some environmental issue with the jenkins workers? I'm not
familiar with our jenkins configs. Are we using some dockerized environment
to build, or these are shared worker hosts? Do we have any control about
the installed C libraries?

Best regards,
Mate


On Thu, Aug 27, 2020 at 3:19 PM Enrico Olivelli  wrote:

> Hi ZooKeepers,
> I am checking the new CI system and I noticed this error for branch-3.5 on
> the C client build, the error does not occur on branch-3.6 and on master
>
> TestClient.o `test -f 'tests/TestClient.cc' || echo
>
> '/home/jenkins/jenkins-home/workspace/er-multi-branch-build_branch-3.5/zookeeper-client/zookeeper-client-c/'`tests/TestClient.cc*22:00:06*
>   [exec] Makefile:1397: recipe for target 'zktest_mt-TestClient.o'
> failed*22:00:06*   [exec] make[1]: Leaving directory
>
> '/home/jenkins/jenkins-home/workspace/er-multi-branch-build_branch-3.5/zookeeper-client/zookeeper-client-c/target/c'*22:00:06*
>   [exec] Makefile:2106: recipe for target 'check-am'
> failed*22:00:06*   [exec] In file included from
>
> /home/jenkins/jenkins-home/workspace/er-multi-branch-build_branch-3.5/zookeeper-client/zookeeper-client-c/tests/TestClient.cc:38:0:*22:00:06*
>   [exec]
> /home/jenkins/jenkins-home/workspace/er-multi-branch-build_branch-3.5/zookeeper-client/zookeeper-client-c/include/zookeeper.h:36:10:
> fatal error: openssl/ossl_typ.h: No such file or directory*22:00:06*
> [exec]  #include *22:00:06*   [exec]
> ^~~~*22:00:06*   [exec] compilation
> terminated.*22:00:06*   [exec] make[1]: ***
> [zktest_mt-TestClient.o] Error 1*22:00:06*   [exec] make: ***
> [check-am] Error 2
>
>
> see
>
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.5/44/execution/node/72/log/
>
> Best regards
> Enrico
>

Re: Preparing for 3.6.2 release ?

[Szalay-Bekő Máté]

Hello,

Thank You for working on the release!
one thing that comes to my mind is ZOOKEEPER-3905. It is an important
security fix that would be great to have on 3.6.2 I think. It already
landed on master, but AFAICS Andor haven't pushed the PR yet for branch-3.6.

Best regards,
Mate

On Mon, Aug 10, 2020 at 3:58 PM Enrico Olivelli  wrote:

> Hi,
> I would like to start the release process for 3.6.2.
>
> Any thoughts ?
>
> Enrico
>
> Il giorno ven 24 lug 2020 alle ore 16:30 Enrico Olivelli <
> eolive...@gmail.com> ha scritto:
>
> > Hi,
> > we have committed a few fixes on 3.6.2 release branch, in particular
> > bugfixes and fixes for regressions
> >
> > This is the list:
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.6.2
> >
> > I would like to start doing the necessary housekeeping before the release
> > and I volunteer as release manager.
> >
> > I no one objects this is my proposal:
> > - check all pending PRs, to see if there is something good (ready and
> > valuable) for 3.6.2
> > - clean up JIRA, move to next version issues that someone marked for
> 3.6.2
> > but no one has worked on them
> > - cut a release candidate at the end of next week
> >
> > So if you have marked some issue for inclusion on 3.6.2 please let me
> > know, otherwise I will move each item to 3.6.3
> > I would love to remove fixVersion at all, because if no one is working on
> > an issue and the issue is not worth to be on top o the list, it is only
> > noise, but this is a separate topic.
> >
> > Best regards
> > Enrico
> >
>

Re: jUnit5 migration review

[Szalay-Bekő Máté]

>What's the impact of such a change on projects incorporating//depending
upon ZK test libraries? Is that still a thing?

If we only submit this to the master branch, then it should be fine even if
some projects somehow rely on some of our test utils.
But as far as I know, no downstream project is using our test code through
maven. At least junit is only listed as test dependency in maven:
https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.6.1 So
the JUnit5 migration shouldn't leak out.
Many hadoop components are using ZK Mini Clusters defined in Hadoop or
HBase. But those are just using our ZooKeeperMain class and nothing from
our test code.

Kind regards,
Mate

On Thu, Jul 30, 2020 at 12:08 PM Tamas Penzes 
wrote:

> >
> > > If a project depends on a test jar based on junit5 then the project has
> > to
> > > upgrade to junit5.
> > >
> > Can you explain more?
>
>
> jUnit 5 has fundamental changes e.g. classes have been moved to different
> packages, method signatures have changed, some annotation parameters got
> removed (timeout, expected), it also became modularized, Rules don't exist
> anymore, etc.
> If a test class depends on a test jar using jUnit5 obviously jUnit5 must be
> used.
>
> But there is a special case:
> jUnit5 vintage package is part of jUnit5 and it is basically a jUnit 4.13
> with the important change that it shows every class originating from jUnit4
> deprecated.
> It can be used if someone can live together with a large amount of
> deprecation messages.
> If there is a given testclass which uses jUnit5 vintage (basically junit4)
> in a junit5 based jar and we only depend on this vintage based class in our
> test we are able to use jUnit4.
>
> So jUnit5 with the vintage package is quite tolerant with jUnit4 based
> test, but still motivates towards migration through the deprecation
> messages.
>
> Regards, Tamaas
>
> On Thu, Jul 30, 2020 at 7:06 AM Enrico Olivelli 
> wrote:
>
> > Il Gio 30 Lug 2020, 01:28 Tamas Penzes  ha
> > scritto:
> >
> > > If a project depends on a test jar based on junit5 then the project has
> > to
> > > upgrade to junit5.
> > >
> >
> > Can you explain more?
> >
> >
> >
> > > With junit5 the tests can stay in junit4 format (using junit-vintage),
> > but
> > > then they complain about being deprecated.
> > >
> > > If a test depends on another test class using junit5 the test itself
> must
> > > be converted.
> > >
> > > ZooKeeper is a good candidate to convert first as it is at the bottom
> of
> > > the stack and doesn't deliver a test jar yet.
> > >
> > > After finishing ZooKeeper I'd volunteer to do the junit5 upgrade in
> > Curator
> > > too.
> > >
> >
> > That's awesome, thanks in advance
> > Enrico
> >
> >
> > > Regards, Tamaas
> > >
> > > On Wed, Jul 29, 2020, 23:53 Patrick Hunt  wrote:
> > >
> > > > What's the impact of such a change on projects
> incorporating//depending
> > > > upon ZK test libraries? Is that still a thing?
> > > >
> > > > Patrick
> > > >
> > > > On Wed, Jul 29, 2020 at 2:32 PM Tamas Penzes
> >  > > >
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > If you have (a lot of) free time and would like to review my pull
> > > > request I
> > > > > would be over the seventh heaven.
> > > > > It can be found here:
> https://github.com/apache/zookeeper/pull/1417
> > > > > That's the next step of jUnit4 to 5 migration.
> > > > >
> > > > > Thanks, Tamaas
> > > > >
> > > >
> > >
> >
>
>
> --
> *Tamás Pénzes* | Software Engineering Manager
> e. tam...@cloudera.com
> cloudera.com 
> [image: Cloudera] 
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> 
> --
>

Re: 4 letter words on secureClientPort

[Szalay-Bekő Máté]

FYI: I think this command should work to use 4LW commands on secure port:

(echo "srvr"; sleep 1) | openssl s_client -connect zkhost:1181 -cert
./cert.pem -key ./key.pem

Kind regards,
Mate

On Thu, Jul 23, 2020 at 12:35 AM Norbert Kalmar
 wrote:

> Hi Sankalp,
>
> How are you trying to access zk to run 4lw commands? AFAIK nc(telnet) does
> not support ssl.
> You need something that supports ssl, like openssl s_clien [1] (but I
> haven't actually tested this) or by invoking the java client.
> But as you use secure communication, I would recommend using adminserver,
> if you are on 3.5+ that is.
>
> Regards,
> Norbert
>
> [1]
>
> https://serverfault.com/questions/476068/can-netcat-talk-to-an-encrypted-port/476073
>
> On Wed, Jul 22, 2020 at 11:42 PM Sankalp Bhatia  >
> wrote:
>
> > Hi All,
> >
> > I am getting an exception while trying to ping the secureClientPort in my
> > zookeeper server.
> >
> > io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
> > 737461740a
> >
> > Do the 4 letter words not work with the secureClientPort? Can I tweak
> some
> > system settings to get those to work?
> >
> > Thanks,
> > Sankalp
> >
>

Re: Upgrading existing non-TLS cluster with no downtime

[Szalay-Bekő Máté]

echo "stat" | nc localhost 2182Hi,

I guess this is the part you are referring:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster
(your link was pointing to the 3.3.2 admin guide where this chapter was
missing)

> 1) When I set sslQuorum=true  and portUnification=true on the first
server,
does it go out of the quorum? And when these properties are set in the
second server, a new quorum of first and second server is formed and now
the third server is out of quorum. When the 3rd server follows suit, it is
added back to the quorum.

the "sslQuorum=true  and portUnification=true" setting is needed in step 4
(although the numbering is bad in the markdown...). After step 3 you
already have a 3 server quorum up with portUnification=true, meaning the
cluster can handle both TLS/SSL and regular/non-secure connections. So when
you restart server 1 with sslQuorum=true, then it will be able to re-join
to the quorum, as server 2 and 3 are capable of handling SSL connections
(even if they are not using it for connection initiation). So ideally
between restarting each servers with sslQuorum=true, you always should have
a 3 node full quorum.

> 2) The guideline says to check after restarting every broker that the
quorum is healthy, is there any metric to track that?

I send the "stat" command to all nodes to see if everyone is connected to
the quorum. E.g.: echo "stat" | nc localhost 2181
I usually use 4-letter-word commands but the REST admin API works as well,
and actually that is the officially recommended way, as the 4-letter-words
are / will be deprecated some time.
For the admin server see:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_adminserver

Kind regards,
Mate

On Tue, Jul 14, 2020 at 10:52 PM Sankalp Bhatia 
wrote:

> +users
>
> On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia 
> wrote:
>
> > Hi All,
> >
> > I am trying to follow the section "Upgrading existing non-TLS cluster
> with
> > no downtime" in the zookeeper guide :
> > https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
> >
> > I have an ensemble of 3 servers. I have a couple of questions:
> >
> > 1) When I set sslQuorum=true  and portUnification=true on the first
> > server, does it go out of the quorum? And when these properties are set
> > in the second server, a new quorum of first and second server is formed
> and
> > now the third server is out of quorum. When the 3rd server follows suit,
> it
> > is added back to the quorum.
> >
> > If this is the case, what is the use of a the port-unification feature
> > here?
> >
> > 2) The guideline says to check after restarting every broker that the
> > quorum is healthy, is there any metric to track that?
> >
> > Thanks,
> > Sankalp
> >
> >
> >
> >
>

Re: Use cases of client.portUnification

[Szalay-Bekő Máté]

> Should I also be worried about any performance impacts here in terms of
CPU/Runtime? Will my Plaintext requests be as fast as they are with a
vanilla Plaintext port? Would be helpful if someone can help me with some
documentation around this.

Using SSL vs using unsecure socket does have some performance impact for
sure. But I don't know about any documentation / measurement around this
specific question... I don't know if using non-SSL with or without port
unification makes any difference. I would say most probably not.
I see that the code path for socket / connection initialization is a bit
different for the two cases. But I wouldn't expect real performance impact.
Only the connection initialization part should be impacted, the performance
for the rest of the communication over the socket should be the same, I
assume.

This is the point in the code where the client socket handler gets defined
(either using or not using portUnification):
https://github.com/apache/zookeeper/blob/6ab1822ec431bb3309021c1ddc613a5eaa28d83b/zookeeper-server/src/main/java/org/apache/zookeeper/server/NettyServerCnxnFactory.java#L489

I have some performance measures around SSL vs. unsecure. It is not exactly
what you are asking for, but maybe can be useful:
https://drive.google.com/drive/folders/1uG3JI6sXiuWJ15IjtUMGTJwNY_pP3yhZ?usp=sharing

Cheers,
Mate

On Mon, Jul 6, 2020 at 11:38 AM Sankalp Bhatia 
wrote:

> Thanks Enrico and Mate for the valuable comments.
>
> Mate, regarding your point- *I don't consider the use of
> client.portUnification to be 'bad' or 'unsecure' in itself *
>
> I agree. This is as bad as the case of having a plaintext and TLS port open
> at the same time in terms of security.
>
> Should I also be worried about any performance impacts here in terms of
> CPU/Runtime? Will my Plaintext requests be as fast as they are with a
> vanilla Plaintext port? Would be helpful if someone can help me with some
> documentation around this.
>
> Thanks
> -Sankalp
>
> On Sun, 5 Jul 2020 at 17:09, Enrico Olivelli  wrote:
>
> > In my opinion you can use port unification during a rolling upgrade of
> your
> > ZK cluster and you are moving your servers to TLS.
> >
> > Another case is that you have to connect to two different ZK clusters,
> one
> > with TLS and one with plain connections, some configurations are system
> > properties so it is hard sometimes to implement this scenario.
> >
> > I have not used it, so I am just sharing a couple of ideas.
> >
> > The feature has been contributed by our Facebook friends, I hope that
> > someone from that crew can tell more
> >
> > Regards
> > Enrico
> >
> > Il Dom 5 Lug 2020, 16:41 Szalay-Bekő Máté 
> ha
> > scritto:
> >
> > > Hi Sankalp,
> > >
> > > I think it really depends on your security policies. I don't consider
> the
> > > use of client.portUnification to be 'bad' or 'unsecure' in itself.
> > > Especially, if you can make sure in your cluster that all sensitive
> data
> > is
> > > protected with ACLs and modified / listed using TLS.
> > >
> > > But still the most secure is to use TLS-only connections. In our case
> we
> > > found it tricky to configure all the clients to connect to ZooKeeper
> > using
> > > TLS, so we are following a step-by-step approach to migrate all our
> > clients
> > > to TLS. But instead of using client.portUnification, we decided to
> > maintain
> > > two separate ports in our configs. I don't exactly remember our
> > reasoning,
> > > but e.g for me it seems to be easier to debug connection issues. (so if
> > the
> > > client is able to connect to the secure-only port, then you can make
> sure
> > > TLS is really used - and you don't have a false sense of security)
> > >
> > > Cheers,
> > > Mate
> > >
> > > On Sun, Jul 5, 2020 at 2:35 PM Sankalp Bhatia <
> sankalpbhati...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi Devs,
> > > >
> > > > Can someone share some insights on what is a good use case for the
> > > feature
> > > > *client.portUnification*? I have a use case where clients would want
> > both
> > > > PLAINTEXT and TLS traffic to be served by ZooKeeper server and I want
> > to
> > > > avoid exposing and managing 2 different zookeeper ports. Is this a
> > valid
> > > > use case? or is this feature only supposed to be used for some
> rolling
> > > > upgrades like the one for quorum port unification?
> > > >
> > > > Thanks in advance!
> > > >
> > > > -Sankalp
> > > >
> > >
> >
>

Re: Use cases of client.portUnification

[Szalay-Bekő Máté]

Hi Sankalp,

I think it really depends on your security policies. I don't consider the
use of client.portUnification to be 'bad' or 'unsecure' in itself.
Especially, if you can make sure in your cluster that all sensitive data is
protected with ACLs and modified / listed using TLS.

But still the most secure is to use TLS-only connections. In our case we
found it tricky to configure all the clients to connect to ZooKeeper using
TLS, so we are following a step-by-step approach to migrate all our clients
to TLS. But instead of using client.portUnification, we decided to maintain
two separate ports in our configs. I don't exactly remember our reasoning,
but e.g for me it seems to be easier to debug connection issues. (so if the
client is able to connect to the secure-only port, then you can make sure
TLS is really used - and you don't have a false sense of security)

Cheers,
Mate

On Sun, Jul 5, 2020 at 2:35 PM Sankalp Bhatia 
wrote:

> Hi Devs,
>
> Can someone share some insights on what is a good use case for the feature
> *client.portUnification*? I have a use case where clients would want both
> PLAINTEXT and TLS traffic to be served by ZooKeeper server and I want to
> avoid exposing and managing 2 different zookeeper ports. Is this a valid
> use case? or is this feature only supposed to be used for some rolling
> upgrades like the one for quorum port unification?
>
> Thanks in advance!
>
> -Sankalp
>

Re: Need Help with Maven Build

[Szalay-Bekő Máté]

I saw this problem a few times (usually when I build from IntelliJ console,
after I changed to a different git branch).

My solution is usually:
git clean -xdf
git reset --hard
mvn clean

after these steps usually "mvn clean install -DskipTests" works just fine

On Wed, May 20, 2020 at 1:27 PM Jun Wang  wrote:

> I am using latest maven, build failed with zookeeper source code checked
> out from github. but build is fine with downloaded source code
> apache-zookeeper-3.6.1.tar.gz
>
> $ mvn --version
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: /home/jun/programs/apache-maven-3.6.3
> Java version: 1.8.0_251, vendor: Oracle Corporation, runtime:
> /home/jun/programs/jdk1.8.0_251/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.10.0-38-generic", arch: "amd64", family:
> "unix"
>
>
> 
> From: Michael Han 
> Sent: Wednesday, May 20, 2020 1:08 AM
> To: user 
> Cc: dev@zookeeper.apache.org 
> Subject: Re: Need Help with Maven Build
>
> hi jun - which maven version you are using?
>
> If it's 3.5.x, try upgrade to 3.6.x. I had the exact same issue a while
> back and upgrade maven fixed this, so I didn't bother to debug. That said,
> it's interesting to understand why we failed under specific version of
> maven / env, so cc dev list where we have a few maven experts who might be
> able to help debug.
>
>
> On Tue, May 19, 2020 at 8:34 AM Jun Wang  wrote:
>
> > Hi
> >
> > I got following build error with latest code from github.  But build is
> > fine with downloaded source code.   Any suggestion is appreciated.
> >
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> > (default-compile) on project zookeeper: Fatal error compiling:
> > java.lang.NullPointerException -> [Help 1]
> >
> >
> >
> https://gist.githubusercontent.com/wj1918/b1bcea0473b9ff2096ffa22e3c387e8f/raw/8c2ccfb7919470e0e874abdec5633976720e3dca/zookeeper.build.error.txt
> >
> > Thanks
> > Jun
>