Re: Question regarding first-time contribution to ZK

[Enrico Olivelli]

Hello,
Did you see my question on the PR ?

https://github.com/apache/zookeeper/pull/2085

Enrico


Il giorno mer 21 feb 2024 alle ore 00:00 Rishabh Rai
 ha scritto:
>
> On Mon, Nov 6, 2023 at 11:21 PM Rishabh Rai 
> wrote:
>
> > Hi ZK dev team,
> >
> > Hope you're doing well! I am a developer trying to contribute to ZK for
> > the first time. I recently created a JIRA ticket (ZOOKEEPER-4766) and
> > related PR to improve how ZK handles taking snapshots during leader
> > elections. Any guidance on whether this ticket was filed correctly and on
> > how to get more eyes on the viability of the change would be greatly
> > appreciated. The change itself is fairly trivial (and a bit time sensitive
> > for our use case). Thank you!
> >
> > Best,
> > Rishabh Rai
> >

Re: [VOTE] Apache ZooKeeper release 3.9.2 candidate 0

[Enrico Olivelli]

+1 (binding)

- checked signatures and RAT
- built locally, JDK11, Ubuntu
- run tests
- run some smoke tests using the binaries built locally

Thanks
Enrico

Il giorno mer 28 feb 2024 alle ore 21:36 Andor Molnar
 ha scritto:
>
> +1 (binding)
>
> - verified checksum and gpg signature of the artifacts
> - full build was successful
> - unit tests passed
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
>
> Andor
>
>
>
> On Mon, 2024-02-12 at 22:37 +0100, Damien Diederen wrote:
> > Greetings, all!
> >
> >
> > This is a release candidate for 3.9.2.
> >
> > This is a bugfix release for the 3.9 release line. Includes important
> > dependency upgrades to address CVEs.
> >
> >
> > The full release notes is available at:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353694
> >
> > *** Please download, test and vote by February 16th 2024, 23:59
> > UTC+0. ***
> >
> > Source files:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.2/
> >
> > The release candidate tag in git to be voted upon: release-3.9.2-0
> > https://github.com/apache/zookeeper/releases/tag/release-3.9.2-0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Regards,
> > Damien Diederen
>

Re: [ANNOUNCE] Apache ZooKeeper 3.8.4

[Enrico Olivelli]

Il Mer 6 Mar 2024, 21:31 Andor Molnar  ha scritto:

> Nice job Damien.
> Thanks!
>


Thanks

Enrico

>
>
>
> On Tue, 2024-03-05 at 23:00 +0100, Damien Diederen wrote:
> > The Apache ZooKeeper team is proud to announce Apache ZooKeeper
> > version 3.8.4
> >
> > ZooKeeper is a high-performance coordination service for distributed
> > applications. It exposes common services - such as naming,
> > configuration management, synchronization, and group services - in a
> > simple interface so you don't have to write them from scratch. You
> > can
> > use it off-the-shelf to implement consensus, group management, leader
> > election, and presence protocols. And you can build on it for your
> > own, specific needs.
> >
> > For ZooKeeper release details and downloads, visit:
> > https://zookeeper.apache.org/releases.html
> >
> > ZooKeeper 3.8.4 Release Notes are at:
> > https://zookeeper.apache.org/doc/r3.8.4/releasenotes.html
> >
> > We would like to thank the contributors that made the release
> > possible.
> >
> > Regards,
> >
> > The ZooKeeper Team
>
>

Re: First time contribution to ZK. Attn: Patrick Hunt, Andor Molnar, Flavio Junqueira

[Enrico Olivelli]

George,
thanks for sharing your patch.

A couple of thoughts below





Il giorno mer 21 feb 2024 alle ore 02:57 Liz George
 ha scritto:
>
> Hi Zk dev experts and users.
> We are currently considering this change to Apache Zookeeper code. Details
> are outlined  in this ticket
> https://issues.apache.org/jira/browse/ZOOKEEPER-4766

You did well in sharing the patch here on the mailing list and sending out a PR.
Can you please target the "master" branch in the PR ?

> This code change is to improve how ZK handles taking snapshots during
> leader elections. Any review of this code change would be appreciated.
> Specifically if there are any risks, or if this seems like a valuable fix
> to have that we could push to the community at a later date if needed.

Are all the tests passing ? CI is not running the tests probably
because the PR is not targeting the master branch

>
> Rishabh is the engineer who has made this change in a private env. Please
> can you add your comments to the Jira ticket itself or email him at
> rishabhrai.m...@gmail.com (Cc me as well Pls)

Could Rishabh subscribe to this mailing list ? We could miss adding him
We prefer discussing all together here in the mailing list.
JIRA is not good for "discussions" because folks have to follow JIRA
notifications and for many people they are not taken much into
consideration.

Thanks
Enrico

> Thanks in advance,
> Liz George
> Director, Software Engineering | Salesforce
> Mobile: 408-406-0524

Re: [VOTE] Apache ZooKeeper release 3.8.4 candidate 0

[Enrico Olivelli]

+1 (binding)

- verified checksums and signatures
- full build, on Ubuntu, JDK11
- smoke tests using the binaries I have built from sources

Thanks

Enrico

Il giorno mer 28 feb 2024 alle ore 22:49 Andor Molnar
 ha scritto:
>
> +1 (binding)
>
> - verified checksum and gpg signature of the artifacts
> - full build was successful
> - unit tests passed
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - smoke tests (basic commands, watchers, etc.) passed
>
> Andor
>
>
>
> On Mon, 2024-02-12 at 23:35 +0100, Damien Diederen wrote:
> > Greetings, all!
> >
> >
> > This is a release candidate for 3.8.4.
> >
> > This is a bugfix release for the 3.8 release line. Includes important
> > dependency upgrades to address CVEs.
> >
> >
> > The full release notes is available at:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353693
> >
> > *** Please download, test and vote by February 16th 2024, 23:59
> > UTC+0. ***
> >
> > Source files:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.4/
> >
> > The release candidate tag in git to be voted upon: release-3.8.4-0
> > https://github.com/apache/zookeeper/releases/tag/release-3.8.4-0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Regards,
> > Damien Diederen
>

Re: Lack of support for TLS-only ZK cluster

[Enrico Olivelli]

Il Mar 20 Feb 2024, 22:43 Abhilash Kishore  ha
scritto:

> Yes, that's the PR .
>
> It's ready now. Do you mind taking a look?
>


Reviewed.
Thanks

Enrico


> Regards,
> Abhilash Kishore
>
>
> On Tue, 13 Feb 2024 at 02:28, Andor Molnar  wrote:
>
> > Hi Abhilash,
> >
> > Is this the patch that you're working on?
> >
> > https://github.com/apache/zookeeper/pull/2117
> >
> > I see it's still draft, are u going to finish it soon?
> >
> > Andor
> >
> >
> >
> >
> > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > Thanks Andor, that makes sense. I agree with you, this is a simpler
> > > and
> > > cleaner solution.
> > >
> > > I'll work on the changes and will try to keep it backwards
> > > compatible.
> > >
> > > Regards,
> > > Abhilash Kishore
> > >
> > >
> > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> > >
> > > > Hi Abhilash,
> > > >
> > > > Thanks for looking into this issue.
> > > >
> > > > I wouldn't complicate things by trying to get reconfig parameters
> > > > aligned and mixed with clientPort/secureClientPort. Since the
> > > > documentation says these options are already deprecated I suggest
> > > > to
> > > > upgrade Reconfig config line to support secure client port as well.
> > > >
> > > > So, the following reconfig line:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > >
> > > > will become:
> > > >
> > > > "server.1=abhilash-
> > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > 82".
> > > >
> > > > The 3 scenarios will become:
> > > >
> > > > 1. Non-TLS only:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > >
> > > > 2. TLS-only:
> > > >
> > > > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > >
> > > > 3. TLS/non-TLS mixed:
> > > >
> > > > "server.1=abhilash-
> > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > 82".
> > > >
> > > > In addition to that I would force the user to use either the
> > > > deprecated
> > > > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > > > both.
> > > > Throw an exception and halt the server if both options are
> > > > specified at
> > > > the same time.
> > > >
> > > > Thoughts?
> > > >
> > > > Regards,
> > > > Andor
> > > >
> > > >
> > > >
> > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > Many organizations, large and small, have strict security and
> > > > > compliance
> > > > > requirements to only accept encrypted/TLS connections and not
> > > > > plain
> > > > > text
> > > > > connections.
> > > > >
> > > > > I'd like to discuss an issue which is preventing us from starting
> > > > > our
> > > > > ZK
> > > > > clusters in TLS only mode (for client traffic).
> > > > >
> > > > > As per dynamic reconfig doc
> > > > > 
> > > > > ;;,
> > > > >
> > > > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > > > configuration
> > > > > > parameters should no longer be used. Instead, this information
> > > > > > is
> > > > > > now part
> > > > > > of the server keyword specification, which becomes as follows:
> > > > > > server. =
> > > > > > ::[:role];[ > > > > > port
> > > > > > address>:]
> > > > >
> > > > > Let's say the dynamic config entry of a server is
> > > > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > > > The
> > > > > server
> > > > > starts up with a (plaintext) clientPort listener on 2181.
> > > > >
> > > > > Now, if we want to make this server TLS-only, what options do we
> > > > > have? We
> > > > > want to stop accepting plaintext traffic on 2181 and make the
> > > > > same
> > > > > port
> > > > > accept TLS connections only (make clientPort as
> > > > > secureClientPort).
> > > > >
> > > > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > > > first
> > > > > starts a
> > > > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > > > "server.1"
> > > > > dynamic
> > > > > config entry and then attempts to start a TLS client listener on
> > > > > the
> > > > > same
> > > > > port (2181) and fails. The reason for this behavior is already
> > > > > described in
> > > > > ZOOKEEPER-4276 <
> > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > > > (highly
> > > > > recommended pre-read).
> > > > >
> > > > > It is not possible to just remove the "" part from
> > > > > the
> > > > > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > > > > tried:
> > > > >
> > > > > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > > > > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > > > > server.1=abhilash-ubuntu:3183:4183:participant
> > > > > Arguments are not valid :
> > > > >
> > > > >
> > > > > The reconfig command does not allow us to add a server entry
> > > > > without
> > > > > ";[ > > > > port address>:]".
> > > > >
> > > > > How do we 

Re: ZOOKEEPER-4780 review

[Enrico Olivelli]

Muthu,
thanks for the heads up

I have reviewed the PR

Thanks
Enrico

Il giorno sab 3 feb 2024 alle ore 08:47 Muthuraj Ramalinga kumar
 ha scritto:
>
> Hi,
> I added more commits to address review comments for ZOOKEEPER-4780.
>
> Could someone take a look pls ?
> https://github.com/apache/zookeeper/pull/2100
>
> Thank you
> Muthu

Re: Discussion around ZOOKEEPER-4790

[Enrico Olivelli]

Sönke,

Il giorno gio 1 feb 2024 alle ore 09:26 Sönke Liebau
 ha scritto:
>
> Hi all,
>
> we recently ran into issues with ZooKeeper on Kubernetes which caused us to
> open [1] after a bit of analysis.
>
> We are happy to work on opening a PR to improve this behavior here, but I
> wanted to start a discussion around what "improve" would look like exactly
> before putting any effort into the PR.
>
> I'll keep this mail light on details  - it is hopefully all covered in the
> issue.

It is good to also write some details here, sometimes people are lazy
to open JIRA,
and also it will be easier to add inline questions/answers

>
> My basic question is: would people be okay with adding a check of the
> certificate SAN entries against the hostnames from config?

I think that this is good, especially if that can help people
deploying ZK in k8s with
security enabled. We should remove all the pain points for users.

>
> We cannot simply replace the existing check [2] of course, that'd run a
> high risk of breaking existing setups, obvious options there would be to
> either add a config option to replace the hostname check with this check,
> or run this check in parallel with the hostname check and if either of them
> succeeds allow the connection, but I'm sure there are many other potential
> ways of doing this.

Yes, we must add some flag, then we introduce it in the next major version,
maybe it will become the new default behaviour at some point.

Thanks
Enrico

>
> Any thoughts or opinions on this would be very appreciated.
>
> Best regards,
> Sönke
>
>
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-4790
> [2]
> https://github.com/apache/zookeeper/blob/11c07921c15e2fb7692375327b53f26a583b77ca/zookeeper-server/src/main/java/org/apache/zookeeper/common/ZKTrustManager.java#L158

Re: Moving 3.7 to End-of-Life

[Enrico Olivelli]

Il Sab 27 Gen 2024, 00:27 Patrick Hunt  ha scritto:

> Markmail search for apache mail archive is gone? and unfortunately Apache
> mail archive search seems to be broken (no results coming back...) I
> managed to track this ref down
> https://lists.apache.org/thread/b8sm8gxmohs9gl4vrltd2jr4slqvrg9n
> but I distinctly remember seeing something about this, just can't find it.
>
> According to this it's already eol:
> https://endoflife.date/zookeeper
>
> Our own release page makes it clear that folks should move given stable and
> current have been out for a while. I think we can call it EOL at this
> point.
>


+1 to mark 3.7 release line as EOL

Enrico

>
> Regards,
>
> Patrick
>
> On Fri, Jan 26, 2024 at 7:33 AM Andor Molnar  wrote:
>
> > Hi zk community,
> >
> > According to our Releases [1] page ZooKeeper 3.8.2 became the first
> > stable version of 3.8.x line on 3 Aug, 2023 (when 3.9.0 was released).
> >
> > The previous stable version "in approximately half a year will be
> > announced as End-of-Life". 6 months will pass on 3 Feb, 2024, so we
> > should think about announcing EoL soon.
> >
> > What do you think?
> >
> > Regards,
> > Andor
> >
> >
> > [1] https://zookeeper.apache.org/releases.html
> >
> >
> >
> >
>

Re: New merge script with GH api

[Enrico Olivelli]

Great!

Enrico

Il Ven 1 Dic 2023, 14:34 tison  ha scritto:

> Hi Andor,
>
> Thanks for driving this effort. And Thank Szucs Villo for implementing.
>
> I like this improvement :D
>
> Best,
> tison.
>
> Andor Molnar  于2023年12月1日周五 21:12写道：
> >
> > Hi folks,
> >
> > We've just submitted
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4756 to all active
> > branches. It's about an improvement for the merge script to use GH api
> > for merging PRs instead of manually pushing and leaving the PR is
> > "closed" state.
> >
> > The PR itself has been merged with the new script and it nicely turned
> > into "Merged" status.
> >
> > Please use this new script for merging PRs in the future rather than
> > merging on GitHub. It creates a nice commit message and updates Jira
> > ticket as well. Feedbacks are welcome.
> >
> > We continue working on further improvement which I spotted in Spark
> > project and listed in the Jira's description.
> >
> > Regards,
> > Andor
> >
> >
> >
>

[jira] [Created] (ZOOKEEPER-4775) Add a version of check_zookeeper that works with Python 3

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4775:
--

 Summary: Add a version of check_zookeeper that works with Python 3
 Key: ZOOKEEPER-4775
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4775
 Project: ZooKeeper
  Issue Type: Improvement
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

API to get a whole subtree

[Enrico Olivelli]

Hello,
today I was discussing with a friend from the Solr community, they
would need to read a whole subtree in one shot.

I can't remember if we have something like that, do you have any pointers ?

Cheers
Enrico

Re: [VOTE] Apache ZooKeeper release 3.9.1 candidate 0

[Enrico Olivelli]

+1 (binding)

- Built from sources, run all the tests
- Validated checksums and signatures
- Run a single node cluster from the binaries

Enrico

Il giorno mer 4 ott 2023 alle ore 19:12 Damien Diederen
 ha scritto:
>
>
> Greetings, all,
>
> +1 (non-binding)
>
>   * Verified checksum/signature;
>   * Upgraded a 5-node ensemble to apache-zookeeper-3.9.1-bin.tar.gz,
> ran a few smoke tests;
>   * Reviewed the Release Notes;
>   * Ran dependency-check:check on the source tree.
>
> Cheers, -D
>
>
> Andor Molnar  writes:
> > Hi team,
> >
> > This is a release candidate for 3.9.1.
> >
> > This is a bugfix release for the 3.9 release line. Includes important
> > dependency upgrades to address CVEs.
> >
> >
> > The full release notes is available at:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480
> >
> > *** Please download, test and vote by October 6th 2023, 23:59 UTC+0.
> > ***
> >
> > Source files:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachezookeeper-1096/
> >
> > The release candidate tag in git to be voted upon: release-3.9.1-0
> > https://github.com/apache/zookeeper/releases/tag/release-3.9.1-0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Best regards,
> >
> > Andor

Re: TLS 1.3 support

[Enrico Olivelli]

Scott,

Il Lun 25 Set 2023, 17:20 Scott Guminy  ha scritto:

> Hi,
>
> We have a requirement to support TLS 1.3 starting in January.
>
> I noticed there is a pull request for TLS 1.3 support
>  that's been around for a
> long time.
>

I think that we only need one more reviewer.

I agree that we should move forward as soon as possible.

Enrico


> How can we get support for TLS 1.3 in ZooKeeper?
>

[jira] [Created] (ZOOKEEPER-4735) set the RMI port to address issues with monitoring Zookeeper running in containers

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4735:
--

 Summary: set the RMI port to address issues with monitoring 
Zookeeper running in containers
 Key: ZOOKEEPER-4735
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4735
 Project: ZooKeeper
  Issue Type: Improvement
  Components: server
Reporter: Enrico Olivelli
 Fix For: 3.10.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Apache ZooKeeper 3.9.0

[Enrico Olivelli]

Congratulations !

This is great step forward

I hope that people will try out soon the Backup/Restore feature and
that they provide feedback

Enrico

Il giorno ven 4 ago 2023 alle ore 13:24 Andor Molnar
 ha scritto:
>
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.9.0
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.9.0 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.9.0/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>
>

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 1

[Enrico Olivelli]

+1 (binding)

- Verified signatures and checksums
- Built  and run all the tests on JDK-17
- Run some smoke tests
- Run tests on some user application


Thanks
Enrico

Il giorno gio 27 lug 2023 alle ore 12:39 Mohammad Arshad
 ha scritto:
>
> +1 (non-binding)
> verified signature -ok
> verified checksum -ok
> run rat,checkstyle and spotbugs -ok
> run all java test cases -ok
> build and installed 3 node cluster, executed few cli commands -ok
>
> -Arshad
>
> On Wed, Jul 26, 2023 at 5:17 AM Patrick Hunt  wrote:
>
> > +1 - xsum/sig verified, rat ran clean, I was able to compile, run the owasp
> > checker, and start various ensemble sizes manually w/o issue. lgtm.
> >
> > Patrick
> >
> > On Wed, Jul 19, 2023 at 2:20 AM Andor Molnar  wrote:
> >
> > > This is release candidate for ZooKeeper 3.9.0.
> > >
> > > It is a major release and it introduces a lot of new features, most
> > > notably:
> > > - Admin server API for taking snapshot and stream out the data
> > > - Communicate the Zxid that triggered a WatchEvent to fire
> > > - TLS - dynamic loading for client trust/key store
> > > - Add Netty-TcNative OpenSSL Support
> > > - Adding SSL support to Zktreeutil
> > > - Improve syncRequestProcessor performance
> > > - Updates to all the third party dependencies to get rid of every known
> > > CVE.
> > >
> > > The full release notes is available at:
> > >
> > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > >
> > > *** Please download, test and vote by July 30th 2023, 23:59 UTC+0. ***
> > >
> > > Source files:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/
> > >
> > > Maven staging repo:
> > >
> > >
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > >
> > > The release candidate tag in git to be voted upon: release-3.9.0-1
> > > https://github.com/apache/zookeeper/tree/release-3.9.0-1
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
> > >
> >

Re: Review request for ZOOKEEPER-4409?

[Enrico Olivelli]

Hello

Il Dom 25 Giu 2023, 10:00 Yong-Hao Zou  ha scritto:

> Hi,
>
> I wonder if someone can continue the review of ZOOKEEPER-4409 which
> has been stuck for a while.
>

I have reviewers the PR

>
> Besides, is it possible to request a CVE for this issue, which would
> benefit my research greatly.
>

I don't think that this bug deserves a CVE. In that case I suggest you to
contact secur...@zookeeper.apache.org explaining your reasons.
Disclosing publicly a security issue without a fix/workaround may generate
confusion

Enrico

>
> https://github.com/apache/zookeeper/pull/1774
>
> Best regards,
> Yonghao
>

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Enrico Olivelli]

For what it's worth, all my tests passed today :-)

I would have cast a +1.
But I agree that we should fix the problem reported by Mate

Enrico

Il giorno mar 18 lug 2023 alle ore 14:48 Szalay-Bekő Máté
 ha scritto:
>
> Thanks Andor!
>
> > Since we don't have better idea, I opened a pull request to upgrade OWASP
> to the latest (8.3.1) version.
>
> Yes, I tested the PR, upgrading OWASP is solving the issue also on my
> machine. I approved the PR.
>
> Regards,
> Máté
>
> On Tue, Jul 18, 2023 at 1:45 PM Andor Molnar  wrote:
>
> > Hi Mate,
> >
> > I take your e-mail as a -1 vote, so this RC VOTE is CANCELLED.
> > I'll prepare another rc.
> >
> > Regards,
> > Andor
> >
> >
> > On Mon, 2023-07-17 at 22:50 +0200, Szalay-Bekő Máté wrote:
> > > Hello Andor!
> > >
> > > Thanks for this great release!
> > >
> > > I found two issues with RC0:
> > >
> > > 1) OWASP CVE check (mvn dependency-check:check) failed with
> > > "netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:
> > > CVE-2011-1797(9.3)"
> > >
> > > This seems to be a false positive to me (looks to be some security
> > > issue
> > > affecting old safari / chromium web browser versions?). I didn't get
> > > deep
> > > into this, but I guess we see this since
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> > >
> > > Interestingly, the CI pipeline doesn't catch this CVE (
> > >
> > https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/
> > ),
> > > maybe this is some bug in OWASP that is triggered only with certain
> > > maven
> > > versions or during building on certain platforms? I ran OWASP on
> > > Ubuntu
> > > 18.04.2 with maven 3.9.3.
> > >
> > > 2) Also I see that the website (
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
> > )
> > > is still showing "ZooKeeper 3.8 Documentation" on the top
> > >
> > >
> > > What do you think? We shouldn't pass the RC until we are certain
> > > about the
> > > CVE issue. (unless this is something happening only on my setup... it
> > > is
> > > strange that OWAPS is green on CI)
> > >
> > >
> > > Beside these, I ran all my usual RC test steps, and found no other
> > > issues
> > > with the RC:
> > > - verified checksum and gpg signature of the artifacts
> > > - I built the source code (incl. the C-client, using -Pfull-build) on
> > > Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> > > - all the unit tests passed (both Java and C-client)
> > > - I also built and executed unit tests for zkpython
> > > - I also built the java code (without -Pfull-build) using other JDK
> > > versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this
> > > time, just
> > > used 'clean install -DskipTests')
> > > - checkstyle and spotbugs passed
> > > - apache-rat passed
> > > - fatjar built
> > > - I executed quick rolling-upgrade tests (using
> > > https://github.com/symat/zk-rolling-upgrade-test):
> > >  - rolling upgrade from 3.5.10 to 3.9.0
> > >  - rolling upgrade from 3.6.4 to 3.9.0
> > >  - rolling upgrade from 3.7.1 to 3.9.0
> > >  - rolling upgrade from 3.8.2 to 3.9.0
> > > - compared generated release notes (
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/releasenotes.html
> > > ) with Jira (
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > > )
> > >
> > >
> > > Best regards,
> > > Máté
> > >
> > > On Mon, Jul 17, 2023 at 3:11 PM Andor Molnar 
> > > wrote:
> > >
> > > > Hi team,
> > > >
> > > > This is a release candidate for 3.9.0.
> > > >
> > > > It is a major release and it introduces a lot of new features, most
> > > > notably:
> > > > - Admin server API for taking snapshot and stream out the data
> > > > - Communicate the Zxid that triggered a WatchEvent to fire
> > > > - TLS - dynamic loading for client trust/key store
> > > > - Add Netty-TcNative OpenSSL Support
> > > > - Adding SSL support to Zktreeutil
> > > > - Improve syncRequestProcessor performance
> > > > - Updates to all the third party dependencies to get rid of every
> > > > known
> > > > CVE.
> > > >
> > > > The full release notes is available at:
> > > >
> > > >
> > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > > >
> > > > *** Please download, test and vote by July 30th 2023, 23:59 UTC+0.
> > > > ***
> > > >
> > > > Source files:
> > > >
> > > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/
> > > >
> > > > Maven staging repo:
> > > >
> > > >
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.8.0-1
> > > > https://github.com/apache/zookeeper/tree/release-3.9.0-0
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > > > release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > 

Re: [ANNOUNCE] Apache ZooKeeper 3.8.2

[Enrico Olivelli]

Congratulations !

Thank you Mate for driving the release

Enrico

Il giorno mar 18 lug 2023 alle ore 17:19 Szalay-Bekő Máté
 ha scritto:
>
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.8.2
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> Release 3.8.2 is a bugfix release, solving 12 issues, including CVE fixes
> and
> additional test, security and other improvements.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.8.2 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.8.2/releasenotes.html
>
> We would like to thank the contributors who made the release possible.
>
> Regards,
> The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Enrico Olivelli]

+1 (binding)

- verified checksums and signatures
- built from the sources
- run all the tests on JDK11 on Mac M1
- run some application/service (and all the unit/integration tests)
thats uses ZooKeeper (HerdDB.org + Apache BookKeeper) using the
binaries that I have built locally


Thank you for driving the release

Enrico

Il giorno mer 5 lug 2023 alle ore 23:22 Szalay-Bekő Máté
 ha scritto:
>
> +1 (binding)
>
> I did the following tests for the release candidate:
> - verified checksum and gpg signature of the artifacts
> - I built the source code (incl. the C-client, using -Pfull-build) on
> Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> - all the unit tests passed (both Java and C-client)
> - I also built and executed unit tests for zkpython
> - I also built the java code (without -Pfull-build) using other JDK
> versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this time, just
> used 'clean install -DskipTests')
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - fatjar built
> - I executed quick rolling-upgrade tests (using
> https://github.com/symat/zk-rolling-upgrade-test):
> - rolling upgrade from 3.5.10 to 3.8.2
> - rolling upgrade from 3.6.4 to 3.8.2
> - rolling upgrade from 3.7.1 to 3.8.2
> - rolling upgrade from 3.8.1 to 3.8.1
> - checked the generated documentation (zookeeper-docs/target/html)
> - compared generated release notes (
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/releasenotes.html)
> with Jira (
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> )
>
> Best regards,
> Máté
>
> On Wed, Jul 5, 2023 at 11:20 PM Szalay-Bekő Máté 
> wrote:
>
> > This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> > including CVE fixes and additional test, security and other improvements.
> >
> > Please find the full release notes in the following link:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> >
> > *** Please download, test and vote by July 14th 2023, 23:59 UTC+0. ***
> >
> >
> > Source files:
> > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
> >
> > Maven staging repo:
> >
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
> >
> > The release candidate tag in git to be voted upon: release-3.8.2-0
> > (please note, branch-3.8.2 will move here only after the vote)
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
> >
> >
> > Should we release this candidate?
> >
> >
> > Best regards,
> > Máté
> >

Re: ZooKeeper release 3.9.0

[Enrico Olivelli]

Sirius,

Il giorno ven 16 giu 2023 alle ore 15:09 Yang Sirius
 ha scritto:
>
> Hi Andor,
>
> I am writing to inquire about the possibility to include the TLA+ 
> specifications for ZooKeeper in the upcoming release, version 3.9.0. The 
> proposal to provide TLA+ specifications for ZooKeeper was raised in 
> ZOOKEEPER-3615, and 
> addressed via pull request 
> #1690 on github.
>
> Formal specifications can serve as precise documentation of the Zab design 
> and implementation, and can help eliminate any ambiguities in the informal 
> protocol description, which would be beneficial for ZooKeeper learners and 
> developers. Popular consensus protocols like Paxos and Raft also provide 
> their TLA+ specifications. It would be great to merge the pull request and 
> include the TLA+ specifications for ZooKeeper in the new version.
>
> I have also raised a request for the review of pull request 
> #1690 to the ZooKeeper 
> developer mailing list. More details can be found at 
> https://lists.apache.org/thread/ww4v1r733whcds64jg5wt7ozclbjhdr0 .
>
> Looking forward to your feedback!

I would like to commit that patch, but unfortunately there is an open
discussion and we need some reviewers to formally approve it.

That said, that patch is mostly about "documentation" and it can be
committed after the 3.9.0 release.

Enrico

>
> Best regards,
>
> Sirius
>
>
> 2023年6月15日 19:57，Andor Molnar mailto:an...@apache.org>> 写道：
>
> Hi folks,
>
> There're 64 open tickets which has fixVersion = 3.9.0
> I'll remove the fixVersion from all of them except the ones that we
> marked as release blockers.
>
> Currently:
>
> - ZOOKEEPER-4393 Problem to connect to zookeeper in FIPS mode
> - ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> - ZOOKEEPER-4655 Communicate the Zxid that triggered a WatchEvent to
> fire
>
> Please let me know if you would like to add anything to this list.
>
> Regards,
> Andor
>
>
>
>

Current master branch is broker

[Enrico Olivelli]

Hello ZooKeepers,

Current master branch is broken, see the errors below.

Maybe we committed patches and there were some unexpected conflicts
undetected by git.

Enrico

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
(default-testCompile) on project zookeeper: Compilation failure:
Compilation failure:

[ERROR] 
/Users/enricoolivelli/dev/zookeeper/zookeeper-server/src/test/java/org/apache/zookeeper/test/PersistentRecursiveWatcherTest.java:[198,13]
no suitable method found for
assertEvent(java.util.concurrent.BlockingQueue,org.apache.zookeeper.Watcher.Event.EventType,java.lang.String)

[ERROR] method
org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEvent(java.util.concurrent.BlockingQueue,org.apache.zookeeper.Watcher.Event.EventType,java.lang.String,org.apache.zookeeper.data.Stat)
is not applicable

[ERROR]   (actual and formal argument lists differ in length)

[ERROR] method
org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEvent(java.util.concurrent.BlockingQueue,org.apache.zookeeper.Watcher.Event.EventType,java.lang.String,long)
is not applicable

[ERROR]   (actual and formal argument lists differ in length)

[ERROR] method
org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEvent(java.util.concurrent.BlockingQueue,org.apache.zookeeper.Watcher.Event.EventType,org.apache.zookeeper.Watcher.Event.KeeperState,java.lang.String,long)
is not applicable

Re: Name of sequence node is not unique

[Enrico Olivelli]

 could take a
> > look at ZOOKEEPER-4332[4]. ZooKeeper does not work well with massive
> > children when listing. BookKeeper's HierarchicalLedgerManager[5] is a
> > real world example for this.
> >
> >
> > > New
> > > ===
> > > if (parentCVersion > currentParentCVersion
> > > *|| parentCVersion == Integer.MIN_VALUE &&
> > > currentParentCVersion == Integer.MAX_VALUE) *{
> > > parent.stat.setCversion(parent
> > > CVersion);
> > > parent.stat.setPzxid(zxid);
> > >   }
> >
> > This breaks "monotonically increasing" and gains no "uniqueness". The
> > cversion will wrap around again given your cases.
> >
> > > It would be better to use a longer integer.
> > >   1.  Increasing to a 64-bit counter certainly solves the problem, but
> > this might require conversion of zk data when the current counter is stored
> > as 32-bit
> >
> > I support this, but it demands massive work and probably relates to a
> > long term goal[6].  The "int" fact of "cversion" is exposed both in
> > API(Stat) and storage(StatPersisted).
> >
> >
> > [1]:
> > https://github.com/kezhuw/zookeeper/commit/755b1168156c28e4fc2813be593ac67514e8bdc7#diff-1af986ce48b5d4bb4b8e51374a70cc6e109a04c70d9f450be3df8f302010341cR59
> > [2]:
> > https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java#L675
> > [3]:
> > https://zookeeper.apache.org/doc/r3.8.1/zookeeperProgrammers.html#Sequence+Nodes+--+Unique+Naming
> > [4]: https://issues.apache.org/jira/browse/ZOOKEEPER-4332
> > [5]:
> > https://bookkeeper.apache.org/docs/getting-started/concepts/#hierarchical-ledger-manager
> > [6]: https://issues.apache.org/jira/browse/ZOOKEEPER-102
> >
> > Kezhu Wang
> >
> >
> > On Fri, Jun 16, 2023 at 11:33 AM Josef Roehrl
> >  wrote:
> > >
> > > I wanted to add 2 things.
> > >
> > >
> > >   1.  Increasing to a 64-bit counter certainly solves the problem, but
> > this might require conversion of zk data when the current counter is stored
> > as 32-bit
> > >   2.  A client that relies on a unique version that it uses as a
> > reference outside of zk should verify that a version that it receives does
> > not already exist outside zk. This applies even if 1. is considered, should
> > a zk quorum be reset or lose its data.
> > >
> > > Josef Roehrl
> > > FuseForward | Senior Architect - Professional Services
> > > [
> > https://fuseforward.atlassian.net/wiki/download/attachments/512327681/image001.png?version=1=1537397840684=1=v2
> > ]
> > > Website<
> > https://fuseforward.com/?utm_source=Email%20Signature_medium=email%20signature_campaign=email%20signature>
> > | Newsletter<
> > https://fuseforward.com/subscribe-to-our-newsletter/?utm_source=Email%20Signature_medium=Email%20Signature_campaign=Email%20Signature>
> > | Twitter<https://twitter.com/fuseforward> | LinkedIn<
> > https://www.linkedin.com/company/fuseforward/?originalSubdomain=ca>
> > >
> > > 
> > > From: Ted Dunning 
> > > Sent: Thursday, June 15, 2023 6:53 PM
> > > To: dev@zookeeper.apache.org 
> > > Subject: Re: Name of sequence node is not unique
> > >
> > > The invariant is that the value should be increasing except in failure
> > > modes. You found a somewhat surprising failure mode.
> > >
> > > Please compute how long it would take for a 64-bit counter to overflow if
> > > incremented a million times per second. (hint, half a million years).
> > > Remember that zk only does things at less than 100,000 per second
> > >
> > > On Thu, Jun 15, 2023, 17:03 Li Wang  wrote:
> > >
> > > > Thanks a lot for your inputs, Ted.
> > > >
> > > > On Thu, Jun 15, 2023 at 2:52 PM Ted Dunning 
> > wrote:
> > > >
> > > > > Breaking a semantic invariant is a dangerous solution here.
> > > >
> > > > Totally agree. We should not break a semantic invariant if there is
> > one.
> > > > What's the semantic invariant here and how ZK is supposed to behave in
> > the
> > > > overflow case?
> > > >
> > > >
> > > > > It would be better to use a longer integer.
> > > > >
> > > > Yes, I thought about this too. Lo

Re: Netty native libraries in ZooKeeper

[Enrico Olivelli]

I may be wrong but Epoll is only a Linux thing probably.

You don't have it on Mac or Windows

Enrico

Il Gio 15 Giu 2023, 19:49 Andor Molnar  ha scritto:

> Interesting that with only the BOM included and the dependencies
> without the classifier, Netty doesn't load the native epoll selector,
> but loads the native SSL library.
>
> I'm confused.
>
> In the Pulsar example the sub-project dependency has a hardcoded
> classifier:
>
> 
>   io.netty
>   netty-transport-native-epoll
>   linux-x86_64
> 
>
> Andor
>
>
>
> On Thu, 2023-06-15 at 14:31 +0200, Enrico Olivelli wrote:
> > I think that the best way currently is to add these dependencies:
> >
> > Import the Netty BOM in the main pom.xml
> > 
> >   io.netty
> >   netty-bom
> >   ${netty.version}
> >   pom
> >   import
> > 
> >
> >
> > declare netty dependencies without setting the version and the
> > classifier
> > 
> >   io.netty
> >   netty-tcnative-boringssl-static
> > 
> >
> > This is the way we are doing it in Pulsar
> >
> https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pom.xml#L647
> >
> https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pulsar-common/pom.xml#L146
> >
> > This way Maven should bundle all the native libraries for all the
> > supported platforms
> >
> >
> >
> > Enrico
> >
> > Il giorno gio 15 giu 2023 alle ore 12:50 Andor Molnar
> >  ha scritto:
> > > Hi,
> > >
> > > I've come across the following when working on the support of
> > > native
> > > SSL libraries. Currently ZooKeeper supports loading the native
> > > epoll-
> > > based event loop of Netty, but a build profile which would download
> > > the
> > > required dependencies is not shipped with our product.
> > >
> > > This is perfectly okay since the feature of using native libraries
> > > is
> > > not a build-time requirement, but in this case the user has to
> > > download
> > > the required and appropriate versions of Netty jars and put them on
> > > the
> > > classpath.
> > >
> > > Shall we add a Maven build profile to ease this process?
> > >
> > > 
> > >   netty-native
> > >   
> > > fedora > > erWi
> > > thLikes>
> > >   
> > >   
> > > 
> > >   
> > > io.netty
> > > netty-tcnative-boringssl-static
> > > ${netty-tcnative.version}
> > > ${os.detected.classifier}
> > >   
> > >   
> > > io.netty
> > > netty-transport-native-epoll
> > > ${netty.version}
> > > ${os.detected.classifier}
> > >   
> > > 
> > >   
> > > 
> > >
> > > What do you think?
> > >
> > > Andor
> > >
> > >
> > >
>
>

Re: Name of sequence node is not unique

[Enrico Olivelli]

Li,
thanks for reporting your problem.

Most likely you have found a bug.

I have one question, related to your use case,
is the problem that the numbers are not "unique" or that the number is
not monotonically increasing ?

Do you have 2147483647 concurrent sessions and you found that two
sessions got the same sequenceId ?
or are you storing the sequenceId somewhere and you use it as a
globally unique id, not only among the connected sessions but also
among all the sessions that are ever connected to the cluster ?

Enrico

Il giorno ven 9 giu 2023 alle ore 21:10 Li Wang  ha scritto:
>
> Hello,
>
> We are running 3.7.1 in production and running into an "issue" that the
> names of sequence nodes are not unique after the counter hits the max int
> (i.e 2147483647) and overflows.  I would like to start a thread to discuss
> the following
>
> 1. Is this a bug or "expected" behavior?
> 2. Is ZK supposed to support the overflow scenario and need to make sure
> the name is unique when overflow happens?
>
> The name is not unique after hitting the max int value because of we
> have the following in zk  code base:
>
> 1.  The cversion of parent znode is used to build the child name in
> PrepRequestProcessor
>
> int parentCVersion = parentRecord.stat.getCversion();
> if (createMode.isSequential()) {
> path = path + String.format(Locale.ENGLISH, "%010d",
> parentCVersion);
> }
>
>
> https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/
> java/org/apache/zookeeper/server/PrepRequestProcessor.java#L668-L671
>
>
> 2. The parent znode is read from either zks.outstandingChangesForPath map
> or zk database/datatree.
>
>lastChange = zks.outstandingChangesForPath.get(path);
> if (lastChange == null) {
> DataNode n = zks.getZKDatabase().getNode(path);
>
>
> https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java#L168-L170
>
>
>
> 3. The cversion of the parent node in outstandingChangesForPath map is
> always updated  but not in zk database as we added the following code in 3.6
>
> if (parentCVersion > parent.stat.getCversion()) {
> parent.stat.setCversion(parentCVersion);
> parent.stat.setPzxid(zxid);
> }
>
> https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java#L477-L480
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-3249
>
>
> When overflow happens, the new parentCversion is changed to -2147483648.
> It's updated in the outstandingChangesForPath map. It's not updated in
> DataTree and the value stays as 2147483647  because -2147483648 is less
> than 2147483647, so the cVerson is inconsistent in  ZK.
>
> Due to the inconsistent cVersion, when the next request comes in after
> overflow, the sequence number is non-deterministic and not unique depending
> on where the parent node is read from.  It can be 2147483647 if the
> parent node is read from DataTree or -2147483648,  -2147483647 and so on if
> it's from the outstandingChangesForPath map.
>
> We have the following doc about unique naming but no info on  "expected"
> behavior after overflow.
>
> Sequence Nodes -- Unique Naming
>
>
> When creating a znode you can also request that ZooKeeper append a
> monotonically increasing counter to the end of path. This counter is unique
> to the parent znode. The counter has a format of %010d -- that is 10 digits
> with 0 (zero) padding (the counter is formatted in this way to simplify
> sorting), i.e. "01". See Queue Recipe for an example use of this
> feature. Note: the counter used to store the next sequence number is a
> signed int (4bytes) maintained by the parent node, the counter will
> overflow when incremented beyond 2147483647 (resulting in a name
> "-2147483648").
>
>
>
> Please let me know if you have any comments or inputs.
>
>
> Thanks,
>
>
> Li

Re: Review request for ZOOKEEPER-4471 and ZOOKEEPER-4472

[Enrico Olivelli]

Your patches have been merged to master branch and they will be
shipped with 3.9.0.

Thank you
Enrico

Il giorno sab 10 giu 2023 alle ore 18:13 Kezhu Wang 
ha scritto:
>
> Hi all and committers,
>
> [ZOOKEEPER-4471][1] reported that `AddWatchMode.PERSISTENT` could be
> partially removed by `WatcherType.Data` or `WatcherType.Children`.
>
> [ZOOKEEPER-4472][2] proposed to add `WatcherType.Persistent` and
> `WatcherType.PersistentRecursive` to remove `AddWatchMode.PERSISTENT`
> and `AddWatchMode.PERSISTENT_RECURSIVE` respectively.
>
> [ZOOKEEPER-4466][3] rescues us from conflict among different watch
> types on the same path. So, clients can watch whatever paths in
> whatever modes. They will not ruin each other.
>
> On the other hand, [ZOOKEEPER-4472][2] complements the removing part
> of [ZOOKEEPER-4466][3]. If a client wants to remove a persistent
> watch, it will issue a `removeWatches` `WatcherType.Persistent`. This
> will not affect any other watcher types. Currently, clients have to
> resort to `WatcherType.Any` to remove them. This could potentially
> affect other ongoing watcher types.
>
> Ideally, ZOOKEEPER-4472 is independent of ZOOKEEPER-4471, but it would
> be hard to ship only ZOOKEEPER-4472 without a fix to ZOOKEEPER-4471.
> That will require particular attention to avoid trigger paths for
> ZOOKEEPER-4471. That is why I delayed it until now.
>
> I saw discussions about [cut release for 3.9][4]. I really hope we can
> merge [pr#1998][5] for ZOOKEEPER-4471 and [pr#2006][6] for
> ZOOKEEPER-4472 in 3.9.0. It would be impossible for us to merge
> ZOOKEEPER-4472 to patch versions of 3.9 series as it touches both
> server logic and api side while ZOOKEEPER-4466 is shipped into 3.9. It
> looks weird if we support different watcher types one same path in
> watching but not all of them in removing. That is why I hope we can
> ship them along with ZOOKEEPER-4466 in 3.9.0.
>
> I plan to reply with a short message about this possibility in the
> release discussion thread to ref back this mail. Hope it won't bore
> you in there or here.
>
> Anyway, please take your time to review pr#1998 and pr#2006 no matter
> whether they will be included in 3.9.0.
>
> Last, if you are curious how ZOOKEEPER-4472 could affect a real
> program. You can take a look at [try_remove_watcher][7] and
> [dispatch_path_event][8] in [zookeeper-rust-client][9]. The
> `try_remove_watcher` ignores dropping of persistent watches if there
> are other watches remaining. But this will leak persistent watches on
> the server side. `dispatch_path_event` handles this, it will issue
> `removeWatches` with `WatcherType.Any` if it receives events to a path
> with no watchers. For no persistent watches, the client will issue
> corresponding `WatcherType`s if there are more watchers with them. But
> it can't do the same for persistent watches due to leak of
> `WatcherType.Persistent` and `WatcherType.PersistentRecursive`. Hope
> this convinces you of  ZOOKEEPER-4472.
>
> [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> [3]: https://issues.apache.org/jira/browse/ZOOKEEPER-4466
> [4]: https://lists.apache.org/thread/dgnt1xnlf5n9tzt7m4otbodg2qdx3fz4
> [5]: https://github.com/apache/zookeeper/pull/1998
> [6]: https://github.com/apache/zookeeper/pull/2006
> [7]: 
> https://github.com/kezhuw/zookeeper-client-rust/blob/7466e867fb1b229d6b6ffb230e2e682c49ff494e/src/session/watch.rs#L368-L388
> [8]: 
> https://github.com/kezhuw/zookeeper-client-rust/blob/7466e867fb1b229d6b6ffb230e2e682c49ff494e/src/session/watch.rs#L341-L366
> [9]: https://github.com/kezhuw/zookeeper-client-rust
>
> Best,
> Kezhu Wang

Re: planning release 3.8.2

[Enrico Olivelli]

Thanks for volounteering

Il giorno mer 14 giu 2023 alle ore 14:31 Szalay-Bekő Máté
 ha scritto:
>
> Hello ZooKeepers,
>
> release 3.8.1 happened this January and I volunteered to do 3.8.2 soon,
> maybe in June or early July. Let me know if someone would also like to do
> it, I am happy to hand it over! :)
>
> Of course we need to do all the 3pp CVE / vulnerability fixes first.
> And Andor already started a thread about a FIPS TLS improvement which we
> should wait for.
>
> Does anyone know about any other open ticket we should wait for in 3.8.2?

Nothing that I am aware of.

Best regards
Enrico

>
> Best regards,
> Máté

Re: FIPS: removing ZKTrustManager

[Enrico Olivelli]

Il giorno mer 14 giu 2023 alle ore 13:43 Andor Molnar
 ha scritto:
>
> PR has been created with the proposed resolution:
>
> https://github.com/apache/zookeeper/pull/2008


Committed to master and branch-3.8

Thank you
Enrico

> Please review.
>
> Thanks,
> Andor
>
>
>
> On Sat, 2023-06-10 at 11:25 +0200, Andor Molnar wrote:
> > "we use this method dozens of other places in the code"
> >
> > Checked. Mostly logging and output formatting like 4lws, etc.
> >
> >
> >
> > On Sat, 2023-06-10 at 11:18 +0200, Andor Molnar wrote:
> > > First, I've created a pull request for ZOOKEEPER-3860:
> > >
> > > https://github.com/apache/zookeeper/pull/2005
> > >
> > > To improve the logging in ZKTrustManager without altering the
> > > behaviour. The patch also contains a change in
> > > NetUtils.formatInetAddr() which, I believe, should use the hostname
> > > when creating textual representation of an InetAddress. I'm not
> > > 100%
> > > sure about this, because while it certainly helps in TLS cases to
> > > avoid
> > > unnecessary reverse DNS lookups, we use this method dozens of other
> > > places in the code. Unit tests are passsing.
> > >
> > > ZOOKEEPER-4268
> > >
> > > It's about reverse lookups in the client code, but I haven't found
> > > the
> > > reported behaviour on latest master, so just closed the ticket.
> > >
> > > Andor
> > >
> > >
> > >
> > > On Fri, 2023-06-09 at 18:29 +0200, Szalay-Bekő Máté wrote:
> > > > yeah, I remember these tickets, thanks for picking them up!
> > > > I agree and like the solution you proposed, in general in the
> > > > long
> > > > term it
> > > > is good not to use a custom trust manager, but rely on the
> > > > standard
> > > > one.
> > > >
> > > > Máté
> > > >
> > > >
> > > > On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli <
> > > > eolive...@gmail.com
> > > > wrote:
> > > >
> > > > > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> > > > >  ha scritto:
> > > > > > I'd like to backport this to the 3.8 branch too.
> > > > > >
> > > > > > Let's say I'll add new "zookeeper.fips-mode" parameter which
> > > > > > will
> > > > > > be
> > > > > > "false" by default in 3.8 and "true" for 3.9.0.
> > > > >
> > > > > I am +1
> > > > > ZK 3.9 will take time to be adopted and this is an important
> > > > > security
> > > > > related topic
> > > > >
> > > > > Enrico
> > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Andor
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > > > > I think that switching to
> > > > > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
> > > > > > > is
> > > > > > > a
> > > > > > > good
> > > > > > > option.
> > > > > > > The less tweaks we have about Security code the better.
> > > > > > >
> > > > > > >
> > > > > > > It would be great to see this in 3.9.0.
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > > > > > >  ha scritto:
> > > > > > > > Hi zk folks,
> > > > > > > >
> > > > > > > > Problem(s)
> > > > > > > > ==
> > > > > > > >
> > > > > > > > One problem that we're having with a custom Trust Manager
> > > > > > > > in
> > > > > > > > ZK is
> > > > > > > > that
> > > > > > > > FIPS doesn't allow that:
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > > > > > >
> > > > > > > > In FIPS mode the only allowed 

Re: JDK 21 is in Rampdown / The importance of testing with Early-Access Builds

[Enrico Olivelli]

This is our Jenkins job
https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/

The latest build passed.

We are using openjdk-21-ea+21 on Linux.

it seems that it is not the very latest EA (is it +26 ?)

>From the output of the jenkins job run:

[ZooKeeper-Java-EA] $ /home/jenkins/tools/maven/apache-maven-3.9.2/bin/mvn -v
Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
Java version: 21-ea, vendor: Oracle Corporation, runtime:
/usr/local/asfpackages/java/openjdk-21-ea+21
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-206-generic", arch: "amd64", family: "unix"

Enrico

Il giorno mer 14 giu 2023 alle ore 10:44 David Delabassee
 ha scritto:
>
> Welcome to the OpenJDK Quality Outreach June update.
>
> JDK 21 has entered Rampdown Phase One (RDP1) [1], which means that the 
> main-line has been forked into a dedicated JDK 21 stabilization repository. 
> At this point, the overall JDK 21 feature set is frozen. Any changes pushed 
> to the main line are now bound for JDK 22. The stabilization repository is 
> open for select bug fixes and, with approval, late low-risk enhancements per 
> the JDK Release Process [2]. And in contrast to past practice, most 
> stabilization changes will be integrated via backports from the main-line 
> repository [1].
>
> The coming weeks are critical to identify and resolve as many issues as 
> possible, i.e. before JDK 21 enters the Release Candidates phase in August. 
> We need to make sure those few weeks are leveraged to test both existing code 
> running on top of JDK 21 and new JDK 21 features. The heads-up below 
> illustrates the importance and the benefits of doing such tests.
>
> [1] https://mail.openjdk.org/pipermail/jdk-dev/2023-June/007911.html
> [2] https://openjdk.org/jeps/3#Integrating-fixes-and-enhancements
>
>
> ## Heads-up: On the Importance of Doing Tests With OpenJDK Early-Access Builds
>
> The following is a recent example that demonstrates the benefits of testing 
> an existing codebase using the OpenJDK early-access builds.
>
> Last month, we published a heads-up focused on Sequenced Collections [3] as 
> they could potentially introduce some incompatibilities.
> The Eclipse Collections (EC) team did their homework and sure enough, EC was 
> impacted as it was now throwing compilation errors with JDK 21 early-access 
> builds. The EC team was able to quickly fix those compilation errors, i.e., 
> it was mainly about adding overriding default methods. But once those 
> compilation errors were fixed, and this is where it gets interesting, another 
> issue surfaced. This time, the problem was related to LinkedHashMap 
> serialization. After some investigation, the EC team identified that second 
> issue as JDK one and a JBS ticket was opened. That issue was then confirmed 
> as a JDK regression and was promptly fixed in OpenJDK main-line, i.e., JDK 
> 22. The fix was then backported into the JDK 21 stabilization repository. 
> This EC pull request [4] provides additional details.
> In this case, the JDK fix was easy but it is nevertheless the kind of issues 
> that could have easily fallen through the crack if the EC team wasn’t 
> pro-actively testing with OpenJDK early-access builds. The EC issue would 
> have then surfaced after the JDK 21 General Availability... and who knows 
> when the JDK LinkedHashMap serialization regression would have been fixed?
> TL; DR; Testing an existing codebase with OpenJDK early-access builds is a 
> win-win situation. It helps the project itself, Eclipse Collections in this 
> case, as it enables developers to identify issues in their own codebase 
> before that new JDK version is Generally Available. It helps the JDK too as 
> any JDK issue detected early enough in the development cycle gives the 
> OpenJDK engineers a chance to address it before the General Availability of 
> that new JDK version. And last but not least, having a robust JDK is also a 
> win for the Java community at large.
>
> And thanks to the Eclipse Collections team and especially to Don Raab for 
> helping to make the Java platform more robust!
>
> [3] https://inside.java/2023/05/12/quality-heads-up/
> [4] https://github.com/eclipse/eclipse-collections/pull/1461
>
>
> ## JDK 21 Early-Access Builds
>
> JDK 21 Early-Access builds 26 are now available [5], and are provided under 
> the GNU General Public License v2, with the Classpath Exception. The Release 
> Notes are available here [6] and the javadocs here [7].
>
> ### JEPs integrated into JDK 21:
> - 430: String Templates (Preview)
> - 431: Sequenced Collections
> - 439: Generational ZGC
> - 440: Record Patterns
> - 441: Pattern Matching for switch
> - 442: Foreign Function & Memory API (3rd Preview)
> - 443: Unnamed Patterns and Variables (Preview)
> - 444: Virtual Threads
> - 445: Unnamed Classes and Instance Main Methods (Preview)
> - 446: Scoped Values (Preview)
> - 448: Vector 

Re: Netty native libraries in ZooKeeper

[Enrico Olivelli]

I think that the best way currently is to add these dependencies:

Import the Netty BOM in the main pom.xml

  io.netty
  netty-bom
  ${netty.version}
  pom
  import



declare netty dependencies without setting the version and the classifier

  io.netty
  netty-tcnative-boringssl-static


This is the way we are doing it in Pulsar
https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pom.xml#L647
https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pulsar-common/pom.xml#L146

This way Maven should bundle all the native libraries for all the
supported platforms



Enrico

Il giorno gio 15 giu 2023 alle ore 12:50 Andor Molnar
 ha scritto:
>
> Hi,
>
> I've come across the following when working on the support of native
> SSL libraries. Currently ZooKeeper supports loading the native epoll-
> based event loop of Netty, but a build profile which would download the
> required dependencies is not shipped with our product.
>
> This is perfectly okay since the feature of using native libraries is
> not a build-time requirement, but in this case the user has to download
> the required and appropriate versions of Netty jars and put them on the
> classpath.
>
> Shall we add a Maven build profile to ease this process?
>
> 
>   netty-native
>   
> fedora thLikes>
>   
>   
> 
>   
> io.netty
> netty-tcnative-boringssl-static
> ${netty-tcnative.version}
> ${os.detected.classifier}
>   
>   
> io.netty
> netty-transport-native-epoll
> ${netty.version}
> ${os.detected.classifier}
>   
> 
>   
> 
>
> What do you think?
>
> Andor
>
>
>

Re: ZooKeeper release 3.9.0

[Enrico Olivelli]

Il giorno gio 15 giu 2023 alle ore 13:58 Andor Molnar
 ha scritto:
>
> Hi folks,
>
> There're 64 open tickets which has fixVersion = 3.9.0
> I'll remove the fixVersion from all of them except the ones that we
> marked as release blockers.

I agree.
It is useless to keep the "fixVersion" on an issue that is not a
blocker for a release.
it adds only noise

Enrico

>
> Currently:
>
> - ZOOKEEPER-4393 Problem to connect to zookeeper in FIPS mode
> - ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> - ZOOKEEPER-4655 Communicate the Zxid that triggered a WatchEvent to
> fire
>
> Please let me know if you would like to add anything to this list.
>
> Regards,
> Andor
>
>
>

Re: Volounteers for releases ?

[Enrico Olivelli]

Il Mer 14 Giu 2023, 08:06 Szalay-Bekő Máté  ha
scritto:

> Hello!
>
> I also might be able to manage a release. I let Andor to do the 3.9.0 ;)
> but I can make a 3.7 or 3.8 release.
>
> Based on our policy, when 3.9.0 will be released, we will deprecate the 3.7
> line. 3.7.1 happened one year ago. Normally I would say it would be good to
> make a last 3.7 release (3.7.2) before we terminate 3.7. But 3.7 and 3.8
> are very similar (we cut 3.8 only for log4j vulnerability). Maybe we can
> deprecate 3.7 without  3.7.2 and I should focus on 3.8.2 instead?
>
> what do you think?
>

Let's focus on 3.8.2.
Users on 3.7 can easily migrate to 3.8


Thanks

Enrico




> Mate
>
> On Tue, Jun 13, 2023 at 10:58 AM Andor Molnar  wrote:
>
> > Awesome. Thanks Enrico!
> >
> > I owe you an apology: found an important TLS ticket which is another
> > low hanging fruit:
> >
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> >
> > ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> >
> > We've already done this for HBase and I always wanted to port this back
> > to ZooKeeper. It's a very cool performance improvement for Linux-based
> > installations (or whereever else Netty has OpenSSL support),
> > unfortunately for ZooKeeper this is only for the server-client stack.
> >
> > Let me land this for 3.9.0.
> >
> > Andor
> >
> >
> > On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> > > Tests on master branch are passing on JDK20
> > >
> > > Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> > > Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> > > Java version: 20, vendor: Eclipse Adoptium, runtime:
> > > /usr/local/asfpackages/java/adoptium-jdk-20+36
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> > > family: "unix"
> > >
> > > https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> > >
> > > Enrico
> > >
> > > Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
> > >  ha scritto:
> > > > Sure. I've just noticed that the patch has been outstanding for a
> > > > year
> > > > now, small and ready to be submitted.
> > > >
> > > > Andor
> > > >
> > > >
> > > >
> > > > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > > > >  ha scritto:
> > > > > > I came across the graceful termination patch yesterday. Sounds
> > > > > > like
> > > > > > important for K8s environments. Enrico, what do you think?
> > > > > > Looks
> > > > > > like
> > > > > > you're not a fan of that.
> > > > > >
> > > > > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > > > > >
> > > > > > https://github.com/apache/zookeeper/pull/1898
> > > > >
> > > > > I have taken a look and I have posted some feedback about
> > > > > renaming
> > > > > the
> > > > > configuration flag.
> > > > > I am not sure that we are in a hurry to commit that patch, we can
> > > > > release it with 3.9.1.
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > > Andor
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > > > > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > > > > > >  ha scritto:
> > > > > > > > Hi Kezhu,
> > > > > > > >
> > > > > > > > Sure, I'll take a look at the open PRs before cutting 3.9.0
> > > > > > > > from
> > > > > > > > master. Let's mark these tickets release blockers as you
> > > > > > > > suggested.
> > > > > > > >
> > > > > > > > Any more blockers of 3.9.0 that anyone knows about?
> > > > > > >
> > > > > > > No, there are no critical issues at the moment.
> > > > > > >
> > > > > > > I will double check on compatibility with the latest 

Re: Volounteers for releases ?

[Enrico Olivelli]

Tests on master branch are passing on JDK20

Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
Java version: 20, vendor: Eclipse Adoptium, runtime:
/usr/local/asfpackages/java/adoptium-jdk-20+36
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-206-generic", arch: "amd64", family: "unix"

https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/

Enrico

Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
 ha scritto:
>
> Sure. I've just noticed that the patch has been outstanding for a year
> now, small and ready to be submitted.
>
> Andor
>
>
>
> On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> >  ha scritto:
> > > I came across the graceful termination patch yesterday. Sounds like
> > > important for K8s environments. Enrico, what do you think? Looks
> > > like
> > > you're not a fan of that.
> > >
> > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > >
> > > https://github.com/apache/zookeeper/pull/1898
> >
> > I have taken a look and I have posted some feedback about renaming
> > the
> > configuration flag.
> > I am not sure that we are in a hurry to commit that patch, we can
> > release it with 3.9.1.
> >
> > Enrico
> >
> >
> > > Andor
> > >
> > >
> > >
> > > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > > >  ha scritto:
> > > > > Hi Kezhu,
> > > > >
> > > > > Sure, I'll take a look at the open PRs before cutting 3.9.0
> > > > > from
> > > > > master. Let's mark these tickets release blockers as you
> > > > > suggested.
> > > > >
> > > > > Any more blockers of 3.9.0 that anyone knows about?
> > > >
> > > > No, there are no critical issues at the moment.
> > > >
> > > > I will double check on compatibility with the latest JDKs, it is
> > > > better that when we cut a new major release
> > > > it works well with the newer JDKs
> > > >
> > > > Enrico
> > > >
> > > >
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > Sorry for the disruption.
> > > > > >
> > > > > > I want to ask if there is any possibility for us to include
> > > > > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > > > >
> > > > > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> > > > > > `WatcherType.PersistentRecursive` to remove
> > > > > > `AddWatchMode.PERSISTENT`
> > > > > > and
> > > > > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > > > > complementary to
> > > > > > [ZOOKEEPER-4466][3] which supports multiple different watches
> > > > > > on
> > > > > > one
> > > > > > path
> > > > > > and was merged to master one month ago. If we don't get it to
> > > > > > 3.9.0,
> > > > > > it is
> > > > > > probably impossible for us to merge it to patch versions of
> > > > > > 3.9
> > > > > > series as
> > > > > > it touches both server logic and api side. It might be
> > > > > > strange to
> > > > > > ship
> > > > > > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to
> > > > > > support
> > > > > > different
> > > > > > watcher types on one path in watching but not all of them in
> > > > > > removing.
> > > > > >
> > > > > > ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver
> > > > > > comprehensive
> > > > > > tests.
> > > > > >
> > > > > > I have already sent a [review request][3] to the dev mailing
> > > > > > list. It
> > > > > > has
> > > > > > more descriptional information and a real world use case
> > &g

Re: Volounteers for releases ?

[Enrico Olivelli]

Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
 ha scritto:
>
> I came across the graceful termination patch yesterday. Sounds like
> important for K8s environments. Enrico, what do you think? Looks like
> you're not a fan of that.
>
> ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
>
> https://github.com/apache/zookeeper/pull/1898

I have taken a look and I have posted some feedback about renaming the
configuration flag.
I am not sure that we are in a hurry to commit that patch, we can
release it with 3.9.1.

Enrico


>
> Andor
>
>
>
> On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> >  ha scritto:
> > > Hi Kezhu,
> > >
> > > Sure, I'll take a look at the open PRs before cutting 3.9.0 from
> > > master. Let's mark these tickets release blockers as you suggested.
> > >
> > > Any more blockers of 3.9.0 that anyone knows about?
> >
> > No, there are no critical issues at the moment.
> >
> > I will double check on compatibility with the latest JDKs, it is
> > better that when we cut a new major release
> > it works well with the newer JDKs
> >
> > Enrico
> >
> >
> > > Andor
> > >
> > >
> > >
> > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > Hi all,
> > > >
> > > > Sorry for the disruption.
> > > >
> > > > I want to ask if there is any possibility for us to include
> > > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > >
> > > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> > > > `WatcherType.PersistentRecursive` to remove
> > > > `AddWatchMode.PERSISTENT`
> > > > and
> > > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > > complementary to
> > > > [ZOOKEEPER-4466][3] which supports multiple different watches on
> > > > one
> > > > path
> > > > and was merged to master one month ago. If we don't get it to
> > > > 3.9.0,
> > > > it is
> > > > probably impossible for us to merge it to patch versions of 3.9
> > > > series as
> > > > it touches both server logic and api side. It might be strange to
> > > > ship
> > > > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to support
> > > > different
> > > > watcher types on one path in watching but not all of them in
> > > > removing.
> > > >
> > > > ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver comprehensive
> > > > tests.
> > > >
> > > > I have already sent a [review request][3] to the dev mailing
> > > > list. It
> > > > has
> > > > more descriptional information and a real world use case
> > > > of  ZOOKEEPER-4472.
> > > >
> > > > Look forward to your feedback in either thread!
> > > >
> > > > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> > > > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> > > > [3]:
> > > > https://lists.apache.org/thread/m7gxcffsnjy2lm8g52nssfxb6t800o3r
> > > >
> > > >
> > > > Best,
> > > > Kezhu Wang
> > > >
> > > >
> > > >
> > > > Best,
> > > > Kezhu Wang
> > > >
> > > >
> > > > On Fri, Jun 9, 2023 at 6:17 PM Andor Molnar 
> > > > wrote:
> > > >
> > > > > Hi Enrico,
> > > > >
> > > > > I can take the master cut next week, but let me put together an
> > > > > email
> > > > > about a TLS topic first. I'd like to propose a fix to resolve
> > > > > the
> > > > > problem of FIPS (custome trust manager in ZK) and reverse DNS
> > > > > lookups.
> > > > > I'd like to include it in 3.9.0 and 3.8.2.
> > > > >
> > > > > Andor
> > > > >
> > > > > p.s. Whoever is making a change on the webpage, please remove
> > > > > the
> > > > > 3.8.0
> > > > > release.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> > > > > > Hello ZooKeepers,
> > > > > > I think that it is time to do a round of releases.
> > > > > >
> > > > > > We should cut a release out of the master branch, 3.9.0 and
> > > > > > main
> > > > > > cutting a release out of 3.7.x and 3.8.x would be useful.
> > > > > >
> > > > > > Before cutting the release please ensure that third party
> > > > > > libraries
> > > > > > are not reported against CVEs
> > > > > >
> > > > > >
> > > > > > This is the list of the latest releases
> > > > > > https://zookeeper.apache.org/releases.html
> > > > > >
> > > > > > Would anyone volunteer ?
> > > > > >
> > > > > > Enrico
>

Re: Volounteers for releases ?

[Enrico Olivelli]

Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
 ha scritto:
>
> Hi Kezhu,
>
> Sure, I'll take a look at the open PRs before cutting 3.9.0 from
> master. Let's mark these tickets release blockers as you suggested.
>
> Any more blockers of 3.9.0 that anyone knows about?

No, there are no critical issues at the moment.

I will double check on compatibility with the latest JDKs, it is
better that when we cut a new major release
it works well with the newer JDKs

Enrico


>
> Andor
>
>
>
> On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > Hi all,
> >
> > Sorry for the disruption.
> >
> > I want to ask if there is any possibility for us to include
> > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> >
> > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> > `WatcherType.PersistentRecursive` to remove `AddWatchMode.PERSISTENT`
> > and
> > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > complementary to
> > [ZOOKEEPER-4466][3] which supports multiple different watches on one
> > path
> > and was merged to master one month ago. If we don't get it to 3.9.0,
> > it is
> > probably impossible for us to merge it to patch versions of 3.9
> > series as
> > it touches both server logic and api side. It might be strange to
> > ship
> > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to support
> > different
> > watcher types on one path in watching but not all of them in
> > removing.
> >
> > ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver comprehensive
> > tests.
> >
> > I have already sent a [review request][3] to the dev mailing list. It
> > has
> > more descriptional information and a real world use case
> > of  ZOOKEEPER-4472.
> >
> > Look forward to your feedback in either thread!
> >
> > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> > [3]: https://lists.apache.org/thread/m7gxcffsnjy2lm8g52nssfxb6t800o3r
> >
> >
> > Best,
> > Kezhu Wang
> >
> >
> >
> > Best,
> > Kezhu Wang
> >
> >
> > On Fri, Jun 9, 2023 at 6:17 PM Andor Molnar  wrote:
> >
> > > Hi Enrico,
> > >
> > > I can take the master cut next week, but let me put together an
> > > email
> > > about a TLS topic first. I'd like to propose a fix to resolve the
> > > problem of FIPS (custome trust manager in ZK) and reverse DNS
> > > lookups.
> > > I'd like to include it in 3.9.0 and 3.8.2.
> > >
> > > Andor
> > >
> > > p.s. Whoever is making a change on the webpage, please remove the
> > > 3.8.0
> > > release.
> > >
> > >
> > >
> > > On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> > > > Hello ZooKeepers,
> > > > I think that it is time to do a round of releases.
> > > >
> > > > We should cut a release out of the master branch, 3.9.0 and main
> > > > cutting a release out of 3.7.x and 3.8.x would be useful.
> > > >
> > > > Before cutting the release please ensure that third party
> > > > libraries
> > > > are not reported against CVEs
> > > >
> > > >
> > > > This is the list of the latest releases
> > > > https://zookeeper.apache.org/releases.html
> > > >
> > > > Would anyone volunteer ?
> > > >
> > > > Enrico
>

Re: FIPS: removing ZKTrustManager

[Enrico Olivelli]

Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
 ha scritto:
>
> I'd like to backport this to the 3.8 branch too.
>
> Let's say I'll add new "zookeeper.fips-mode" parameter which will be
> "false" by default in 3.8 and "true" for 3.9.0.

I am +1
ZK 3.9 will take time to be adopted and this is an important security
related topic

Enrico

>
> Thoughts?
>
> Andor
>
>
>
> On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > I think that switching to
> > sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is a good
> > option.
> > The less tweaks we have about Security code the better.
> >
> >
> > It would be great to see this in 3.9.0.
> >
> > Enrico
> >
> > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> >  ha scritto:
> > > Hi zk folks,
> > >
> > > Problem(s)
> > > ==
> > >
> > > One problem that we're having with a custom Trust Manager in ZK is
> > > that
> > > FIPS doesn't allow that:
> > >
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > >
> > > In FIPS mode the only allowed TrustManager in the JDK is
> > > X509TrustManagerImpl which is the default implementation. The class
> > > is
> > > final, so extending it is not an option unfortunately.
> > >
> > > The intention behind implementing a custom trust manager in ZK was,
> > > I
> > > believe, the need for server and client-side hostname verification.
> > > Hostname verification officially is not part of the SSL/TLS
> > > protocol,
> > > it's the responsibility of an upper level protocol like HTTPS.
> > >
> > > Hacking hostname verification in the SSL handshake is nice and was
> > > working fine so far, but unfortunately breaks the FIPS standard.
> > >
> > > Another annoying issue with ZKTrustManager is the need for reverse
> > > DNS
> > > lookup. This is usually needed when the hostname of the certificate
> > > provider is not known at the time of handshake. For instance, when
> > > somebody connects the client via IP address, which is generally not
> > > recommended when TLS is active in the client-server protocol.
> > >
> > > The bigger problem I've found is in the leader election: when a
> > > peer
> > > connects with a smaller id, the node will close the existing
> > > connection
> > > and opens a new one in the other direction, based on the
> > > information
> > > received in the InitialMessage from the peer which only contains
> > > the IP
> > > address, not the hostname. Therefore TrustManager needs to perform
> > > reverse DNS lookup.
> > >
> > > Tickets about reverse DNS lookup issues:
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-3860
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4268
> > >
> > > Proposal
> > > 
> > >
> > > I suggest to remove ZKTrustManager entirely from the codebase and
> > > use
> > > the built-in, FIPS-Enabled X509TrustManagerImpl instead. It has the
> > > downside of losing hostname verification, but we have an option to
> > > re-
> > > enable it in client-server communication: Netty has built-in
> > > support
> > > for it, we just need to do
> > >
> > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
> > >
> > > when creating the SSLEngine and that will result in a behaviour
> > > very
> > > similar to what we provide currently. I can show some examples.
> > >
> > > What we will truly lose is the hostname verification option in the
> > > Quorum and Leader Election protocols. Since in these protocols we
> > > manipulate the sockets directly, we would need to implement the
> > > verification manually.
> > >
> > > What do you think about this trade-off?
> > >
> > > Of course, we can put this change behind a feature flag "fips-
> > > mode",
> > > which will lead to a new mode in ZooKeeper that is actually less
> > > strict
> > > as the original behaviour.
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
>

Re: FIPS: removing ZKTrustManager

[Enrico Olivelli]

I think that switching to
sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is a good
option.
The less tweaks we have about Security code the better.


It would be great to see this in 3.9.0.

Enrico

Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
 ha scritto:
>
> Hi zk folks,
>
> Problem(s)
> ==
>
> One problem that we're having with a custom Trust Manager in ZK is that
> FIPS doesn't allow that:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-4393
>
> In FIPS mode the only allowed TrustManager in the JDK is
> X509TrustManagerImpl which is the default implementation. The class is
> final, so extending it is not an option unfortunately.
>
> The intention behind implementing a custom trust manager in ZK was, I
> believe, the need for server and client-side hostname verification.
> Hostname verification officially is not part of the SSL/TLS protocol,
> it's the responsibility of an upper level protocol like HTTPS.
>
> Hacking hostname verification in the SSL handshake is nice and was
> working fine so far, but unfortunately breaks the FIPS standard.
>
> Another annoying issue with ZKTrustManager is the need for reverse DNS
> lookup. This is usually needed when the hostname of the certificate
> provider is not known at the time of handshake. For instance, when
> somebody connects the client via IP address, which is generally not
> recommended when TLS is active in the client-server protocol.
>
> The bigger problem I've found is in the leader election: when a peer
> connects with a smaller id, the node will close the existing connection
> and opens a new one in the other direction, based on the information
> received in the InitialMessage from the peer which only contains the IP
> address, not the hostname. Therefore TrustManager needs to perform
> reverse DNS lookup.
>
> Tickets about reverse DNS lookup issues:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3860
> https://issues.apache.org/jira/browse/ZOOKEEPER-4268
>
> Proposal
> 
>
> I suggest to remove ZKTrustManager entirely from the codebase and use
> the built-in, FIPS-Enabled X509TrustManagerImpl instead. It has the
> downside of losing hostname verification, but we have an option to re-
> enable it in client-server communication: Netty has built-in support
> for it, we just need to do
>
> sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
>
> when creating the SSLEngine and that will result in a behaviour very
> similar to what we provide currently. I can show some examples.
>
> What we will truly lose is the hostname verification option in the
> Quorum and Leader Election protocols. Since in these protocols we
> manipulate the sockets directly, we would need to implement the
> verification manually.
>
> What do you think about this trade-off?
>
> Of course, we can put this change behind a feature flag "fips-mode",
> which will lead to a new mode in ZooKeeper that is actually less strict
> as the original behaviour.
>
> Regards,
> Andor
>
>
>

Volounteers for releases ?

[Enrico Olivelli]

Hello ZooKeepers,
I think that it is time to do a round of releases.

We should cut a release out of the master branch, 3.9.0 and main
cutting a release out of 3.7.x and 3.8.x would be useful.

Before cutting the release please ensure that third party libraries
are not reported against CVEs


This is the list of the latest releases
https://zookeeper.apache.org/releases.html

Would anyone volunteer ?

Enrico

Re: Zookeeper support to log4j 2.X

[Enrico Olivelli]

Could you upgrade to 3.8.0?

Otherwise, IIRC on the latest releases od older branches we switched to a
compatible replacement for log4j 1 but without the security issues: reloadj4


Enrico

Il Ven 26 Mag 2023, 17:54 jitesh chavan  ha
scritto:

> Hello Team,
>
> We are using Zookeeper version 3.7. Kindly confirm whether this version
> support log4J 2.17.
> We tried coupling Log4J 2.X but it failed. Log4J 1.2 is working fine.
>
> Please advise, thanks !
>
> Thank you,
> Jitesh P. CHavan
>

Re: Query - Facing

[Enrico Olivelli]

Jeya,

Il Mer 22 Mar 2023, 17:29 Jeyalakshmi Rajagopal 
ha scritto:

> Hi Team,
> I am installing zookeeper -3.8.0 version on my Linux server.
> Zookeeper server has started successfully. But zookeeper status is showing
> as
> below
>
> #./zkServer.sh status
> Zookeeper JMX enabled by default
> Using config: /opt/zookeeper/bin/..conf/zoo.cfg
> myid could not be determined , will not able to locate client port in the
> server configs. Client address:localhost. Client SSL: false.
> Error contacting service.It is probably not running.
>
> But i was keeping the myid in server1 has 1, server2 has 2 and server3 has
> 3 .
>
> I am expecting the leader and 2 followers in zookeeper status .
>
> Could you please help me?
>

I am sorry, I missed this message.
Have you been able to solve your problem?

Enrico


> Thanks & Regards,
> Jeya
>

Re: What path getConfig watchers should receive from ZooKeeper with chroot “/zookeeper/config” ?

[Enrico Olivelli]

Il Ven 14 Apr 2023, 20:47 Chris Nauroth  ha scritto:

> Thank you for the discussion, Kezhu.
>
> My take is that the behavior you proposed, with the received path always
> /zookeeper/config regardless of chroot, is the correct one. However, it's
> hard to know if anyone has coded workarounds on top of the current
> behavior, which would then break if we made the change.
>
> The safest course of action would be to target a fix for 3.9.0 and announce
> it as a backward-incompatible change. I don't think anyone has a specific
> timeline yet for a 3.9.0 release though.
>
> I'm curious to hear if others disagree with me and instead think the risk
> is low enough to make the change in existing release lines.
>

I agree with this proposal.


Enrico



> Chris Nauroth
>
>
> On Mon, Apr 3, 2023 at 9:34 AM Kezhu Wang  wrote:
>
> > Hi all,
> >
> > Any thoughts on this?
> >
> > Curator has similar issue reported as CURATOR-666[1]. I think it might be
> > worth to get Curator and ZooKeeper behave same in `getConfig`.
> >
> > I replied this also to dev@curator for joint discussion. See my quotes
> and
> > ZOOKEEPER-4565[2], ZOOKEEPER-4601[3] for context.
> >
> > [1]: https://issues.apache.org/jira/browse/CURATOR-666
> > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4565
> > [3]: https://issues.apache.org/jira/browse/ZOOKEEPER-4601
> >
> > On Wed, Jul 27, 2022 at 8:35 PM Kezhu Wang  wrote:
> >
> > > Hi Devs,
> > >
> > > Before ZOOKEEPER-4565[1], `ClientCnxn` uses following code to strip
> > chroot:
> > >
> > > ```
> > > // convert from a server path to a client path
> > > if (chrootPath != null) {
> > > String serverPath = event.getPath();
> > > if (serverPath.compareTo(chrootPath) == 0) {
> > > event.setPath("/");
> > > } else if (serverPath.length() > chrootPath.length()) {
> > > event.setPath(serverPath.substring(chrootPath.length()));
> > > } else {
> > > LOG.warn("Got server path {} which is too short for chroot path
> > > {}.",
> > > event.getPath(), chrootPath);
> > > }
> > > }
> > > ```
> > >
> > > This results in behavior:
> > > * For chroot "/zookeeper", watcher will receive event path "/config".
> > > * For chroot "/short", watcher will receive illegal path "eper/config".
> > > This causes in ZOOKEEPER-4601.
> > > * For chroot "/pretty-long-chroot-path", watcher will receive event
> path
> > > "/zookeeper/config".
> > >
> > > ZOOKEEPEER-4601 changed the stripping code to fix illegal path:
> > >
> > > ```
> > > private String stripChroot(String serverPath) {
> > > if (serverPath.startsWith(chrootPath)) {
> > > if (serverPath.length() == chrootPath.length()) {
> > > return "/";
> > > }
> > > return serverPath.substring(chrootPath.length());
> > > } else if (serverPath.startsWith(ZooDefs.ZOOKEEPER_NODE_SUBTREE)) {
> > > return serverPath;
> > > }
> > > LOG.warn("Got server path {} which is not descendant of chroot path
> > > {}.", serverPath, chrootPath);
> > > return serverPath;
> > > }
> > > ```
> > >
> > > This results in behavior:
> > > * For chroot "/zookeeper", watcher will receive event path "/config".
> > > * For chroot "/other-chroot"(eg. "/short", "/pretty-long-chroot-path",
> > > etc.), watcher will receive event path "/zookeeper/config".
> > >
> > > The path `getConfig` watcher received was not changed in
> ZOOKEEPER-4565.
> > >
> > > It is a bit of surprising to me that event path of `getConfig` watcher
> is
> > > not "/zookeeper/config" . I guess the current behavior might not be by
> > > design. Personally, I prefer to `getConfig` watchers(whether it is the
> > > default one or not) to receive path "/zookeeper/config". But,
> obviously,
> > > such change is a breaking change in behavior(though might not be by
> > design)
> > > of public API. @eolivelli mentioned that such a change needs a mailing
> > list
> > > decision, so I post it here for discussion in addition to
> > ZOOKEEPER-4601[2].
> > >
> > > Any thoughts ?
> > >
> > > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4565
> > > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4601
> > >
> > >
> > > Best,
> > > Kezhu Wang
> > >
> >
> >
> > --
> > Best,
> > Kezhu Wang
> >
>

[ANNOUNCE] New committer: Li Wang

[Enrico Olivelli]

The Project Management Committee (PMC) for Apache Zookeeper
has invited Li Wang to become a committer and we are pleased
to announce that they have accepted.

Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.
A PMC member helps manage and guide the direction of the project.

In order to know more about how the ASF work please checkout this
link: https://www.apache.org/foundation/how-it-works.html

Please join me in celebrating this event for our community !

Congratulations Li !

Enrico Olivelli

Fwd: [ANNOUNCE] New self-serve portal for projects, committers, and contributors

[Enrico Olivelli]

FYI

-- Forwarded message -
Da: Daniel Gruno 
Date: gio 2 mar 2023 alle ore 01:17
Subject: [ANNOUNCE] New self-serve portal for projects, committers,
and contributors
To: 


Hi folks,
During today's Infrastructure round-table session[1] we launched a new
infrastructure self-serve portal that replaces our old self-serve site.

The URL remains the same, https://selfserve.apache.org/ , and while the
self-serve offerings are largely the same as well, I do want to call out
the new "Request a Jira account" feature on there.

Prior to today, projects were tasked with managing new Jira accounts
themselves, including handling the initial request from users. The new
self-serve workflow for Jira accounts is aimed at making this a faster,
more streamlined experience for all projects, while also addressing data
privacy concerns and time constraints of projects. Any project using
Jira can now refer their users/contributors to
https://selfserve.apache.org/jira-account.html for new Jira account
requests, which will take care of all the needed paperwork for new
account requests and loop in projects for the final review and approval
of the accounts.

We hope this new feature will be well received. If there are any
questions, concerns, or other feedback regarding the new self-serve
portal, please do reach out to us at: us...@infra.apache.org

With regards,
Daniel on behalf of ASF Infra.


[1] https://infra.apache.org/roundtable.html

Re: [ANNOUNCE] Apache ZooKeeper 3.8.1 released

[Enrico Olivelli]

Andor,

Il giorno gio 23 feb 2023 alle ore 15:39 Andor Molnar
 ha scritto:
>
> Hi Enrico,
>
> Thanks for the release, nice work.
> Looks like 3.8.0 downloads and documentation have been kept on the
> website. Is that intentional?

I missed that we have to remove the old release from the download page.
I will fix it soon and I will double check the instructions on the wiki

thanks for the heads up

Enrico

>
> Regards,
> Andor
>
>
>
> On Mon, 2023-01-30 at 08:55 +0100, Enrico Olivelli wrote:
> > The Apache ZooKeeper team is proud to announce Apache ZooKeeper
> > version 3.8.1
> >
> > ZooKeeper is a high-performance coordination service for distributed
> > applications. It exposes common services - such as naming,
> > configuration management, synchronization, and group services - in a
> > simple interface so you don't have to write them from scratch. You
> > can
> > use it off-the-shelf to implement consensus, group management, leader
> > election, and presence protocols. And you can build on it for your
> > own, specific needs.
> >
> > For ZooKeeper release details and downloads, visit:
> > https://zookeeper.apache.org/releases.html
> >
> > ZooKeeper 3.8.1 Release Notes are at:
> > https://zookeeper.apache.org/doc/r3.8.1/releasenotes.html
> >
> > We would like to thank the contributors that made the release
> > possible.
> >
> > Regards,
> >
> > The ZooKeeper Team
>

[jira] [Created] (ZOOKEEPER-4676) ReadOnlyModeTest doesn't not compile on JDK20 (Thread.suspend has been removed)

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4676:
--

 Summary: ReadOnlyModeTest doesn't not compile on JDK20 
(Thread.suspend has been removed)
 Key: ZOOKEEPER-4676
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4676
 Project: ZooKeeper
  Issue Type: Test
  Components: tests
Reporter: Enrico Olivelli
Assignee: Enrico Olivelli
 Fix For: 3.9.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Last part for Backup/Restore

[Enrico Olivelli]

Hello ZooKeepers,
This is the last part of the Backup/Restore feature

https://github.com/apache/zookeeper/pull/1966


Please take a look, it would be great to commit it before cutting the
3.9.0 release


Cheers
Enrico

Re: Metrics Missing !!

[Enrico Olivelli]

Hello,

Il Sab 18 Feb 2023, 19:45 vaibhav tewari  ha
scritto:

> Hi everyone
> After I enabled prometheus metrics exporter in zookeeper 3.7.1, one
> instance of zookeeper, running in a cluster of 3 , has stopped sending
> metrics for file_descriptor_count, avg_latency and outstanding_request.
> When I deploy it on a single instance it seems to be working fine sending
> all the metric data.
>

Only one out of 3?
What happens if you restart it?
Maybe if you haven't restarted it taking  jstack dump will help
troubleshooting, maybe there is one thread blocked


Enrico


> I am currently stuck at this,It would be great if someone can help me...
> Thanks in advance!!

Re: JDK 20 Rampdown Phase 2 & JMX Heads-up

[Enrico Olivelli]

Hello,
I have found a new test that doesn't pass on JDK20

https://issues.apache.org/jira/browse/ZOOKEEPER-4675

The problem is that Thread.suspend() now throws UnsupportedOperationException

Any volunteers to fix it ?

Enrico

Il giorno mer 25 gen 2023 alle ore 06:23 David Delabassee
 ha scritto:
>
> Hi,
>
> First off, on behalf of Oracle’s Java Team, I’d like to wish you a happy
> and prosperous new year!
>
> In 2023, two Java releases will be made available: JDK 20 (March) &  JDK
> 21 (September).
>
> JDK 20 [1] has entered Rampdown Phase Two (RDP2) [2], its initial
> Release Candidate is planned for February 9. Given that and to be better
> prepared for the future, it makes sense to begin testing your project(s)
> using JDK 21 early-access (EA) builds. Your feedback allows us to
> evaluate and address issues you find while testing EA builds.
>
> [1] https://jdk.java.net/20/
> [2] https://mail.openjdk.org/pipermail/jdk-dev/2023-January/007308.html
> [3] https://jdk.java.net/21/
>
>
> ## Heads-up - JDK 21: JMX Subject Delegation & Fine-grained Security
> Deprecation
>
> JMX has some features that rely on Security Manager APIs which are
> deprecated for removal (see JEP 411 [4]). These features are "Subject
> Delegation" and "Fine-grained Security", which both seem to be generally
> unused, and would require significant investment to implement without
> touching the deprecated APIs. As a consequence, "Subject Delegation" is
> being proposed for deprecation in JDK 21 [5].
>
> Fine-grained Security is also being considered for deprecation at the
> same time. This feature [6] has allowed configuration of a security
> policy to restrict or permit access to specific MBean actions. It is
> expected that this feature is generally unused, possibly because there
> is simply no demand for such detailed control, and that it is too
> complex to create and maintain the policies.
>
> [4] https://openjdk.org/jeps/411
> [5] https://bugs.openjdk.org/browse/JDK-8298966
> [6]
> https://docs.oracle.com/en/java/javase/19/jmx/fine-grained-security-example.html
>
>
> ## JDK 20 Early-Access builds
>
> The latest early-access builds of JDK 20 (builds 32) are available [7],
> and are provided under the GNU General Public License v2, with the
> Classpath Exception. The Release Notes are available here [8].
>
> [7] https://openjdk.org/projects/jdk/20/
> [8] https://jdk.java.net/20/release-notes
>
> ### JEPs integrated into JDK 20:
>
> - JEP 429: Scoped Values (Incubator)
> - JEP 432: Record Patterns (2nd Preview)
> - JEP 433: Pattern Matching for switch (4th Preview)
> - JEP 434: Foreign Function & Memory API (2nd Preview)
> - JEP 436: Virtual Threads (2nd Preview)
> - JEP 437: Structured Concurrency (2nd Incubator)
>
> ### Changes in recent JDK 20 builds that may be of interest:
>
> - JDK-8298525: javadoc crashes with "UnsupportedOperationException: Not
> yet implemented" in SeeTaglet.inherit [Reported by Apache Ant]
> - JDK-8298893: Rename option UsePolyIntrinsics to UsePoly1305Intrinsics
> - JDK-8287411: Enhance DTLS Performance
> - JDK-8293554: Enhanced DH Key Exchanges
>
>
> ## JDK 21 Early-Access builds
>
> The latest early-access builds of JDK 21 (builds 6) are available [9],
> and are provided under the GNU General Public License v2, with the
> Classpath Exception. The related EA API Javadoc is also available [10].
>
> [9] https://jdk.java.net/21/
> [10] https://download.java.net/java/early_access/jdk21/docs/api/
>
> ### Changes in recent JDK 21 builds that may be of interest:
>
> - JDK-8297295: Remove ThreadGroup.allowThreadSuspension
> - JDK-8287411: Enhance DTLS performance
> - JDK-8233269: Improve handling of JAVA_ARGS
> - JDK-8297933: Compiler should only use verified interface types for
> optimization
> - JDK-8298381: Improve handling of session tickets for multiple SSLContexts
> - JDK-8299501: Usage of constructors of primitive wrapper classes should
> be avoided in java.util API docs
> - JDK-8299475: Enhance SocketException by cause where it is missing in
> net and nio area
> - JDK-8299544: Improve performance of CRC32C intrinsics (non-AVX-512)
> for small inputs
> - JDK-8299576: Reimplement java.io.Bits using VarHandle access
> - JDK-8278326: Socket close is not thread safe and other cleanup
> - JDK-8299673: Simplify object pinning interactions with string
> deduplication
>
>
> ## JavaFX 20 & 21 Early-Access Builds
>
> These are early-access builds of the JavaFX Runtime, built from
> openjdk/jfx [11]. Those EA builds are intended to allow JavaFX
> application developers to build and test their applications with JavaFX
> 20 on JDK 20. The latest EA builds (JavaFX 20 EA b16 2023/1/14) are now
> available [12] and are provided under the GNU General Public License,
> version 2, with the Classpath Exception. Please note that initial JavaFX
> 21 early-access builds (JavaFX 21 b1 2023/1/19) are now available [13]
> as well.
> Feedback should be reported to the openjfx-dev mailing list [14].
>
> [11] 

[jira] [Created] (ZOOKEEPER-4675) Tests don't pass on JDK20: Thread.suspend is no more supported

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4675:
--

 Summary: Tests don't pass on JDK20: Thread.suspend is no more 
supported
 Key: ZOOKEEPER-4675
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4675
 Project: ZooKeeper
  Issue Type: Test
  Components: tests
Reporter: Enrico Olivelli
 Fix For: 3.9.0, 3.8.2


This test doesn't pass on JDK20 

[ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 86.683 
s <<< FAILURE! - in org.apache.zookeeper.test.ReadOnlyModeTest [ERROR] 
testSeekForRwServer Time elapsed: 3.323 s <<< ERROR! 
java.lang.UnsupportedOperationException at 
java.base/java.lang.Thread.suspend(Thread.java:1865) at 
org.apache.zookeeper.test.ReadOnlyModeTest.testSeekForRwServer(ReadOnlyModeTest.java:303)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4674) C client tests don't pass on CI

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4674:
--

 Summary: C client tests don't pass on CI
 Key: ZOOKEEPER-4674
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4674
 Project: ZooKeeper
  Issue Type: Test
  Components: c client, tests
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Reporting a Potential Bug in Apache ZooKeeper Code

[Enrico Olivelli]

Marcos,

Il Sab 11 Feb 2023, 21:12 Marcos Rico Peng  ha
scritto:

> Dear Apache ZooKeeper Committee,
>
>
>
> I hope this email finds you well. I am reaching out to bring to your
> attention a potential bug that I have discovered in the Apache ZooKeeper
> code while working on a project.
>
>
>
> I have noticed that the behavior of PERSISTENT, EPHEMERAL, CONTAINER, and
> TTL nodes is as expected. However, when trying to access the type by
> checking the EphemeralOwner parameter of a node's stat, it returns the same
> value (0x0) for all node types except EPHEMERAL. This suggests that there
> may be an issue with the implementation of EphemeralType.get(). I am using
> Zookeeper version 3.6.
>


What's your usecase?
Why the application needs to know the type?


Enrico

>
>
>
> To reproduce this issue I activated Zookeeper extended usage and created
> different nodes of different types. They all behave as expected but the
> EphemeralOwner parameter is 0x0. This conflicts with the documentation
> https://zookeeper.apache.org/doc/r3.6.3/apidocs/zookeeper-server/index.html
> .
>
>
>
> I would like to inquire if there is another way of retrieving the node
> type or if this is indeed a bug in the code. If so, I would be grateful if
> you could provide me with any relevant documentation or code references
> that I can use to further understand the issue.
>
>
>
> Thank you for your time and efforts in maintaining Apache ZooKeeper. I
> look forward to hearing back from you soon.
>
>
>
> Best regards,
>
> Marcos
>
>

Re: Add s390x to Jenkins CI

[Enrico Olivelli]

This is the job
https://ci-hadoop.apache.org/job/zookeeper-multi-branch-build-s390x/job/master/

All tests passed !

Vivian,
Maybe you could keep an eye to the job and report here when it fails

Thank you very much

Enrico

Il giorno mer 8 feb 2023 alle ore 14:16 Vivian Kong
 ha scritto:
>
> Thank you for taking this on Enrico!  If there’s anything I can help please 
> let me know.
>
>
> Regards,
>
> Vivian Kong
> Linux on IBM Z Open Source Ecosystem
> IBM Canada Toronto Lab
>
> From: Enrico Olivelli 
> Date: Wednesday, February 8, 2023 at 3:29 AM
> To: dev@zookeeper.apache.org 
> Subject: [EXTERNAL] Re: Add s390x to Jenkins CI
> Vivian,
>
> Il giorno lun 30 gen 2023 alle ore 15:45 Vivian Kong
>  ha scritto:
> >
> > Hi all,
> >
> > As s390x nodes are available 
> > (https://cwiki.apache.org/confluence/display/INFRA/ci-hadoop.apache.org ), 
> > I’m wondering if it is possible to add a s390x CI job for Zookeeper?  I’ve 
> > raised a PR (https://github.com/apache/zookeeper/pull/1975 ) to add the 
> > Jenkinsfile we might need for this and I’d appreciate any feedback.
>
> I agree that we should restore those tests.
>
> It would be great to move all of our CI tests to GH actions (I
> recently heard about the support for ARM)
>
> I am going to merge your patch and try to configure a pipeline on
> Jenkins and see the results
>
> your patch does no harm and we can revert it
>
> Does anyone have other solutions ?
>
> Enrico
>
> >
> > s390x was part of Travis CI 
> > (https://issues.apache.org/jira/browse/ZOOKEEPER-3681 ) and it will be 
> > great if CI can be run again on s390x.
> >
> > Thank you.
> >
> > Regards,
> >
> > Vivian Kong
> > Linux on IBM Z Open Source Ecosystem
> > IBM Canada Toronto Lab

Please review tests fix for JDK19 and JDK20

[Enrico Olivelli]

Hello ZooKeepers,
please review again this PR to make CI pass on JDK19 and JDK20
https://github.com/apache/zookeeper/pull/1959

Thanks

Enrico

Re: Add s390x to Jenkins CI

[Enrico Olivelli]

Vivian,

Il giorno lun 30 gen 2023 alle ore 15:45 Vivian Kong
 ha scritto:
>
> Hi all,
>
> As s390x nodes are available 
> (https://cwiki.apache.org/confluence/display/INFRA/ci-hadoop.apache.org), I’m 
> wondering if it is possible to add a s390x CI job for Zookeeper?  I’ve raised 
> a PR (https://github.com/apache/zookeeper/pull/1975) to add the Jenkinsfile 
> we might need for this and I’d appreciate any feedback.

I agree that we should restore those tests.

It would be great to move all of our CI tests to GH actions (I
recently heard about the support for ARM)

I am going to merge your patch and try to configure a pipeline on
Jenkins and see the results

your patch does no harm and we can revert it

Does anyone have other solutions ?

Enrico

>
> s390x was part of Travis CI 
> (https://issues.apache.org/jira/browse/ZOOKEEPER-3681) and it will be great 
> if CI can be run again on s390x.
>
> Thank you.
>
> Regards,
>
> Vivian Kong
> Linux on IBM Z Open Source Ecosystem
> IBM Canada Toronto Lab

Re: ZOOKEEPER-4306 CloseSessionTxn contains too many ephemal nodes cause cluster crash

[Enrico Olivelli]

Li,

Il giorno mer 8 feb 2023 alle ore 03:49 Li Wang  ha scritto:
>
> Hello,
>
>
> We had a production outage due to the issue reported in
> https://issues.apache.org/jira/browse/ZOOKEEPER-4306 and some other users
> also ran into the same issue. I wonder if we can use this thread to discuss
> and come to a consensus on how to fix it. :-)
>
>
>
> Thanks Damien Diederen
>  for the
> contribution and patch. Limiting the number of ephemeral nodes that can be
> created in a session looks like a simple and reasonable solution to me.
> Having a way to enforce it will protect the system from potential OOM
> issues.

How does the client recover from having created too many ephemeral nodes ?
This seems not trivial to do. Let me share some ideas:

Solution one: fail the creation of the node
If we fail the creation of the node then the application will probably
enter a loop and continue to create it.
There is no way to say that some znode is "more important" than other
znodes, so the application will keep failing in the creation
of random znodes.

Solution two: force expires the session (and reset ephemeral nodes)
In this case some applications would probably recover in a better way
(ZK client applications are supposed to deal with session expiration
somehow).
and some applications will auto-restart (because session expired is a
symptom of network partition and suicide is the best thing to do)
In any case the application will try to create the znodes, work for
some time, and then die again (or recreate the session)

I agree that a short term solution is a server side protection, but it
is better to think to a better plan.


>
>
> I've also looked into the possibility of splitting CloseSessionTxn into
> smaller ones. Unfortunately, it didn't work, as currently in Zookeeper, one
> request can only have one txn. Even though we can split the paths to be
> deleted into multiple batches and define sub-txn for each batch, we have to
> wrap all sub-txn(s) into a single wrapper txn and associate it to the
> request. At the end, when loading zk database, we still have to deserialize
> the large wrapper txn, which can fail the length check (jute.maxBuffer +
> zookeeper.jute.maxbuffer.extrasize).


Unfortunately there are few users that say that zookeeper doesn't
scale and probably here we are hitting one of such cases,
and most of these cases are due to the write protocol (JUTE), that
puts unneeded constraints on Zookeeper

Enrico

>
>
> Changing ZK to allow multiple txns for a single request looks quite
> involved and it may have other implications.
>
>
> I wonder if anyone has any input or any better ideas?
>
>
>
> Thanks,
>
>
> Li

[jira] [Created] (ZOOKEEPER-4672) Code cleanup - apply JDK7 Diamond operator consistently

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4672:
--

 Summary: Code cleanup - apply JDK7 Diamond operator consistently
 Key: ZOOKEEPER-4672
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4672
 Project: ZooKeeper
  Issue Type: Task
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4669) Upgrade snappy-java to 1.1.9.1 (in order to support M1 macs)

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4669:
--

 Summary: Upgrade snappy-java to 1.1.9.1 (in order to support M1 
macs)
 Key: ZOOKEEPER-4669
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4669
 Project: ZooKeeper
  Issue Type: Task
  Components: java client
Reporter: Enrico Olivelli
 Fix For: 3.9.0, 3.7.2, 3.8.2






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New ZooKeeper Committer - Zili Chen

[Enrico Olivelli]

Congratulations !

Well deserved

Enrico

Il giorno mar 31 gen 2023 alle ore 01:12 Chris Nauroth
 ha scritto:
>
> On behalf of the Apache ZooKeeper PMC, I am pleased to announce that
> Zili Chen has been elected as a committer on the Apache ZooKeeper
> project. We appreciate all of Zili's hard work thus far, and we
> look forward to continued contributions.
>
> Welcome Zili!
>
> Chris Nauroth

Help needed on CI - C client tests are failing only only there

[Enrico Olivelli]

Hello ZooKeepers,
on GH Actions there is one test on the C client that is failing

Zookeeper_simpleSystem::testAsyncWatcherAutoResetFAIL: zktest-mt

The test passes on my laptop

The tests fails also on other branches and not only master branch

Are there any volunteers ?

This is a problem for PR validation and generally in order to say that
our branches are in good shape

Enrico

[ANNOUNCE] Apache ZooKeeper 3.8.1 released

[Enrico Olivelli]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.8.1

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.8.1 Release Notes are at:
https://zookeeper.apache.org/doc/r3.8.1/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

[RESULT] [VOTE] Apache ZooKeeper release 3.8.1 candidate 1

[Enrico Olivelli]

The VOTE passed with 4 votes, 3 of them are binding:
- Patrick Hunt
- Enrico Olivelli
- Szalay-Bekő Máté
- Andrey Yegorov

I will promote the artifacts and close the release

Thank you to everyone who spent time to validate the release

Enrico

Il giorno lun 30 gen 2023 alle ore 08:33 Enrico Olivelli
 ha scritto:
>
> +1 (binding)
>
> - staged and self verified the artifacts
> - built from sources and did some smoke tests with the built artifacts
>
> I don't like much that there is a CPP test that is often failing on GH
> actions (I will open a ticket),
> but the test doesn't fail locally
>
>
>
> Enrico
>
> Il giorno dom 29 gen 2023 alle ore 01:00 Szalay-Bekő Máté
>  ha scritto:
> >
> > +1 (binding)
> >
> > I did the following tests for the release candidate:
> > - verified checksum and gpg signature of the artifacts
> > - built the source code (incl. the C-client, using -Pfull-build) on Ubuntu
> > 18.04.5 using OpenJDK 8u265, maven 3.6.3 and GCC version 7.5.0
> > - all the unit tests passed eventually (both Java and C-client)
> > - also built and executed unit tests for zkpython
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built
> > - executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> >   - rolling upgrade from 3.5.10 to 3.8.1
> >   - rolling upgrade from 3.6.4 to 3.8.1
> >   - rolling upgrade from 3.7.1 to 3.8.1
> >   - rolling upgrade from 3.8.0 to 3.8.1
> >
> > Best regards,
> > Máté
> >
> > On Fri, Jan 27, 2023 at 6:54 PM Patrick Hunt  wrote:
> >
> > > +1 - xsums validated, rat ran clean, built/compiled fine and I was able to
> > > run some manual tests on various cluster sizes.
> > >
> > > Regards,
> > >
> > > Patrick
> > >
> > > On Wed, Jan 25, 2023 at 8:39 AM Enrico Olivelli 
> > > wrote:
> > >
> > > > This is the second release candidate for 3.8.1.
> > > >
> > > > This is a bugfix release. The full release notes is available at:
> > > >
> > > >
> > > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> > > >
> > > > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0.
> > > ***
> > > >
> > > > Source files:
> > > >
> > > >
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/
> > > >
> > > > Maven staging repo:
> > > >
> > > https://repository.apache.org/content/repositories/orgapachezookeeper-1088/
> > > >
> > > > The release candidate tag in git to be voted upon: release-3.8.1-1
> > > > https://github.com/apache/zookeeper/tree/release-3.8.1-1
> > > >
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > >
> > > > The staging version of the website is:
> > > >
> > > >
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/website/index.html
> > > >
> > > >
> > > > Should we release this candidate?
> > > >
> > > >
> > > > Enrico Olivelli
> > > >
> > >

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 1

[Enrico Olivelli]

+1 (binding)

- staged and self verified the artifacts
- built from sources and did some smoke tests with the built artifacts

I don't like much that there is a CPP test that is often failing on GH
actions (I will open a ticket),
but the test doesn't fail locally



Enrico

Il giorno dom 29 gen 2023 alle ore 01:00 Szalay-Bekő Máté
 ha scritto:
>
> +1 (binding)
>
> I did the following tests for the release candidate:
> - verified checksum and gpg signature of the artifacts
> - built the source code (incl. the C-client, using -Pfull-build) on Ubuntu
> 18.04.5 using OpenJDK 8u265, maven 3.6.3 and GCC version 7.5.0
> - all the unit tests passed eventually (both Java and C-client)
> - also built and executed unit tests for zkpython
> - checkstyle and spotbugs passed
> - apache-rat passed
> - owasp (CVE check) passed
> - fatjar built
> - executed quick rolling-upgrade tests (using
> https://github.com/symat/zk-rolling-upgrade-test):
>   - rolling upgrade from 3.5.10 to 3.8.1
>   - rolling upgrade from 3.6.4 to 3.8.1
>   - rolling upgrade from 3.7.1 to 3.8.1
>   - rolling upgrade from 3.8.0 to 3.8.1
>
> Best regards,
> Máté
>
> On Fri, Jan 27, 2023 at 6:54 PM Patrick Hunt  wrote:
>
> > +1 - xsums validated, rat ran clean, built/compiled fine and I was able to
> > run some manual tests on various cluster sizes.
> >
> > Regards,
> >
> > Patrick
> >
> > On Wed, Jan 25, 2023 at 8:39 AM Enrico Olivelli 
> > wrote:
> >
> > > This is the second release candidate for 3.8.1.
> > >
> > > This is a bugfix release. The full release notes is available at:
> > >
> > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> > >
> > > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0.
> > ***
> > >
> > > Source files:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/
> > >
> > > Maven staging repo:
> > >
> > https://repository.apache.org/content/repositories/orgapachezookeeper-1088/
> > >
> > > The release candidate tag in git to be voted upon: release-3.8.1-1
> > > https://github.com/apache/zookeeper/tree/release-3.8.1-1
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > >
> > >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Enrico Olivelli
> > >
> >

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 1

[Enrico Olivelli]

Actually this VOTE will end on January 28th and not on 26th, that is
72 hours from now

Enrico

Il giorno mer 25 gen 2023 alle ore 17:38 Enrico Olivelli
 ha scritto:
>
> This is the second release candidate for 3.8.1.
>
> This is a bugfix release. The full release notes is available at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
>
> *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***
>
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1088/
>
> The release candidate tag in git to be voted upon: release-3.8.1-1
> https://github.com/apache/zookeeper/tree/release-3.8.1-1
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/website/index.html
>
>
> Should we release this candidate?
>
>
> Enrico Olivelli

[VOTE] Apache ZooKeeper release 3.8.1 candidate 1

[Enrico Olivelli]

This is the second release candidate for 3.8.1.

This is a bugfix release. The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801

*** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***

Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachezookeeper-1088/

The release candidate tag in git to be voted upon: release-3.8.1-1
https://github.com/apache/zookeeper/tree/release-3.8.1-1

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-1/website/index.html


Should we release this candidate?


Enrico Olivelli

Re: Release steps for 3.8.1

[Enrico Olivelli]

Il giorno mer 11 gen 2023 alle ore 18:24 John Muczynski
 ha scritto:
>
> I see that 3.6.4 went through a RC (Release Candidate) and an approval 
> process for its release, including some download, test, and approve.
> Will the same be happening for the 3.8.1 release?

Yes,
look for VOTE threads

Best regards
Enrico

>

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 0

[Enrico Olivelli]

I am cancelling the VOTE

I am going to prepare a new RC soon

Enrico

Il giorno mar 24 gen 2023 alle ore 09:15 Enrico Olivelli
 ha scritto:
>
> Andrey,
>
> Il giorno lun 23 gen 2023 alle ore 23:58 Andrey Yegorov
>  ha scritto:
> >
> >
> > -1 (non-binding) from me.
> >
> > I created a couple of PRs to test Apache Pulsar and Apache BookKeeper with 
> > this RC.
> > https://github.com/apache/bookkeeper/pull/3754
> > https://github.com/dlg99/pulsar/pull/8
> >
> > There are new test failures related to "Invalid Watcher, shouldn't be 
> > null!".
> > The problem is caused by validateWatcher in constructor (added in 3.8.1)
> > ```
> > private void validateWatcher(Watcher watcher) {
> > if (watcher == null) {
> > throw new IllegalArgumentException("Invalid Watcher, shouldn't 
> > be null!");
> > }
> > }
> >
> > public ZooKeeper(String connectString, int sessionTimeout, Watcher 
> > watcher, boolean canBeReadOnly, HostProvider hostProvider, ZKClientConfig 
> > clientConfig) throws IOException {
> > LOG.info("Initiating client connection, connectString={} 
> > sessionTimeout={} watcher={}", new Object[]{connectString, sessionTimeout, 
> > watcher});
> > this.validateWatcher(watcher);
> > ...
> > ```
> >
> > So the calls like
> > ```
> > ZooKeeper zkc = new ZooKeeper(zkUtil.getZooKeeperConnectString(), 1, 
> > null);
> > ```
> > (where watcher is null) are no longer valid.
> >
> > Personally, I think it is a bit too much for a minor 3.8.x release, 3.9.0 
> > would be a more appropriate version to require such changes.
>
> I agree that this looks like a breaking change and we should not allow
> it for a minor release.
> Also I generally think that it is better to NOT introduce any breaking
> changes in ZooKeeper at all as the library is really widely used.
>
> This is the PR that introduced the change
> https://github.com/apache/zookeeper/pull/1855
>
> This is the JIRA
> https://issues.apache.org/jira/browse/ZOOKEEPER-1875
>
> The validation prevents NPE to happen in the event dispatcher thread
>
> I am going to revert that change from the branch-3.8 branch and on master 
> branch
>
> We can think to a better solution (not to be discussed here)
>
> Enrico
>
>
>
> >
> > --
> > Andrey Yegorov
> >
> > On 2023/01/23 13:50:08 Enrico Olivelli wrote:
> > > This is a release candidate for 3.8.1.
> > >
> > > This is a bugfix release. The full release notes is available at:
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> > >
> > > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***
> > >
> > > Source files:
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/
> > >
> > > Maven staging repo:
> > > https://repository.apache.org/content/repositories/orgapachezookeeper-1085/
> > >
> > > The release candidate tag in git to be voted upon: release-3.8.1-0
> > > https://github.com/apache/zookeeper/tree/release-3.8.1-0
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/website/index.html
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Enrico Olivelli
> > >

[jira] [Created] (ZOOKEEPER-4666) Fix license files for Netty: some files are for 4.1.76 and some other files are for 4.1.86

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4666:
--

 Summary: Fix license files for Netty: some files are for 4.1.76 
and some other files are for 4.1.86
 Key: ZOOKEEPER-4666
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4666
 Project: ZooKeeper
  Issue Type: Task
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 0

[Enrico Olivelli]

Andrey,

Il giorno lun 23 gen 2023 alle ore 23:58 Andrey Yegorov
 ha scritto:
>
>
> -1 (non-binding) from me.
>
> I created a couple of PRs to test Apache Pulsar and Apache BookKeeper with 
> this RC.
> https://github.com/apache/bookkeeper/pull/3754
> https://github.com/dlg99/pulsar/pull/8
>
> There are new test failures related to "Invalid Watcher, shouldn't be null!".
> The problem is caused by validateWatcher in constructor (added in 3.8.1)
> ```
> private void validateWatcher(Watcher watcher) {
> if (watcher == null) {
> throw new IllegalArgumentException("Invalid Watcher, shouldn't be 
> null!");
> }
> }
>
> public ZooKeeper(String connectString, int sessionTimeout, Watcher 
> watcher, boolean canBeReadOnly, HostProvider hostProvider, ZKClientConfig 
> clientConfig) throws IOException {
> LOG.info("Initiating client connection, connectString={} 
> sessionTimeout={} watcher={}", new Object[]{connectString, sessionTimeout, 
> watcher});
> this.validateWatcher(watcher);
> ...
> ```
>
> So the calls like
> ```
> ZooKeeper zkc = new ZooKeeper(zkUtil.getZooKeeperConnectString(), 1, 
> null);
> ```
> (where watcher is null) are no longer valid.
>
> Personally, I think it is a bit too much for a minor 3.8.x release, 3.9.0 
> would be a more appropriate version to require such changes.

I agree that this looks like a breaking change and we should not allow
it for a minor release.
Also I generally think that it is better to NOT introduce any breaking
changes in ZooKeeper at all as the library is really widely used.

This is the PR that introduced the change
https://github.com/apache/zookeeper/pull/1855

This is the JIRA
https://issues.apache.org/jira/browse/ZOOKEEPER-1875

The validation prevents NPE to happen in the event dispatcher thread

I am going to revert that change from the branch-3.8 branch and on master branch

We can think to a better solution (not to be discussed here)

Enrico



>
> --
> Andrey Yegorov
>
> On 2023/01/23 13:50:08 Enrico Olivelli wrote:
> > This is a release candidate for 3.8.1.
> >
> > This is a bugfix release. The full release notes is available at:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> >
> > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***
> >
> > Source files:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachezookeeper-1085/
> >
> > The release candidate tag in git to be voted upon: release-3.8.1-0
> > https://github.com/apache/zookeeper/tree/release-3.8.1-0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Enrico Olivelli
> >

Re: [VOTE] Apache ZooKeeper release 3.8.1 candidate 0

[Enrico Olivelli]

Patrick

Il giorno lun 23 gen 2023 alle ore 23:11 Patrick Hunt
 ha scritto:
>
> Thanks Enrico - off the bat I did notice a couple license file mis-matches,
> not sure how you want to handle those:
>
> -rw-r--r--   1 phunt  staff11366 Jan 23 05:25
> netty-common-4.1.76.Final.LICENSE.txt
> -rw-r--r--   1 phunt  staff   654571 Dec 16 05:34
> netty-common-4.1.86.Final.jar
> -rw-r--r--   1 phunt  staff11366 Jan 23 05:25
> netty-transport-4.1.76.Final.LICENSE.txt
> -rw-r--r--   1 phunt  staff   488341 Dec 16 05:34
> netty-transport-4.1.86.Final.jar
>
> Should I continue verification or are you going to address/respin?

I will fix them and send a new RC., after addressing Andrey's problem
about the watcher

Continuing the verification would help in order to find other problems earlier.
But if you have limited time then you can hold on and wait for the new RC

Enrico


>
> Patrick
>
>
> On Mon, Jan 23, 2023 at 5:51 AM Enrico Olivelli  wrote:
>
> > This is a release candidate for 3.8.1.
> >
> > This is a bugfix release. The full release notes is available at:
> >
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801
> >
> > *** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***
> >
> > Source files:
> >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachezookeeper-1085/
> >
> > The release candidate tag in git to be voted upon: release-3.8.1-0
> > https://github.com/apache/zookeeper/tree/release-3.8.1-0
> >
> > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > https://www.apache.org/dist/zookeeper/KEYS
> >
> > The staging version of the website is:
> >
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/website/index.html
> >
> >
> > Should we release this candidate?
> >
> >
> > Enrico Olivelli
> >

[VOTE] Apache ZooKeeper release 3.8.1 candidate 0

[Enrico Olivelli]

This is a release candidate for 3.8.1.

This is a bugfix release. The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12351398=Html=12310801

*** Please download, test and vote by Thursday 26th 2023, 23:59 UTC+0. ***

Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachezookeeper-1085/

The release candidate tag in git to be voted upon: release-3.8.1-0
https://github.com/apache/zookeeper/tree/release-3.8.1-0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.1-candidate-0/website/index.html


Should we release this candidate?


Enrico Olivelli

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Il giorno lun 23 gen 2023 alle ore 13:54 Enrico Olivelli
 ha scritto:
>
> Actually I think that I am falling into a rabbit hole.
>
> The Contrib packages have many CVEs against third party libraries
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-4663 - OWASP is
> failing on loggraph due to yui-min.js: CVE-2013-4940, CVE-2013-4939
> https://issues.apache.org/jira/browse/ZOOKEEPER-4664 - OWASP is
> failing on zookeeper zookeeper-contrib-rest due to some third party
> dependencies
> https://issues.apache.org/jira/browse/ZOOKEEPER-4665 - OWASP is
> failing on zooinspector due to some third party dependencies
>
> There is too much work to do at the moment, and we can't blindly
> upgrade dependencies without proper testing.
>
> I am leaning towards creating the RC and ignoring all these problems.
> They don't affect the core code package, and they are optional
> modules, not deployed to Maven central or releases as binaries

During the release I have found that we actually stage the artifacts
in Maven central
but they are not supposed to be consumed from there.

This is a minor release, I think it is better to not change the layout.
We could improve the list of stuff that we send to Maven central in a
next major release

Enrico

>
> Enrico
>
> Il giorno lun 23 gen 2023 alle ore 13:30 Enrico Olivelli
>  ha scritto:
> >
> > Unfortunately I missed these OWASP failures on the contrib packages
> >
> > [ERROR] Failed to execute goal
> > org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> > zookeeper-it:
> > [ERROR]
> > [ERROR] One or more dependencies were identified with vulnerabilities
> > that have a CVSS score greater than or equal to '0.0':
> > [ERROR]
> > [ERROR] junit-4.13.jar: CVE-2020-15250(5.5)
> > [ERROR] junit-platform-engine-1.6.2.jar: CVE-2022-31514(9.3)
> > [ERROR]
> > [ERROR] See the dependency-check report for more details.
> >
> > I will send other PRs
> >
> > Enrico
> >
> > Il giorno gio 19 gen 2023 alle ore 12:07 Enrico Olivelli
> >  ha scritto:
> > >
> > > I have opened a few PRs,
> > > please help me review
> > >
> > > https://github.com/apache/zookeeper/pull/1972
> > > https://github.com/apache/zookeeper/pull/1971
> > > https://github.com/apache/zookeeper/pull/1970
> > >
> > > Enrico
> > >
> > > Il giorno gio 19 gen 2023 alle ore 11:43 Enrico Olivelli
> > >  ha scritto:
> > > >
> > > > Unfortunately OWASP check is failing on branch-3.8
> > > >
> > > > [ERROR] Failed to execute goal
> > > > org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> > > > zookeeper:
> > > > [ERROR]
> > > > [ERROR] One or more dependencies were identified with vulnerabilities
> > > > that have a CVSS score greater than or equal to '0.0':
> > > > [ERROR]
> > > > [ERROR] commons-cli-1.4.jar: CVE-2021-37533(6.5)
> > > > [ERROR] commons-io-2.11.0.jar: CVE-2021-37533(6.5)
> > > > [ERROR] jackson-databind-2.13.2.1.jar: CVE-2022-42003(7.5), 
> > > > CVE-2022-42004(7.5)
> > > > [ERROR] netty-transport-4.1.76.Final.jar: CVE-2022-41915(6.5),
> > > > CVE-2022-24823(5.5), CVE-2022-41881(7.5)
> > > > [ERROR]
> > > > [ERROR] See the dependency-check report for more details.
> > > > [ERROR]
> > > >
> > > > I will take a look if there are already patches to be cherry-picked.
> > > >
> > > > I guess it will take some time, I hoped to cut the release candidate 
> > > > today :-(
> > > >
> > > > Enrico
> > > >
> > > > Il giorno mar 17 gen 2023 alle ore 23:06 Chris Nauroth
> > > >  ha scritto:
> > > > >
> > > > > +1
> > > > >
> > > > > Thank you for taking this up, Enrico!
> > > > >
> > > > > Chris Nauroth
> > > > >
> > > > >
> > > > > On Tue, Jan 17, 2023 at 9:24 AM Enrico Olivelli  
> > > > > wrote:
> > > > >
> > > > > > Hello ZooKeepers,
> > > > > > We have received a few requests to cut a 3.8.1 release.
> > > > > >
> > > > > > I will start the release procedure by the end of this week,
> > > > > > if there anything that blocks the release or that you would like to
> > > > > > cherry-pick please let me know
> > > > > >
> > > > > > Best regards
> > > > > > Enrico
> > > > > >

[jira] [Created] (ZOOKEEPER-4665) OWASP is failing on zooinspector due to some third party dependencies

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4665:
--

 Summary: OWASP is failing on zooinspector due to some third party 
dependencies
 Key: ZOOKEEPER-4665
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4665
 Project: ZooKeeper
  Issue Type: Task
Reporter: Enrico Olivelli


 
{code:java}
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '0.0': 
[ERROR] 
[ERROR] apache-rat-tasks-0.6.jar: CVE-2020-22475(6.8), CVE-2022-39349(5.5)
[ERROR] commons-collections-3.2.jar: CVE-2015-6420(7.5)
[ERROR] guava-18.0.jar: CVE-2020-8908(3.3), CVE-2018-10237(5.9)
[ERROR] 
[ERROR] See the dependency-check report for more details.
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Actually I think that I am falling into a rabbit hole.

The Contrib packages have many CVEs against third party libraries

https://issues.apache.org/jira/browse/ZOOKEEPER-4663 - OWASP is
failing on loggraph due to yui-min.js: CVE-2013-4940, CVE-2013-4939
https://issues.apache.org/jira/browse/ZOOKEEPER-4664 - OWASP is
failing on zookeeper zookeeper-contrib-rest due to some third party
dependencies
https://issues.apache.org/jira/browse/ZOOKEEPER-4665 - OWASP is
failing on zooinspector due to some third party dependencies

There is too much work to do at the moment, and we can't blindly
upgrade dependencies without proper testing.

I am leaning towards creating the RC and ignoring all these problems.
They don't affect the core code package, and they are optional
modules, not deployed to Maven central or releases as binaries

Enrico

Il giorno lun 23 gen 2023 alle ore 13:30 Enrico Olivelli
 ha scritto:
>
> Unfortunately I missed these OWASP failures on the contrib packages
>
> [ERROR] Failed to execute goal
> org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> zookeeper-it:
> [ERROR]
> [ERROR] One or more dependencies were identified with vulnerabilities
> that have a CVSS score greater than or equal to '0.0':
> [ERROR]
> [ERROR] junit-4.13.jar: CVE-2020-15250(5.5)
> [ERROR] junit-platform-engine-1.6.2.jar: CVE-2022-31514(9.3)
> [ERROR]
> [ERROR] See the dependency-check report for more details.
>
> I will send other PRs
>
> Enrico
>
> Il giorno gio 19 gen 2023 alle ore 12:07 Enrico Olivelli
>  ha scritto:
> >
> > I have opened a few PRs,
> > please help me review
> >
> > https://github.com/apache/zookeeper/pull/1972
> > https://github.com/apache/zookeeper/pull/1971
> > https://github.com/apache/zookeeper/pull/1970
> >
> > Enrico
> >
> > Il giorno gio 19 gen 2023 alle ore 11:43 Enrico Olivelli
> >  ha scritto:
> > >
> > > Unfortunately OWASP check is failing on branch-3.8
> > >
> > > [ERROR] Failed to execute goal
> > > org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> > > zookeeper:
> > > [ERROR]
> > > [ERROR] One or more dependencies were identified with vulnerabilities
> > > that have a CVSS score greater than or equal to '0.0':
> > > [ERROR]
> > > [ERROR] commons-cli-1.4.jar: CVE-2021-37533(6.5)
> > > [ERROR] commons-io-2.11.0.jar: CVE-2021-37533(6.5)
> > > [ERROR] jackson-databind-2.13.2.1.jar: CVE-2022-42003(7.5), 
> > > CVE-2022-42004(7.5)
> > > [ERROR] netty-transport-4.1.76.Final.jar: CVE-2022-41915(6.5),
> > > CVE-2022-24823(5.5), CVE-2022-41881(7.5)
> > > [ERROR]
> > > [ERROR] See the dependency-check report for more details.
> > > [ERROR]
> > >
> > > I will take a look if there are already patches to be cherry-picked.
> > >
> > > I guess it will take some time, I hoped to cut the release candidate 
> > > today :-(
> > >
> > > Enrico
> > >
> > > Il giorno mar 17 gen 2023 alle ore 23:06 Chris Nauroth
> > >  ha scritto:
> > > >
> > > > +1
> > > >
> > > > Thank you for taking this up, Enrico!
> > > >
> > > > Chris Nauroth
> > > >
> > > >
> > > > On Tue, Jan 17, 2023 at 9:24 AM Enrico Olivelli  
> > > > wrote:
> > > >
> > > > > Hello ZooKeepers,
> > > > > We have received a few requests to cut a 3.8.1 release.
> > > > >
> > > > > I will start the release procedure by the end of this week,
> > > > > if there anything that blocks the release or that you would like to
> > > > > cherry-pick please let me know
> > > > >
> > > > > Best regards
> > > > > Enrico
> > > > >

[jira] [Created] (ZOOKEEPER-4664) OWASP is failing on zookeeper zookeeper-contrib-rest due to some third party deopendencies

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4664:
--

 Summary: OWASP is failing on zookeeper zookeeper-contrib-rest due 
to some third party deopendencies
 Key: ZOOKEEPER-4664
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4664
 Project: ZooKeeper
  Issue Type: Task
Reporter: Enrico Olivelli


 
{code:java}
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '0.0': 
[ERROR] 
[ERROR] grizzly-utils-1.9.8.jar: CVE-2014-0099(4.3), CVE-2021-4277(5.3)
[ERROR] jersey-core-1.1.5.1.jar: CVE-2014-3643(7.5)
[ERROR] jettison-1.1.jar: CVE-2022-40149(7.5), CVE-2022-45693(7.5), 
CVE-2022-40150(7.5), CVE-2022-45685(7.5)
[ERROR] 
[ERROR] See the dependency-check report for more details.
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4663) OWASP is failing on loggraph due to yui-min.js: CVE-2013-4940, CVE-2013-4939

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4663:
--

 Summary: OWASP is failing on loggraph due to yui-min.js: 
CVE-2013-4940, CVE-2013-4939
 Key: ZOOKEEPER-4663
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4663
 Project: ZooKeeper
  Issue Type: Task
  Components: contrib
Reporter: Enrico Olivelli


https://nvd.nist.gov/vuln/detail/CVE-2013-4939



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4662) OWASP is failing on the zookeeper-it module due to JUnit CVEs

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4662:
--

 Summary: OWASP is failing on the zookeeper-it module due to JUnit 
CVEs
 Key: ZOOKEEPER-4662
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4662
 Project: ZooKeeper
  Issue Type: Test
  Components: tests
Affects Versions: 3.8.0
Reporter: Enrico Olivelli
 Fix For: 3.9.0, 3.8.1, 3.7.2


 
{code:java}
[ERROR] One or more dependencies were identified with vulnerabilities that have 
a CVSS score greater than or equal to '0.0': 
[ERROR] 
[ERROR] junit-4.13.jar: CVE-2020-15250(5.5)
[ERROR] junit-platform-engine-1.6.2.jar: CVE-2022-31514(9.3)
[ERROR] 
[ERROR] See the dependency-check report for more details.
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Unfortunately I missed these OWASP failures on the contrib packages

[ERROR] Failed to execute goal
org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
zookeeper-it:
[ERROR]
[ERROR] One or more dependencies were identified with vulnerabilities
that have a CVSS score greater than or equal to '0.0':
[ERROR]
[ERROR] junit-4.13.jar: CVE-2020-15250(5.5)
[ERROR] junit-platform-engine-1.6.2.jar: CVE-2022-31514(9.3)
[ERROR]
[ERROR] See the dependency-check report for more details.

I will send other PRs

Enrico

Il giorno gio 19 gen 2023 alle ore 12:07 Enrico Olivelli
 ha scritto:
>
> I have opened a few PRs,
> please help me review
>
> https://github.com/apache/zookeeper/pull/1972
> https://github.com/apache/zookeeper/pull/1971
> https://github.com/apache/zookeeper/pull/1970
>
> Enrico
>
> Il giorno gio 19 gen 2023 alle ore 11:43 Enrico Olivelli
>  ha scritto:
> >
> > Unfortunately OWASP check is failing on branch-3.8
> >
> > [ERROR] Failed to execute goal
> > org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> > zookeeper:
> > [ERROR]
> > [ERROR] One or more dependencies were identified with vulnerabilities
> > that have a CVSS score greater than or equal to '0.0':
> > [ERROR]
> > [ERROR] commons-cli-1.4.jar: CVE-2021-37533(6.5)
> > [ERROR] commons-io-2.11.0.jar: CVE-2021-37533(6.5)
> > [ERROR] jackson-databind-2.13.2.1.jar: CVE-2022-42003(7.5), 
> > CVE-2022-42004(7.5)
> > [ERROR] netty-transport-4.1.76.Final.jar: CVE-2022-41915(6.5),
> > CVE-2022-24823(5.5), CVE-2022-41881(7.5)
> > [ERROR]
> > [ERROR] See the dependency-check report for more details.
> > [ERROR]
> >
> > I will take a look if there are already patches to be cherry-picked.
> >
> > I guess it will take some time, I hoped to cut the release candidate today 
> > :-(
> >
> > Enrico
> >
> > Il giorno mar 17 gen 2023 alle ore 23:06 Chris Nauroth
> >  ha scritto:
> > >
> > > +1
> > >
> > > Thank you for taking this up, Enrico!
> > >
> > > Chris Nauroth
> > >
> > >
> > > On Tue, Jan 17, 2023 at 9:24 AM Enrico Olivelli  
> > > wrote:
> > >
> > > > Hello ZooKeepers,
> > > > We have received a few requests to cut a 3.8.1 release.
> > > >
> > > > I will start the release procedure by the end of this week,
> > > > if there anything that blocks the release or that you would like to
> > > > cherry-pick please let me know
> > > >
> > > > Best regards
> > > > Enrico
> > > >

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Huang

Il giorno ven 20 gen 2023 alle ore 13:33 Binyu Huang
 ha scritto:
>
> Hi Enrico,
> I wish our specification for ZooKeeper and Zab could be merged in later 
> versions(like 3.8.1).
> It will be nice if someone could review our works.
>
> (with Key ZOOKEEPER-3615, and pull request 
> https://github.com/apache/zookeeper/pull/1690)

I think that this is only about the website, we can commit it outside
of the release cycle.

I have pinged a few folks, we need some reviewer to confirm the content

Thanks
Enrico

>
> Best wishes,
> Huang
>
> On 2023/01/17 17:22:38 Enrico Olivelli wrote:
> > Hello ZooKeepers,
> > We have received a few requests to cut a 3.8.1 release.
> >
> > I will start the release procedure by the end of this week,
> > if there anything that blocks the release or that you would like to
> > cherry-pick please let me know
> >
> > Best regards
> > Enrico
> >

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

I have opened a few PRs,
please help me review

https://github.com/apache/zookeeper/pull/1972
https://github.com/apache/zookeeper/pull/1971
https://github.com/apache/zookeeper/pull/1970

Enrico

Il giorno gio 19 gen 2023 alle ore 11:43 Enrico Olivelli
 ha scritto:
>
> Unfortunately OWASP check is failing on branch-3.8
>
> [ERROR] Failed to execute goal
> org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
> zookeeper:
> [ERROR]
> [ERROR] One or more dependencies were identified with vulnerabilities
> that have a CVSS score greater than or equal to '0.0':
> [ERROR]
> [ERROR] commons-cli-1.4.jar: CVE-2021-37533(6.5)
> [ERROR] commons-io-2.11.0.jar: CVE-2021-37533(6.5)
> [ERROR] jackson-databind-2.13.2.1.jar: CVE-2022-42003(7.5), 
> CVE-2022-42004(7.5)
> [ERROR] netty-transport-4.1.76.Final.jar: CVE-2022-41915(6.5),
> CVE-2022-24823(5.5), CVE-2022-41881(7.5)
> [ERROR]
> [ERROR] See the dependency-check report for more details.
> [ERROR]
>
> I will take a look if there are already patches to be cherry-picked.
>
> I guess it will take some time, I hoped to cut the release candidate today :-(
>
> Enrico
>
> Il giorno mar 17 gen 2023 alle ore 23:06 Chris Nauroth
>  ha scritto:
> >
> > +1
> >
> > Thank you for taking this up, Enrico!
> >
> > Chris Nauroth
> >
> >
> > On Tue, Jan 17, 2023 at 9:24 AM Enrico Olivelli  wrote:
> >
> > > Hello ZooKeepers,
> > > We have received a few requests to cut a 3.8.1 release.
> > >
> > > I will start the release procedure by the end of this week,
> > > if there anything that blocks the release or that you would like to
> > > cherry-pick please let me know
> > >
> > > Best regards
> > > Enrico
> > >

[jira] [Created] (ZOOKEEPER-4661) Upgrade Jackson Databind to 2.13.4.2 for CVE-2022-42003 CVE-2022-42004

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4661:
--

 Summary: Upgrade Jackson Databind to 2.13.4.2 for CVE-2022-42003 
CVE-2022-42004
 Key: ZOOKEEPER-4661
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4661
 Project: ZooKeeper
  Issue Type: Improvement
  Components: server
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4660) Suppress false positive OWASP failure for CVE-2021-37533

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4660:
--

 Summary: Suppress false positive OWASP failure for CVE-2021-37533
 Key: ZOOKEEPER-4660
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4660
 Project: ZooKeeper
  Issue Type: Improvement
Reporter: Enrico Olivelli


OWAPS reports a failure against CVE-2021-37533 for apache commons-io-2.11.0.

The CVE is for commons-net, not used by ZooKeeper



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4659) Upgrade Commons CLI to 1.5.0 due to OWASP failing on 1.4 CVE-2021-37533

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4659:
--

 Summary: Upgrade Commons CLI to 1.5.0 due to OWASP failing on 1.4 
CVE-2021-37533
 Key: ZOOKEEPER-4659
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4659
 Project: ZooKeeper
  Issue Type: Improvement
  Components: tools
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Unfortunately OWASP check is failing on branch-3.8

[ERROR] Failed to execute goal
org.owasp:dependency-check-maven:7.1.0:check (default-cli) on project
zookeeper:
[ERROR]
[ERROR] One or more dependencies were identified with vulnerabilities
that have a CVSS score greater than or equal to '0.0':
[ERROR]
[ERROR] commons-cli-1.4.jar: CVE-2021-37533(6.5)
[ERROR] commons-io-2.11.0.jar: CVE-2021-37533(6.5)
[ERROR] jackson-databind-2.13.2.1.jar: CVE-2022-42003(7.5), CVE-2022-42004(7.5)
[ERROR] netty-transport-4.1.76.Final.jar: CVE-2022-41915(6.5),
CVE-2022-24823(5.5), CVE-2022-41881(7.5)
[ERROR]
[ERROR] See the dependency-check report for more details.
[ERROR]

I will take a look if there are already patches to be cherry-picked.

I guess it will take some time, I hoped to cut the release candidate today :-(

Enrico

Il giorno mar 17 gen 2023 alle ore 23:06 Chris Nauroth
 ha scritto:
>
> +1
>
> Thank you for taking this up, Enrico!
>
> Chris Nauroth
>
>
> On Tue, Jan 17, 2023 at 9:24 AM Enrico Olivelli  wrote:
>
> > Hello ZooKeepers,
> > We have received a few requests to cut a 3.8.1 release.
> >
> > I will start the release procedure by the end of this week,
> > if there anything that blocks the release or that you would like to
> > cherry-pick please let me know
> >
> > Best regards
> > Enrico
> >

Cutting Apache ZooKeeper 3.8.1 release

[Enrico Olivelli]

Hello ZooKeepers,
We have received a few requests to cut a 3.8.1 release.

I will start the release procedure by the end of this week,
if there anything that blocks the release or that you would like to
cherry-pick please let me know

Best regards
Enrico

Re: Next release date

[Enrico Olivelli]

Marco,
3.9.0 has some big features still in progress, like adding
Backup/Restore support, so we would probably wait for that feature
to be fully merged before cutting a release.

I see that a few users like you are requesting a release out of the
3.8.x release line.
If nobody chimes in I will start the process to cut a 3.8.1 release,
it will take some time, probably in a couple of weeks we will have a
release

Thanks
Enrico

Il giorno mar 17 gen 2023 alle ore 18:17 Marco Barboza
 ha scritto:
>
> Hello.
> What is the estimated time for the release of the new version?
> Either 3.8.1 or 3.9.0. Looking forward from some changes that are fixing
> some CVE.
>
> Thanks in advance!
> --
> // Marco Barboza

Re: JDK 20 Rampdown Phase 1 & Valhalla LW4 Early-Access builds

[Enrico Olivelli]

Hello folks,
I have submitted a PR that fixes the problem
https://github.com/apache/zookeeper/pull/1959

With those changes the tests that failed now pass on JDK19 and JDK20

Please take a look

Enrico

Il giorno lun 12 dic 2022 alle ore 12:04 Enrico Olivelli
 ha scritto:
>
> David,
> we have one problem with our tests:
>
> See here:
> https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/82/testReport/junit/org.apache.zookeeper.test/StaticHostProviderTest/testEmptyResolution/
>
> The error is (even with latest Mockito):
> Cannot mock/spy class java.net.InetAddress
> Mockito cannot mock/spy because :
>  - sealed class
>
> My understanding is that we have to change our tests and stop mocking
> InetAddress, correct ?
>
> Enrico
>
> Il giorno lun 12 dic 2022 alle ore 11:23 David Delabassee
>  ha scritto:
> >
> > Welcome to the final OpenJDK Quality Outreach update for 2022!
> >
> > JDK 20, scheduled for General Availability on March 21 2023, is now in
> > Rampdown Phase One (RDP1) [1]. At this point, the overall JDK 20 [2]
> > feature set is frozen (see below the final list of JEPs integrated into
> > JDK 20) and only low-risk enhancements might still be considered. The
> > coming weeks should be used to identify and resolve as many issues as
> > possible, i.e. before JDK 20 enters the Release Candidates phase in
> > early February 2023.
> >
> >
> > ## JDK 20 Early-Access builds
> >
> > The latest Early-Access (builds 27) are available [2] with the Release
> > Notes here [3]. Those builds are provided under the GNU GPL v2, with the
> > Classpath Exception.
> >
> > ### JEPs integrated into JDK 20:
> >
> > JEP 429: Scoped Values (Incubator)
> > JEP 432: Record Patterns (2nd Preview)
> > JEP 433: Pattern Matching for switch (4th Preview)
> > JEP 434: Foreign Function & Memory API (2nd Preview)
> > JEP 436: Virtual Threads (2nd Preview)
> > JEP 437: Structured Concurrency (2nd Incubator)
> >
> > [1] https://mail.openjdk.org/pipermail/jdk-dev/2022-December/007233.html
> > [2] https://jdk.java.net/20/
> > [3] https://jdk.java.net/20/release-notes
> >
> >
> > ### Changes in recent JDK 20 builds that may be of interest:
> >
> >  Build 27:
> > - JDK-8297794: Deprecate JMX Management Applets for Removal
> > - JDK-8297118: Change IncompatibleClassChangeError to MatchException for
> > exhaustive switch statements and switch expressions
> > - JDK-8294047: HttpResponseInputStream swallows interrupts
> > - JDK-8281236: (D)TLS key exchange named groups
> > - JDK-8280798: com.sun.jdi.ObjectReference::setValue spec should
> > prohibit any final field modification
> > - JDK-8295350: JFR: Add stop methods for recording streams
> > - JDK-8295044: Implementation of Foreign Function and Memory API (2nd
> > Preview)
> > - JDK-8296896: Change virtual Thread.yield to use external submit
> > - JDK-8297804: (tz) Update Timezone Data to 2022g
> > - JDK-8295803: Console should be usable in jshell and other environments
> > - JDK-828: Implementation of Scoped Values (Incubator)
> > - JDK-8296672: Implementation of Virtual Threads (2nd Preview)
> >
> >  Build 26:
> > - JDK-8297276: Remove thread text from Subject.current
> > - JDK-8297030: Reduce Default Keep-Alive Timeout Value for httpclient
> > - JDK-8247645: ChaCha20 Intrinsics
> >
> >  Build 25:
> > - JDK-8296472: Remove ObjectLocker around
> > appendToClassPathForInstrumentation call
> > - JDK-8290313: Produce warning when user specified java.io.tmpdir
> > directory doesn't exist
> > - JDK-8288717: Add a means to close idle connections in HTTP/2
> > connection pool
> > - JDK-8288047: Accelerate Poly1305 on x86_64 using AVX512 instructions
> > - JDK-8059632: Method reference compilation uses incorrect qualifying type
> > - JDK-8297161: Add additional Service Attributes to Standard Algorithm
> > Names guide
> > - JDK-8294073: Performance improvement for message digest implementations
> >
> >  Build 24:
> > - JDK-8294731: Improve multiplicative inverse for secp256r1 implementation
> > - JDK-8296715: CLDR v42 update for tzdata 2022f
> > - JDK-8296958: [JVMCI] add API for retrieving ConstantValue attributes
> >
> >  Build 23:
> > - JDK-8296226: Add constructors (String,Throwable) and (Throwable) to
> > InvalidParameterException
> > - JDK-8295673: Deprecate and disable legacy parallel class loading
> > workaround for non-parallel-capable class loaders
> > - JDK-8294241: Deprecate URL public constructors
> &

Re: Hope to release 3.8.1 to solve ZOOKEEPER-4460 related to JDK17

[Enrico Olivelli]

Sorry for late reply.
We should definitely cut a release

Any one who volunteers ?

Enrico

Il giorno sab 7 gen 2023 alle ore 07:26 Ling Hengqian
 ha scritto:
>
> Dear community:
>
>Hi!
>
>I expect our community to release Zookeeper 3.8.1 or Zookeeper 3.9.0 
> in the near future to carry the corresponding PR of ZOOKEEPER-4460, namely 
> https://github.com/apache/zookeeper/pull/1942 .
>
> This will help resolve the issue that occurred at 
> https://github.com/oracle/graalvm-reachability-metadata/issues/163 , an issue 
> related to ZOOKEEPER-4460 that prevents Zookeeper Server from running 
> nativeTest under GraalVM 22.3 For JDK17, and It caused the packaging of 
> GraalVM Native Image to fail.
>
> A minimal reproducible Example and corresponding reproduction steps are 
> located at https://github.com/linghengqian/zookeeper-jdk17-nativetest .
>
> Unfortunately I can't fix this problem by specifying Zookeeper 3.6.4 in 
> Gradle, because Zookeeper 3.6.4 is using Log4j, which involves problems that 
> must be solved in Log4j2 3.x, refer to 
> https://github.com/oracle/graalvm-reachability-metadata/issues/50 .
>
> With best wishes.
>
> Ling Hengqian
> 2022.01.07
>
> 从 Windows 版邮件发送
>

Re: [ANNOUNCE] Year 2023 :)

[Enrico Olivelli]

LOL

Enrico

Il giorno sab 31 dic 2022 alle ore 17:26 Szalay-Bekő Máté
 ha scritto:
>
> The Apache ZooKeeper team is proud to announce Year 2023
>
> You may have noticed that the year 2022 also gets EoL very soon. The new
> 2023 version was released successfully and - depending on time zones -
> people started to use it already.
>
> Based on the feedbacks the upgrades seemed to be smooth so far, at least no
> incompatibilities found. Sadly, we were not able to test this version
> before the release. Most likely it will introduce some new issues but
> hopefully also will solve some of the old ones. Let's enjoy it anyway...
>
> Happy new year! ;)

Re: Request for review [jira](ZOOKEEPER-3615)

[Enrico Olivelli]

Huang,
Thank you very much.

I hope we can merge it soon.

I have pinged a few people who participated in the reviews

Enrico

Il Gio 12 Gen 2023, 12:56 黄 彬寓  ha scritto:

> Hi everyone!
> I have completed writing TLA+ specification for ZooKeeper.(see
> ZOOKEEPER-3615, with links <
> https://issues.apache.org/jira/browse/ZOOKEEPER-3615> and
> https://github.com/apache/zookeeper/pull/1690).
>
> I have discussed this issue with community members on github, and now it
> has been shelved. I hope anyone interested in it could give me some
> suggestions for improvement, and finally merge this issue. Please
> communicate with me anytime if you have suggestions.
>
> Best wishes,
> Huang
>

Re: Request to join slack

[Enrico Olivelli]

Ben,

Il Gio 5 Gen 2023, 20:45 Ben Johnston  ha
scritto:

> Hello,
>
> I am an app sec engineer for a company who uses Zookeeper. I would like to
> join the slack as a guest to get some visibility on the release process,
>
We are not using slack for communication about those things.

There is a global ASF slack space, usually open only to committers +
guests, and we have a zookeeper channel. But it is only meant for informal
quick chats, like pinging someone for review.

Discussions happen here on dev@ and if you want to report a new issue you
have to use security zookeeper.apache.org that is a private list.



especially as it relates to updating library versions to fix CVEs. My team
> is tracking several CVEs
>
> CVE-2022-42003/4
> CVE-2020-36518
> CVE-2022-41915
>
> We’re on the 3.8 version. Thanks!
>
 Those CVEs are about library upgrades.

Can you please double check if we already upgraded those libraries?

I think that it would only be a matter of cutting a release. It has been
quite some time that we didn't cut a release out of the 3.8 branch. We can
do it.

Jackson is usually easily upgradable and Netty requires only some testing.

If you can't wait for a release you could upgrade those libraries and build
your package

Thanks

Enrico



>
> *Ben Johnston, GCIH, GCFA, GPEN*
>
> Application Security Engineer
>
> *COFENSE*
>
> *o.* 785-250-4412
>
> *e.* ben.johns...@cofense.com
>
>
>
> *Connect with Cofense:*
>
>
>
> [image: https://cofense.com/wp-content/uploads/2019/07/cofense.png]
> [image:
> https://cofense.com/wp-content/uploads/2019/06/fb.png]
> [image:
> https://cofense.com/wp-content/uploads/2019/06/tw.png]
> [image:
> https://cofense.com/wp-content/uploads/2019/06/li.png]
> [image:
> https://cofense.com/wp-content/uploads/2019/06/ig.png]
> [image:
> https://cofense.com/wp-content/uploads/2019/06/m.png]
> 
>
>
>
>
>
>

So many good PRs open !

[Enrico Olivelli]

Hello ZooKeepers,

I am very happy to see that recently there are many new contributions
coming to our GitHub space.
https://github.com/apache/zookeeper/pulls

I would like to gently ping all the committers and suggest we spend a
little time in reviews, this way we can leverage all the big value
coming from the community.

Also, it would be enormously useful for anyone who wants to share
their opinions/reviews on open pull requests.

The community is actually made of the people who participate in the
project, and the opinion of everyone is always welcome.


I will be happy to drive the upcoming 3.9.0 release and it will be
great to see all the precious contributions committed soon.

Thank you to everyone who is helping our project

Enrico

[jira] [Created] (ZOOKEEPER-4656) Allow ZooKeeperAdmin creation with custom HostProvider

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4656:
--

 Summary: Allow ZooKeeperAdmin creation with custom HostProvider
 Key: ZOOKEEPER-4656
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4656
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4655) Communicate the Zxid that triggered a WatchEvent to fire

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4655:
--

 Summary:  Communicate the Zxid that triggered a WatchEvent to fire
 Key: ZOOKEEPER-4655
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4655
 Project: ZooKeeper
  Issue Type: New Feature
Reporter: Enrico Olivelli






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Call for reviews for ZOOKEEPER-4475 and ZOOKEEPER-4466

[Enrico Olivelli]

Kezhu,
Sorry for late reply.
We should definitely move forward with this work


Enrico

Il Lun 17 Ott 2022, 16:27 Kezhu Wang  ha scritto:

> Ping.
>
> Best,
> Kezhu Wang
>
>
> On July 1, 2022 at 11:38:16, Kezhu Wang (kez...@gmail.com) wrote:
>
> Hi tison,
>
> Thank you for reviewing.
>
> pr#1859 tries to support standard watches and persistent watches on same
> paths. It has no code conflicts with pr#1820, but test requirement on
> pr#1820. Assumes that:
>
> 1. Persistent watch (and/or child watch) on “/a”
> 2. Persistent recursive watch on “/a”
>
> Ideally, persistent watch and/or child watch should receive
> `NodeChildrenChanged` while persistent recursive watch should not. Without
> pr#1820 which filter out `NodeChildrenChanged` for persistent recursive
> watch in client side, test introduced in pr#1859 will fail.
>
> There are other followups, which are related to watcher removing, I have
> reported but blocked by pr#1859(aka. ZOOKEEPER-4466):
> * ZOOKEEPER-4471[1]: Remove WatcherType.Children break persistent watcher's
> child events
> * ZOOKEEPER-4472[2]: Support persistent watchers removing individually
>
> [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
>
> Best,
> Kezhu Wang
>
> On June 29, 2022 at 17:19:37, tison (wander4...@gmail.com) wrote:
>
> Thanks for your contribution Kezhu!
>
> I've reviewed PR-1820. It looks good to me. PR-1859 seems a followup of
> 1820, will review 1859 after 1820 get accepted.
>
> Best,
> tison.
>
>
> Kezhu Wang  于2022年6月28日周二 23:17写道：
>
> > Hi guys,
> >
> > First, let me summarize changes of these two issues and associated prs
> > here.
> >
> > ZOOKEEPER-4475[1] reports that NodeChildrenChanged could be delivered to
> > persistent recursive watchers if a child watch is created on descendants
> of
> > node being watched using persistent recursive watch. pr#1820[2] solves
> this
> > by filtering out NodeChildrenChanged events for persistent recursive
> > watches on the client side.
> >
> > ZOOKEEPER-4466[3] reports that standard watch and persistent watch could
> > not coexist on same path. pr#1859[4] introduces WatchStats to count and
> > coexist different modes on same path.
> >
> > pr#1820 has been opened for a while but received no reviews. I think it
> is
> > pretty simple and solves a simple bug. It should take a long time to
> > review.
> >
> > For pr#1859, @eolivelli has given valuable comments. But both I and
> > @eolivelli think ZOOKEEPER-4466 deserves more attention. So, basically,
> we
> > need more reviewers to make sure pr#1859 goes in the right direction and
> > breaks no sensible codes.
> >
> > It would be appreciated if any reviewers could take a look at these prs.
> >
> > Best,
> > Kezhu Wang
> >
> > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4475
> > [2]: https://github.com/apache/zookeeper/pull/1820
> > [3]: https://issues.apache.org/jira/browse/ZOOKEEPER-4466
> > [4]: https://github.com/apache/zookeeper/pull/1859
> >
>

Re: Two issues in ZooKeeper that might cause data inconsistency or committed data loss

[Enrico Olivelli]

Yang,
Thanks for your report


Il Mar 13 Dic 2022, 18:16 Yang Sirius  ha
scritto:

> Hi everyone!
>
> Recently we discovered two issues in the ZooKeeper’s latest versions that
> might cause data inconsistency or committed data loss. Details and analysis
> of the issues are presented on JIRA:
>
>
>- ZOOKEEPER-4643  :
> Committed txns may be improperly truncated if follower crashes right
>after updating currentEpoch but before persisting txns to disk.
>- ZOOKEEPER-4646 
> : Committed txns may still be lost if followers crash after replying
>ACK-LD but before writing txns to disk. (This issue is related to the fix
>of ZOOKEEPER-3911
>)
>
>
> The issues seem to be critical since they lead to data loss or
> inconsistency, which violate the properties that ZAB is supposed to
> satisfy. I wonder whether the bugs should get a fix since data consistency
> is of prime importance of ZooKeeper. If so, I will try to fix the code
> together with further testing and verification techniques.
>

Help is always welcome!

I personally don't have time to investigate and code a fix, but I will be
happy to review your work

Thank you very much

Sharing problems and solutions is fundamental for an OSS community like
Apache ZooKeeper

Cheers
Enrico



> Thanks!
>
> Attached are example traces of these two issues that have been generated
> in multiple versions such as 3.8.0 & 3.7.1. (The traces are also provided
> on JIRA.)
> Trace-ZK-4643:
> Trace-ZK-4646:
>

Re: [EXTERNAL]ZooKeeper 3.6.4 Release Date - EoL for 3.6 after 3.6.4?

[Enrico Olivelli]

Badava,
Thanks for your report.

We do our best to never break compatibility, so upgrading to the latest
versions should be doable in all of those projects.

Usually people are afraid to touch this part of the system because it is
always the heart (source of truth).

We are going to release 3.9.0 soon, and keeping too many branches alive it
not feasible for the community.



Enrico


Il Ven 16 Dic 2022, 00:14 Badaya, Dhananjay  ha
scritto:

> Hello Team,
>
> We really appreciate the work to release 3.6.4 as we have been waiting for
> it for some time (due to reload4j changes).
>
> But at the same time, we were surprised to hear that 3.6.x is going to be
> EOL. As per Maven Central
> https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper, 3.7.x
> and 3.8.x versions are relatively less adopted. Also, a lot of Apache
> projects are still using ZK 3.6.x or older. As we use these projects in our
> big data clusters, upgrading to 3.7.x may get difficult due to this.
>
> AppsZK client version in OSS trunk
> Hadoop   3.6.3
> https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L100
> Spark   3.6.2   https://github.com/apache/spark/blob/master/pom.xml#L129
> Hive3.5.5   https://github.com/apache/hive/blob/master/pom.xml#L208
> Presto  3.4.14
> https://github.com/prestodb/presto/blob/master/pom.xml#L1870
> HBase   3.5.7   https://github.com/apache/hbase/blob/master/pom.xml#L831
> Sqoop   3.4.6
> https://github.com/apache/sqoop/blob/trunk/gradle.properties#L26
> Pig 3.5.7
> https://github.com/apache/pig/blob/trunk/ivy/libraries.properties#L74
> Ranger  3.4.14  https://github.com/apache/ranger/blob/master/pom.xml#L208
> Giraph  3.4.5   https://github.com/apache/giraph/blob/trunk/pom.xml#L355
> Hama3.4.5   https://github.com/apache/hama/blob/master/pom.xml#L102
>
> I understand that maintaining an older release is time consuming, but just
> wanted to provide this data highlighting the existing usage of 3.6.x branch
> to make sure we make the right trade-offs.
>
> Thanks,
> Dhananjay
>
>
>
> CAUTION: This email originated from outside of the organization. Do
> not click links or open attachments unless you can confirm the sender and
> know the content is safe.
>
>
>
> Thank you for taking care of Jetty!!
> I'll run CVE checks and also go through all the important dependencies,
> bumping them if needed.
>
> Máté
>
> On Tue, Dec 6, 2022 at 7:00 PM Chris Nauroth 
> wrote:
>
> > I would like to get this Jetty dependency upgrade into 3.6.4:
> >
> > https://github.com/apache/zookeeper/pull/1941
> >
> > This may be a good time to look at locking in any other relevant
> dependency
> > upgrades, as we're considering this to be the last 3.6 release.
> >
> > Chris Nauroth
> >
> >
> > On Mon, Dec 5, 2022 at 9:14 AM Chris Nauroth 
> wrote:
> >
> > > +1 for retiring the 3.6 release line after 3.6.4.
> > >
> > > Máté, thank you for helping with the release!
> > >
> > > Chris Nauroth
> > >
> > >
> > > On Sat, Dec 3, 2022 at 10:21 AM Enrico Olivelli <
> eolive...@gmail.com>
> > > wrote:
> > >
> > >> Mate,
> > >>
> > >> Il Sab 3 Dic 2022, 16:53 Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> > ha
> > >> scritto:
> > >>
> > >> > Hello!
> > >> >
> > >> > I think I can spare the time for starting the release process
> sometime
> > >> in
> > >> > the second part of next week.
> > >> >
> > >>
> > >> Thanks! I will test it and VOTE
> > >>
> > >>
> > >>
> > >> > I see the last 3.6 release happened on 13 April, 2021 - pretty
> long
> > ago.
> > >> > There are plenty of tickets to ship, including the log4j ->
> reload4j
> > >> > migration: https://issues.apache.org/jira/browse/ZOOKEEPER-4455
> > >>
> > >>
> > >>
> > >> This is very important
> > >>
> > >> >
> > >> >
> > >> > All tickets with fixVersion=3.6.4:
> > >> >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/browse/ZOOKEEPER-4602?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.6.4%20ORDER%20BY%20priority%20DESC%2C%20updated%20

Re: [VOTE] Apache ZooKeeper release 3.6.4 candidate 2

[Enrico Olivelli]

+1 (binding)

Built from sources and run all the tests (on Mac M1 with JDK8
1.8.0_352, vendor: Azul Systems)

Enrico Olivelli

Il giorno lun 19 dic 2022 alle ore 18:07 Chris Nauroth
 ha scritto:
>
> +1 (binding)
>
> - Verified all checksums.
> - Verified all signatures.
> - Built from source, including native code on Linux.
> - Tests passed.
> - Ran several small samples successfully.
>
> Máté, thank you for your diligence with these release candidates!
>
> Chris Nauroth
>
>
> On Sun, Dec 18, 2022 at 12:03 PM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > I did the following tests for the release candidate:
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build) on
> > Ubuntu 20.04.5 using OpenJDK 8u352, maven 3.6.3 and GCC version 9.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built and executed unit tests for zkpython
> > - I also built the java code (without -Pfull-build) using other JDK
> > versions: 11.0.15, 17.0.3, 18.0.1, 19.0.1 (but didn't run the tests this
> > time, just used 'clean install -DskipTests')
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - fatjar built
> > - I executed quick rolling-upgrade tests (using
> > https://github.com/symat/zk-rolling-upgrade-test):
> >   - rolling upgrade from 3.5.10 to 3.6.4
> >   - rolling upgrade from 3.6.3 to 3.6.4
> >   - rolling upgrade from 3.6.4 to 3.7.1
> >   - rolling upgrade from 3.6.4 to 3.8.0
> > - checked the generated documentation (zookeeper-docs/target/html)
> > - compared generated release notes (
> >
> > https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/releasenotes.html
> > )
> > with Jira (
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > )
> >
> > Best regards,
> > Máté
> >
> > On Sun, Dec 18, 2022 at 9:01 PM Szalay-Bekő Máté <
> > szalay.beko.m...@gmail.com>
> > wrote:
> >
> > > This is a bugfix release candidate for 3.6.4. It fixes 42 issues,
> > > including CVE fixes,
> > > log4j1 removal (using reload4j from now) and various other bug fixes
> > > (thread leaks, data
> > > corruption, snapshotting and SASL related fixes).
> > >
> > > Please note, that based on our Release Strategy (
> > > https://zookeeper.apache.org/releases.html#release-strategy) branch 3.6
> > > should become end-of-life and most likely 3.6.4 will be our last 3.6
> > > release.
> > >
> > > The full release notes is available at:
> > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12350076
> > >
> > > *** Please download, test and vote by December 29th 2022, 23:59 UTC+0.
> > ***
> > >
> > >
> > > Source files:
> > > https://people.apache.org/~symat/zookeeper-3.6.4-rc2/
> > >
> > > Maven staging repo:
> > >
> > >
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.6.4/
> > >
> > > The release candidate tag in git to be voted upon: release-3.6.4-2
> > > (please note, branch-3.6.4 will move here only after the vote)
> > >
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > >
> > > The staging version of the website is:
> > > https://people.apache.org/~symat/zookeeper-3.6.4-rc2/website/
> > >
> > >
> > > Should we release this candidate?
> > >
> > >
> > > Best regards,
> > > Máté
> > >
> >

[jira] [Created] (ZOOKEEPER-4647) Tests don't pass on JDK20 because we try to mock InetAddress

[Enrico Olivelli (Jira)]

Enrico Olivelli created ZOOKEEPER-4647:
--

 Summary: Tests don't pass on JDK20 because we try to mock 
InetAddress
 Key: ZOOKEEPER-4647
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4647
 Project: ZooKeeper
  Issue Type: Test
Reporter: Enrico Olivelli


This test fails on JDK20-Ea



org.apache.zookeeper.test.StaticHostProviderTest.testEmptyResolution



Mockito cannot mock this class: class java.net.InetAddress. Mockito can only 
mock non-private & non-final classes. If you're not sure why you're getting 
this error, please report to the mailing list.

if I try to upgrade  Mockito to 4.9.0  the error is

org.mockito.exceptions.base.MockitoException: 

Cannot mock/spy class java.net.InetAddress

Mockito cannot mock/spy because :

 - sealed class

 

 at 
org.apache.zookeeper.test.StaticHostProviderTest.testReResolvingSingle(StaticHostProviderTest.jav



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: JDK 20 Rampdown Phase 1 & Valhalla LW4 Early-Access builds

[Enrico Olivelli]

(adding David to the thread, I am not sure that he is subscribe)

Il giorno lun 12 dic 2022 alle ore 12:04 Enrico Olivelli
 ha scritto:
>
> David,
> we have one problem with our tests:
>
> See here:
> https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/82/testReport/junit/org.apache.zookeeper.test/StaticHostProviderTest/testEmptyResolution/
>
> The error is (even with latest Mockito):
> Cannot mock/spy class java.net.InetAddress
> Mockito cannot mock/spy because :
>  - sealed class
>
> My understanding is that we have to change our tests and stop mocking
> InetAddress, correct ?
>
> Enrico
>
> Il giorno lun 12 dic 2022 alle ore 11:23 David Delabassee
>  ha scritto:
> >
> > Welcome to the final OpenJDK Quality Outreach update for 2022!
> >
> > JDK 20, scheduled for General Availability on March 21 2023, is now in
> > Rampdown Phase One (RDP1) [1]. At this point, the overall JDK 20 [2]
> > feature set is frozen (see below the final list of JEPs integrated into
> > JDK 20) and only low-risk enhancements might still be considered. The
> > coming weeks should be used to identify and resolve as many issues as
> > possible, i.e. before JDK 20 enters the Release Candidates phase in
> > early February 2023.
> >
> >
> > ## JDK 20 Early-Access builds
> >
> > The latest Early-Access (builds 27) are available [2] with the Release
> > Notes here [3]. Those builds are provided under the GNU GPL v2, with the
> > Classpath Exception.
> >
> > ### JEPs integrated into JDK 20:
> >
> > JEP 429: Scoped Values (Incubator)
> > JEP 432: Record Patterns (2nd Preview)
> > JEP 433: Pattern Matching for switch (4th Preview)
> > JEP 434: Foreign Function & Memory API (2nd Preview)
> > JEP 436: Virtual Threads (2nd Preview)
> > JEP 437: Structured Concurrency (2nd Incubator)
> >
> > [1] https://mail.openjdk.org/pipermail/jdk-dev/2022-December/007233.html
> > [2] https://jdk.java.net/20/
> > [3] https://jdk.java.net/20/release-notes
> >
> >
> > ### Changes in recent JDK 20 builds that may be of interest:
> >
> >  Build 27:
> > - JDK-8297794: Deprecate JMX Management Applets for Removal
> > - JDK-8297118: Change IncompatibleClassChangeError to MatchException for
> > exhaustive switch statements and switch expressions
> > - JDK-8294047: HttpResponseInputStream swallows interrupts
> > - JDK-8281236: (D)TLS key exchange named groups
> > - JDK-8280798: com.sun.jdi.ObjectReference::setValue spec should
> > prohibit any final field modification
> > - JDK-8295350: JFR: Add stop methods for recording streams
> > - JDK-8295044: Implementation of Foreign Function and Memory API (2nd
> > Preview)
> > - JDK-8296896: Change virtual Thread.yield to use external submit
> > - JDK-8297804: (tz) Update Timezone Data to 2022g
> > - JDK-8295803: Console should be usable in jshell and other environments
> > - JDK-828: Implementation of Scoped Values (Incubator)
> > - JDK-8296672: Implementation of Virtual Threads (2nd Preview)
> >
> >  Build 26:
> > - JDK-8297276: Remove thread text from Subject.current
> > - JDK-8297030: Reduce Default Keep-Alive Timeout Value for httpclient
> > - JDK-8247645: ChaCha20 Intrinsics
> >
> >  Build 25:
> > - JDK-8296472: Remove ObjectLocker around
> > appendToClassPathForInstrumentation call
> > - JDK-8290313: Produce warning when user specified java.io.tmpdir
> > directory doesn't exist
> > - JDK-8288717: Add a means to close idle connections in HTTP/2
> > connection pool
> > - JDK-8288047: Accelerate Poly1305 on x86_64 using AVX512 instructions
> > - JDK-8059632: Method reference compilation uses incorrect qualifying type
> > - JDK-8297161: Add additional Service Attributes to Standard Algorithm
> > Names guide
> > - JDK-8294073: Performance improvement for message digest implementations
> >
> >  Build 24:
> > - JDK-8294731: Improve multiplicative inverse for secp256r1 implementation
> > - JDK-8296715: CLDR v42 update for tzdata 2022f
> > - JDK-8296958: [JVMCI] add API for retrieving ConstantValue attributes
> >
> >  Build 23:
> > - JDK-8296226: Add constructors (String,Throwable) and (Throwable) to
> > InvalidParameterException
> > - JDK-8295673: Deprecate and disable legacy parallel class loading
> > workaround for non-parallel-capable class loaders
> > - JDK-8294241: Deprecate URL public constructors
> > - JDK-8289689: (fs) Re-examine the need for normalization to Unicode
> > Normalization Format D (macOS)
> > - JDK-8279164: Disabl