[ANNOUNCE] New ZooKeeper PMC member: Damien Diederen

[Andor Molnar]

I am happy to announce that Damien Diederen has been invited to join
the Apache ZooKeeper PMC and he accepted.

Damien is doing great work for our community.

Please join me in congratulating with him

Congrats Damien !


If you want to know more about the ASF works and what is a PMC you can
read more here
https://www.apache.org/foundation/how-it-works.html#pmc

[ANNOUNCE] New ZooKeeper PMC member: Zili Chen

[Andor Molnar]

I am happy to announce that Zili Chen (tison) has been invited to join
the Apache ZooKeeper PMC and he accepted.

Zili is doing great work for our community.

Please join me in congratulating with him

Congrats Mate !


If you want to know more about the ASF works and what is a PMC you can
read more here
https://www.apache.org/foundation/how-it-works.html#pmc

Re: Proposal for Updating the Wiki Page and JIRA Ticket (ZOOKEEPER-4805)

[Andor Molnar]

Thanks Villo.

Now the below wiki page should have all the updated information related
to the recent improvements that Villo made on the merge script.

It should be capable of merging PRs via GitHub API, use Jira via access
token, no need to expose passwords anymore, etc.

The script should be the primary tool for committers to submit pull
requests and create backports. Generally preferred over using GH UI.

Changes on the wiki page authored by me, because Villo doesn't have
write permissions. Any feedback appriciated as mentioned below.

Regards,
Andor



On Tue, 2024-03-26 at 10:22 +0100, Villő Szűcs wrote:
> Hi ZooKeeper Dev Team,
> 
> I hope this email finds you well. I wanted to bring to your attention
> some
> updates I have made to the ZooKeeper wiki page titled "Merging Github
> Pull
> Requests." Additionally, I have linked these updates to an associated
> JIRA
> ticket, ZOOKEEPER-4805
> ;.
> 
> You can find the updated wiki page here:
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Merging+Github+Pull+Requests
> 
> 
> I welcome any feedback or suggestions you may have regarding these
> updates.
> Please feel free to review the wiki page and the associated JIRA
> ticket at
> your convenience.
> 
> Thank you for your attention to this matter, and I look forward to
> hearing
> from you soon.
> 
> Best regards,
> Villő

Re: Backport to active branches by default

[Andor Molnar]

Do you mean handle 3.9 and 3.8 slightly differently and be more strict
on branch-3.8?

I can agree with that, but 3.9 can still receive more patches.

Andor




On Wed, 2024-03-20 at 13:16 -0700, Patrick Hunt wrote:
> Shouldn't we only backport critical fixes into the non-mainline
> branch? The
> whole idea is that that's the "stable" release while the mainline is
> the
> most current...
> 
> Regards,
> 
> Patrick
> 
> On Wed, Mar 20, 2024 at 12:54 PM Andor Molnar 
> wrote:
> 
> > Hi ZK committers,
> > 
> > I've come across recently that patch authors keep asking me to
> > backport
> > their patches to active branches, because it was only submitted to
> > the
> > master branch.
> > 
> > I think we should get into the habit of submitting every accepted
> > PRs
> > to all active branches (today it's branch-3.8 and branch-3.9)
> > unless
> > it's explicitly asked otherwise.
> > 
> > For example, in case of a big new feature which requires a major
> > version upgrade, we should not do that automatically, but for
> > everything else, like bug fixes, improvements, code cleanups, doc
> > updates, etc. feel free and submit everywhere.
> > 
> > If we don't do that, we'll end up not shipping anything in minor
> > releases.
> > 
> > What do you think?
> > 
> > Regards,
> > Andor
> > 
> > 
> > 
> > 

Backport to active branches by default

[Andor Molnar]

Hi ZK committers,

I've come across recently that patch authors keep asking me to backport
their patches to active branches, because it was only submitted to the
master branch.

I think we should get into the habit of submitting every accepted PRs
to all active branches (today it's branch-3.8 and branch-3.9) unless
it's explicitly asked otherwise. 

For example, in case of a big new feature which requires a major
version upgrade, we should not do that automatically, but for
everything else, like bug fixes, improvements, code cleanups, doc
updates, etc. feel free and submit everywhere.

If we don't do that, we'll end up not shipping anything in minor
releases.

What do you think?

Regards,
Andor

Re: CVE-2024-23944: Apache ZooKeeper: Information disclosure in persistent watcher handling

[Andor Molnar]

Hi Li,

That's the right ticket.

I've just updated the Jira ticket with the links to the commits.
There's no PR since it was a security fix, but looks like we forgot to
add it to the master branch.

Damien, would you please take care of that?

Btw, we don't plan to fix it in the 3.7 release line, but the patch is
already on the branch for your convenience:
29c7b9462681f47c2ac12e609341cf9f52abac5c

Regards,
Andor



On Thu, 2024-03-14 at 12:58 -0700, Li Wang wrote:
> Thanks, Andor.
> 
> Do you have the PR link for the fix in 3.9.2 and 3.8.4? There is a
> JIRA ticket in the release notes of 3.9.2 and 3.8.4, but the status
> is
> still OPEN and there is no PR link there.
> 
> https://issues.apache.org/jira/browse/ZOOKEEPER-4799
> 
> We are in 3.7.2 and may need to patch it ourselves.
> 
> Best,
> 
> Li
> 
> 
> 
> On Thu, Mar 14, 2024 at 8:52 AM Andor Molnar 
> wrote:
> 
> > Severity: critical
> > 
> > Affected versions:
> > 
> > - Apache ZooKeeper 3.9.0 through 3.9.1
> > - Apache ZooKeeper 3.8.0 through 3.8.3
> > - Apache ZooKeeper 3.6.0 through 3.7.2
> > 
> > Description:
> > 
> > Information disclosure in persistent watchers handling in Apache
> > ZooKeeper
> > due to missing ACL check. It allows an attacker to monitor child
> > znodes by
> > attaching a persistent watcher (addWatch command) to a parent which
> > the
> > attacker has already access to. ZooKeeper server doesn't do ACL
> > check when
> > the persistent watcher is triggered and as a consequence, the full
> > path of
> > znodes that a watch event gets triggered upon is exposed to the
> > owner of
> > the watcher. It's important to note that only the path is exposed
> > by this
> > vulnerability, not the data of znode, but since znode path can
> > contain
> > sensitive information like user name or login ID, this issue is
> > potentially
> > critical.
> > 
> > Users are recommended to upgrade to version 3.9.2, 3.8.4 which
> > fixes the
> > issue.
> > 
> > Credit:
> > 
> > 周吉安(寒泉)  (reporter)
> > 
> > References:
> > 
> > https://zookeeper.apache.org/
> > https://www.cve.org/CVERecord?id=CVE-2024-23944
> > 
> > 

CVE-2024-23944: Apache ZooKeeper: Information disclosure in persistent watcher handling

[Andor Molnar]

Severity: critical

Affected versions:

- Apache ZooKeeper 3.9.0 through 3.9.1
- Apache ZooKeeper 3.8.0 through 3.8.3
- Apache ZooKeeper 3.6.0 through 3.7.2

Description:

Information disclosure in persistent watchers handling in Apache ZooKeeper due 
to missing ACL check. It allows an attacker to monitor child znodes by 
attaching a persistent watcher (addWatch command) to a parent which the 
attacker has already access to. ZooKeeper server doesn't do ACL check when the 
persistent watcher is triggered and as a consequence, the full path of znodes 
that a watch event gets triggered upon is exposed to the owner of the watcher. 
It's important to note that only the path is exposed by this vulnerability, not 
the data of znode, but since znode path can contain sensitive information like 
user name or login ID, this issue is potentially critical.

Users are recommended to upgrade to version 3.9.2, 3.8.4 which fixes the issue.

Credit:

周吉安(寒泉)  (reporter)

References:

https://zookeeper.apache.org/
https://www.cve.org/CVERecord?id=CVE-2024-23944

Re: [ANNOUNCE] Apache ZooKeeper 3.8.4

[Andor Molnar]

Nice job Damien.
Thanks!



On Tue, 2024-03-05 at 23:00 +0100, Damien Diederen wrote:
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper
> version 3.8.4
> 
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You
> can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
> 
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
> 
> ZooKeeper 3.8.4 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.8.4/releasenotes.html
> 
> We would like to thank the contributors that made the release
> possible.
> 
> Regards,
> 
> The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.8.4 candidate 0

[Andor Molnar]

+1 (binding)

- verified checksum and gpg signature of the artifacts
- full build was successful
- unit tests passed
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- smoke tests (basic commands, watchers, etc.) passed

Andor



On Mon, 2024-02-12 at 23:35 +0100, Damien Diederen wrote:
> Greetings, all!
> 
> 
> This is a release candidate for 3.8.4.
> 
> This is a bugfix release for the 3.8 release line. Includes important
> dependency upgrades to address CVEs.
> 
> 
> The full release notes is available at:
> 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353693
> 
> *** Please download, test and vote by February 16th 2024, 23:59
> UTC+0. ***
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.4/
> 
> The release candidate tag in git to be voted upon: release-3.8.4-0
> https://github.com/apache/zookeeper/releases/tag/release-3.8.4-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.4-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Damien Diederen

Re: [VOTE] Apache ZooKeeper release 3.9.2 candidate 0

[Andor Molnar]

+1 (binding)

- verified checksum and gpg signature of the artifacts
- full build was successful
- unit tests passed
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed

Andor



On Mon, 2024-02-12 at 22:37 +0100, Damien Diederen wrote:
> Greetings, all!
> 
> 
> This is a release candidate for 3.9.2.
> 
> This is a bugfix release for the 3.9 release line. Includes important
> dependency upgrades to address CVEs.
> 
> 
> The full release notes is available at:
> 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353694
> 
> *** Please download, test and vote by February 16th 2024, 23:59
> UTC+0. ***
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.2/
> 
> The release candidate tag in git to be voted upon: release-3.9.2-0
> https://github.com/apache/zookeeper/releases/tag/release-3.9.2-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.2-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Damien Diederen

Re: Lack of support for TLS-only ZK cluster

[Andor Molnar]

Yes, I'll take a look also soon.

Andor



On Wed, 2024-02-21 at 13:03 -0800, Abhilash Kishore wrote:
> Thanks for the review Enrico! 
> 
> Do we need another +1 or are we ready to merge? 
> 
> Regards,
> Abhilash Kishore
> 
> 
> On Tue, 20 Feb 2024 at 13:59, Enrico Olivelli 
> wrote:
> > Il Mar 20 Feb 2024, 22:43 Abhilash Kishore 
> > ha
> > scritto:
> > 
> > > Yes, that's the PR <
> > https://github.com/apache/zookeeper/pull/2117>;.
> > >
> > > It's ready now. Do you mind taking a look?
> > >
> > 
> > 
> > Reviewed.
> > Thanks
> > 
> > Enrico
> > 
> > 
> > > Regards,
> > > Abhilash Kishore
> > >
> > >
> > > On Tue, 13 Feb 2024 at 02:28, Andor Molnar 
> > wrote:
> > >
> > > > Hi Abhilash,
> > > >
> > > > Is this the patch that you're working on?
> > > >
> > > > https://github.com/apache/zookeeper/pull/2117
> > > >
> > > > I see it's still draft, are u going to finish it soon?
> > > >
> > > > Andor
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> > > > > Thanks Andor, that makes sense. I agree with you, this is a
> > simpler
> > > > > and
> > > > > cleaner solution.
> > > > >
> > > > > I'll work on the changes and will try to keep it backwards
> > > > > compatible.
> > > > >
> > > > > Regards,
> > > > > Abhilash Kishore
> > > > >
> > > > >
> > > > > On Fri, 5 Jan 2024 at 09:00, Andor Molnar 
> > wrote:
> > > > >
> > > > > > Hi Abhilash,
> > > > > >
> > > > > > Thanks for looking into this issue.
> > > > > >
> > > > > > I wouldn't complicate things by trying to get reconfig
> > parameters
> > > > > > aligned and mixed with clientPort/secureClientPort. Since
> > the
> > > > > > documentation says these options are already deprecated I
> > suggest
> > > > > > to
> > > > > > upgrade Reconfig config line to support secure client port
> > as well.
> > > > > >
> > > > > > So, the following reconfig line:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181"
> > > > > >
> > > > > > will become:
> > > > > >
> > > > > > "server.1=abhilash-
> > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > 82".
> > > > > >
> > > > > > The 3 scenarios will become:
> > > > > >
> > > > > > 1. Non-TLS only:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > > > > >
> > > > > > 2. TLS-only:
> > > > > >
> > > > > > "server.1=abhilash-
> > ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > > > > >
> > > > > > 3. TLS/non-TLS mixed:
> > > > > >
> > > > > > "server.1=abhilash-
> > > > > > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > > > > > 82".
> > > > > >
> > > > > > In addition to that I would force the user to use either
> > the
> > > > > > deprecated
> > > > > > settings (clientPort/secureClientPort) OR reconfig lines,
> > but not
> > > > > > both.
> > > > > > Throw an exception and halt the server if both options are
> > > > > > specified at
> > > > > > the same time.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Regards,
> > > > > > Andor
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > > > > > Many organizations, large and small, have strict security
> > and
> > > > > > > compliance
> > > > > > > requirements to only accept encry

Re: Lack of support for TLS-only ZK cluster

[Andor Molnar]

Hi Abhilash,

Is this the patch that you're working on?

https://github.com/apache/zookeeper/pull/2117

I see it's still draft, are u going to finish it soon?

Andor




On Mon, 2024-01-08 at 18:46 -0800, Abhilash Kishore wrote:
> Thanks Andor, that makes sense. I agree with you, this is a simpler
> and
> cleaner solution.
> 
> I'll work on the changes and will try to keep it backwards
> compatible.
> 
> Regards,
> Abhilash Kishore
> 
> 
> On Fri, 5 Jan 2024 at 09:00, Andor Molnar  wrote:
> 
> > Hi Abhilash,
> > 
> > Thanks for looking into this issue.
> > 
> > I wouldn't complicate things by trying to get reconfig parameters
> > aligned and mixed with clientPort/secureClientPort. Since the
> > documentation says these options are already deprecated I suggest
> > to
> > upgrade Reconfig config line to support secure client port as well.
> > 
> > So, the following reconfig line:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"
> > 
> > will become:
> > 
> > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > 82".
> > 
> > The 3 scenarios will become:
> > 
> > 1. Non-TLS only:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"
> > 
> > 2. TLS-only:
> > 
> > "server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".
> > 
> > 3. TLS/non-TLS mixed:
> > 
> > "server.1=abhilash-
> > ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
> > 82".
> > 
> > In addition to that I would force the user to use either the
> > deprecated
> > settings (clientPort/secureClientPort) OR reconfig lines, but not
> > both.
> > Throw an exception and halt the server if both options are
> > specified at
> > the same time.
> > 
> > Thoughts?
> > 
> > Regards,
> > Andor
> > 
> > 
> > 
> > On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> > > Many organizations, large and small, have strict security and
> > > compliance
> > > requirements to only accept encrypted/TLS connections and not
> > > plain
> > > text
> > > connections.
> > > 
> > > I'd like to discuss an issue which is preventing us from starting
> > > our
> > > ZK
> > > clusters in TLS only mode (for client traffic).
> > > 
> > > As per dynamic reconfig doc
> > > <https://zookeeper.apache.org/doc/current/zookeeperReconfig.html>
> > > ;;,
> > > 
> > > > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > > > configuration
> > > > parameters should no longer be used. Instead, this information
> > > > is
> > > > now part
> > > > of the server keyword specification, which becomes as follows:
> > > > server. =
> > > > ::[:role];[ > > > port
> > > > address>:]
> > > 
> > > Let's say the dynamic config entry of a server is
> > > "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181".
> > > The
> > > server
> > > starts up with a (plaintext) clientPort listener on 2181.
> > > 
> > > Now, if we want to make this server TLS-only, what options do we
> > > have? We
> > > want to stop accepting plaintext traffic on 2181 and make the
> > > same
> > > port
> > > accept TLS connections only (make clientPort as
> > > secureClientPort).
> > > 
> > > If we add "secureClientPort=2181" in zoo.cfg, then ZK server
> > > first
> > > starts a
> > > plaintext listener on 2181 because of ";0.0.0.0:2181" in
> > > "server.1"
> > > dynamic
> > > config entry and then attempts to start a TLS client listener on
> > > the
> > > same
> > > port (2181) and fails. The reason for this behavior is already
> > > described in
> > > ZOOKEEPER-4276 <
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4276'
> > > > (highly
> > > recommended pre-read).
> > > 
> > > It is not possible to just remove the "" part from
> > > the
> > > "server.1" entry as well (I believe it is mandatory from v3.5). I
> > > tried:
> > > 
> > > [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> > > [zk: localhost:2181(CONNECTED) 5] reconfig -add
> > > serv

[jira] [Created] (ZOOKEEPER-4806) Commits have to be refreshed after merging

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4806:
---

 Summary: Commits have to be refreshed after merging
 Key: ZOOKEEPER-4806
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4806
 Project: ZooKeeper
  Issue Type: Sub-task
Reporter: Andor Molnar
Assignee: Szucs Villo


The following error occurs if somebody wants to cherry-pick immediately after 
merge:
{noformat}
All checks have passed on the github.
Pull request #2115 merged. Sha: #18c78cd10bc02d764a46ac1659b263cf69f2671d

Would you like to pick 18c78cd10bc02d764a46ac1659b263cf69f2671d into another 
branch? (y/n): y
Enter a branch name [branch-3.9]:
git fetch apache
>From https://gitbox.apache.org/repos/asf/zookeeper
   72e3d9ce9..e571dd814  master -> apache/master
git checkout -b PR_TOOL_PICK_PR_2115_BRANCH-3.9 apache/branch-3.9
Switched to a new branch 'PR_TOOL_PICK_PR_2115_BRANCH-3.9'
git cherry-pick -sx 18c78cd10bc02d764a46ac1659b263cf69f2671d
fatal: bad object 18c78cd10bc02d764a46ac1659b263cf69f2671d

Error cherry-picking: Command '['git', 'cherry-pick', '-sx', 
'18c78cd10bc02d764a46ac1659b263cf69f2671d']' returned non-zero exit status 
128.{noformat}
The reason for this is, because the local git repo doesn't know about the new 
commit yet.

We should do a {{git fetch}} after successfully merged via GitHub.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4805) Update cwiki page with latest changes

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4805:
---

 Summary: Update cwiki page with latest changes
 Key: ZOOKEEPER-4805
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4805
 Project: ZooKeeper
  Issue Type: Sub-task
  Components: documentation
Reporter: Andor Molnar
Assignee: Szucs Villo


Update the following wiki page with latest changes and instructions how to use 
the script:

[https://cwiki.apache.org/confluence/display/ZOOKEEPER/Merging+Github+Pull+Requests]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[ANNOUNCE] Apache ZooKeeper 3.7 End-of-Life 2nd Feb, 2024

[Andor Molnar]

The Apache ZooKeeper community would like to make the official
announcement of 3.7 release line End-of-Life. It will be effective on
2nd of February, 2024 00:01 AM (PDT). From that day forward the 3.7
version of Apache ZooKeeper won’t be supported by the community which
means we won't

- accept patches on the 3.7.x branch, 
- run automated tests on any JDK version, 
- create new releases from 3.7.x branch, 
- resolve security issues, CVEs or critical bugs.

Latest released version of Apache ZooKeeper 3.7 (currently 3.7.2) will
be available on the download page for another year (until 2nd of
February, 2025), after that it will be accessible among other
historical versions from Apache Archives.

=== Upgrade ===

We recommend users of Apache ZooKeeper 3.7 to plan your production
upgrades according to the following supported upgrade path:

1) Upgrade to latest 3.8.x version
2) (Optional) Upgrade to latest 3.9.x version.

Please find known upgrade issues and workarounds on the following wiki
page: Upgrade FAQ [1]

In addition to that the user@ mailing list is open 24/7 to help and
answer your questions as usual.

=== Compatibility ===

Our backward compatibility rules still apply and can be found here:
Backward compatibility rules [2]

Following the recommended upgrade path with rolling upgrade process
ZooKeeper quorum will be available at all times as long as clients are
not starting to use new features.

Regards,
Andor

[1] https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
[2] 
https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement

Re: Moving 3.7 to End-of-Life

[Andor Molnar]

Hi tison,

I think 3.8+




On Wed, 2024-01-31 at 20:15 +0800, tison wrote:
> I don't have a preference because as long as we don't run new patch
> releases for 3.7, it's effectively dropped.
> 
> Just for your information, I'm upgrading Curator with ZK 3.9 and
> running compatible tests [1]. I'm open to inputs about which ZK
> versions should new Curator versions maintain compatibility.
> 
> Best,
> tison.
> 
> [1] https://github.com/apache/curator/pull/496
> 
> Andor Molnar  于2024年1月31日周三 16:57写道：
> > Hi all,
> > 
> > We didn't get any negative feedback, so I think we're good to
> > proceed
> > with the EoL process.
> > 
> > Bit confused - since we didn't find any warning announcement about
> > 3.7
> > EoL process, shall we announce it with a few month grace period
> > first
> > or just stick to the rules and let it be effective immediately?
> > 
> > Thoughts?
> > 
> > Andor
> > 
> > 
> > 
> > On Mon, 2024-01-29 at 10:30 +0100, Andor Molnar wrote:
> > > Thanks guys.
> > > 
> > > Sounds like we have 3 binding +1 votes.
> > > 
> > > Andor
> > > 
> > > 
> > > 
> > > On Sat, 2024-01-27 at 09:29 +0100, Enrico Olivelli wrote:
> > > > Il Sab 27 Gen 2024, 00:27 Patrick Hunt  ha
> > > > scritto:
> > > > 
> > > > > Markmail search for apache mail archive is gone? and
> > > > > unfortunately
> > > > > Apache
> > > > > mail archive search seems to be broken (no results coming
> > > > > back...)
> > > > > I
> > > > > managed to track this ref down
> > > > > https://lists.apache.org/thread/b8sm8gxmohs9gl4vrltd2jr4slqvrg9n
> > > > > but I distinctly remember seeing something about this, just
> > > > > can't
> > > > > find it.
> > > > > 
> > > > > According to this it's already eol:
> > > > > https://endoflife.date/zookeeper
> > > > > 
> > > > > Our own release page makes it clear that folks should move
> > > > > given
> > > > > stable and
> > > > > current have been out for a while. I think we can call it EOL
> > > > > at
> > > > > this
> > > > > point.
> > > > > 
> > > > 
> > > > +1 to mark 3.7 release line as EOL
> > > > 
> > > > Enrico
> > > > 
> > > > > Regards,
> > > > > 
> > > > > Patrick
> > > > > 
> > > > > On Fri, Jan 26, 2024 at 7:33 AM Andor Molnar <
> > > > > an...@apache.org>
> > > > > wrote:
> > > > > 
> > > > > > Hi zk community,
> > > > > > 
> > > > > > According to our Releases [1] page ZooKeeper 3.8.2 became
> > > > > > the
> > > > > > first
> > > > > > stable version of 3.8.x line on 3 Aug, 2023 (when 3.9.0 was
> > > > > > released).
> > > > > > 
> > > > > > The previous stable version "in approximately half a year
> > > > > > will
> > > > > > be
> > > > > > announced as End-of-Life". 6 months will pass on 3 Feb,
> > > > > > 2024,
> > > > > > so
> > > > > > we
> > > > > > should think about announcing EoL soon.
> > > > > > 
> > > > > > What do you think?
> > > > > > 
> > > > > > Regards,
> > > > > > Andor
> > > > > > 
> > > > > > 
> > > > > > [1] https://zookeeper.apache.org/releases.html
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 

Re: Moving 3.7 to End-of-Life

[Andor Molnar]

Hi all,

We didn't get any negative feedback, so I think we're good to proceed
with the EoL process. 

Bit confused - since we didn't find any warning announcement about 3.7
EoL process, shall we announce it with a few month grace period first
or just stick to the rules and let it be effective immediately?

Thoughts?

Andor



On Mon, 2024-01-29 at 10:30 +0100, Andor Molnar wrote:
> Thanks guys.
> 
> Sounds like we have 3 binding +1 votes.
> 
> Andor
> 
> 
> 
> On Sat, 2024-01-27 at 09:29 +0100, Enrico Olivelli wrote:
> > Il Sab 27 Gen 2024, 00:27 Patrick Hunt  ha
> > scritto:
> > 
> > > Markmail search for apache mail archive is gone? and
> > > unfortunately
> > > Apache
> > > mail archive search seems to be broken (no results coming
> > > back...)
> > > I
> > > managed to track this ref down
> > > https://lists.apache.org/thread/b8sm8gxmohs9gl4vrltd2jr4slqvrg9n
> > > but I distinctly remember seeing something about this, just can't
> > > find it.
> > > 
> > > According to this it's already eol:
> > > https://endoflife.date/zookeeper
> > > 
> > > Our own release page makes it clear that folks should move given
> > > stable and
> > > current have been out for a while. I think we can call it EOL at
> > > this
> > > point.
> > > 
> > 
> > +1 to mark 3.7 release line as EOL
> > 
> > Enrico
> > 
> > > Regards,
> > > 
> > > Patrick
> > > 
> > > On Fri, Jan 26, 2024 at 7:33 AM Andor Molnar 
> > > wrote:
> > > 
> > > > Hi zk community,
> > > > 
> > > > According to our Releases [1] page ZooKeeper 3.8.2 became the
> > > > first
> > > > stable version of 3.8.x line on 3 Aug, 2023 (when 3.9.0 was
> > > > released).
> > > > 
> > > > The previous stable version "in approximately half a year will
> > > > be
> > > > announced as End-of-Life". 6 months will pass on 3 Feb, 2024,
> > > > so
> > > > we
> > > > should think about announcing EoL soon.
> > > > 
> > > > What do you think?
> > > > 
> > > > Regards,
> > > > Andor
> > > > 
> > > > 
> > > > [1] https://zookeeper.apache.org/releases.html
> > > > 
> > > > 
> > > > 
> > > > 

Re: Moving 3.7 to End-of-Life

[Andor Molnar]

Thanks guys.

Sounds like we have 3 binding +1 votes.

Andor



On Sat, 2024-01-27 at 09:29 +0100, Enrico Olivelli wrote:
> Il Sab 27 Gen 2024, 00:27 Patrick Hunt  ha scritto:
> 
> > Markmail search for apache mail archive is gone? and unfortunately
> > Apache
> > mail archive search seems to be broken (no results coming back...)
> > I
> > managed to track this ref down
> > https://lists.apache.org/thread/b8sm8gxmohs9gl4vrltd2jr4slqvrg9n
> > but I distinctly remember seeing something about this, just can't
> > find it.
> > 
> > According to this it's already eol:
> > https://endoflife.date/zookeeper
> > 
> > Our own release page makes it clear that folks should move given
> > stable and
> > current have been out for a while. I think we can call it EOL at
> > this
> > point.
> > 
> 
> 
> +1 to mark 3.7 release line as EOL
> 
> Enrico
> 
> > 
> > Regards,
> > 
> > Patrick
> > 
> > On Fri, Jan 26, 2024 at 7:33 AM Andor Molnar 
> > wrote:
> > 
> > > Hi zk community,
> > > 
> > > According to our Releases [1] page ZooKeeper 3.8.2 became the
> > > first
> > > stable version of 3.8.x line on 3 Aug, 2023 (when 3.9.0 was
> > > released).
> > > 
> > > The previous stable version "in approximately half a year will be
> > > announced as End-of-Life". 6 months will pass on 3 Feb, 2024, so
> > > we
> > > should think about announcing EoL soon.
> > > 
> > > What do you think?
> > > 
> > > Regards,
> > > Andor
> > > 
> > > 
> > > [1] https://zookeeper.apache.org/releases.html
> > > 
> > > 
> > > 
> > > 
> > 

Moving 3.7 to End-of-Life

[Andor Molnar]

Hi zk community,

According to our Releases [1] page ZooKeeper 3.8.2 became the first
stable version of 3.8.x line on 3 Aug, 2023 (when 3.9.0 was released).

The previous stable version "in approximately half a year will be
announced as End-of-Life". 6 months will pass on 3 Feb, 2024, so we
should think about announcing EoL soon.

What do you think?

Regards,
Andor


[1] https://zookeeper.apache.org/releases.html

Re: Lack of support for TLS-only ZK cluster

[Andor Molnar]

Hi Abhilash,

Thanks for looking into this issue.

I wouldn't complicate things by trying to get reconfig parameters
aligned and mixed with clientPort/secureClientPort. Since the
documentation says these options are already deprecated I suggest to
upgrade Reconfig config line to support secure client port as well.

So, the following reconfig line:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181"

will become:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
82".

The 3 scenarios will become:

1. Non-TLS only:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;"

2. TLS-only:

"server.1=abhilash-ubuntu:3183:4183:participant;;0.0.0.0:2182".

3. TLS/non-TLS mixed:

"server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181;0.0.0.0:21
82".

In addition to that I would force the user to use either the deprecated
settings (clientPort/secureClientPort) OR reconfig lines, but not both.
Throw an exception and halt the server if both options are specified at
the same time.

Thoughts?

Regards,
Andor



On Tue, 2024-01-02 at 11:48 -0800, Abhilash Kishore wrote:
> Many organizations, large and small, have strict security and
> compliance
> requirements to only accept encrypted/TLS connections and not plain
> text
> connections.
> 
> I'd like to discuss an issue which is preventing us from starting our
> ZK
> clusters in TLS only mode (for client traffic).
> 
> As per dynamic reconfig doc
> ;,
> 
> > Starting with 3.5.0 the *clientPort* and *clientPortAddress*
> > configuration
> > parameters should no longer be used. Instead, this information is
> > now part
> > of the server keyword specification, which becomes as follows:
> > server. = ::[:role];[ > port
> > address>:]
> 
> 
> Let's say the dynamic config entry of a server is
> "server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181". The
> server
> starts up with a (plaintext) clientPort listener on 2181.
> 
> Now, if we want to make this server TLS-only, what options do we
> have? We
> want to stop accepting plaintext traffic on 2181 and make the same
> port
> accept TLS connections only (make clientPort as secureClientPort).
> 
> If we add "secureClientPort=2181" in zoo.cfg, then ZK server first
> starts a
> plaintext listener on 2181 because of ";0.0.0.0:2181" in "server.1"
> dynamic
> config entry and then attempts to start a TLS client listener on the
> same
> port (2181) and fails. The reason for this behavior is already
> described in
> ZOOKEEPER-4276  > (highly
> recommended pre-read).
> 
> It is not possible to just remove the "" part from the
> "server.1" entry as well (I believe it is mandatory from v3.5). I
> tried:
> 
> [zk: localhost:2181(CONNECTED) 4] reconfig -remove 1
> [zk: localhost:2181(CONNECTED) 5] reconfig -add
> server.1=abhilash-ubuntu:3183:4183:participant
> Arguments are not valid :
> 
> 
> The reconfig command does not allow us to add a server entry without
> ";[ port address>:]".
> 
> How do we support a "TLS-only" cluster in this case?
> 
> My recommendation:
> 
>1. If both clientPort and secureClientPort are not set in zoo.cfg,
> then
>use the client port address from dynamic config.
>2. If only clientPort is set in zoo.cfg, then it has to match the
> port
>in dynamic config and ZK starts a plaintext listener on this port.
>3. If only secureClientPort is set in zoo.cfg, then it has to
> match the
>port in dynamic config and ZK starts a TLS listener on this port.
>4. If both clientPort and secureClientPort are set in zoo.cfg,
> then the
>client port in zoo.cfg should match the port in dynamic config. ZK
> starts a
>plaintext listener on clientPort and TLS listener on
> secureClientPort (dual
>mode).
> 
> 
> This would reintroduce the requirement to set "clientPort" in zoo.cfg
> if
> someone wants to start the cluster in dual mode.
> 
> For example,
> 
> secureClientPort=2182
> server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> 
> will no longer be a valid config because of rule 3 above.
> 
> It has to be:
> 
> clientPort=2181
> secureClientPort=2182
> server.1=abhilash-ubuntu:3183:4183:participant;0.0.0.0:2181
> 
> 
> I can create a PR to make the above changes, but first I'd like to
> know
> your thoughts on this and discuss further on whether there's a better
> way
> to handle this.
> 
> Regards,
> Abhilash Kishore

Re: [DISCUSS] OpenTelemetry for Zookeeper?

[Andor Molnar]

Thanks Chris.

Unfortunately both ZooKeeper client and server code reside in
zookeeper-server Maven project, so I think the separation that you
suggest is not feasible at the moment.

Andor




On Tue, 2024-01-02 at 21:13 -0500, Christopher wrote:
> I think open tracing is a good library to use. Accumulo started using
> it
> after HTrace went to the attic. Accumulo also uses ZooKeeper. It
> remains to
> be seen if open telemetry will be stable in the long term, since it
> is
> relatively new.
> 
> Lots of projects in an application stack may want to use open
> telemetry,
> but it may be difficult because of dependency hell and class path
> issues if
> lots of libraries in an application are all using open telemetry, but
> using
> different incompatible versions. That was certainly a big problem
> with
> HTrace when Hadoop was using one version and Accumulo was using a
> different
> version (notably, this problem arose, even though HTrace was
> originally
> derived from code that started out in Accumulo as "cloudtrace",
> because it
> was modified to be more general purpose and began being used by
> multiple
> applications in the application stack). This is not a unique issue,
> but
> other places where this could be a problem, such as logging, have
> well
> established solutions, like slf4j. It's not clear to me yet whether
> open
> telemetry can be relied upon to be stable like that in the long term.
> If
> zookeeper starts using it, it would be safest to use it only on the
> server
> code, at least to start, and not require it as a dependency of
> zookeeper
> client code, to make sure it doesn't conflict with other applications
> that
> use ZooKeeper.
> 
> If it is used for client code, it should be possible to completely
> disable
> it with a property so nothing breaks if it is disabled and it is
> missing
> from the class path, or if a different version that is incompatible
> exists
> on the class path. That could be hard to do.
> 
> This doesn't mean it shouldn't be done, just that if it is done,
> these are
> some things to consider to try to avoid potential problems down the
> line.
> 
> On Tue, Jan 2, 2024, 10:57 Andor Molnar  wrote:
> 
> > Hi all,
> > 
> > Inspired by the following CURATOR ticket I started to think about
> > what
> > needs to be done for ZooKeeper to support OpenTelemetry.
> > 
> > CURATOR-695 Open Telemetry Tracing Driver [1]
> > 
> > Unfortunately we don't have such generic tracing driver, even
> > ZooTrace
> > class looks unusable for this use case, but we should be able to
> > implement it in a generic fashion. Start the trace in
> > PrepRequestProcessor when request comes in and finish it in
> > FinalRequestProcessor with adding some in-process events too.
> > 
> > It's never that simple obviously, because, for instance, we also
> > need
> > to track the failing code paths too, but looks to me a good
> > starting
> > point and something we should invest into.
> > 
> > Thoughts?
> > 
> > Regards,
> > Andor
> > 
> > [1] https://issues.apache.org/jira/browse/CURATOR-695
> > 
> > 
> > 
> > 

[DISCUSS] OpenTelemetry for Zookeeper?

[Andor Molnar]

Hi all,

Inspired by the following CURATOR ticket I started to think about what
needs to be done for ZooKeeper to support OpenTelemetry.

CURATOR-695 Open Telemetry Tracing Driver [1]

Unfortunately we don't have such generic tracing driver, even ZooTrace
class looks unusable for this use case, but we should be able to
implement it in a generic fashion. Start the trace in
PrepRequestProcessor when request comes in and finish it in
FinalRequestProcessor with adding some in-process events too.

It's never that simple obviously, because, for instance, we also need
to track the failing code paths too, but looks to me a good starting
point and something we should invest into.

Thoughts?

Regards,
Andor

[1] https://issues.apache.org/jira/browse/CURATOR-695

New merge script with GH api

[Andor Molnar]

Hi folks,

We've just submitted 
https://issues.apache.org/jira/browse/ZOOKEEPER-4756 to all active
branches. It's about an improvement for the merge script to use GH api
for merging PRs instead of manually pushing and leaving the PR is
"closed" state.

The PR itself has been merged with the new script and it nicely turned
into "Merged" status.

Please use this new script for merging PRs in the future rather than
merging on GitHub. It creates a nice commit message and updates Jira
ticket as well. Feedbacks are welcome.

We continue working on further improvement which I spotted in Spark
project and listed in the Jira's description.

Regards,
Andor

ZOOKEEPER-2053 Zookeeper scripts should honor ZOOKEEPER_HOME

[Andor Molnar]

Hi ZK folks,

I have an relative old PR still open to address ZOOKEEPER-2053
Shall we commit it?

https://github.com/apache/zookeeper/pull/2037

Thanks,
Andor

CVE-2023-44981: Apache ZooKeeper: Authorization bypass in SASL Quorum Peer Authentication

[Andor Molnar]

Severity: critical

Affected versions:

- Apache ZooKeeper 3.9.0
- Apache ZooKeeper 3.8.0 through 3.8.2
- Apache ZooKeeper 3.7.0 through 3.7.1
- Apache ZooKeeper before 3.7.0

Description:

Authorization Bypass Through User-Controlled Key vulnerability in Apache 
ZooKeeper. If SASL Quorum Peer authentication is enabled in ZooKeeper 
(quorum.auth.enableSasl=true), the authorization is done by verifying that the 
instance part in SASL authentication ID is listed in zoo.cfg server list. The 
instance part in SASL auth ID is optional and if it's missing, like 
'e...@example.com', the authorization check will be skipped. As a result an 
arbitrary endpoint could join the cluster and begin propagating counterfeit 
changes to the leader, essentially giving it complete read-write access to the 
data tree. Quorum Peer authentication is not enabled by default.

Users are recommended to upgrade to version 3.9.1, 3.8.3, 3.7.2, which fixes 
the issue.

Alternately ensure the ensemble election/quorum communication is protected by a 
firewall as this will mitigate the issue.

See the documentation for more details on correct cluster administration.

Credit:

Damien Diederen  (reporter)

References:

https://zookeeper.apache.org/
https://www.cve.org/CVERecord?id=CVE-2023-44981

[ANNOUNCE] Apache ZooKeeper 3.7.2

[Andor Molnar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.7.2

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.7.2 Release Notes are at:
https://zookeeper.apache.org/doc/r3.7.2/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

[ANNOUNCE] Apache ZooKeeper 3.8.3

[Andor Molnar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.8.3

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.8.3 Release Notes are at:
https://zookeeper.apache.org/doc/r3.8.3/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

[ANNOUNCE] Apache ZooKeeper 3.9.1

[Andor Molnar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.9.1

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.9.1 Release Notes are at:
https://zookeeper.apache.org/doc/r3.9.1/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

[RESULT] [VOTE] Apache ZooKeeper release 3.7.2 candidate 0

[Andor Molnar]

I'm happy to announce that we have unanimously approved this release.
 
There are 4 approving votes, 3 of which are binding:
 
- Damien Diederen (non-binding)
- Mate Szalay-Beko (binding)
- Patrick Hunt (binding)
- Andor Molnar (binding)
 
There are no disapproving votes.
 
I will promote the artifacts and complete the release procedure.
 
Thanks to every one who contributed to this great release !
 
Andor



On Fri, 2023-10-06 at 12:04 +0200, Andor Molnar wrote:
> Hi ZK folks,
> 
> This is a release candidate for 3.7.2.
> 
> This is a bugfix release for the 3.7 release line. Includes important
> bugfixes and dependency upgrades to address CVEs.
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351732
> 
> 
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
> 
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1098/
> 
> The release candidate tag in git to be voted upon: release-3.7.2-0
> https://github.com/apache/zookeeper/releases/tag/release-3.7.2-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Andor
> 
> 
> 

[RESULT] [VOTE] Apache ZooKeeper release 3.8.3 candidate 0

[Andor Molnar]

I'm happy to announce that we have unanimously approved this release.
 
There are 4 approving votes, 3 of which are binding:
 
- Damien Diederen (non-binding)
- Mate Szalay-Beko (binding)
- Patrick Hunt (binding)
- Andor Molnar (binding)
 
There are no disapproving votes.
 
I will promote the artifacts and complete the release procedure.
 
Thanks to every one who contributed to this great release !
 
Andor



On Thu, 2023-10-05 at 12:50 +0200, Andor Molnar wrote:
> Hi,
> 
> This is a release candidate for 3.8.3.
> 
> This is a bugfix release for the 3.8 release line. Includes important
> dependency upgrades to address CVEs.
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353400
> 
> 
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
> 
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1097/
> 
> The release candidate tag in git to be voted upon: release-3.8.3-0
> https://github.com/apache/zookeeper/releases/tag/release-3.8.3-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Andor
> 
> 
> 

[RESULT] [VOTE] Apache ZooKeeper release 3.9.1 candidate 0

[Andor Molnar]

I'm happy to announce that we have unanimously approved this release.
 
There are 5 approving votes, 4 of which are binding:
 
- Enrico Olivelli (binding)
- Damien Diederen (non-binding)
- Mate Szalay-Beko (binding)
- Patrick Hunt (binding)
- Andor Molnar (binding)
 
There are no disapproving votes.
 
I will promote the artifacts and complete the release procedure.
 
Thanks to every one who contributed to this great release !
 
Andor



On Wed, 2023-10-04 at 14:28 +0200, Andor Molnar wrote:
> Hi team,
> 
> This is a release candidate for 3.9.1.
> 
> This is a bugfix release for the 3.9 release line. Includes important
> dependency upgrades to address CVEs.
> 
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480
> 
> *** Please download, test and vote by October 6th 2023, 23:59 UTC+0.
> ***
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1096/
> 
> The release candidate tag in git to be voted upon: release-3.9.1-0
> https://github.com/apache/zookeeper/releases/tag/release-3.9.1-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Best regards,
> 
> Andor
> 
> 
> 

Re: [VOTE] Apache ZooKeeper release 3.8.3 candidate 0

[Andor Molnar]

+1

- checksum / signature good
- rat run clean
- owasp dependency check passed
- java unit tests passed on Ubuntu 20.04 / openjdk version "11.0.12"
- c++ unit tests passed
- created 3-node SSL ensemble and ran a few smoke tests
- release notes looks fine
- verified CLI commands

Regards,
Andor




On Thu, 2023-10-05 at 12:50 +0200, Andor Molnar wrote:
> Hi,
> 
> This is a release candidate for 3.8.3.
> 
> This is a bugfix release for the 3.8 release line. Includes important
> dependency upgrades to address CVEs.
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353400
> 
> 
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
> 
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1097/
> 
> The release candidate tag in git to be voted upon: release-3.8.3-0
> https://github.com/apache/zookeeper/releases/tag/release-3.8.3-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Andor
> 
> 
> 

Re: [VOTE] Apache ZooKeeper release 3.7.2 candidate 0

[Andor Molnar]

+1

- checksum / signature good
- rat run clean
- owasp dependency check passed
- java unit tests passed on Ubuntu 20.04 / openjdk version "11.0.12"
- one of C++ tests is still failing for me, but CI looks good:
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.7.2/

- created 3-node SSL ensemble and ran a few smoke tests
- release notes looks fine, but history lacks the notes of 3.7.0 - I'll
make a note in the release guide
- verified CLI commands

Regards,
Andor


On Fri, 2023-10-06 at 12:04 +0200, Andor Molnar wrote:
> Hi ZK folks,
> 
> This is a release candidate for 3.7.2.
> 
> This is a bugfix release for the 3.7 release line. Includes important
> bugfixes and dependency upgrades to address CVEs.
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351732
> 
> 
> *** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
> ***
> 
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1098/
> 
> The release candidate tag in git to be voted upon: release-3.7.2-0
> https://github.com/apache/zookeeper/releases/tag/release-3.7.2-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Regards,
> Andor
> 
> 
> 

[VOTE] Apache ZooKeeper release 3.7.2 candidate 0

[Andor Molnar]

Hi ZK folks,

This is a release candidate for 3.7.2.

This is a bugfix release for the 3.7 release line. Includes important
bugfixes and dependency upgrades to address CVEs.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351732


*** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
***


Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachezookeeper-1098/

The release candidate tag in git to be voted upon: release-3.7.2-0
https://github.com/apache/zookeeper/releases/tag/release-3.7.2-0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.7.2-candidate-0/website/index.html


Should we release this candidate?


Regards,
Andor

[jira] [Created] (ZOOKEEPER-4756) Merge script should use GitHub api to merge pull requests

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4756:
---

 Summary: Merge script should use GitHub api to merge pull requests
 Key: ZOOKEEPER-4756
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4756
 Project: ZooKeeper
  Issue Type: Improvement
  Components: tools
Affects Versions: 3.9.0
Reporter: Andor Molnar


Github merge script (zk-merge-pr.py) is a nice tool which does a lot of 
housekeeping tasks when merging a PR including fixing the commit message or 
closing the Jira. Merging on the Github UI is also possible, but could lead to 
mistakes like leaving the commit message without the Jira id.

Unfortunately when the script merges the PR it does that without Github and 
leaving the PR in 'Closed' rather than 'Merged'. This is misleading. Let's 
improve the script to use Github API for merging PRs and possibly disable 
merging on the Github UI.

Email thread:

[https://lists.apache.org/thread/cbmktklydtlylkybvq6jrx5m4l8b2cm5]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] Apache ZooKeeper release 3.8.3 candidate 0

[Andor Molnar]

Hi,

This is a release candidate for 3.8.3.

This is a bugfix release for the 3.8 release line. Includes important
dependency upgrades to address CVEs.

The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353400


*** Please download, test and vote by October 9th 2023, 23:59 UTC+0.
***


Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachezookeeper-1097/

The release candidate tag in git to be voted upon: release-3.8.3-0
https://github.com/apache/zookeeper/releases/tag/release-3.8.3-0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.3-candidate-0/website/index.html


Should we release this candidate?


Regards,
Andor

Re: [VOTE] Apache ZooKeeper release 3.9.1 candidate 0

[Andor Molnar]

+1

- checksum / signature good
- rat run clean
- owasp dependency check passed
- java unit tests passed on Ubuntu 20.04 / openjdk version "11.0.12"
- one of C++ tests failed, but based on CI builds everything is fine
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.9.1/

[exec] terminate called after throwing an instance of
'CppUnit::Exception'
[exec]   what():  equality assertion failed
[exec] - Expected: -101
[exec] - Actual  : -4
[exec]
[exec] Zookeeper_simpleSystem::testAsyncWatcherAutoResetFAIL: zktest-mt
[exec] ==
[exec] 1 of 2 tests failed
[exec] Please report to u...@zookeeper.apache.org
[exec] ==

- created 3-node SSL ensemble and ran a few smoke tests
- release notes looking fine
- verified CLI commands

Minor glitch in the client: the commit id is not replaced by Maven.

[zk: andor-5560-ubuntu:2183(CONNECTED) 5] version
ZooKeeper CLI version: 3.9.1-${mvngit.commit.id}, built on 2023-10-04
15:03 UTC

Andor



On Wed, 2023-10-04 at 14:28 +0200, Andor Molnar wrote:
> Hi team,
> 
> This is a release candidate for 3.9.1.
> 
> This is a bugfix release for the 3.9 release line. Includes important
> dependency upgrades to address CVEs.
> 
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480
> 
> *** Please download, test and vote by October 6th 2023, 23:59 UTC+0.
> ***
> 
> Source files:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachezookeeper-1096/
> 
> The release candidate tag in git to be voted upon: release-3.9.1-0
> https://github.com/apache/zookeeper/releases/tag/release-3.9.1-0
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/index.html
> 
> 
> Should we release this candidate?
> 
> 
> Best regards,
> 
> Andor
> 
> 
> 

[VOTE] Apache ZooKeeper release 3.9.1 candidate 0

[Andor Molnar]

Hi team,

This is a release candidate for 3.9.1.

This is a bugfix release for the 3.9 release line. Includes important
dependency upgrades to address CVEs.


The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12353480

*** Please download, test and vote by October 6th 2023, 23:59 UTC+0.
***

Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachezookeeper-1096/

The release candidate tag in git to be voted upon: release-3.9.1-0
https://github.com/apache/zookeeper/releases/tag/release-3.9.1-0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.1-candidate-0/website/index.html


Should we release this candidate?


Best regards,

Andor

[ANNOUNCE] Apache ZooKeeper 3.9.0

[Andor Molnar]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.9.0

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.9.0 Release Notes are at:
https://zookeeper.apache.org/doc/r3.9.0/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 1

[Andor Molnar]

Thank you for the reviews!

There were 4 approving votes, 3 of them are binding:

- Enrico Olivelli (binding)
- Mohammad Arshad (non-binding)
- Patrick Hunt (binding)
- Máté Szalay-Bekő (binding)

There were no disapproving votes.
I will promote the artifacts and complete the release procedure.

Thanks to everyone who contributed to this release!

Best Regards,
Andor


On Mon, 2023-07-31 at 11:42 +0200, Enrico Olivelli wrote:
> +1 (binding)
> 
> - Verified signatures and checksums
> - Built  and run all the tests on JDK-17
> - Run some smoke tests
> - Run tests on some user application
> 
> 
> Thanks
> Enrico
> 
> Il giorno gio 27 lug 2023 alle ore 12:39 Mohammad Arshad
>  ha scritto:
> > +1 (non-binding)
> > verified signature -ok
> > verified checksum -ok
> > run rat,checkstyle and spotbugs -ok
> > run all java test cases -ok
> > build and installed 3 node cluster, executed few cli commands -ok
> > 
> > -Arshad
> > 
> > On Wed, Jul 26, 2023 at 5:17 AM Patrick Hunt 
> > wrote:
> > 
> > > +1 - xsum/sig verified, rat ran clean, I was able to compile, run
> > > the owasp
> > > checker, and start various ensemble sizes manually w/o issue.
> > > lgtm.
> > > 
> > > Patrick
> > > 
> > > On Wed, Jul 19, 2023 at 2:20 AM Andor Molnar 
> > > wrote:
> > > 
> > > > This is release candidate for ZooKeeper 3.9.0.
> > > > 
> > > > It is a major release and it introduces a lot of new features,
> > > > most
> > > > notably:
> > > > - Admin server API for taking snapshot and stream out the data
> > > > - Communicate the Zxid that triggered a WatchEvent to fire
> > > > - TLS - dynamic loading for client trust/key store
> > > > - Add Netty-TcNative OpenSSL Support
> > > > - Adding SSL support to Zktreeutil
> > > > - Improve syncRequestProcessor performance
> > > > - Updates to all the third party dependencies to get rid of
> > > > every known
> > > > CVE.
> > > > 
> > > > The full release notes is available at:
> > > > 
> > > > 
> > > > 
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > > > *** Please download, test and vote by July 30th 2023, 23:59
> > > > UTC+0. ***
> > > > 
> > > > Source files:
> > > > 
> > > > 
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/
> > > > Maven staging repo:
> > > > 
> > > > 
> > > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > > > The release candidate tag in git to be voted upon: release-
> > > > 3.9.0-1
> > > > https://github.com/apache/zookeeper/tree/release-3.9.0-1
> > > > 
> > > > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > > > release:
> > > > https://www.apache.org/dist/zookeeper/KEYS
> > > > 
> > > > The staging version of the website is:
> > > > 
> > > > 
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/website/index.html
> > > > 
> > > > Should we release this candidate?
> > > > 
> > > > Regards,
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > 

[VOTE] Apache ZooKeeper release 3.9.0 candidate 1

[Andor Molnar]

This is release candidate for ZooKeeper 3.9.0.

It is a major release and it introduces a lot of new features, most
notably:
- Admin server API for taking snapshot and stream out the data
- Communicate the Zxid that triggered a WatchEvent to fire
- TLS - dynamic loading for client trust/key store
- Add Netty-TcNative OpenSSL Support
- Adding SSL support to Zktreeutil
- Improve syncRequestProcessor performance
- Updates to all the third party dependencies to get rid of every known
CVE.

The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304

*** Please download, test and vote by July 30th 2023, 23:59 UTC+0. ***

Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/

The release candidate tag in git to be voted upon: release-3.9.0-1
https://github.com/apache/zookeeper/tree/release-3.9.0-1

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-1/website/index.html


Should we release this candidate?

Regards,
Andor

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Andor Molnar]

Hi Mate,

I take your e-mail as a -1 vote, so this RC VOTE is CANCELLED.
I'll prepare another rc.

Regards,
Andor


On Mon, 2023-07-17 at 22:50 +0200, Szalay-Bekő Máté wrote:
> Hello Andor!
> 
> Thanks for this great release!
> 
> I found two issues with RC0:
> 
> 1) OWASP CVE check (mvn dependency-check:check) failed with
> "netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:
> CVE-2011-1797(9.3)"
> 
> This seems to be a false positive to me (looks to be some security
> issue
> affecting old safari / chromium web browser versions?). I didn't get
> deep
> into this, but I guess we see this since
> https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> 
> Interestingly, the CI pipeline doesn't catch this CVE (
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/),
> maybe this is some bug in OWASP that is triggered only with certain
> maven
> versions or during building on certain platforms? I ran OWASP on
> Ubuntu
> 18.04.2 with maven 3.9.3.
> 
> 2) Also I see that the website (
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html)
> is still showing "ZooKeeper 3.8 Documentation" on the top
> 
> 
> What do you think? We shouldn't pass the RC until we are certain
> about the
> CVE issue. (unless this is something happening only on my setup... it
> is
> strange that OWAPS is green on CI)
> 
> 
> Beside these, I ran all my usual RC test steps, and found no other
> issues
> with the RC:
> - verified checksum and gpg signature of the artifacts
> - I built the source code (incl. the C-client, using -Pfull-build) on
> Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> - all the unit tests passed (both Java and C-client)
> - I also built and executed unit tests for zkpython
> - I also built the java code (without -Pfull-build) using other JDK
> versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this
> time, just
> used 'clean install -DskipTests')
> - checkstyle and spotbugs passed
> - apache-rat passed
> - fatjar built
> - I executed quick rolling-upgrade tests (using
> https://github.com/symat/zk-rolling-upgrade-test):
>  - rolling upgrade from 3.5.10 to 3.9.0
>  - rolling upgrade from 3.6.4 to 3.9.0
>  - rolling upgrade from 3.7.1 to 3.9.0
>  - rolling upgrade from 3.8.2 to 3.9.0
> - compared generated release notes (
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/releasenotes.html
> ) with Jira (
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> )
> 
> 
> Best regards,
> Máté
> 
> On Mon, Jul 17, 2023 at 3:11 PM Andor Molnar 
> wrote:
> 
> > Hi team,
> > 
> > This is a release candidate for 3.9.0.
> > 
> > It is a major release and it introduces a lot of new features, most
> > notably:
> > - Admin server API for taking snapshot and stream out the data
> > - Communicate the Zxid that triggered a WatchEvent to fire
> > - TLS - dynamic loading for client trust/key store
> > - Add Netty-TcNative OpenSSL Support
> > - Adding SSL support to Zktreeutil
> > - Improve syncRequestProcessor performance
> > - Updates to all the third party dependencies to get rid of every
> > known
> > CVE.
> > 
> > The full release notes is available at:
> > 
> > 
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > 
> > *** Please download, test and vote by July 30th 2023, 23:59 UTC+0.
> > ***
> > 
> > Source files:
> > 
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/
> > 
> > Maven staging repo:
> > 
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > 
> > The release candidate tag in git to be voted upon: release-3.8.0-1
> > https://github.com/apache/zookeeper/tree/release-3.9.0-0
> > 
> > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > release:
> > https://www.apache.org/dist/zookeeper/KEYS
> > 
> > The staging version of the website is:
> > 
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
> > 
> > 
> > Should we release this candidate?
> > 
> > 
> > Regards,
> > Andor
> > 
> > 
> > 

Re: [VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Andor Molnar]

Hi Mate,

Since we don't have better idea, I opened a pull request to upgrade
OWASP to the latest (8.3.1) version.

https://github.com/apache/zookeeper/pull/2035

Please approve.

Andor



On Mon, 2023-07-17 at 22:50 +0200, Szalay-Bekő Máté wrote:
> Hello Andor!
> 
> Thanks for this great release!
> 
> I found two issues with RC0:
> 
> 1) OWASP CVE check (mvn dependency-check:check) failed with
> "netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:
> CVE-2011-1797(9.3)"
> 
> This seems to be a false positive to me (looks to be some security
> issue
> affecting old safari / chromium web browser versions?). I didn't get
> deep
> into this, but I guess we see this since
> https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> 
> Interestingly, the CI pipeline doesn't catch this CVE (
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/),
> maybe this is some bug in OWASP that is triggered only with certain
> maven
> versions or during building on certain platforms? I ran OWASP on
> Ubuntu
> 18.04.2 with maven 3.9.3.
> 
> 2) Also I see that the website (
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html)
> is still showing "ZooKeeper 3.8 Documentation" on the top
> 
> 
> What do you think? We shouldn't pass the RC until we are certain
> about the
> CVE issue. (unless this is something happening only on my setup... it
> is
> strange that OWAPS is green on CI)
> 
> 
> Beside these, I ran all my usual RC test steps, and found no other
> issues
> with the RC:
> - verified checksum and gpg signature of the artifacts
> - I built the source code (incl. the C-client, using -Pfull-build) on
> Ubuntu 18.04.2 using OpenJDK 8u372, maven 3.9.3 and GCC version 7.4.0
> - all the unit tests passed (both Java and C-client)
> - I also built and executed unit tests for zkpython
> - I also built the java code (without -Pfull-build) using other JDK
> versions: 11.0.19, 17.0.7, 20.0.1 (but didn't run the tests this
> time, just
> used 'clean install -DskipTests')
> - checkstyle and spotbugs passed
> - apache-rat passed
> - fatjar built
> - I executed quick rolling-upgrade tests (using
> https://github.com/symat/zk-rolling-upgrade-test):
>  - rolling upgrade from 3.5.10 to 3.9.0
>  - rolling upgrade from 3.6.4 to 3.9.0
>  - rolling upgrade from 3.7.1 to 3.9.0
>  - rolling upgrade from 3.8.2 to 3.9.0
> - compared generated release notes (
> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/releasenotes.html
> ) with Jira (
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> )
> 
> 
> Best regards,
> Máté
> 
> On Mon, Jul 17, 2023 at 3:11 PM Andor Molnar 
> wrote:
> 
> > Hi team,
> > 
> > This is a release candidate for 3.9.0.
> > 
> > It is a major release and it introduces a lot of new features, most
> > notably:
> > - Admin server API for taking snapshot and stream out the data
> > - Communicate the Zxid that triggered a WatchEvent to fire
> > - TLS - dynamic loading for client trust/key store
> > - Add Netty-TcNative OpenSSL Support
> > - Adding SSL support to Zktreeutil
> > - Improve syncRequestProcessor performance
> > - Updates to all the third party dependencies to get rid of every
> > known
> > CVE.
> > 
> > The full release notes is available at:
> > 
> > 
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304
> > 
> > *** Please download, test and vote by July 30th 2023, 23:59 UTC+0.
> > ***
> > 
> > Source files:
> > 
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/
> > 
> > Maven staging repo:
> > 
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/
> > 
> > The release candidate tag in git to be voted upon: release-3.8.0-1
> > https://github.com/apache/zookeeper/tree/release-3.9.0-0
> > 
> > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > release:
> > https://www.apache.org/dist/zookeeper/KEYS
> > 
> > The staging version of the website is:
> > 
> > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html
> > 
> > 
> > Should we release this candidate?
> > 
> > 
> > Regards,
> > Andor
> > 
> > 
> > 

[jira] [Created] (ZOOKEEPER-4721) Upgrade OWASP Dependency Check to 8.3.1

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4721:
---

 Summary: Upgrade OWASP Dependency Check to 8.3.1
 Key: ZOOKEEPER-4721
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4721
 Project: ZooKeeper
  Issue Type: Bug
  Components: build
Affects Versions: 3.5.4, 3.6.0, 3.4.12
Reporter: Abraham Fine
Assignee: Patrick D. Hunt
 Fix For: 3.6.0, 3.4.13, 3.5.5






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Andor Molnar]

Thanks Mate!

Please don't forget to cleanup the website: documentation both for
3.8.0 and 3.8.1 should be replaced with 3.8.2 in the main menu.

Regards,
Andor



On Mon, 2023-07-17 at 19:41 +0200, Szalay-Bekő Máté wrote:
> Thank you for the reviews!
> 
> I'm happy to announce that we have unanimously approved this release.
> There were 3 approving votes, all of them binding:
> 
> - Enrico Olivelli (binding)
> - Andor Molnár (binding)
> - Máté Szalay-Bekő (binding)
> 
> There were no disapproving votes.
> 
> I will promote the artifacts and complete the release procedure.
> 
> Thanks to everyone who contributed to this release!
> 
> Best Regards,
> Máté
> 
> On Mon, Jul 17, 2023 at 4:08 PM Andor Molnar 
> wrote:
> 
> > +1 (binding)
> > 
> > - verified checksum and gpg signature of the artifacts
> > - I built the source code (incl. the C-client, using -Pfull-build)
> > on
> > Ubuntu 20.04 using OpenJDK 8u302, maven 3.6.3 and GCC version 9.4.0
> > - all the unit tests passed (both Java and C-client)
> > - I also built the code using Oracle JDK 20.0.1
> > - checkstyle and spotbugs passed
> > - apache-rat passed
> > - owasp (CVE check) passed
> > - checked the generated documentation (zookeeper-docs/target/html)
> > - checked release notes
> > - created 3-node cluster with TLS enabled and ran some smoke tests
> > - run zk-smoketest.py (https://github.com/phunt/zk-smoketest)
> > - run zk-latencies.py (https://github.com/phunt/zk-smoketest)
> > 
> > Thanks,
> > Andor
> > 
> > 
> > 
> > 
> > On Wed, 2023-07-05 at 23:20 +0200, Szalay-Bekő Máté wrote:
> > > This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> > > including
> > > CVE fixes and additional test, security and other improvements.
> > > 
> > > Please find the full release notes in the following link:
> > > 
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> > > *** Please download, test and vote by July 14th 2023, 23:59
> > > UTC+0.
> > > ***
> > > 
> > > 
> > > Source files:
> > > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
> > > 
> > > Maven staging repo:
> > > 
> > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
> > > The release candidate tag in git to be voted upon: release-3.8.2-
> > > 0
> > > (please note, branch-3.8.2 will move here only after the vote)
> > > 
> > > ZooKeeper's KEYS file containing PGP keys we use to sign the
> > > release:
> > > https://www.apache.org/dist/zookeeper/KEYS
> > > 
> > > The staging version of the website is:
> > > https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
> > > 
> > > 
> > > Should we release this candidate?
> > > 
> > > 
> > > Best regards,
> > > Máté

Re: [VOTE] Apache ZooKeeper release 3.8.2 candidate 0

[Andor Molnar]

+1 (binding)

- verified checksum and gpg signature of the artifacts
- I built the source code (incl. the C-client, using -Pfull-build) on
Ubuntu 20.04 using OpenJDK 8u302, maven 3.6.3 and GCC version 9.4.0
- all the unit tests passed (both Java and C-client)
- I also built the code using Oracle JDK 20.0.1 
- checkstyle and spotbugs passed
- apache-rat passed
- owasp (CVE check) passed
- checked the generated documentation (zookeeper-docs/target/html)
- checked release notes
- created 3-node cluster with TLS enabled and ran some smoke tests
- run zk-smoketest.py (https://github.com/phunt/zk-smoketest)
- run zk-latencies.py (https://github.com/phunt/zk-smoketest)

Thanks,
Andor




On Wed, 2023-07-05 at 23:20 +0200, Szalay-Bekő Máté wrote:
> This is a bugfix release candidate for 3.8.2. It fixes 12 issues,
> including
> CVE fixes and additional test, security and other improvements.
> 
> Please find the full release notes in the following link:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12352866
> 
> *** Please download, test and vote by July 14th 2023, 23:59 UTC+0.
> ***
> 
> 
> Source files:
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/
> 
> Maven staging repo:
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.8.2/
> 
> The release candidate tag in git to be voted upon: release-3.8.2-0
> (please note, branch-3.8.2 will move here only after the vote)
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.8.2-rc0/website/
> 
> 
> Should we release this candidate?
> 
> 
> Best regards,
> Máté

[VOTE] Apache ZooKeeper release 3.9.0 candidate 0

[Andor Molnar]

Hi team,

This is a release candidate for 3.9.0.

It is a major release and it introduces a lot of new features, most
notably:
- Admin server API for taking snapshot and stream out the data
- Communicate the Zxid that triggered a WatchEvent to fire
- TLS - dynamic loading for client trust/key store
- Add Netty-TcNative OpenSSL Support
- Adding SSL support to Zktreeutil
- Improve syncRequestProcessor performance
- Updates to all the third party dependencies to get rid of every known
CVE.

The full release notes is available at:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12351304

*** Please download, test and vote by July 30th 2023, 23:59 UTC+0. ***

Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/

Maven staging repo:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.9.0/

The release candidate tag in git to be voted upon: release-3.8.0-1
https://github.com/apache/zookeeper/tree/release-3.9.0-0

ZooKeeper's KEYS file containing PGP keys we use to sign the release:
https://www.apache.org/dist/zookeeper/KEYS

The staging version of the website is:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.9.0-candidate-0/website/index.html


Should we release this candidate?


Regards,
Andor

Re: ZOOKEEPER-4714 and ZOOKEEPER-4715

[Andor Molnar]

ZOOKEEPER-4714 is merged now. No more blockers, I'll cut the release.

Andor



On Fri, 2023-07-14 at 16:46 +0200, Andor Molnar wrote:
> In my understanding 
> 
> ZOOKEEPER-4714 <-- depends on <-- ZOOKEEPER-4715 
> 
> which is already submitted to the master branch. Additionally
> ZOOKEEPER-4714 is being actively reviewed by folks, so just keep up
> the
> good work, I'll wait for it for the 3.9.0 cut.
> 
> Regards,
> Andor
> 
> 
> 
> On Wed, 2023-07-05 at 11:49 +0300, Andor Molnar wrote:
> > Hi Yan Zhao,
> > 
> > I noticed you opened these 2 new tickets against the 3.9.0 fix
> > version
> > which is next release of ZooKepeer I'm just about to cut this week
> > from
> > the master branch.
> > 
> > Do you think these tickets are blockers of 3.9.0?
> > 
> > Regards,
> > Andor
> > 
> > 
> > 

Re: ZOOKEEPER-4714 and ZOOKEEPER-4715

[Andor Molnar]

In my understanding 

ZOOKEEPER-4714 <-- depends on <-- ZOOKEEPER-4715 

which is already submitted to the master branch. Additionally
ZOOKEEPER-4714 is being actively reviewed by folks, so just keep up the
good work, I'll wait for it for the 3.9.0 cut.

Regards,
Andor



On Wed, 2023-07-05 at 11:49 +0300, Andor Molnar wrote:
> Hi Yan Zhao,
> 
> I noticed you opened these 2 new tickets against the 3.9.0 fix
> version
> which is next release of ZooKepeer I'm just about to cut this week
> from
> the master branch.
> 
> Do you think these tickets are blockers of 3.9.0?
> 
> Regards,
> Andor
> 
> 
> 

Re: ZooKeeper release 3.9.0

[Andor Molnar]

Hi tison,

No need for new version yet, I still haven't cut 3.9.0 yet and probably
won't get there this week. Quite busy.

Please continue fileing patches for 3.9.0 until I send a notification
here about the start of the release process.

Regards,
Andor



On Fri, 2023-07-07 at 09:59 +0800, tison wrote:
> Hi Andor,
> 
> Since the latest unrelease version is 3.9.0, I recently mark two new
> issues
> as fixed 3.9.0.
> 
> Please check if we will carry them or we will create a 3.10.0
> unreleased
> version and move them there
> 
> * https://issues.apache.org/jira/browse/ZOOKEEPER-4718
> * https://issues.apache.org/jira/browse/ZOOKEEPER-4715
> 
> Best,
> tison.
> 
> 
> Andor Molnar  于2023年7月1日周六 21:41写道：
> 
> > I'm fine with that. Thanks Damien.
> > 
> > Andor
> > 
> > 
> > 
> > On Wed, 2023-06-21 at 14:53 +0200, Damien Diederen wrote:
> > > Hi Andor, Máté, All,
> > > 
> > > > There're 64 open tickets which has fixVersion = 3.9.0
> > > > I'll remove the fixVersion from all of them except the ones
> > > > that we
> > > > marked as release blockers.
> > > […]
> > > > Please let me know if you would like to add anything to this
> > > > list.
> > > 
> > > I have just merged this one in 'master', 'branch-3.8' and
> > > 'branch-
> > > 3.7':
> > > 
> > >   * ZOOKEEPER-4026 CREATE2 requests embeded in a MULTI request
> > > only
> > > get a regular CREATE response
> > > 
> > > which was not a blocker, but was labeled "Major" and had been
> > > lingering
> > > for a while.  I set fixVersion = 3.9.0, 3.8.2, 3.7.2; I hope this
> > > doesn't get in the way.
> > > 
> > > Cheers, -D

ZOOKEEPER-4714 and ZOOKEEPER-4715

[Andor Molnar]

Hi Yan Zhao,

I noticed you opened these 2 new tickets against the 3.9.0 fix version
which is next release of ZooKepeer I'm just about to cut this week from
the master branch.

Do you think these tickets are blockers of 3.9.0?

Regards,
Andor

Re: Netty CVE-2023-34462 (SniHandler)

[Andor Molnar]

Hi Colin,

Thanks for the heads-up. We just committed the upgrade of Netty on
master and branch-3.8:
https://github.com/apache/zookeeper/pull/2019

That means the new Netty version can be expected in 3.9.0 and 3.8.2
versions of ZooKeeper soon.

I think we should backport it to branch-3.7 too, however it's going to
be EoL soon.

3.6 is not maintained anymore, so I don't expect it to be upgraded and
new release issued.

Andor



On Wed, 2023-06-21 at 12:58 +0100, Colvin Cowie wrote:
> Hello
> 
> CVE-2023-34462 for Netty has been announced yesterday and there's a
> new
> release of Netty that patches it. There's a GH advisory for it
> https://github.com/advisories/GHSA-6mjq-h674-j845.
> 
> Is SNI enabled (by default) in ZooKeeper?
> Can the version of netty included in existing releases of ZooKeeper
> be
> replaced without code changes? I see 3.6.2 and later all include
> Netty
> 4.1.86,
> 
> Thanks
> Colvin

Re: ZooKeeper release 3.9.0

[Andor Molnar]

I'm fine with that. Thanks Damien.

Andor



On Wed, 2023-06-21 at 14:53 +0200, Damien Diederen wrote:
> Hi Andor, Máté, All,
> 
> > There're 64 open tickets which has fixVersion = 3.9.0
> > I'll remove the fixVersion from all of them except the ones that we
> > marked as release blockers.
> […]
> > Please let me know if you would like to add anything to this list.
> 
> I have just merged this one in 'master', 'branch-3.8' and 'branch-
> 3.7':
> 
>   * ZOOKEEPER-4026 CREATE2 requests embeded in a MULTI request only
> get a regular CREATE response
> 
> which was not a blocker, but was labeled "Major" and had been
> lingering
> for a while.  I set fixVersion = 3.9.0, 3.8.2, 3.7.2; I hope this
> doesn't get in the way.
> 
> Cheers, -D

zk-merge-pr.py

[Andor Molnar]

Hi all,

The greatest advantage of our merge script is the ability of
backporting to other branches. 

The greatest disadvantage of our merge script is it closes the pull
request (red color) instead of "merging" (purple color).

We could replace the current pure git approach of merging the main
commit and call a github API to squash-and-merge. Do you know about a
good python lib for doing this?

Andor

Re: ZooKeeper release 3.9.0

[Andor Molnar]

Submitted.

(was a great opportunity to verify the new Merge button restrictions)

Thanks,
Andor



On Fri, 2023-06-16 at 16:27 +0200, Andor Molnar wrote:
> Looks like Michael and Flavio have already reviewed and approved it,
> so
> let's merge it then.
> 
> Andor
> 
> 
> 
> On Fri, 2023-06-16 at 16:12 +0200, Enrico Olivelli wrote:
> > Sirius,
> > 
> > Il giorno ven 16 giu 2023 alle ore 15:09 Yang Sirius
> >  ha scritto:
> > > Hi Andor,
> > > 
> > > I am writing to inquire about the possibility to include the TLA+
> > > specifications for ZooKeeper in the upcoming release, version
> > > 3.9.0. The proposal to provide TLA+ specifications for ZooKeeper
> > > was raised in ZOOKEEPER-3615<
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-3615>;;, and
> > > addressed via pull request #1690<
> > > https://github.com/apache/zookeeper/pull/1690> on github.
> > > 
> > > Formal specifications can serve as precise documentation of the
> > > Zab
> > > design and implementation, and can help eliminate any ambiguities
> > > in the informal protocol description, which would be beneficial
> > > for
> > > ZooKeeper learners and developers. Popular consensus protocols
> > > like
> > > Paxos and Raft also provide their TLA+ specifications. It would
> > > be
> > > great to merge the pull request and include the TLA+
> > > specifications
> > > for ZooKeeper in the new version.
> > > 
> > > I have also raised a request for the review of pull request
> > > #1690<
> > > https://github.com/apache/zookeeper/pull/1690> to the ZooKeeper
> > > developer mailing list. More details can be found at 
> > > https://lists.apache.org/thread/ww4v1r733whcds64jg5wt7ozclbjhdr0
> > > .
> > > 
> > > Looking forward to your feedback!
> > 
> > I would like to commit that patch, but unfortunately there is an
> > open
> > discussion and we need some reviewers to formally approve it.
> > 
> > That said, that patch is mostly about "documentation" and it can be
> > committed after the 3.9.0 release.
> > 
> > Enrico
> > 
> > > Best regards,
> > > 
> > > Sirius
> > > 
> > > 
> > > 2023年6月15日 19:57，Andor Molnar  > > an...@apache.org>> 写道：
> > > 
> > > Hi folks,
> > > 
> > > There're 64 open tickets which has fixVersion = 3.9.0
> > > I'll remove the fixVersion from all of them except the ones that
> > > we
> > > marked as release blockers.
> > > 
> > > Currently:
> > > 
> > > - ZOOKEEPER-4393 Problem to connect to zookeeper in FIPS mode
> > > - ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> > > - ZOOKEEPER-4655 Communicate the Zxid that triggered a WatchEvent
> > > to
> > > fire
> > > 
> > > Please let me know if you would like to add anything to this
> > > list.
> > > 
> > > Regards,
> > > Andor
> > > 
> > > 
> > > 
> > > 

Re: ZooKeeper release 3.9.0

[Andor Molnar]

Looks like Michael and Flavio have already reviewed and approved it, so
let's merge it then.

Andor



On Fri, 2023-06-16 at 16:12 +0200, Enrico Olivelli wrote:
> Sirius,
> 
> Il giorno ven 16 giu 2023 alle ore 15:09 Yang Sirius
>  ha scritto:
> > Hi Andor,
> > 
> > I am writing to inquire about the possibility to include the TLA+
> > specifications for ZooKeeper in the upcoming release, version
> > 3.9.0. The proposal to provide TLA+ specifications for ZooKeeper
> > was raised in ZOOKEEPER-3615<
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3615>;, and
> > addressed via pull request #1690<
> > https://github.com/apache/zookeeper/pull/1690> on github.
> > 
> > Formal specifications can serve as precise documentation of the Zab
> > design and implementation, and can help eliminate any ambiguities
> > in the informal protocol description, which would be beneficial for
> > ZooKeeper learners and developers. Popular consensus protocols like
> > Paxos and Raft also provide their TLA+ specifications. It would be
> > great to merge the pull request and include the TLA+ specifications
> > for ZooKeeper in the new version.
> > 
> > I have also raised a request for the review of pull request #1690<
> > https://github.com/apache/zookeeper/pull/1690> to the ZooKeeper
> > developer mailing list. More details can be found at 
> > https://lists.apache.org/thread/ww4v1r733whcds64jg5wt7ozclbjhdr0 .
> > 
> > Looking forward to your feedback!
> 
> I would like to commit that patch, but unfortunately there is an open
> discussion and we need some reviewers to formally approve it.
> 
> That said, that patch is mostly about "documentation" and it can be
> committed after the 3.9.0 release.
> 
> Enrico
> 
> > Best regards,
> > 
> > Sirius
> > 
> > 
> > 2023年6月15日 19:57，Andor Molnar  > an...@apache.org>> 写道：
> > 
> > Hi folks,
> > 
> > There're 64 open tickets which has fixVersion = 3.9.0
> > I'll remove the fixVersion from all of them except the ones that we
> > marked as release blockers.
> > 
> > Currently:
> > 
> > - ZOOKEEPER-4393 Problem to connect to zookeeper in FIPS mode
> > - ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> > - ZOOKEEPER-4655 Communicate the Zxid that triggered a WatchEvent
> > to
> > fire
> > 
> > Please let me know if you would like to add anything to this list.
> > 
> > Regards,
> > Andor
> > 
> > 
> > 
> > 

Re: Current master branch is broker

[Andor Molnar]

Fix submitted.


On Fri, 2023-06-16 at 14:07 +0200, Andor Molnar wrote:
> Yeah, sorry, I've submitted the patch which broke the build. My first
> thought was to revert immediately and let the contributor to fix it,
> but Kezhu already opened a PR with the fix.
> 
> Once the build is greeen, I'm submitting it with the title:
> ZOOKEEPER-4655: [ADDENDUM] fix build error
> 
> Thanks Kezhu.
> 
> Andor
> 
> 
> 
> On Fri, 2023-06-16 at 19:44 +0800, tison wrote:
> > Thank you Kezhu!
> > 
> > This case somehow indicates that we're under a "high" traffic. lol.
> > 
> > Best,
> > tison.
> > 
> > 
> > Kezhu Wang  于2023年6月16日周五 19:20写道：
> > 
> > > Hi Enrico,
> > > 
> > > I presented a fix https://github.com/apache/zookeeper/pull/2012.
> > > 
> > > On Fri, Jun 16, 2023 at 6:47 PM Enrico Olivelli <
> > > eolive...@gmail.com>
> > > wrote:
> > > > Hello ZooKeepers,
> > > > 
> > > > Current master branch is broken, see the errors below.
> > > > 
> > > > Maybe we committed patches and there were some unexpected
> > > > conflicts
> > > > undetected by git.
> > > > 
> > > > Enrico
> > > > 
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-compiler-
> > > > plugin:3.8.1:testCompile
> > > > (default-testCompile) on project zookeeper: Compilation
> > > > failure:
> > > > Compilation failure:
> > > > 
> > > > [ERROR]
> > > /Users/enricoolivelli/dev/zookeeper/zookeeper-
> > > server/src/test/java/org/apache/zookeeper/test/PersistentRecursiv
> > > eW
> > > atcherTest.java:[198,13]
> > > > no suitable method found for
> > > > 
> > > assertEvent(java.util.concurrent.BlockingQueue > > er
> > > .WatchedEvent>,org.apache.zookeeper.Watcher.Event.EventType,java.
> > > la
> > > ng.String)
> > > > [ERROR] method
> > > > 
> > > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEv
> > > en
> > > t(java.util.concurrent.BlockingQueue > > Ev
> > > ent>,org.apache.zookeeper.Watcher.Event.EventType,java.lang.Strin
> > > g,
> > > org.apache.zookeeper.data.Stat)
> > > > is not applicable
> > > > 
> > > > [ERROR]   (actual and formal argument lists differ in
> > > > length)
> > > > 
> > > > [ERROR] method
> > > > 
> > > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEv
> > > en
> > > t(java.util.concurrent.BlockingQueue > > Ev
> > > ent>,org.apache.zookeeper.Watcher.Event.EventType,java.lang.Strin
> > > g,
> > > long)
> > > > is not applicable
> > > > 
> > > > [ERROR]   (actual and formal argument lists differ in
> > > > length)
> > > > 
> > > > [ERROR] method
> > > > 
> > > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEv
> > > en
> > > t(java.util.concurrent.BlockingQueue > > Ev
> > > ent>,org.apache.zookeeper.Watcher.Event.EventType,org.apache.zook
> > > ee
> > > per.Watcher.Event.KeeperState,java.lang.String,long)
> > > > is not applicable

Re: Current master branch is broker

[Andor Molnar]

Yeah, sorry, I've submitted the patch which broke the build. My first
thought was to revert immediately and let the contributor to fix it,
but Kezhu already opened a PR with the fix.

Once the build is greeen, I'm submitting it with the title:
ZOOKEEPER-4655: [ADDENDUM] fix build error

Thanks Kezhu.

Andor



On Fri, 2023-06-16 at 19:44 +0800, tison wrote:
> Thank you Kezhu!
> 
> This case somehow indicates that we're under a "high" traffic. lol.
> 
> Best,
> tison.
> 
> 
> Kezhu Wang  于2023年6月16日周五 19:20写道：
> 
> > Hi Enrico,
> > 
> > I presented a fix https://github.com/apache/zookeeper/pull/2012.
> > 
> > On Fri, Jun 16, 2023 at 6:47 PM Enrico Olivelli <
> > eolive...@gmail.com>
> > wrote:
> > > Hello ZooKeepers,
> > > 
> > > Current master branch is broken, see the errors below.
> > > 
> > > Maybe we committed patches and there were some unexpected
> > > conflicts
> > > undetected by git.
> > > 
> > > Enrico
> > > 
> > > [ERROR] Failed to execute goal
> > > org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> > > (default-testCompile) on project zookeeper: Compilation failure:
> > > Compilation failure:
> > > 
> > > [ERROR]
> > /Users/enricoolivelli/dev/zookeeper/zookeeper-
> > server/src/test/java/org/apache/zookeeper/test/PersistentRecursiveW
> > atcherTest.java:[198,13]
> > > no suitable method found for
> > > 
> > assertEvent(java.util.concurrent.BlockingQueue > .WatchedEvent>,org.apache.zookeeper.Watcher.Event.EventType,java.la
> > ng.String)
> > > [ERROR] method
> > > 
> > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEven
> > t(java.util.concurrent.BlockingQueue > ent>,org.apache.zookeeper.Watcher.Event.EventType,java.lang.String,
> > org.apache.zookeeper.data.Stat)
> > > is not applicable
> > > 
> > > [ERROR]   (actual and formal argument lists differ in length)
> > > 
> > > [ERROR] method
> > > 
> > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEven
> > t(java.util.concurrent.BlockingQueue > ent>,org.apache.zookeeper.Watcher.Event.EventType,java.lang.String,
> > long)
> > > is not applicable
> > > 
> > > [ERROR]   (actual and formal argument lists differ in length)
> > > 
> > > [ERROR] method
> > > 
> > org.apache.zookeeper.test.PersistentRecursiveWatcherTest.assertEven
> > t(java.util.concurrent.BlockingQueue > ent>,org.apache.zookeeper.Watcher.Event.EventType,org.apache.zookee
> > per.Watcher.Event.KeeperState,java.lang.String,long)
> > > is not applicable

[jira] [Created] (ZOOKEEPER-4705) Restrict GitHub merge butten to allow squash commit only

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4705:
---

 Summary: Restrict GitHub merge butten to allow squash commit only
 Key: ZOOKEEPER-4705
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4705
 Project: ZooKeeper
  Issue Type: Improvement
  Components: build-infrastructure
Affects Versions: 3.8.1, 3.7.1
Reporter: Andor Molnar
Assignee: Andor Molnar
 Fix For: 3.9.0, 3.7.2, 3.8.2


Based on 
[https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA=git+-+.asf.yaml+features#Git.asf.yamlfeatures-Mergebuttons]

 

Add limitation to .asf.yaml.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Netty native libraries in ZooKeeper

[Andor Molnar]

Yeah, it works fine with the hardcoded classifier.
I'll make this Pulsar approach in the patch.

Andor



On Thu, 2023-06-15 at 19:50 +0200, Enrico Olivelli wrote:
> I may be wrong but Epoll is only a Linux thing probably.
> 
> You don't have it on Mac or Windows
> 
> Enrico
> 
> Il Gio 15 Giu 2023, 19:49 Andor Molnar  ha scritto:
> 
> > Interesting that with only the BOM included and the dependencies
> > without the classifier, Netty doesn't load the native epoll
> > selector,
> > but loads the native SSL library.
> > 
> > I'm confused.
> > 
> > In the Pulsar example the sub-project dependency has a hardcoded
> > classifier:
> > 
> > 
> >   io.netty
> >   netty-transport-native-epoll
> >   linux-x86_64
> > 
> > 
> > Andor
> > 
> > 
> > 
> > On Thu, 2023-06-15 at 14:31 +0200, Enrico Olivelli wrote:
> > > I think that the best way currently is to add these dependencies:
> > > 
> > > Import the Netty BOM in the main pom.xml
> > > 
> > >   io.netty
> > >   netty-bom
> > >   ${netty.version}
> > >   pom
> > >   import
> > > 
> > > 
> > > 
> > > declare netty dependencies without setting the version and the
> > > classifier
> > > 
> > >   io.netty
> > >   netty-tcnative-boringssl-static
> > > 
> > > 
> > > This is the way we are doing it in Pulsar
> > > 
> > https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pom.xml#L647
> > https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pulsar-common/pom.xml#L146
> > > This way Maven should bundle all the native libraries for all the
> > > supported platforms
> > > 
> > > 
> > > 
> > > Enrico
> > > 
> > > Il giorno gio 15 giu 2023 alle ore 12:50 Andor Molnar
> > >  ha scritto:
> > > > Hi,
> > > > 
> > > > I've come across the following when working on the support of
> > > > native
> > > > SSL libraries. Currently ZooKeeper supports loading the native
> > > > epoll-
> > > > based event loop of Netty, but a build profile which would
> > > > download
> > > > the
> > > > required dependencies is not shipped with our product.
> > > > 
> > > > This is perfectly okay since the feature of using native
> > > > libraries
> > > > is
> > > > not a build-time requirement, but in this case the user has to
> > > > download
> > > > the required and appropriate versions of Netty jars and put
> > > > them on
> > > > the
> > > > classpath.
> > > > 
> > > > Shall we add a Maven build profile to ease this process?
> > > > 
> > > > 
> > > >   netty-native
> > > >   
> > > > fedora > > > sifi
> > > > erWi
> > > > thLikes>
> > > >   
> > > >   
> > > > 
> > > >   
> > > > io.netty
> > > > netty-tcnative-boringssl-
> > > > static
> > > > ${netty-tcnative.version}
> > > > ${os.detected.classifier}
> > > >   
> > > >   
> > > > io.netty
> > > > netty-transport-native-epoll
> > > > ${netty.version}
> > > > ${os.detected.classifier}
> > > >   
> > > > 
> > > >   
> > > > 
> > > > 
> > > > What do you think?
> > > > 
> > > > Andor
> > > > 
> > > > 
> > > > 

Re: Netty native libraries in ZooKeeper

[Andor Molnar]

Interesting that with only the BOM included and the dependencies
without the classifier, Netty doesn't load the native epoll selector,
but loads the native SSL library.

I'm confused.

In the Pulsar example the sub-project dependency has a hardcoded
classifier: 


  io.netty
  netty-transport-native-epoll
  linux-x86_64


Andor



On Thu, 2023-06-15 at 14:31 +0200, Enrico Olivelli wrote:
> I think that the best way currently is to add these dependencies:
> 
> Import the Netty BOM in the main pom.xml
> 
>   io.netty
>   netty-bom
>   ${netty.version}
>   pom
>   import
> 
> 
> 
> declare netty dependencies without setting the version and the
> classifier
> 
>   io.netty
>   netty-tcnative-boringssl-static
> 
> 
> This is the way we are doing it in Pulsar
> https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pom.xml#L647
> https://github.com/apache/pulsar/blob/d7f355881b2b1eebf2be6ea262c202660d684fb7/pulsar-common/pom.xml#L146
> 
> This way Maven should bundle all the native libraries for all the
> supported platforms
> 
> 
> 
> Enrico
> 
> Il giorno gio 15 giu 2023 alle ore 12:50 Andor Molnar
>  ha scritto:
> > Hi,
> > 
> > I've come across the following when working on the support of
> > native
> > SSL libraries. Currently ZooKeeper supports loading the native
> > epoll-
> > based event loop of Netty, but a build profile which would download
> > the
> > required dependencies is not shipped with our product.
> > 
> > This is perfectly okay since the feature of using native libraries
> > is
> > not a build-time requirement, but in this case the user has to
> > download
> > the required and appropriate versions of Netty jars and put them on
> > the
> > classpath.
> > 
> > Shall we add a Maven build profile to ease this process?
> > 
> > 
> >   netty-native
> >   
> > fedora > erWi
> > thLikes>
> >   
> >   
> > 
> >   
> > io.netty
> > netty-tcnative-boringssl-static
> > ${netty-tcnative.version}
> > ${os.detected.classifier}
> >   
> >   
> > io.netty
> > netty-transport-native-epoll
> > ${netty.version}
> > ${os.detected.classifier}
> >   
> > 
> >   
> > 
> > 
> > What do you think?
> > 
> > Andor
> > 
> > 
> > 

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

Thanks Enrico, we've made a mistake though: discussed that fips-mode
will be enabled by default on master branch and disabled by default on
branch-3.8.

Let me create a separate pull request for that.

Andor



On Thu, 2023-06-15 at 14:39 +0200, Enrico Olivelli wrote:
> Il giorno mer 14 giu 2023 alle ore 13:43 Andor Molnar
>  ha scritto:
> > PR has been created with the proposed resolution:
> > 
> > https://github.com/apache/zookeeper/pull/2008
> 
> Committed to master and branch-3.8
> 
> Thank you
> Enrico
> 
> > Please review.
> > 
> > Thanks,
> > Andor
> > 
> > 
> > 
> > On Sat, 2023-06-10 at 11:25 +0200, Andor Molnar wrote:
> > > "we use this method dozens of other places in the code"
> > > 
> > > Checked. Mostly logging and output formatting like 4lws, etc.
> > > 
> > > 
> > > 
> > > On Sat, 2023-06-10 at 11:18 +0200, Andor Molnar wrote:
> > > > First, I've created a pull request for ZOOKEEPER-3860:
> > > > 
> > > > https://github.com/apache/zookeeper/pull/2005
> > > > 
> > > > To improve the logging in ZKTrustManager without altering the
> > > > behaviour. The patch also contains a change in
> > > > NetUtils.formatInetAddr() which, I believe, should use the
> > > > hostname
> > > > when creating textual representation of an InetAddress. I'm not
> > > > 100%
> > > > sure about this, because while it certainly helps in TLS cases
> > > > to
> > > > avoid
> > > > unnecessary reverse DNS lookups, we use this method dozens of
> > > > other
> > > > places in the code. Unit tests are passsing.
> > > > 
> > > > ZOOKEEPER-4268
> > > > 
> > > > It's about reverse lookups in the client code, but I haven't
> > > > found
> > > > the
> > > > reported behaviour on latest master, so just closed the ticket.
> > > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > On Fri, 2023-06-09 at 18:29 +0200, Szalay-Bekő Máté wrote:
> > > > > yeah, I remember these tickets, thanks for picking them up!
> > > > > I agree and like the solution you proposed, in general in the
> > > > > long
> > > > > term it
> > > > > is good not to use a custom trust manager, but rely on the
> > > > > standard
> > > > > one.
> > > > > 
> > > > > Máté
> > > > > 
> > > > > 
> > > > > On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli <
> > > > > eolive...@gmail.com
> > > > > wrote:
> > > > > 
> > > > > > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> > > > > >  ha scritto:
> > > > > > > I'd like to backport this to the 3.8 branch too.
> > > > > > > 
> > > > > > > Let's say I'll add new "zookeeper.fips-mode" parameter
> > > > > > > which
> > > > > > > will
> > > > > > > be
> > > > > > > "false" by default in 3.8 and "true" for 3.9.0.
> > > > > > 
> > > > > > I am +1
> > > > > > ZK 3.9 will take time to be adopted and this is an
> > > > > > important
> > > > > > security
> > > > > > related topic
> > > > > > 
> > > > > > Enrico
> > > > > > 
> > > > > > > Thoughts?
> > > > > > > 
> > > > > > > Andor
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > > > > > I think that switching to
> > > > > > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS
> > > > > > > > ");
> > > > > > > > is
> > > > > > > > a
> > > > > > > > good
> > > > > > > > option.
> > > > > > > > The less tweaks we have about Security code the better.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > It would be great to see this in 3.9.0.
> > > > > > > > 
&g

ZooKeeper release 3.9.0

[Andor Molnar]

Hi folks,

There're 64 open tickets which has fixVersion = 3.9.0
I'll remove the fixVersion from all of them except the ones that we
marked as release blockers.

Currently:

- ZOOKEEPER-4393 Problem to connect to zookeeper in FIPS mode
- ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support 
- ZOOKEEPER-4655 Communicate the Zxid that triggered a WatchEvent to
fire

Please let me know if you would like to add anything to this list.

Regards,
Andor

Netty native libraries in ZooKeeper

[Andor Molnar]

Hi,

I've come across the following when working on the support of native
SSL libraries. Currently ZooKeeper supports loading the native epoll-
based event loop of Netty, but a build profile which would download the
required dependencies is not shipped with our product.

This is perfectly okay since the feature of using native libraries is
not a build-time requirement, but in this case the user has to download
the required and appropriate versions of Netty jars and put them on the
classpath.

Shall we add a Maven build profile to ease this process?


  netty-native
  
fedora
  
  

  
io.netty
netty-tcnative-boringssl-static
${netty-tcnative.version}
${os.detected.classifier}
  
  
io.netty
netty-transport-native-epoll
${netty.version}
${os.detected.classifier}
  

  


What do you think?

Andor

Re: Volounteers for releases ?

[Andor Molnar]

Created a patch for this:

https://github.com/apache/zookeeper/pull/2009

Andor



On Tue, 2023-06-13 at 10:57 +0200, Andor Molnar wrote:
> Awesome. Thanks Enrico!
> 
> I owe you an apology: found an important TLS ticket which is another
> low hanging fruit:
> 
> https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> 
> ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> 
> We've already done this for HBase and I always wanted to port this
> back
> to ZooKeeper. It's a very cool performance improvement for Linux-
> based
> installations (or whereever else Netty has OpenSSL support),
> unfortunately for ZooKeeper this is only for the server-client stack.
> 
> Let me land this for 3.9.0.
> 
> Andor
> 
> 
> On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> > Tests on master branch are passing on JDK20
> > 
> > Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> > Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> > Java version: 20, vendor: Eclipse Adoptium, runtime:
> > /usr/local/asfpackages/java/adoptium-jdk-20+36
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> > family: "unix"
> > 
> > https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> > 
> > Enrico
> > 
> > Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
> >  ha scritto:
> > > Sure. I've just noticed that the patch has been outstanding for a
> > > year
> > > now, small and ready to be submitted.
> > > 
> > > Andor
> > > 
> > > 
> > > 
> > > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > > >  ha scritto:
> > > > > I came across the graceful termination patch yesterday.
> > > > > Sounds
> > > > > like
> > > > > important for K8s environments. Enrico, what do you think?
> > > > > Looks
> > > > > like
> > > > > you're not a fan of that.
> > > > > 
> > > > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > > > > 
> > > > > https://github.com/apache/zookeeper/pull/1898
> > > > 
> > > > I have taken a look and I have posted some feedback about
> > > > renaming
> > > > the
> > > > configuration flag.
> > > > I am not sure that we are in a hurry to commit that patch, we
> > > > can
> > > > release it with 3.9.1.
> > > > 
> > > > Enrico
> > > > 
> > > > 
> > > > > Andor
> > > > > 
> > > > > 
> > > > > 
> > > > > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > > > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > > > > >  ha scritto:
> > > > > > > Hi Kezhu,
> > > > > > > 
> > > > > > > Sure, I'll take a look at the open PRs before cutting
> > > > > > > 3.9.0
> > > > > > > from
> > > > > > > master. Let's mark these tickets release blockers as you
> > > > > > > suggested.
> > > > > > > 
> > > > > > > Any more blockers of 3.9.0 that anyone knows about?
> > > > > > 
> > > > > > No, there are no critical issues at the moment.
> > > > > > 
> > > > > > I will double check on compatibility with the latest JDKs,
> > > > > > it
> > > > > > is
> > > > > > better that when we cut a new major release
> > > > > > it works well with the newer JDKs
> > > > > > 
> > > > > > Enrico
> > > > > > 
> > > > > > 
> > > > > > > Andor
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > Sorry for the disruption.
> > > > > > > > 
> > > > > > > > I want to ask if there is any possibility for us to
> > > > > > > > include
> > > > > > > > [ZOOKEEPER-4471

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

PR has been created with the proposed resolution:

https://github.com/apache/zookeeper/pull/2008

Please review.

Thanks,
Andor



On Sat, 2023-06-10 at 11:25 +0200, Andor Molnar wrote:
> "we use this method dozens of other places in the code"
> 
> Checked. Mostly logging and output formatting like 4lws, etc.
> 
> 
> 
> On Sat, 2023-06-10 at 11:18 +0200, Andor Molnar wrote:
> > First, I've created a pull request for ZOOKEEPER-3860:
> > 
> > https://github.com/apache/zookeeper/pull/2005
> > 
> > To improve the logging in ZKTrustManager without altering the
> > behaviour. The patch also contains a change in
> > NetUtils.formatInetAddr() which, I believe, should use the hostname
> > when creating textual representation of an InetAddress. I'm not
> > 100%
> > sure about this, because while it certainly helps in TLS cases to
> > avoid
> > unnecessary reverse DNS lookups, we use this method dozens of other
> > places in the code. Unit tests are passsing.
> > 
> > ZOOKEEPER-4268
> > 
> > It's about reverse lookups in the client code, but I haven't found
> > the
> > reported behaviour on latest master, so just closed the ticket.
> > 
> > Andor
> > 
> > 
> > 
> > On Fri, 2023-06-09 at 18:29 +0200, Szalay-Bekő Máté wrote:
> > > yeah, I remember these tickets, thanks for picking them up!
> > > I agree and like the solution you proposed, in general in the
> > > long
> > > term it
> > > is good not to use a custom trust manager, but rely on the
> > > standard
> > > one.
> > > 
> > > Máté
> > > 
> > > 
> > > On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli <
> > > eolive...@gmail.com
> > > wrote:
> > > 
> > > > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> > > >  ha scritto:
> > > > > I'd like to backport this to the 3.8 branch too.
> > > > > 
> > > > > Let's say I'll add new "zookeeper.fips-mode" parameter which
> > > > > will
> > > > > be
> > > > > "false" by default in 3.8 and "true" for 3.9.0.
> > > > 
> > > > I am +1
> > > > ZK 3.9 will take time to be adopted and this is an important
> > > > security
> > > > related topic
> > > > 
> > > > Enrico
> > > > 
> > > > > Thoughts?
> > > > > 
> > > > > Andor
> > > > > 
> > > > > 
> > > > > 
> > > > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > > > I think that switching to
> > > > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
> > > > > > is
> > > > > > a
> > > > > > good
> > > > > > option.
> > > > > > The less tweaks we have about Security code the better.
> > > > > > 
> > > > > > 
> > > > > > It would be great to see this in 3.9.0.
> > > > > > 
> > > > > > Enrico
> > > > > > 
> > > > > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > > > > >  ha scritto:
> > > > > > > Hi zk folks,
> > > > > > > 
> > > > > > > Problem(s)
> > > > > > > ==
> > > > > > > 
> > > > > > > One problem that we're having with a custom Trust Manager
> > > > > > > in
> > > > > > > ZK is
> > > > > > > that
> > > > > > > FIPS doesn't allow that:
> > > > > > > 
> > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > > > > > 
> > > > > > > In FIPS mode the only allowed TrustManager in the JDK is
> > > > > > > X509TrustManagerImpl which is the default implementation.
> > > > > > > The
> > > > > > > class
> > > > > > > is
> > > > > > > final, so extending it is not an option unfortunately.
> > > > > > > 
> > > > > > > The intention behind implementing a custom trust manager
> > > > > > > in
> > > > > > > ZK was,
> > > > > > > I
> > > > > > > believe, the need for server a

Re: Volounteers for releases ?

[Andor Molnar]

Hi Mate,

Not sure how much room do we have, but TLS patches are going to be
backported to 3.8 too.

Andor




On Wed, 2023-06-14 at 08:34 +0200, Enrico Olivelli wrote:
> Il Mer 14 Giu 2023, 08:06 Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com> ha
> scritto:
> 
> > Hello!
> > 
> > I also might be able to manage a release. I let Andor to do the
> > 3.9.0 ;)
> > but I can make a 3.7 or 3.8 release.
> > 
> > Based on our policy, when 3.9.0 will be released, we will deprecate
> > the 3.7
> > line. 3.7.1 happened one year ago. Normally I would say it would be
> > good to
> > make a last 3.7 release (3.7.2) before we terminate 3.7. But 3.7
> > and 3.8
> > are very similar (we cut 3.8 only for log4j vulnerability). Maybe
> > we can
> > deprecate 3.7 without  3.7.2 and I should focus on 3.8.2 instead?
> > 
> > what do you think?
> > 
> 
> Let's focus on 3.8.2.
> Users on 3.7 can easily migrate to 3.8
> 
> 
> Thanks
> 
> Enrico
> 
> 
> 
> 
> > Mate
> > 
> > On Tue, Jun 13, 2023 at 10:58 AM Andor Molnar 
> > wrote:
> > 
> > > Awesome. Thanks Enrico!
> > > 
> > > I owe you an apology: found an important TLS ticket which is
> > > another
> > > low hanging fruit:
> > > 
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-4622
> > > 
> > > ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support
> > > 
> > > We've already done this for HBase and I always wanted to port
> > > this back
> > > to ZooKeeper. It's a very cool performance improvement for Linux-
> > > based
> > > installations (or whereever else Netty has OpenSSL support),
> > > unfortunately for ZooKeeper this is only for the server-client
> > > stack.
> > > 
> > > Let me land this for 3.9.0.
> > > 
> > > Andor
> > > 
> > > 
> > > On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> > > > Tests on master branch are passing on JDK20
> > > > 
> > > > Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> > > > Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> > > > Java version: 20, vendor: Eclipse Adoptium, runtime:
> > > > /usr/local/asfpackages/java/adoptium-jdk-20+36
> > > > Default locale: en_US, platform encoding: UTF-8
> > > > OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> > > > family: "unix"
> > > > 
> > > > https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> > > > 
> > > > Enrico
> > > > 
> > > > Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
> > > >  ha scritto:
> > > > > Sure. I've just noticed that the patch has been outstanding
> > > > > for a
> > > > > year
> > > > > now, small and ready to be submitted.
> > > > > 
> > > > > Andor
> > > > > 
> > > > > 
> > > > > 
> > > > > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > > > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > > > > >  ha scritto:
> > > > > > > I came across the graceful termination patch yesterday.
> > > > > > > Sounds
> > > > > > > like
> > > > > > > important for K8s environments. Enrico, what do you
> > > > > > > think?
> > > > > > > Looks
> > > > > > > like
> > > > > > > you're not a fan of that.
> > > > > > > 
> > > > > > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > > > > > > 
> > > > > > > https://github.com/apache/zookeeper/pull/1898
> > > > > > 
> > > > > > I have taken a look and I have posted some feedback about
> > > > > > renaming
> > > > > > the
> > > > > > configuration flag.
> > > > > > I am not sure that we are in a hurry to commit that patch,
> > > > > > we can
> > > > > > release it with 3.9.1.
> > > > > > 
> > > > > > Enrico
> > > > > > 
> > > > > > 
> > > > > > > Andor
> > > > > > > 
> > > > > > > 
> &g

Re: Volounteers for releases ?

[Andor Molnar]

Awesome. Thanks Enrico!

I owe you an apology: found an important TLS ticket which is another
low hanging fruit:

https://issues.apache.org/jira/browse/ZOOKEEPER-4622

ZOOKEEPER-4622 Add Netty-TcNative OpenSSL Support

We've already done this for HBase and I always wanted to port this back
to ZooKeeper. It's a very cool performance improvement for Linux-based
installations (or whereever else Netty has OpenSSL support),
unfortunately for ZooKeeper this is only for the server-client stack.

Let me land this for 3.9.0.

Andor


On Mon, 2023-06-12 at 15:41 +0200, Enrico Olivelli wrote:
> Tests on master branch are passing on JDK20
> 
> Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
> Maven home: /home/jenkins/tools/maven/apache-maven-3.9.2
> Java version: 20, vendor: Eclipse Adoptium, runtime:
> /usr/local/asfpackages/java/adoptium-jdk-20+36
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.15.0-206-generic", arch: "amd64",
> family: "unix"
> 
> https://ci-hadoop.apache.org/view/ZooKeeper/job/ZooKeeper-Java-EA/113/
> 
> Enrico
> 
> Il giorno lun 12 giu 2023 alle ore 15:16 Andor Molnar
>  ha scritto:
> > Sure. I've just noticed that the patch has been outstanding for a
> > year
> > now, small and ready to be submitted.
> > 
> > Andor
> > 
> > 
> > 
> > On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> > > Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
> > >  ha scritto:
> > > > I came across the graceful termination patch yesterday. Sounds
> > > > like
> > > > important for K8s environments. Enrico, what do you think?
> > > > Looks
> > > > like
> > > > you're not a fan of that.
> > > > 
> > > > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > > > 
> > > > https://github.com/apache/zookeeper/pull/1898
> > > 
> > > I have taken a look and I have posted some feedback about
> > > renaming
> > > the
> > > configuration flag.
> > > I am not sure that we are in a hurry to commit that patch, we can
> > > release it with 3.9.1.
> > > 
> > > Enrico
> > > 
> > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > > > >  ha scritto:
> > > > > > Hi Kezhu,
> > > > > > 
> > > > > > Sure, I'll take a look at the open PRs before cutting 3.9.0
> > > > > > from
> > > > > > master. Let's mark these tickets release blockers as you
> > > > > > suggested.
> > > > > > 
> > > > > > Any more blockers of 3.9.0 that anyone knows about?
> > > > > 
> > > > > No, there are no critical issues at the moment.
> > > > > 
> > > > > I will double check on compatibility with the latest JDKs, it
> > > > > is
> > > > > better that when we cut a new major release
> > > > > it works well with the newer JDKs
> > > > > 
> > > > > Enrico
> > > > > 
> > > > > 
> > > > > > Andor
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > > > > Hi all,
> > > > > > > 
> > > > > > > Sorry for the disruption.
> > > > > > > 
> > > > > > > I want to ask if there is any possibility for us to
> > > > > > > include
> > > > > > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > > > > > 
> > > > > > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent`
> > > > > > > and
> > > > > > > `WatcherType.PersistentRecursive` to remove
> > > > > > > `AddWatchMode.PERSISTENT`
> > > > > > > and
> > > > > > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > > > > > complementary to
> > > > > > > [ZOOKEEPER-4466][3] which supports multiple different
> > > > > > > watches
> > > > > > > on
> > > > > > > one
> > > > > > > p

Re: Volounteers for releases ?

[Andor Molnar]

Sure. I've just noticed that the patch has been outstanding for a year
now, small and ready to be submitted. 

Andor



On Mon, 2023-06-12 at 14:29 +0200, Enrico Olivelli wrote:
> Il giorno lun 12 giu 2023 alle ore 11:13 Andor Molnar
>  ha scritto:
> > I came across the graceful termination patch yesterday. Sounds like
> > important for K8s environments. Enrico, what do you think? Looks
> > like
> > you're not a fan of that.
> > 
> > ZOOKEEPER-4400 Zookeeper not getting Graceful Termination
> > 
> > https://github.com/apache/zookeeper/pull/1898
> 
> I have taken a look and I have posted some feedback about renaming
> the
> configuration flag.
> I am not sure that we are in a hurry to commit that patch, we can
> release it with 3.9.1.
> 
> Enrico
> 
> 
> > Andor
> > 
> > 
> > 
> > On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> > > Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
> > >  ha scritto:
> > > > Hi Kezhu,
> > > > 
> > > > Sure, I'll take a look at the open PRs before cutting 3.9.0
> > > > from
> > > > master. Let's mark these tickets release blockers as you
> > > > suggested.
> > > > 
> > > > Any more blockers of 3.9.0 that anyone knows about?
> > > 
> > > No, there are no critical issues at the moment.
> > > 
> > > I will double check on compatibility with the latest JDKs, it is
> > > better that when we cut a new major release
> > > it works well with the newer JDKs
> > > 
> > > Enrico
> > > 
> > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > > > Hi all,
> > > > > 
> > > > > Sorry for the disruption.
> > > > > 
> > > > > I want to ask if there is any possibility for us to include
> > > > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > > > 
> > > > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> > > > > `WatcherType.PersistentRecursive` to remove
> > > > > `AddWatchMode.PERSISTENT`
> > > > > and
> > > > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > > > complementary to
> > > > > [ZOOKEEPER-4466][3] which supports multiple different watches
> > > > > on
> > > > > one
> > > > > path
> > > > > and was merged to master one month ago. If we don't get it to
> > > > > 3.9.0,
> > > > > it is
> > > > > probably impossible for us to merge it to patch versions of
> > > > > 3.9
> > > > > series as
> > > > > it touches both server logic and api side. It might be
> > > > > strange to
> > > > > ship
> > > > > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to
> > > > > support
> > > > > different
> > > > > watcher types on one path in watching but not all of them in
> > > > > removing.
> > > > > 
> > > > > ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver
> > > > > comprehensive
> > > > > tests.
> > > > > 
> > > > > I have already sent a [review request][3] to the dev mailing
> > > > > list. It
> > > > > has
> > > > > more descriptional information and a real world use case
> > > > > of  ZOOKEEPER-4472.
> > > > > 
> > > > > Look forward to your feedback in either thread!
> > > > > 
> > > > > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> > > > > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> > > > > [3]:
> > > > > https://lists.apache.org/thread/m7gxcffsnjy2lm8g52nssfxb6t800o3r
> > > > > 
> > > > > 
> > > > > Best,
> > > > > Kezhu Wang
> > > > > 
> > > > > 
> > > > > 
> > > > > Best,
> > > > > Kezhu Wang
> > > > > 
> > > > > 
> > > > > On Fri, Jun 9, 2023 at 6:17 PM Andor Molnar  > > > > >
> > > > > wrote:
> > > > > 
> > > > > > Hi Enrico,
> > > > > > 
> > > > > > I can take the master cut next week, but let me put
> > > > > > together an
> > > > > > email
> > > > > > about a TLS topic first. I'd like to propose a fix to
> > > > > > resolve
> > > > > > the
> > > > > > problem of FIPS (custome trust manager in ZK) and reverse
> > > > > > DNS
> > > > > > lookups.
> > > > > > I'd like to include it in 3.9.0 and 3.8.2.
> > > > > > 
> > > > > > Andor
> > > > > > 
> > > > > > p.s. Whoever is making a change on the webpage, please
> > > > > > remove
> > > > > > the
> > > > > > 3.8.0
> > > > > > release.
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> > > > > > > Hello ZooKeepers,
> > > > > > > I think that it is time to do a round of releases.
> > > > > > > 
> > > > > > > We should cut a release out of the master branch, 3.9.0
> > > > > > > and
> > > > > > > main
> > > > > > > cutting a release out of 3.7.x and 3.8.x would be useful.
> > > > > > > 
> > > > > > > Before cutting the release please ensure that third party
> > > > > > > libraries
> > > > > > > are not reported against CVEs
> > > > > > > 
> > > > > > > 
> > > > > > > This is the list of the latest releases
> > > > > > > https://zookeeper.apache.org/releases.html
> > > > > > > 
> > > > > > > Would anyone volunteer ?
> > > > > > > 
> > > > > > > Enrico

Re: Volounteers for releases ?

[Andor Molnar]

I came across the graceful termination patch yesterday. Sounds like
important for K8s environments. Enrico, what do you think? Looks like
you're not a fan of that.

ZOOKEEPER-4400 Zookeeper not getting Graceful Termination

https://github.com/apache/zookeeper/pull/1898

Andor



On Mon, 2023-06-12 at 08:49 +0200, Enrico Olivelli wrote:
> Il giorno lun 12 giu 2023 alle ore 08:19 Andor Molnar
>  ha scritto:
> > Hi Kezhu,
> > 
> > Sure, I'll take a look at the open PRs before cutting 3.9.0 from
> > master. Let's mark these tickets release blockers as you suggested.
> > 
> > Any more blockers of 3.9.0 that anyone knows about?
> 
> No, there are no critical issues at the moment.
> 
> I will double check on compatibility with the latest JDKs, it is
> better that when we cut a new major release
> it works well with the newer JDKs
> 
> Enrico
> 
> 
> > Andor
> > 
> > 
> > 
> > On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> > > Hi all,
> > > 
> > > Sorry for the disruption.
> > > 
> > > I want to ask if there is any possibility for us to include
> > > [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> > > 
> > > ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> > > `WatcherType.PersistentRecursive` to remove
> > > `AddWatchMode.PERSISTENT`
> > > and
> > > `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> > > complementary to
> > > [ZOOKEEPER-4466][3] which supports multiple different watches on
> > > one
> > > path
> > > and was merged to master one month ago. If we don't get it to
> > > 3.9.0,
> > > it is
> > > probably impossible for us to merge it to patch versions of 3.9
> > > series as
> > > it touches both server logic and api side. It might be strange to
> > > ship
> > > ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to support
> > > different
> > > watcher types on one path in watching but not all of them in
> > > removing.
> > > 
> > > ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver comprehensive
> > > tests.
> > > 
> > > I have already sent a [review request][3] to the dev mailing
> > > list. It
> > > has
> > > more descriptional information and a real world use case
> > > of  ZOOKEEPER-4472.
> > > 
> > > Look forward to your feedback in either thread!
> > > 
> > > [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> > > [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> > > [3]: 
> > > https://lists.apache.org/thread/m7gxcffsnjy2lm8g52nssfxb6t800o3r
> > > 
> > > 
> > > Best,
> > > Kezhu Wang
> > > 
> > > 
> > > 
> > > Best,
> > > Kezhu Wang
> > > 
> > > 
> > > On Fri, Jun 9, 2023 at 6:17 PM Andor Molnar 
> > > wrote:
> > > 
> > > > Hi Enrico,
> > > > 
> > > > I can take the master cut next week, but let me put together an
> > > > email
> > > > about a TLS topic first. I'd like to propose a fix to resolve
> > > > the
> > > > problem of FIPS (custome trust manager in ZK) and reverse DNS
> > > > lookups.
> > > > I'd like to include it in 3.9.0 and 3.8.2.
> > > > 
> > > > Andor
> > > > 
> > > > p.s. Whoever is making a change on the webpage, please remove
> > > > the
> > > > 3.8.0
> > > > release.
> > > > 
> > > > 
> > > > 
> > > > On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> > > > > Hello ZooKeepers,
> > > > > I think that it is time to do a round of releases.
> > > > > 
> > > > > We should cut a release out of the master branch, 3.9.0 and
> > > > > main
> > > > > cutting a release out of 3.7.x and 3.8.x would be useful.
> > > > > 
> > > > > Before cutting the release please ensure that third party
> > > > > libraries
> > > > > are not reported against CVEs
> > > > > 
> > > > > 
> > > > > This is the list of the latest releases
> > > > > https://zookeeper.apache.org/releases.html
> > > > > 
> > > > > Would anyone volunteer ?
> > > > > 
> > > > > Enrico

Re: Volounteers for releases ?

[Andor Molnar]

Hi Kezhu,

Sure, I'll take a look at the open PRs before cutting 3.9.0 from
master. Let's mark these tickets release blockers as you suggested.

Any more blockers of 3.9.0 that anyone knows about?

Andor



On Sun, 2023-06-11 at 00:41 +0800, Kezhu Wang wrote:
> Hi all,
> 
> Sorry for the disruption.
> 
> I want to ask if there is any possibility for us to include
> [ZOOKEEPER-4471][1] and [ZOOKEEPER-4472][2] in 3.9.0.
> 
> ZOOKEEPER-4472 proposed to add `WatcherType.Persistent` and
> `WatcherType.PersistentRecursive` to remove `AddWatchMode.PERSISTENT`
> and
> `AddWatchMode.PERSISTENT_RECURSIVE` respectively. It is a
> complementary to
> [ZOOKEEPER-4466][3] which supports multiple different watches on one
> path
> and was merged to master one month ago. If we don't get it to 3.9.0,
> it is
> probably impossible for us to merge it to patch versions of 3.9
> series as
> it touches both server logic and api side. It might be strange to
> ship
> ZOOKEEPER-4466 without ZOOKEEPER-4472, as we are going to support
> different
> watcher types on one path in watching but not all of them in
> removing.
> 
> ZOOKEEPER-4472 relies on ZOOKEEPER-4471 to deliver comprehensive
> tests.
> 
> I have already sent a [review request][3] to the dev mailing list. It
> has
> more descriptional information and a real world use case
> of  ZOOKEEPER-4472.
> 
> Look forward to your feedback in either thread!
> 
> [1]: https://issues.apache.org/jira/browse/ZOOKEEPER-4471
> [2]: https://issues.apache.org/jira/browse/ZOOKEEPER-4472
> [3]: https://lists.apache.org/thread/m7gxcffsnjy2lm8g52nssfxb6t800o3r
> 
> 
> Best,
> Kezhu Wang
> 
> 
> 
> Best,
> Kezhu Wang
> 
> 
> On Fri, Jun 9, 2023 at 6:17 PM Andor Molnar  wrote:
> 
> > Hi Enrico,
> > 
> > I can take the master cut next week, but let me put together an
> > email
> > about a TLS topic first. I'd like to propose a fix to resolve the
> > problem of FIPS (custome trust manager in ZK) and reverse DNS
> > lookups.
> > I'd like to include it in 3.9.0 and 3.8.2.
> > 
> > Andor
> > 
> > p.s. Whoever is making a change on the webpage, please remove the
> > 3.8.0
> > release.
> > 
> > 
> > 
> > On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> > > Hello ZooKeepers,
> > > I think that it is time to do a round of releases.
> > > 
> > > We should cut a release out of the master branch, 3.9.0 and main
> > > cutting a release out of 3.7.x and 3.8.x would be useful.
> > > 
> > > Before cutting the release please ensure that third party
> > > libraries
> > > are not reported against CVEs
> > > 
> > > 
> > > This is the list of the latest releases
> > > https://zookeeper.apache.org/releases.html
> > > 
> > > Would anyone volunteer ?
> > > 
> > > Enrico

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

"we use this method dozens of other places in the code"

Checked. Mostly logging and output formatting like 4lws, etc.



On Sat, 2023-06-10 at 11:18 +0200, Andor Molnar wrote:
> First, I've created a pull request for ZOOKEEPER-3860:
> 
> https://github.com/apache/zookeeper/pull/2005
> 
> To improve the logging in ZKTrustManager without altering the
> behaviour. The patch also contains a change in
> NetUtils.formatInetAddr() which, I believe, should use the hostname
> when creating textual representation of an InetAddress. I'm not 100%
> sure about this, because while it certainly helps in TLS cases to
> avoid
> unnecessary reverse DNS lookups, we use this method dozens of other
> places in the code. Unit tests are passsing.
> 
> ZOOKEEPER-4268
> 
> It's about reverse lookups in the client code, but I haven't found
> the
> reported behaviour on latest master, so just closed the ticket.
> 
> Andor
> 
> 
> 
> On Fri, 2023-06-09 at 18:29 +0200, Szalay-Bekő Máté wrote:
> > yeah, I remember these tickets, thanks for picking them up!
> > I agree and like the solution you proposed, in general in the long
> > term it
> > is good not to use a custom trust manager, but rely on the standard
> > one.
> > 
> > Máté
> > 
> > 
> > On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli  > >
> > wrote:
> > 
> > > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> > >  ha scritto:
> > > > I'd like to backport this to the 3.8 branch too.
> > > > 
> > > > Let's say I'll add new "zookeeper.fips-mode" parameter which
> > > > will
> > > > be
> > > > "false" by default in 3.8 and "true" for 3.9.0.
> > > 
> > > I am +1
> > > ZK 3.9 will take time to be adopted and this is an important
> > > security
> > > related topic
> > > 
> > > Enrico
> > > 
> > > > Thoughts?
> > > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > > I think that switching to
> > > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is
> > > > > a
> > > > > good
> > > > > option.
> > > > > The less tweaks we have about Security code the better.
> > > > > 
> > > > > 
> > > > > It would be great to see this in 3.9.0.
> > > > > 
> > > > > Enrico
> > > > > 
> > > > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > > > >  ha scritto:
> > > > > > Hi zk folks,
> > > > > > 
> > > > > > Problem(s)
> > > > > > ==
> > > > > > 
> > > > > > One problem that we're having with a custom Trust Manager
> > > > > > in
> > > > > > ZK is
> > > > > > that
> > > > > > FIPS doesn't allow that:
> > > > > > 
> > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > > > > 
> > > > > > In FIPS mode the only allowed TrustManager in the JDK is
> > > > > > X509TrustManagerImpl which is the default implementation.
> > > > > > The
> > > > > > class
> > > > > > is
> > > > > > final, so extending it is not an option unfortunately.
> > > > > > 
> > > > > > The intention behind implementing a custom trust manager in
> > > > > > ZK was,
> > > > > > I
> > > > > > believe, the need for server and client-side hostname
> > > > > > verification.
> > > > > > Hostname verification officially is not part of the SSL/TLS
> > > > > > protocol,
> > > > > > it's the responsibility of an upper level protocol like
> > > > > > HTTPS.
> > > > > > 
> > > > > > Hacking hostname verification in the SSL handshake is nice
> > > > > > and was
> > > > > > working fine so far, but unfortunately breaks the FIPS
> > > > > > standard.
> > > > > > 
> > > > > > Another annoying issue with ZKTrustManager is the need for
> > > > > > reverse
> > > > > > DNS
> > > > > > lookup. This i

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

First, I've created a pull request for ZOOKEEPER-3860:

https://github.com/apache/zookeeper/pull/2005

To improve the logging in ZKTrustManager without altering the
behaviour. The patch also contains a change in
NetUtils.formatInetAddr() which, I believe, should use the hostname
when creating textual representation of an InetAddress. I'm not 100%
sure about this, because while it certainly helps in TLS cases to avoid
unnecessary reverse DNS lookups, we use this method dozens of other
places in the code. Unit tests are passsing.

ZOOKEEPER-4268

It's about reverse lookups in the client code, but I haven't found the
reported behaviour on latest master, so just closed the ticket.

Andor



On Fri, 2023-06-09 at 18:29 +0200, Szalay-Bekő Máté wrote:
> yeah, I remember these tickets, thanks for picking them up!
> I agree and like the solution you proposed, in general in the long
> term it
> is good not to use a custom trust manager, but rely on the standard
> one.
> 
> Máté
> 
> 
> On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli 
> wrote:
> 
> > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> >  ha scritto:
> > > I'd like to backport this to the 3.8 branch too.
> > > 
> > > Let's say I'll add new "zookeeper.fips-mode" parameter which will
> > > be
> > > "false" by default in 3.8 and "true" for 3.9.0.
> > 
> > I am +1
> > ZK 3.9 will take time to be adopted and this is an important
> > security
> > related topic
> > 
> > Enrico
> > 
> > > Thoughts?
> > > 
> > > Andor
> > > 
> > > 
> > > 
> > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > I think that switching to
> > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is a
> > > > good
> > > > option.
> > > > The less tweaks we have about Security code the better.
> > > > 
> > > > 
> > > > It would be great to see this in 3.9.0.
> > > > 
> > > > Enrico
> > > > 
> > > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > > >  ha scritto:
> > > > > Hi zk folks,
> > > > > 
> > > > > Problem(s)
> > > > > ==
> > > > > 
> > > > > One problem that we're having with a custom Trust Manager in
> > > > > ZK is
> > > > > that
> > > > > FIPS doesn't allow that:
> > > > > 
> > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > > > 
> > > > > In FIPS mode the only allowed TrustManager in the JDK is
> > > > > X509TrustManagerImpl which is the default implementation. The
> > > > > class
> > > > > is
> > > > > final, so extending it is not an option unfortunately.
> > > > > 
> > > > > The intention behind implementing a custom trust manager in
> > > > > ZK was,
> > > > > I
> > > > > believe, the need for server and client-side hostname
> > > > > verification.
> > > > > Hostname verification officially is not part of the SSL/TLS
> > > > > protocol,
> > > > > it's the responsibility of an upper level protocol like
> > > > > HTTPS.
> > > > > 
> > > > > Hacking hostname verification in the SSL handshake is nice
> > > > > and was
> > > > > working fine so far, but unfortunately breaks the FIPS
> > > > > standard.
> > > > > 
> > > > > Another annoying issue with ZKTrustManager is the need for
> > > > > reverse
> > > > > DNS
> > > > > lookup. This is usually needed when the hostname of the
> > > > > certificate
> > > > > provider is not known at the time of handshake. For instance,
> > > > > when
> > > > > somebody connects the client via IP address, which is
> > > > > generally not
> > > > > recommended when TLS is active in the client-server protocol.
> > > > > 
> > > > > The bigger problem I've found is in the leader election: when
> > > > > a
> > > > > peer
> > > > > connects with a smaller id, the node will close the existing
> > > > > connection
> > > > > and opens a new one in the other direction, based on the
> > > > > information
> > > > > received i

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

Sure, good point. I don't want to wipe it completely, just putting it
behind a feature flag.



On Fri, 2023-06-09 at 10:03 -0700, Patrick Hunt wrote:
> "remove ZKTrustManager entirely from the codebase" - what is the
> impact on
> backward compatibility if this is done? Why wouldn't we keep this as
> an
> option (not the default?) to ensure folks won't experience a "gap"
> when
> migrating to new versions. We could phase it out over time as part of
> such
> a plan (at least n-1 compatibility is something we have guaranteed in
> the
> past)
> 
> Patrick
> 
> On Fri, Jun 9, 2023 at 9:29 AM Szalay-Bekő Máté <
> szalay.beko.m...@gmail.com>
> wrote:
> 
> > yeah, I remember these tickets, thanks for picking them up!
> > I agree and like the solution you proposed, in general in the long
> > term it
> > is good not to use a custom trust manager, but rely on the standard
> > one.
> > 
> > Máté
> > 
> > 
> > On Fri, Jun 9, 2023 at 2:08 PM Enrico Olivelli  > >
> > wrote:
> > 
> > > Il giorno ven 9 giu 2023 alle ore 14:07 Andor Molnar
> > >  ha scritto:
> > > > I'd like to backport this to the 3.8 branch too.
> > > > 
> > > > Let's say I'll add new "zookeeper.fips-mode" parameter which
> > > > will be
> > > > "false" by default in 3.8 and "true" for 3.9.0.
> > > 
> > > I am +1
> > > ZK 3.9 will take time to be adopted and this is an important
> > > security
> > > related topic
> > > 
> > > Enrico
> > > 
> > > > Thoughts?
> > > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> > > > > I think that switching to
> > > > > sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is
> > > > > a good
> > > > > option.
> > > > > The less tweaks we have about Security code the better.
> > > > > 
> > > > > 
> > > > > It would be great to see this in 3.9.0.
> > > > > 
> > > > > Enrico
> > > > > 
> > > > > Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
> > > > >  ha scritto:
> > > > > > Hi zk folks,
> > > > > > 
> > > > > > Problem(s)
> > > > > > ==
> > > > > > 
> > > > > > One problem that we're having with a custom Trust Manager
> > > > > > in ZK is
> > > > > > that
> > > > > > FIPS doesn't allow that:
> > > > > > 
> > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > > > > > 
> > > > > > In FIPS mode the only allowed TrustManager in the JDK is
> > > > > > X509TrustManagerImpl which is the default implementation.
> > > > > > The class
> > > > > > is
> > > > > > final, so extending it is not an option unfortunately.
> > > > > > 
> > > > > > The intention behind implementing a custom trust manager in
> > > > > > ZK was,
> > > > > > I
> > > > > > believe, the need for server and client-side hostname
> > > > > > verification.
> > > > > > Hostname verification officially is not part of the SSL/TLS
> > > > > > protocol,
> > > > > > it's the responsibility of an upper level protocol like
> > > > > > HTTPS.
> > > > > > 
> > > > > > Hacking hostname verification in the SSL handshake is nice
> > > > > > and was
> > > > > > working fine so far, but unfortunately breaks the FIPS
> > > > > > standard.
> > > > > > 
> > > > > > Another annoying issue with ZKTrustManager is the need for
> > > > > > reverse
> > > > > > DNS
> > > > > > lookup. This is usually needed when the hostname of the
> > > > > > certificate
> > > > > > provider is not known at the time of handshake. For
> > > > > > instance, when
> > > > > > somebody connects the client via IP address, which is
> > > > > > generally not
> > > > > > recommended when TLS is active in the client-server
> > > > &

Re: FIPS: removing ZKTrustManager

[Andor Molnar]

I'd like to backport this to the 3.8 branch too.

Let's say I'll add new "zookeeper.fips-mode" parameter which will be
"false" by default in 3.8 and "true" for 3.9.0.

Thoughts?

Andor



On Fri, 2023-06-09 at 13:55 +0200, Enrico Olivelli wrote:
> I think that switching to
> sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); is a good
> option.
> The less tweaks we have about Security code the better.
> 
> 
> It would be great to see this in 3.9.0.
> 
> Enrico
> 
> Il giorno ven 9 giu 2023 alle ore 13:42 Andor Molnar
>  ha scritto:
> > Hi zk folks,
> > 
> > Problem(s)
> > ==
> > 
> > One problem that we're having with a custom Trust Manager in ZK is
> > that
> > FIPS doesn't allow that:
> > 
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4393
> > 
> > In FIPS mode the only allowed TrustManager in the JDK is
> > X509TrustManagerImpl which is the default implementation. The class
> > is
> > final, so extending it is not an option unfortunately.
> > 
> > The intention behind implementing a custom trust manager in ZK was,
> > I
> > believe, the need for server and client-side hostname verification.
> > Hostname verification officially is not part of the SSL/TLS
> > protocol,
> > it's the responsibility of an upper level protocol like HTTPS.
> > 
> > Hacking hostname verification in the SSL handshake is nice and was
> > working fine so far, but unfortunately breaks the FIPS standard.
> > 
> > Another annoying issue with ZKTrustManager is the need for reverse
> > DNS
> > lookup. This is usually needed when the hostname of the certificate
> > provider is not known at the time of handshake. For instance, when
> > somebody connects the client via IP address, which is generally not
> > recommended when TLS is active in the client-server protocol.
> > 
> > The bigger problem I've found is in the leader election: when a
> > peer
> > connects with a smaller id, the node will close the existing
> > connection
> > and opens a new one in the other direction, based on the
> > information
> > received in the InitialMessage from the peer which only contains
> > the IP
> > address, not the hostname. Therefore TrustManager needs to perform
> > reverse DNS lookup.
> > 
> > Tickets about reverse DNS lookup issues:
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3860
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4268
> > 
> > Proposal
> > 
> > 
> > I suggest to remove ZKTrustManager entirely from the codebase and
> > use
> > the built-in, FIPS-Enabled X509TrustManagerImpl instead. It has the
> > downside of losing hostname verification, but we have an option to
> > re-
> > enable it in client-server communication: Netty has built-in
> > support
> > for it, we just need to do
> > 
> > sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
> > 
> > when creating the SSLEngine and that will result in a behaviour
> > very
> > similar to what we provide currently. I can show some examples.
> > 
> > What we will truly lose is the hostname verification option in the
> > Quorum and Leader Election protocols. Since in these protocols we
> > manipulate the sockets directly, we would need to implement the
> > verification manually.
> > 
> > What do you think about this trade-off?
> > 
> > Of course, we can put this change behind a feature flag "fips-
> > mode",
> > which will lead to a new mode in ZooKeeper that is actually less
> > strict
> > as the original behaviour.
> > 
> > Regards,
> > Andor
> > 
> > 
> > 

FIPS: removing ZKTrustManager

[Andor Molnar]

Hi zk folks,

Problem(s)
==

One problem that we're having with a custom Trust Manager in ZK is that
FIPS doesn't allow that:

https://issues.apache.org/jira/browse/ZOOKEEPER-4393

In FIPS mode the only allowed TrustManager in the JDK is
X509TrustManagerImpl which is the default implementation. The class is
final, so extending it is not an option unfortunately.

The intention behind implementing a custom trust manager in ZK was, I
believe, the need for server and client-side hostname verification.
Hostname verification officially is not part of the SSL/TLS protocol,
it's the responsibility of an upper level protocol like HTTPS.

Hacking hostname verification in the SSL handshake is nice and was
working fine so far, but unfortunately breaks the FIPS standard.

Another annoying issue with ZKTrustManager is the need for reverse DNS
lookup. This is usually needed when the hostname of the certificate
provider is not known at the time of handshake. For instance, when
somebody connects the client via IP address, which is generally not
recommended when TLS is active in the client-server protocol. 

The bigger problem I've found is in the leader election: when a peer
connects with a smaller id, the node will close the existing connection
and opens a new one in the other direction, based on the information
received in the InitialMessage from the peer which only contains the IP
address, not the hostname. Therefore TrustManager needs to perform
reverse DNS lookup.

Tickets about reverse DNS lookup issues:
https://issues.apache.org/jira/browse/ZOOKEEPER-3860
https://issues.apache.org/jira/browse/ZOOKEEPER-4268

Proposal


I suggest to remove ZKTrustManager entirely from the codebase and use
the built-in, FIPS-Enabled X509TrustManagerImpl instead. It has the
downside of losing hostname verification, but we have an option to re-
enable it in client-server communication: Netty has built-in support
for it, we just need to do 

sslParameters.setEndpointIdentificationAlgorithm("HTTPS");

when creating the SSLEngine and that will result in a behaviour very
similar to what we provide currently. I can show some examples.

What we will truly lose is the hostname verification option in the
Quorum and Leader Election protocols. Since in these protocols we
manipulate the sockets directly, we would need to implement the
verification manually.

What do you think about this trade-off?

Of course, we can put this change behind a feature flag "fips-mode",
which will lead to a new mode in ZooKeeper that is actually less strict
as the original behaviour.

Regards,
Andor

Re: Volounteers for releases ?

[Andor Molnar]

Hi Enrico,

I can take the master cut next week, but let me put together an email
about a TLS topic first. I'd like to propose a fix to resolve the
problem of FIPS (custome trust manager in ZK) and reverse DNS lookups.
I'd like to include it in 3.9.0 and 3.8.2.

Andor

p.s. Whoever is making a change on the webpage, please remove the 3.8.0
release.



On Fri, 2023-06-09 at 09:11 +0200, Enrico Olivelli wrote:
> Hello ZooKeepers,
> I think that it is time to do a round of releases.
> 
> We should cut a release out of the master branch, 3.9.0 and main
> cutting a release out of 3.7.x and 3.8.x would be useful.
> 
> Before cutting the release please ensure that third party libraries
> are not reported against CVEs
> 
> 
> This is the list of the latest releases
> https://zookeeper.apache.org/releases.html
> 
> Would anyone volunteer ?
> 
> Enrico

Re: [jira] [Created] (ZOOKEEPER-4696) Update for Zookeeper latest version

[Andor Molnar]

Owasp build reported the following:

[ERROR] jackson-core-2.13.4.jar: CVE-2022-45688(7.5)
[ERROR] jetty-io-9.4.49.v20220914.jar: CVE-2023-26048(5.3), CVE-2023-
26049(5.3)
[ERROR] jetty-server-9.4.49.v20220914.jar: CVE-2023-26048(5.3), CVE-
2023-26049(5.3)

Thanks Ben for letting us now. Would you please kindly update the Jira
with the listed CVEs and the affected version (3.8.1)?

We'll check if these CVEs should be fixed on ZooKeeper side and if
needed, you should expect a new release from the 3.8.x branch, since
it's an active release branch.

Andor



On Fri, 2023-05-26 at 08:33 +0200, Andor Molnar wrote:
> Hi Ben,
> 
> Let me check this.
> I triggered an owasp check build on Apache CI:
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/branch-3.8.1/7/
> 
> Btw, Enrico, we're still having both 3.8.0 and 3.8.1 releases on the
> web page as separate release lines. Would you mind if I submit a
> change
> to the webpage to remove 3.8.0?
> 
> Not sure who I talked about it, it was long time ago.
> 
> Regards,
> Andor
> 
> 
> 
> 
> On Thu, 2023-05-18 at 17:54 +, Ben Johnston wrote:
> > > version of zookeeper we are using is 3.8.0
> >  
> > The latest zookeeper release is 3.8.1 (
> > https://github.com/apache/zookeeper/releases/tag/release-3.8.1)
> > that
> > included a number of bugfixes, probably some that are in your list
> > 
> > The 3.8.1 does have a medium and low CVE that are on the jetty
> > server. CVE-2023-26048 and CVE-2023-26049. When might the team do a
> > release to do security fixes?
> >  
> > Thanks,
> >  
> > Ben Johnston, GCIH, GCFA, GPEN
> > Application Security Engineer
> > COFENSE
> > o. 785-250-4412
> > e. ben.johns...@cofense.com
> >  
> > Connect with Cofense:
> >  
> >  
> >  
> >  
> >  
> > From: Dilip anand (Jira) 
> > Date: Tuesday, May 16, 2023 at 11:34 AM
> > To: dev@zookeeper.apache.org 
> > Subject: [jira] [Created] (ZOOKEEPER-4696) Update for Zookeeper
> > latest version
> > 
> > External Email
> > 
> > Dilip anand created ZOOKEEPER-4696:
> > --
> > 
> >  Summary: Update for Zookeeper latest version
> >  Key: ZOOKEEPER-4696
> >  URL: 
> > https://issues.apache.org/jira/browse/ZOOKEEPER-4696
> >  Project: ZooKeeper
> >   Issue Type: Bug
> > Reporter: Dilip anand
> > 
> > 
> > Hi team,
> > 
> >We ran a scan for security vulnerability fixes,we have seen
> > CVE's that are affected for zookeeper and version of zookeeper we
> > are
> > using is 3.8.0 .Here are the CVE's which are affected with
> > zookeeper
> > CVE-2022-32221,CVE-2023-23914,CVE-2023-27533,CVE-2023-27534,CVE-
> > 2022-
> > 22576,CVE-2020-8169,CVE-2020-8285,CVE-2020-8286,CVE-2021-22926,CVE-
> > 2021-22946,CVE-2022-27775,CVE-2022-27781,CVE-2022-27782,CVE-2023-
> > 23916 which do not have any reports in red hat website. we want to
> > know what version of zookeeper will clear these CVEs and when it'll
> > be released?
> > 
> > Regards,
> > Dilip
> > 
> > 
> > 
> > --
> > This message was sent by Atlassian Jira
> > (v8.20.10#820010)

Re: [jira] [Created] (ZOOKEEPER-4696) Update for Zookeeper latest version

[Andor Molnar]

Hi Ben,

Let me check this.
I triggered an owasp check build on Apache CI:
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/branch-3.8.1/7/

Btw, Enrico, we're still having both 3.8.0 and 3.8.1 releases on the
web page as separate release lines. Would you mind if I submit a change
to the webpage to remove 3.8.0?

Not sure who I talked about it, it was long time ago.

Regards,
Andor




On Thu, 2023-05-18 at 17:54 +, Ben Johnston wrote:
> > version of zookeeper we are using is 3.8.0
>  
> The latest zookeeper release is 3.8.1 (
> https://github.com/apache/zookeeper/releases/tag/release-3.8.1) that
> included a number of bugfixes, probably some that are in your list
> 
> The 3.8.1 does have a medium and low CVE that are on the jetty
> server. CVE-2023-26048 and CVE-2023-26049. When might the team do a
> release to do security fixes?
>  
> Thanks,
>  
> Ben Johnston, GCIH, GCFA, GPEN
> Application Security Engineer
> COFENSE
> o. 785-250-4412
> e. ben.johns...@cofense.com
>  
> Connect with Cofense:
>  
>  
>  
>  
>  
> From: Dilip anand (Jira) 
> Date: Tuesday, May 16, 2023 at 11:34 AM
> To: dev@zookeeper.apache.org 
> Subject: [jira] [Created] (ZOOKEEPER-4696) Update for Zookeeper
> latest version
> 
> External Email
> 
> Dilip anand created ZOOKEEPER-4696:
> --
> 
>  Summary: Update for Zookeeper latest version
>  Key: ZOOKEEPER-4696
>  URL: 
> https://issues.apache.org/jira/browse/ZOOKEEPER-4696
>  Project: ZooKeeper
>   Issue Type: Bug
> Reporter: Dilip anand
> 
> 
> Hi team,
> 
>We ran a scan for security vulnerability fixes,we have seen
> CVE's that are affected for zookeeper and version of zookeeper we are
> using is 3.8.0 .Here are the CVE's which are affected with zookeeper
> CVE-2022-32221,CVE-2023-23914,CVE-2023-27533,CVE-2023-27534,CVE-2022-
> 22576,CVE-2020-8169,CVE-2020-8285,CVE-2020-8286,CVE-2021-22926,CVE-
> 2021-22946,CVE-2022-27775,CVE-2022-27781,CVE-2022-27782,CVE-2023-
> 23916 which do not have any reports in red hat website. we want to
> know what version of zookeeper will clear these CVEs and when it'll
> be released?
> 
> Regards,
> Dilip
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.20.10#820010)

[jira] [Created] (ZOOKEEPER-4683) CVE-2022-45688

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4683:
---

 Summary: CVE-2022-45688
 Key: ZOOKEEPER-4683
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4683
 Project: ZooKeeper
  Issue Type: Bug
  Components: security
Affects Versions: 3.8.1, 3.7.1, 3.6.2
Reporter: Andor Molnar


Latest OWASP checks fails on master:

[https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-owasp/job/master/246/]

Please check the impact on ZooKeeper and apply the necessary fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] Apache ZooKeeper 3.8.1 released

[Andor Molnar]

Hi Enrico,

Thanks for the release, nice work.
Looks like 3.8.0 downloads and documentation have been kept on the
website. Is that intentional?

Regards,
Andor



On Mon, 2023-01-30 at 08:55 +0100, Enrico Olivelli wrote:
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper
> version 3.8.1
> 
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You
> can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
> 
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
> 
> ZooKeeper 3.8.1 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.8.1/releasenotes.html
> 
> We would like to thank the contributors that made the release
> possible.
> 
> Regards,
> 
> The ZooKeeper Team

Re: CI and Pull request validation - too many CI servers ?

[Andor Molnar]

People was excited to try out new technologies. :)
ASF Jenkins is the "official" one, if we can say that.

Andor




On Thu, 2022-09-29 at 15:38 +0200, Enrico Olivelli wrote:
> Hello ZooKeepers,
> Currently we are testing our PRs using:
> - GitHub actions
> - Travis
> - ASF Jenkins
> 
> Honestly I can't remember why we have so many different technologies.
> 
> Does anyone remember ?
> 
> I think that GH is the most comprehensive suite.
> Any volunteer for tidying up this situation ?
> 
> Enrico

New releases page: endoflife.date

[Andor Molnar]

Hi ZK folks,

I'm lettig you know that I've added ZooKeeper to this page:

https://endoflife.date/zookeeper

It's pretty neat to track releases there. We don't need to manually
update it, because it monitors the tags on GitHub mirror page, so it
should be all automatic.

Hope you like it.

Andor

Re: [VOTE] Apache ZooKeeper release 3.5.10 candidate 1

[Andor Molnar]

+1 (binding)

- verified signatures, checksums,
- opened some web pages,
- rat, spotbugs, checkstyle clean,
- build successful with unit tests on Ubuntu 22.04,
- verified local 3-node cluster w/ and w/o SSL,
- logging looks good, TRACE logging works.

Thanks,
Andor



On Sun, 2022-05-29 at 19:08 +0200, Szalay-Bekő Máté wrote:
> This is a bugfix release candidate for 3.5.10. It fixes 44 issues,
> including CVE fixes,
> log4j1 removal (using reload4j from now) and various other bug fixes
> (thread leaks, data
> corruption, snapshotting and SASL related fixes).
> 
> Please note, we announced 3.5 to be EOL from June 1st 2022, so most
> likely
> this will be our
> last 3.5 release.
> 
> The full release notes is available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349434
> 
> *** Please download, test and vote by June 3rd 2022, 23:59 UTC+0. ***
> 
> 
> Source files:
> https://people.apache.org/~symat/zookeeper-3.5.10-rc1/
> 
> Maven staging repo:
> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.10/
> 
> The release candidate tag in git to be voted upon: release-3.5.10-rc1
> 
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
> https://www.apache.org/dist/zookeeper/KEYS
> 
> The staging version of the website is:
> https://people.apache.org/~symat/zookeeper-3.5.10-rc1/website/
> 
> 
> Should we release this candidate?
> 
> 
> Best regards,
> Máté

Re: [ANNOUNCE] new ZooKeeper PMC member: Mate Szalay-Beko

[Andor Molnar]

Congratulations Mate!
Well deserved!

Andor



> On 2022. Mar 28., at 8:42, Enrico Olivelli  wrote:
> 
> I am happy to announce that Mate Szalay-Beko has been invited to join
> the Apache ZooKeeper PMC and he accepted.
> 
> Mate is doing great work for our community.
> 
> Please join me in congratulating with him
> 
> Congrats Mate !
> 
> 
> If you want to know more about the ASF works and what is a PMC you can
> read more here
> https://www.apache.org/foundation/how-it-works.html#pmc
> 
> Enrico

[ANNOUNCE] Apache ZooKeeper 3.5 End-of-Life 1st June, 2022

[Andor Molnar]

Hi,

The Apache ZooKeeper community would like to make the official announcement of
3.5 release line End-of-Life. It will be effective on 1st of June, 2022 00:01 AM
(PDT). From that day forward the 3.5 version of Apache ZooKeeper won’t be
supported by the community which means we won’t 

- accept patches on the 3.5.x branch,
- run automated tests on any JDK version,
- create new releases from 3.5.x branch,
- resolve security issues, CVEs or critical bugs.

Latest released version of Apache ZooKeeper 3.5 (currently 3.5.9) will be
available on the download page for another year (until 1st of June, 2023), after
that it will be accessible among other historical versions from Apache Archives.

=== Upgrade ===

We recommend users of Apache ZooKeeper 3.5 to plan your production upgrades
according to the following supported upgrade path:

1) Upgrade to latest 3.5.x version 
2) Upgrade to latest 3.6.x version
3) (Optional) Upgrade to latest 3.7.x version.

Please find known upgrade issues and workarounds on the following wiki page:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ

In addition to that the user@ mailing list is open 24/7 to help and answer your
questions as usual.

=== Compatibility ===

Our backward compatibility rules still apply and can be found here:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement

Following the recommended upgrade path with rolling upgrade process ZooKeeper
quorum will be available at all times as long as clients are not starting to use
new features.

Best Regards,

-Andor

Re: [VOTE] Apache ZooKeeper release 3.8.0 candidate 1

[Andor Molnar]

+1 (binding)

- checksum / signatures verified
- compiled the full build on Mac,
- unit test run on Java 11 - I had a few tests which was constantly failing on 
my Mac, but given that CI already passed all tests and others reported the 
same, I take it as passed,
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-multi-branch-build/job/branch-3.8.0/34/
- rat run clean
- spotbugs, owasp checks passed
- 3-node TLS quorum up and running with some basic manual tests

Thanks,
Andor




> On 2022. Feb 28., at 22:02, Patrick Hunt  wrote:
> 
> +1 - xsum/sig verified. Rat ran clean, compiled fine and I was able to run
> some manual clusters successfully.
> 
> Regards,
> 
> Patrick
> 
> On Fri, Feb 25, 2022 at 2:32 AM Enrico Olivelli  wrote:
> 
>> This is the second release candidate for 3.8.0.
>> 
>> It is a major release and it introduces a lot of new features, most
>> notably:
>> - Migration of the logging framework from Apache Log4j1 to LogBack
>> - Read Key/trust store password from file (and other security related
>> improvements)
>> - Restored support for OSGI
>> - Reduced the performance impact of Prometheus metrics
>> - Official support for JDK17 (all tests are passing)
>> - Updates to all the third party dependencies to get rid of every known
>> CVE.
>> 
>> The full release notes is available at:
>> 
>> 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12349587
>> 
>> *** Please download, test and vote by February 28th 2022, 23:59 UTC+0. ***
>> 
>> Source files:
>> 
>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/
>> 
>> Maven staging repo:
>> https://repository.apache.org/content/repositories/orgapachezookeeper-1073/
>> 
>> The release candidate tag in git to be voted upon: release-3.8.0-1
>> https://github.com/apache/zookeeper/tree/release-3.8.0-1
>> 
>> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>> https://www.apache.org/dist/zookeeper/KEYS
>> 
>> The staging version of the website is:
>> 
>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.8.0-candidate-1/website/index.html
>> 
>> 
>> Should we release this candidate?
>> Enrico Olivelli
>> 

Re: Moving 3.5 to EOL

[Andor Molnar]

Hi folks,

I’ve created a pull request for the website change:

https://github.com/apache/zookeeper/pull/1834

Once the website is updated, I’ll make the announcement for 3.5 EoL.

Thanks,
Andor




> On 2022. Feb 17., at 16:54, Szalay-Bekő Máté  
> wrote:
> 
> Thanks for the clarification, I like the plan!
> 
>> having 2 active versions (stable and current) and when a new minor
> version is announced, the least recent will get another 6 months of support
> 
> What does this mean exactly? Just to be on the same page, this is what you
> propose if we release 3.8.0 until let's say end of February 2022?
> - 3.5 EoL 1st of June 2022
> - 3.6 EoL 1st of Sept 2022 (~6 months after 3.8.0 release)
> - 3.7 will become "stable"
> - 3.8 will become "current"
> 
> Did anyone in the community test the latest 3.7 (which is still 3.7.0) with
> large clusters in production? Are we confident saying 3.7 is stable?
> (on the other hand, if we don't do the announcement, most likely people
> won't start to migrate to 3.7)
> 
> Mate
> 
> On Wed, Feb 16, 2022 at 1:33 PM Enrico Olivelli  wrote:
> 
>> Andor,
>> 
>> Il Mer 16 Feb 2022, 12:47 Andor Molnar  ha scritto:
>> 
>>> Okay, I agree that keeping 2 active versions rather than tying ourselves
>>> to some fixed deadlines makes more sense for ZooKeeper. Let’s go with
>> this
>>> approach then if there’s no other objections:
>>> 
>>> 1) Add this information to the Releases web page: I’ll describe that
>>> ZooKeeper is having 2 active versions (stable and current) and when a new
>>> minor version is announced, the least recent will get another 6 months of
>>> support (security and bugfixes), but after that it will become EoL. That
>>> means no further releases are expected from the community and users
>> should
>>> follow the supported upgrade path. I’ll send this out for review soon.
>>> 
>> 
>> +1
>> 
>> 
>>> 2) Announce 3.5 EoL 1st of June 2022. (sorry Enrico, the end of the long
>>> discussion is essentially what you originally proposed)
>>> 
>> 
>> +1
>> Thanks
>> 
>> 
>> Enrico
>> 
>> 
>> 
>>> Please let me know if you have concerns with this path.
>>> 
>>> Andor
>>> 
>>> 
>>> 
>>>> On 2022. Feb 14., at 17:07, Patrick Hunt  wrote:
>>>> 
>>>> "Define what EOL means" - whatever we do let's make sure it gets onto
>> the
>>>> "releases" page so that folks have official information they can
>>> reference
>>>> from the project.
>>>> 
>>>> I like having a max of 2 versions. Stable and current. I agree that due
>>> to
>>>> our lack of communication/policy so far we should ensure that people
>> have
>>>> opportunity to move/support on the release versions (3.x minors) we
>>> current
>>>> support.
>>>> 
>>>> I like the idea of tying old releases to new ones. I don't think tying
>>>> ourselves to a specific, long term is good though. It definitely
>> reduces
>>>> flexibility. Same with saying that new minors are going to be released
>>>> every Y time. Can't we just say that a stable release will be supported
>>> for
>>>> a minimum of 6 months (other timeframe?) after moving the stable
>>> indicator
>>>> from 3.x to 3.x+1. We then have the flexibility to keep it around
>> longer
>>> if
>>>> there is a reason why folks want to stick for a longer time (eg major
>>>> changes in the more recent versions)
>>>> 
>>>> Patrick
>>>> 
>>>> On Fri, Feb 11, 2022 at 8:08 AM Christopher 
>> wrote:
>>>> 
>>>>> Regarding the suggestion: "Maybe we can also communicate that we’re
>>> going
>>>>> to officially EoL the least recent ZK version every 2 years." If you
>>>>> release new versions less frequently than that, the number of
>>> maintenance
>>>>> versions will go to 0 (though, in practice, you wouldn't EOL your
>>> current
>>>>> release). If you release more frequently, you'll be stuck maintaining
>> an
>>>>> increasing number of versions.
>>>>> 
>>>>> To keep the maintenance burden relatively consistent, I suggest tying
>>> your
>>>>> EOL schedule to your release schedule, so when you release a new
>>> version,
>>>>>

Re: Moving 3.5 to EOL

[Andor Molnar]

Okay, I agree that keeping 2 active versions rather than tying ourselves to 
some fixed deadlines makes more sense for ZooKeeper. Let’s go with this 
approach then if there’s no other objections:

1) Add this information to the Releases web page: I’ll describe that ZooKeeper 
is having 2 active versions (stable and current) and when a new minor version 
is announced, the least recent will get another 6 months of support (security 
and bugfixes), but after that it will become EoL. That means no further 
releases are expected from the community and users should follow the supported 
upgrade path. I’ll send this out for review soon.

2) Announce 3.5 EoL 1st of June 2022. (sorry Enrico, the end of the long 
discussion is essentially what you originally proposed)

Please let me know if you have concerns with this path.

Andor



> On 2022. Feb 14., at 17:07, Patrick Hunt  wrote:
> 
> "Define what EOL means" - whatever we do let's make sure it gets onto the
> "releases" page so that folks have official information they can reference
> from the project.
> 
> I like having a max of 2 versions. Stable and current. I agree that due to
> our lack of communication/policy so far we should ensure that people have
> opportunity to move/support on the release versions (3.x minors) we current
> support.
> 
> I like the idea of tying old releases to new ones. I don't think tying
> ourselves to a specific, long term is good though. It definitely reduces
> flexibility. Same with saying that new minors are going to be released
> every Y time. Can't we just say that a stable release will be supported for
> a minimum of 6 months (other timeframe?) after moving the stable indicator
> from 3.x to 3.x+1. We then have the flexibility to keep it around longer if
> there is a reason why folks want to stick for a longer time (eg major
> changes in the more recent versions)
> 
> Patrick
> 
> On Fri, Feb 11, 2022 at 8:08 AM Christopher  wrote:
> 
>> Regarding the suggestion: "Maybe we can also communicate that we’re going
>> to officially EoL the least recent ZK version every 2 years." If you
>> release new versions less frequently than that, the number of maintenance
>> versions will go to 0 (though, in practice, you wouldn't EOL your current
>> release). If you release more frequently, you'll be stuck maintaining an
>> increasing number of versions.
>> 
>> To keep the maintenance burden relatively consistent, I suggest tying your
>> EOL schedule to your release schedule, so when you release a new version,
>> you drop the oldest one. If you release every 2 years, then it works out
>> the same. But if you release more or less often, your maintenance burden
>> stays consistent.
>> 
>> I would start by deciding the minimum number of concurrent versions you
>> want to maintain. I suggest no more than 2, but ZK currently has 3, and is
>> about to be 4 soon. If you're not marking specific versions as long-term
>> stable, then the default would be to assume you're maintaining the most
>> recent versions.
>> 
>> Then, consider churn. If you release frequently, you may want to set a
>> minimum age for maintenance, so users aren't forced to upgrade too often.
>> So, if you start with 2 concurrent versions and you have a few versions
>> released rapidly, you may temporarily need to support up to 3 or 4 releases
>> until the oldest ones reach the minimum age, like 2 years for example, and
>> are able to be EOL'd.
>> 
>> Then, consider upgrade overlap. When you release, you could EOL the oldest
>> version right away. But, it might be nicer to wait a few months, or maybe
>> up to a year, before the oldest one is EOL'd.
>> 
>> I previously mentioned Accumulo's "LTM" strategy. These are the core
>> considerations we had in mind. So, for example, we support a minimum of 1
>> LTM version, with a 1 year overlap. We don't release frequently enough for
>> the minimum age to be of concern. However, we did want to allow for
>> intermediate feature preview releases that are immediately EOL as soon as a
>> newer version is available. So, at any given time, we are maintaining
>> between 1 and 2 LTM releases, and no more than 1 non-LTM release. We also
>> use this to provide users with information about supported upgrade paths so
>> users can upgrade from LTM to LTM, skipping over non-LTM releases, or they
>> can stay on the latest (whether or not it is LTM).
>> 
>> For ZooKeeper, I would suggest:
>> * maintain at least 2 versions (currently 3.6 and 3.7)
>> * maintain for at least 3 years before EOL
>> 
>> 
>> On Fri, Feb 11, 2022 at 9:10 AM Andor Molnar  

Re: Moving 3.5 to EOL

[Andor Molnar]

Thanks for the pointers. It was good to help refreshing my memory.

We definitely missed the communication when stable and current links were 
flipped from one version to another. Things will get more interesting when 
Enrico finally releases 3.8.0. We’ll end up having 3 different “stable" 
branches and 3.8 will become the “current”.

What can we do with this?

Announcing 3.5 EoL
~~

This should have been done before flipping the stable pointer, but anyway, 
here’re the points that we considered when doing the same for 3.4:
- Discussion happened in March/April 2020, EoL was announced for 1st of June, 
2020 (3 months ahead).

- Define what EOL means - This is already discussed, text can be copy-pasted 
from 3.4 EoL message,

- Provide guidelines for upgrading paths,

- State interoperability guarantees 
   - Previous version of ZooKeeper client is able to connect to server as long
as there’s no new feature enforced on server side,
   - Previous version of ZooKeeper server is able to accept connections from
clients as long as they don’t want to use new features.

- Curator already supports later versions - Is it true for 3.6, 3.7?

It’s February now, so if we nail down the above points, I don’t see any 
objections against announcing 3.5 EoL for 1st of June, 2022 (2 years after 3.4 
EoL, providing 4 months to upgrade). Maybe we can also communicate that we’re 
going to officially EoL the least recent ZK version every 2 years.

Andor




> On 2022. Feb 9., at 20:28, Patrick Hunt  wrote:
> 
> On Wed, Feb 9, 2022 at 3:07 AM Andor Molnar  wrote:
> 
>> Hi Pat,
>> 
>> Yeah, I asked for a more specific suggestion from you. If we avoid using
>> the LTS in ZooKeeper releases and stay with the stable/latest labels, how
>> would you label the current maintained versions?
>> 
> 
> Ah, ok. No worries Andor, I misunderstood. My 0.02:
> 
> We have "stable" and "current" already identified.
> https://dlcdn.apache.org/zookeeper/
> Stable was last updated in April of 2021. My recommendation is that we
> should change the process to notify EOL prior to updating e.g. "stable"
> reference. Stable is our indication w/o using the LTS label. As long as we
> have a public policy & associated announcements, I think that's fine.
> 
> I also bring your attention to this conversation thread from March 2020 for
> the previous EOL'd (3.4) release line:
> https://markmail.org/message/b2pqcztlb2ixoyjp
> Some good ideas in there from many folks, I think we settled on a timeframe
> we felt comfortable with, at least at the time. Unfortunately we did not
> follow through with a plan for future releases. Perhaps we can do that now.
> 
> Regards,
> 
> Patrick
> 
> 
>> 
>> Enrico is about to release 3.8.0 soon, so we’ll end up having four
>> versions in maintenance. What should we do with it to reduce the
>> maintenance cost?
>> 
>> Andor
>> 
>> 
>> 
>> 
>>> On 2022. Feb 4., at 17:58, Patrick Hunt  wrote:
>>> 
>>> On Fri, Feb 4, 2022 at 8:19 AM Andor Molnar  wrote:
>>> 
>>>> More specifically?
>>>> 
>>> 
>>> Are you asking me? :-)  "LTS" literally has a definition in wikipedia:
>>> https://en.wikipedia.org/wiki/Long-term_support
>>> 
>>> 
>>>> 
>>>> Stable 3.5, 3.6, 3.7, 3.8 and EoL 3.5 at the end of the year (1st of
>> Jan,
>>>> 2023)?
>>>> 
>>>> Andor
>>>> 
>>>> 
>>>> 
>>>>> On 2022. Feb 1., at 16:41, Patrick Hunt  wrote:
>>>>> 
>>>>> "LTS" typically has meaning for folks beyond just what the words say.
>> JDK
>>>>> LTS. Ubuntu LTS. etc... I think it would be less confusing to stay with
>>>> the
>>>>> stable/latest labels we have had in the past and plan ahead a bit in
>>>> terms
>>>>> of giving notice when releases will be removed from support.
>>>>> 
>>>>> Patrick
>>>>> 
>>>>> On Tue, Feb 1, 2022 at 3:12 AM Andor Molnar  wrote:
>>>>> 
>>>>>> Hi Andrew,
>>>>>> 
>>>>>> I think that wasn’t a general plan from the community at that time,
>> just
>>>>>> my opinion based on how long 3.4 was the stable release of ZooKeeper
>> (4
>>>>>> years). Since then the release schedule has become much faster and to
>> be
>>>>>> honest I’m not participating in it.
>>>>>> 
>>>>>> As mentioned 3.6 and 3.7 releases are not much different. 3.6 is the
>>>>&

Re: [VOTE] Apache ZooKeeper release 3.8.0 candidate 0

[Andor Molnar]

I agree with Pat. Though adding exclusions doesn’t make any difference in the 
quality of our code, but a build is a build. It’s either green or red (not 
green). No excuse.

Andor



> On 2022. Feb 10., at 16:51, Patrick Hunt  wrote:
> 
> On Thu, Feb 10, 2022 at 12:22 AM Enrico Olivelli 
> wrote:
> 
>> Patrick,
>> If you prefer I can send a patch for. the exclusion of
>> [ERROR] netty-tcnative-2.0.48.Final.jar: CVE-2021-43797, CVE-2019-16869,
>> CVE-2015-2156, CVE-2021-37136, CVE-2014-3488, CVE-2021-37137,
>> CVE-2019-20445, CVE-2019-20444, CVE-2021-21295, CVE-2021-21409,
>> CVE-2021-21290
>> 
>> That said, this won't affect the goodness of the RC.
>> 
>> Our code is safe and the dependencies we use are safe:
>> - to me it looks like those are false positive or at least not related
>> to ZooKeeper
>> - we are not using Netty TC Native features, it is a dependency we
>> inherit, and probably ZooKeeper works well without it
>> 
>> Thank you all of taking time to test the release
>> 
>> 
> NP. My concern is highlighted by this (your) response. You had to say all
> this to explain why the build is failing on a simple security check.
> Post-log4shell folks are really sensitive to security issues, as they
> should be, as we all should be. Its very important that we take security
> seriously. If I download the release, and run the owasp check it fails. I
> then have questions in my mind why. All that you explained here, while
> perfectly reasonable, it won't be available to me at that point. I think
> rather we should ensure that releases are solid/clean before we push them.
> This is a simple thing to fix before we go through the entire process of
> verifying/releasing a new version.
> 
> Hopefully this explains my concerns.
> 
> Regards,
> 
> Patrick
> 
> 
>> Enrico
>> 
>> Il giorno gio 10 feb 2022 alle ore 09:13 Szalay-Bekő Máté
>>  ha scritto:
>>> 
>>> Thanks Enrico for working on the release candidate!
>>> 
>>> The RC looks good to me if we are sure that the OWASP problem is a false
>>> positive and we can skip this netty-tcnative jar check. However, these
>> CVEs
>>> are old... Is it possible that we just added this jar by accident with
>> the
>>> recent netty upgrade? If we don't need it, should we exclude it?
>>> 
>>> I wouldn't vote with +1 until we clarify the state of these CVEs.
>>> 
>>> My RC check:
>>> 
>>> - apache-rat passed
>>> - I built the source code (-Pfull-build) on dockerized Ubuntu 18.04.6
>> using
>>> OpenJDK 11.0.13 and maven 3.6.0.
>>> - all the java unit tests passed eventually. I had 4-8 tests failing in
>>> each run, but after 4 runs all tests passed at least once. (I used
>>> -Dsurefire-forkcount=1) We should somehow fix these flakies. There are
>>> flakies on the CI, but not this many. I executed in docker, maybe this is
>>> the reason or the CI is using a different java version?
>>> - checkstyle and spotbugs passed
>>> - OWASP (CVE check) failed with the mentioned
>>> netty-tcnative-2.0.48.Final.jar failures.
>>> - I built the fatjar
>>> - I executed C client tests. Two of these failed constantly for me:
>>> Zookeeper_simpleSystem::testIPV6 and
>>> Zookeeper_SASLAuth::testClientSASLOverIPv6. (I think these fail for me
>>> because I execute C unit tests on docker, there might be some issues with
>>> the IPv6 interface) I see these passed on CI running on the
>> branch-3.8.0. (
>>> 
>> https://github.com/apache/zookeeper/runs/5048875668?check_suite_focus=true
>> )
>>> - I also built and executed unit tests for zkpython
>>> - I executed quick rolling-upgrade tests (using
>>> https://github.com/symat/zk-rolling-upgrade-test):
>>>  - rolling upgrade from 3.5.9 to 3.8.0
>>>  - rolling upgrade from 3.6.3 to 3.8.0
>>>  - rolling upgrade from 3.7.0 to 3.8.0
>>> - The web page looks OK
>>> 
>>> Best regards,
>>> Máté
>>> 
>>> On Wed, Feb 9, 2022 at 8:04 PM Chris Nauroth 
>> wrote:
>>> 
 Enrico, thank you for putting together a release candidate.
 
 I briefly looked at the OWASP check failure. It's flagging multiple old
 CVEs against netty-tcnative-2.0.48.Final.jar. I can't imagine how
>> these are
 still applicable. This is the newest version of the dependency, so we
>> don't
 have another upgrade path we can try.
 
 I don't understand it. Unfortunately, I haven't found a solution yet.
 
 Chris Nauroth
 
 
 On Wed, Feb 9, 2022 at 2:05 AM Szalay-Bekő Máté <
 szalay.beko.m...@gmail.com>
 wrote:
 
> I started to test it. apache-rat passed for me, but owasp first
>> failed
 due
> to some environment issue:
> 
> [ERROR] Failed to execute goal
 org.owasp:dependency-check-maven:5.3.0:check
> (default-cli) on project parent: Fatal exception(s) analyzing Apache
> ZooKeeper: One or more exceptions occurred during analysis:
> [ERROR] Unable to download meta file:
> https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2004.meta
> [ERROR] No documents exist
> [ERROR] -> [Help 1]
> 
> 

Re: Moving 3.5 to EOL

[Andor Molnar]

Hi Pat,

Yeah, I asked for a more specific suggestion from you. If we avoid using the 
LTS in ZooKeeper releases and stay with the stable/latest labels, how would you 
label the current maintained versions?

Enrico is about to release 3.8.0 soon, so we’ll end up having four versions in 
maintenance. What should we do with it to reduce the maintenance cost?

Andor




> On 2022. Feb 4., at 17:58, Patrick Hunt  wrote:
> 
> On Fri, Feb 4, 2022 at 8:19 AM Andor Molnar  wrote:
> 
>> More specifically?
>> 
> 
> Are you asking me? :-)  "LTS" literally has a definition in wikipedia:
> https://en.wikipedia.org/wiki/Long-term_support
> 
> 
>> 
>> Stable 3.5, 3.6, 3.7, 3.8 and EoL 3.5 at the end of the year (1st of Jan,
>> 2023)?
>> 
>> Andor
>> 
>> 
>> 
>>> On 2022. Feb 1., at 16:41, Patrick Hunt  wrote:
>>> 
>>> "LTS" typically has meaning for folks beyond just what the words say. JDK
>>> LTS. Ubuntu LTS. etc... I think it would be less confusing to stay with
>> the
>>> stable/latest labels we have had in the past and plan ahead a bit in
>> terms
>>> of giving notice when releases will be removed from support.
>>> 
>>> Patrick
>>> 
>>> On Tue, Feb 1, 2022 at 3:12 AM Andor Molnar  wrote:
>>> 
>>>> Hi Andrew,
>>>> 
>>>> I think that wasn’t a general plan from the community at that time, just
>>>> my opinion based on how long 3.4 was the stable release of ZooKeeper (4
>>>> years). Since then the release schedule has become much faster and to be
>>>> honest I’m not participating in it.
>>>> 
>>>> As mentioned 3.6 and 3.7 releases are not much different. 3.6 is the
>>>> “Facebook” version which is well tested and contains lots of patches
>> that
>>>> improves robustness. Both versions are good candidates for upgrade, so
>>>> announcing 3.5 EoL (at least half year from now) is not necessarily bad.
>>>> 
>>>> As an alternative, staying with the LT(S|M) / non-LT(S|M) terms, I think
>>>> the following could also be considered for the community:
>>>> 
>>>> Now:
>>>> 
>>>> master
>>>> --
>>>> 3.7
>>>> 3.6
>>>> 3.5 LTS
>>>> --
>>>> 3.4 EoL
>>>> 
>>>> Can become:
>>>> 
>>>> master
>>>> --
>>>> 3.8 LTS
>>>> 3.7
>>>> 3.5 LTS
>>>> --
>>>> 3.6 EoL
>>>> 3.4 EoL
>>>> 
>>>> In order to keep the number of maintained branches low.
>>>> 
>>>> What do you think?
>>>> 
>>>> Andor
>>>> 
>>>> 
>>>> 
>>>>> On 2022. Jan 31., at 19:41, Andrew Purtell 
>> wrote:
>>>>> 
>>>>> Just to be clear I meant 'you' as the ZooKeeper project as a whole, but
>>>>> maybe I have misunderstood this response:
>>>>> 
>>>> 
>> https://issues.apache.org/jira/browse/HADOOP-17612?focusedCommentId=17311792=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17311792
>>>>> 
>>>>> 
>>>>> On Sun, Jan 30, 2022 at 10:29 AM Enrico Olivelli 
>>>>> wrote:
>>>>> 
>>>>>> Il Dom 30 Gen 2022, 17:51 Andrew Purtell 
>> ha
>>>>>> scritto:
>>>>>> 
>>>>>>> Previously in various contexts - specifically, I am thinking of a
>>>> Hadoop
>>>>>>> JIRA where we once had a conversation on this topic, but I believe
>>>> there
>>>>>>> have been others - you have declared 3.5 a long term stable (LTS)
>>>>>> release.
>>>>>>> 
>>>>>>> A sudden EOL of an LTS is jarring and makes future promise of LTS
>>>>>>> untrustworthy. What I would recommend for what it’s worth is a
>>>> timetable
>>>>>> to
>>>>>>> EOL of 3.5 that is reasonably long, like one or two years, should you
>>>>>>> decide to EOL it.
>>>>>> 
>>>>>> 
>>>>>> I am sorry,
>>>>>> I forgot about such conversation.
>>>>>> 
>>>>>> Can you share some pointers ?
>>>>>> 
>>>>>> No problem from my side as soon as there is someone who needs 3.5 an

Re: Moving 3.5 to EOL

[Andor Molnar]

More specifically?

Stable 3.5, 3.6, 3.7, 3.8 and EoL 3.5 at the end of the year (1st of Jan, 2023)?

Andor



> On 2022. Feb 1., at 16:41, Patrick Hunt  wrote:
> 
> "LTS" typically has meaning for folks beyond just what the words say. JDK
> LTS. Ubuntu LTS. etc... I think it would be less confusing to stay with the
> stable/latest labels we have had in the past and plan ahead a bit in terms
> of giving notice when releases will be removed from support.
> 
> Patrick
> 
> On Tue, Feb 1, 2022 at 3:12 AM Andor Molnar  wrote:
> 
>> Hi Andrew,
>> 
>> I think that wasn’t a general plan from the community at that time, just
>> my opinion based on how long 3.4 was the stable release of ZooKeeper (4
>> years). Since then the release schedule has become much faster and to be
>> honest I’m not participating in it.
>> 
>> As mentioned 3.6 and 3.7 releases are not much different. 3.6 is the
>> “Facebook” version which is well tested and contains lots of patches that
>> improves robustness. Both versions are good candidates for upgrade, so
>> announcing 3.5 EoL (at least half year from now) is not necessarily bad.
>> 
>> As an alternative, staying with the LT(S|M) / non-LT(S|M) terms, I think
>> the following could also be considered for the community:
>> 
>> Now:
>> 
>> master
>> --
>> 3.7
>> 3.6
>> 3.5 LTS
>> --
>> 3.4 EoL
>> 
>> Can become:
>> 
>> master
>> --
>> 3.8 LTS
>> 3.7
>> 3.5 LTS
>> --
>> 3.6 EoL
>> 3.4 EoL
>> 
>> In order to keep the number of maintained branches low.
>> 
>> What do you think?
>> 
>> Andor
>> 
>> 
>> 
>>> On 2022. Jan 31., at 19:41, Andrew Purtell  wrote:
>>> 
>>> Just to be clear I meant 'you' as the ZooKeeper project as a whole, but
>>> maybe I have misunderstood this response:
>>> 
>> https://issues.apache.org/jira/browse/HADOOP-17612?focusedCommentId=17311792=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17311792
>>> 
>>> 
>>> On Sun, Jan 30, 2022 at 10:29 AM Enrico Olivelli 
>>> wrote:
>>> 
>>>> Il Dom 30 Gen 2022, 17:51 Andrew Purtell  ha
>>>> scritto:
>>>> 
>>>>> Previously in various contexts - specifically, I am thinking of a
>> Hadoop
>>>>> JIRA where we once had a conversation on this topic, but I believe
>> there
>>>>> have been others - you have declared 3.5 a long term stable (LTS)
>>>> release.
>>>>> 
>>>>> A sudden EOL of an LTS is jarring and makes future promise of LTS
>>>>> untrustworthy. What I would recommend for what it’s worth is a
>> timetable
>>>> to
>>>>> EOL of 3.5 that is reasonably long, like one or two years, should you
>>>>> decide to EOL it.
>>>> 
>>>> 
>>>> I am sorry,
>>>> I forgot about such conversation.
>>>> 
>>>> Can you share some pointers ?
>>>> 
>>>> No problem from my side as soon as there is someone who needs 3.5 and
>> that
>>>> is willing to help.
>>>> 
>>>> Our codebase is pretty stable and we usually pay much attention  to
>>>> compatibility. So I am sure that 3.5 clients will be able to connect to
>> new
>>>> servers (and vice versa)
>>>> 
>>>> I opened up this discussion to see how much interest is in the
>> community,
>>>> so from your response I understand that there is such interest.
>>>> 
>>>> Thanks for answering
>>>> 
>>>> Enrico
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>>> On Jan 30, 2022, at 5:00 AM, Enrico Olivelli 
>>>>> wrote:
>>>>>> 
>>>>>> Hello,
>>>>>> We are going to release 3.8.0.
>>>>>> It is time to think about moving 3.5 to EOL.
>>>>>> 
>>>>>> Key points:
>>>>>> - we already have a few other "active" branches, 3.6 and 3.7
>>>>>> - 3.5 still has "ant" files, and cherry picking libraries upgrade is
>>>>>> awkward  (you always have to create a separate patch)
>>>>>> - moving to 3.6 is quite easy, so people should not be stuck if
>>>>>> requested to upgrade to 3.6
>>>>>> 
>>>>>> Thoughts ?
>>>>>> 
>>>>>> 
>>>>>> Enrico
>>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> Best regards,
>>> Andrew
>>> 
>>> Unrest, ignorance distilled, nihilistic imbeciles -
>>>   It's what we’ve earned
>>> Welcome, apocalypse, what’s taken you so long?
>>> Bring us the fitting end that we’ve been counting on
>>>  - A23, Welcome, Apocalypse
>> 
>> 

Re: Moving 3.5 to EOL

[Andor Molnar]

Hi Andrew,

I think that wasn’t a general plan from the community at that time, just my 
opinion based on how long 3.4 was the stable release of ZooKeeper (4 years). 
Since then the release schedule has become much faster and to be honest I’m not 
participating in it.

As mentioned 3.6 and 3.7 releases are not much different. 3.6 is the “Facebook” 
version which is well tested and contains lots of patches that improves 
robustness. Both versions are good candidates for upgrade, so announcing 3.5 
EoL (at least half year from now) is not necessarily bad.

As an alternative, staying with the LT(S|M) / non-LT(S|M) terms, I think the 
following could also be considered for the community:

Now:

master
--
3.7
3.6
3.5 LTS
--
3.4 EoL

Can become:

master
--
3.8 LTS
3.7
3.5 LTS
--
3.6 EoL
3.4 EoL

In order to keep the number of maintained branches low.

What do you think?

Andor



> On 2022. Jan 31., at 19:41, Andrew Purtell  wrote:
> 
> Just to be clear I meant 'you' as the ZooKeeper project as a whole, but
> maybe I have misunderstood this response:
> https://issues.apache.org/jira/browse/HADOOP-17612?focusedCommentId=17311792=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17311792
> 
> 
> On Sun, Jan 30, 2022 at 10:29 AM Enrico Olivelli 
> wrote:
> 
>> Il Dom 30 Gen 2022, 17:51 Andrew Purtell  ha
>> scritto:
>> 
>>> Previously in various contexts - specifically, I am thinking of a Hadoop
>>> JIRA where we once had a conversation on this topic, but I believe there
>>> have been others - you have declared 3.5 a long term stable (LTS)
>> release.
>>> 
>>> A sudden EOL of an LTS is jarring and makes future promise of LTS
>>> untrustworthy. What I would recommend for what it’s worth is a timetable
>> to
>>> EOL of 3.5 that is reasonably long, like one or two years, should you
>>> decide to EOL it.
>> 
>> 
>> I am sorry,
>> I forgot about such conversation.
>> 
>> Can you share some pointers ?
>> 
>> No problem from my side as soon as there is someone who needs 3.5 and that
>> is willing to help.
>> 
>> Our codebase is pretty stable and we usually pay much attention  to
>> compatibility. So I am sure that 3.5 clients will be able to connect to new
>> servers (and vice versa)
>> 
>> I opened up this discussion to see how much interest is in the community,
>> so from your response I understand that there is such interest.
>> 
>> Thanks for answering
>> 
>> Enrico
>> 
>> 
>> 
>> 
>> 
>>> 
>>> 
 On Jan 30, 2022, at 5:00 AM, Enrico Olivelli 
>>> wrote:
 
 Hello,
 We are going to release 3.8.0.
 It is time to think about moving 3.5 to EOL.
 
 Key points:
 - we already have a few other "active" branches, 3.6 and 3.7
 - 3.5 still has "ant" files, and cherry picking libraries upgrade is
 awkward  (you always have to create a separate patch)
 - moving to 3.6 is quite easy, so people should not be stuck if
 requested to upgrade to 3.6
 
 Thoughts ?
 
 
 Enrico
>>> 
>> 
> 
> 
> -- 
> Best regards,
> Andrew
> 
> Unrest, ignorance distilled, nihilistic imbeciles -
>It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>   - A23, Welcome, Apocalypse

Re: Cutting 3.8.0 release

[Andor Molnar]

What’s the reason for cutting a new minor release?
The logback migration?

3.7 only has a single patch release so far: 3.7.0

Isn’t that too early?

Andor




> On 2022. Jan 28., at 16:28, Enrico Olivelli  wrote:
> 
> Sure.
> 
> Il giorno ven 28 gen 2022 alle ore 14:19 Szalay-Bekő Máté
>  ha scritto:
>> 
>> Great news, thanks for the work, Enrico!!
>> 
>> I think we should wait for https://github.com/apache/zookeeper/pull/1807 (
>> https://issues.apache.org/jira/browse/ZOOKEEPER-4461) so that we can
>> eliminate all references for log4j1 from our pom.xml files. What do
>> you think?
> 
> good catch
> 
> the patch looks good, let's commit it as soon as CI passes
> 
> Enrico
> 
>> 
>> Regards,
>> Máté
>> 
>> 
>> On Fri, Jan 28, 2022 at 5:24 AM Chris Nauroth  wrote:
>> 
>>> +1
>>> 
>>> Thanks for driving this, Enrico!
>>> 
>>> Chris Nauroth
>>> 
>>> 
>>> On Thu, Jan 27, 2022 at 7:08 AM Enrico Olivelli 
>>> wrote:
>>> 
 Hello ZooKeepers,
 I believe that the master branch is in good shape.
 
 I would like to start the release procedure for 3.8.0.
 
 This is the list of issues for 3.8.0
 
 
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZOOKEEPER%20AND%20fixVersion%20%3D%203.8.0
 
 We recently addressed all of the CVEs by updating some key
 dependencies, like Netty, and moving away from Log4j1 (we switched to
 LogBack)
 
 If no one has objections I will start the release procedure on Monday
 
 Regards
 
 Enrico
 
>>> 

Logback phase #2

[Andor Molnar]

Hi folks,

I’ve created the second and hopefully final patch for the Logback migration 
covering the rest of maven projects. Please review.
https://github.com/apache/zookeeper/pull/1807

Regards,
Andor

[jira] [Created] (ZOOKEEPER-4461) Migrate zookeeper-contrib and -recipes projects.

[Andor Molnar (Jira)]

Andor Molnar created ZOOKEEPER-4461:
---

 Summary: Migrate zookeeper-contrib and -recipes projects.
 Key: ZOOKEEPER-4461
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4461
 Project: ZooKeeper
  Issue Type: Sub-task
Reporter: Andor Molnar






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Logback

[Andor Molnar]

Thanks for the quick review Chris.

I agree with the second part of your e-mail completely. I’m not sure either 
that the community has given a thumbs-up for logback, but I wanted to finalize 
my patch sooner, because I have other duties to take care of.

I feel like logback is generally acceptable for ZK, but log4j2 would be more 
convenient, because most projects will eventually swap for it.

Andor



> On 2022. Jan 20., at 2:42, Chris Nauroth  wrote:
> 
> Thank you, Andor. I entered one more round of very minor feedback.
> 
> I'm not sure about the licensing changes. I responded on the PR with my
> thoughts, but I'd appreciate a second set of eyes on the licensing in
> particular.
> 
> After resolving that feedback, I'll be ready to +1 from a code perspective,
> but it sounds like the discussion of direction is not necessarily settled
> here. Can others who have raised red flags please clarify the degree of
> their objections? Is anyone actually -1 on a move to Logback? For my part,
> even though I raised objections, I'm OK proceeding with Logback.  I'll
> likely swap it for the Log4J 2 SLF4J back-end in my deployments. (I
> specifically tested this on your branch and confirmed it works.)
> 
> Chris Nauroth
> 
> 
> On Wed, Jan 19, 2022 at 1:46 PM Andor Molnar  wrote:
> 
>> I’m done with all the changes that I wanted to include in the first
>> logback patch.
>> Most of Chris’ feedback has also been addressed as well as the licensing
>> changes.
>> We have binary distribution which includes the logback jar, so I added EPL
>> v1.0
>> to LINCENSE.txt and mentioned Logback in the NOTICE.txt file. Hope all
>> done correctly.
>> 
>> Documentation has also been updated according to the new logging backend.
>> 
>> Migration of zookeeper-recipes and zookeeper-contrib projects will come in
>> the upcoming patch.
>> 
>> Andor
>> 
>> 
>> 
>>> On 2022. Jan 19., at 1:45, Ted Dunning  wrote:
>>> 
>>> I believe that the primary contributor to logback was highly skeptical
>> that
>>> the recent problems could possible affect logback. That isn't a good
>>> attitude for security problems.
>>> 
>>> It isn't just a matter of patch rate. There is also the question of
>>> community size. Is logback effectively a one-man show?
>>> 
>>> 
>>> 
>>> On Tue, Jan 18, 2022 at 3:25 PM Christopher  wrote:
>>> 
>>>> While it has had recent activity, it is notable that logback only
>> recently
>>>> became active again for patches to the stable 1.2 releases. After
>> several
>>>> releases in early 2017, it did not have a stable release for over four
>>>> years between 31-Mar-2017 (v1.2.3) and  19-Jul-2021 (v1.2.4).
>>>> 
>>>> On Tue, Jan 18, 2022 at 6:20 PM Christopher 
>> wrote:
>>>> 
>>>>> Yes. It looks like logback is still actively being developed. 1.2 had a
>>>>> release in December. The 1.3 line is still alpha and has also seen
>> recent
>>>>> releases (interestingly, it requires at least Java 9 to build, but will
>>>> run
>>>>> on Java 8, which is similar to what I had recommended for ZK in a
>>>> different
>>>>> thread). 1.2 only requires Java 1.6 or later. Since it's still
>> receiving
>>>>> patches, and it's not alpha, that's probably the best version to use.
>>>>> Currently, it seems to be at 1.2.9.
>>>>> 
>>>>> On Tue, Jan 18, 2022 at 2:25 PM Andor Molnar  wrote:
>>>>> 
>>>>>> I agree with you completely and this is crucial for logback too, so
>>>>>> correct me if I'm wrong. Logback is current and actively maintained.
>> Is
>>>>>> that correct?
>>>>>> 
>>>>>> Andor
>>>>>> 
>>>>>> 
>>>>>> On Tue, 2022-01-18 at 12:43 -0500, Christopher wrote:
>>>>>>> I do think these are more good reasons to adopt
>>>>>>> something that is current and actively maintained, though, rather
>>>>>>> than
>>>>>>> something that is old and not active.
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>> 
>> 

Re: Logback

[Andor Molnar]

I’m done with all the changes that I wanted to include in the first logback 
patch.
Most of Chris’ feedback has also been addressed as well as the licensing 
changes.
We have binary distribution which includes the logback jar, so I added EPL v1.0
to LINCENSE.txt and mentioned Logback in the NOTICE.txt file. Hope all done 
correctly.

Documentation has also been updated according to the new logging backend.

Migration of zookeeper-recipes and zookeeper-contrib projects will come in the 
upcoming patch.

Andor



> On 2022. Jan 19., at 1:45, Ted Dunning  wrote:
> 
> I believe that the primary contributor to logback was highly skeptical that
> the recent problems could possible affect logback. That isn't a good
> attitude for security problems.
> 
> It isn't just a matter of patch rate. There is also the question of
> community size. Is logback effectively a one-man show?
> 
> 
> 
> On Tue, Jan 18, 2022 at 3:25 PM Christopher  wrote:
> 
>> While it has had recent activity, it is notable that logback only recently
>> became active again for patches to the stable 1.2 releases. After several
>> releases in early 2017, it did not have a stable release for over four
>> years between 31-Mar-2017 (v1.2.3) and  19-Jul-2021 (v1.2.4).
>> 
>> On Tue, Jan 18, 2022 at 6:20 PM Christopher  wrote:
>> 
>>> Yes. It looks like logback is still actively being developed. 1.2 had a
>>> release in December. The 1.3 line is still alpha and has also seen recent
>>> releases (interestingly, it requires at least Java 9 to build, but will
>> run
>>> on Java 8, which is similar to what I had recommended for ZK in a
>> different
>>> thread). 1.2 only requires Java 1.6 or later. Since it's still receiving
>>> patches, and it's not alpha, that's probably the best version to use.
>>> Currently, it seems to be at 1.2.9.
>>> 
>>> On Tue, Jan 18, 2022 at 2:25 PM Andor Molnar  wrote:
>>> 
>>>> I agree with you completely and this is crucial for logback too, so
>>>> correct me if I'm wrong. Logback is current and actively maintained. Is
>>>> that correct?
>>>> 
>>>> Andor
>>>> 
>>>> 
>>>> On Tue, 2022-01-18 at 12:43 -0500, Christopher wrote:
>>>>> I do think these are more good reasons to adopt
>>>>> something that is current and actively maintained, though, rather
>>>>> than
>>>>> something that is old and not active.
>>>> 
>>>> 
>>>> 
>> 

Re: Logback

[Andor Molnar]

I agree with you completely and this is crucial for logback too, so
correct me if I'm wrong. Logback is current and actively maintained. Is
that correct?

Andor


On Tue, 2022-01-18 at 12:43 -0500, Christopher wrote:
> I do think these are more good reasons to adopt
> something that is current and actively maintained, though, rather
> than
> something that is old and not active.