Re: ZooKeeper 3.6.1...is it time ?

[Patrick Hunt]

Thanks for the ping - I commented on the jira.

Patrick

On Fri, Mar 20, 2020 at 10:47 AM Enrico Olivelli 
wrote:

> Patrick
> We have this issue that you created
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-3696
>
> I feel it is better to address the problem in 3.7 and not on 3.6.1
>
> Do you agree?
>
> Enrico
>
>
> Il Ven 20 Mar 2020, 11:11 Norbert Kalmar  ha
> scritto:
>
> > I just saw the discussion here on ZOOKEEPER-3739, which I committed
> about a
> > minute ago. But please do check if anyone is interested in it (we can
> > always revert).
> > My very short take on the issue: Removed an unsupported api class that
> use
> > was *supposedly* to speed up file modification watch event on OSX
> systems.
> >
> > - Norbert
> >
> > On Fri, Mar 20, 2020 at 8:07 AM Christopher  wrote:
> >
> > > Ted, please see the JIRA issue and linked PR for detailed answers to
> > those
> > > questions. My side issue has already hijacked this thread enough :) We
> > can
> > > continue in the forked thread that Enrico created, if you wish.
> > >
> > > On Fri, Mar 20, 2020 at 1:32 AM Ted Dunning 
> > wrote:
> > >
> > > > What internal class?
> > > >
> > > > Where is it used?
> > > >
> > > > Why are others using JDK 13 not seeing the problem that you
> described?
> > > >
> > > >
> > > >
> > > > On Thu, Mar 19, 2020 at 8:58 PM Christopher 
> > wrote:
> > > >
> > > > > Apologies, but it's a bit hard for me to explain briefly. Here's an
> > > > attempt
> > > > > to be as brief as I can:
> > > > >
> > > > > Newer JDKs add a `-release` option to cross-compile with strict
> > > > > compatibility enforced. When this flag is enabled, as my PR does
> > > > > automatically with the profiles (when it detects building with a
> > newer
> > > > JDK
> > > > > or in m2e Eclipse), the compiler identifies the use of a class that
> > is
> > > > not
> > > > > strictly compatible with Java 8. My PR removes this not strictly
> > > > compatible
> > > > > class. The use of this class should be considered a bug because it
> > will
> > > > > cause a failure in some JDKs (those without the com.sun class)... a
> > bug
> > > > > that is caught by the strict compatibility checks of the `-release`
> > > flag
> > > > in
> > > > > newer JDKs.
> > > > >
> > > > > The problem with the class is that it only is available on some
> > > versions
> > > > of
> > > > > Java (those with com.sun internals). It is not only not strictly
> > > > compatible
> > > > > with Java 8 ("not guaranteed to work"), it is specifically
> > "guaranteed
> > > to
> > > > > not work" in future Java versions, because it is planned for
> explicit
> > > > > removal. It already causes problems in current versions if you have
> > an
> > > > > application that uses modules and depends on ZooKeeper, because the
> > > > > unsupported class is hidden away in an internal module.
> > > > >
> > > > > What you say about ensuring that releases are built with JDK8 is
> > > actually
> > > > > not necessary any longer with the use of the `-release` flag. Newer
> > > JDKs
> > > > > enforce cross-compilation compatibility better. This release flag
> > > option
> > > > > eliminates the need to hold yourself back, doing release builds on
> an
> > > > older
> > > > > end-of-life JDK to retain compatibility. Now, you can build with a
> > > newer
> > > > > JDK (taking advantage of newer Maven plugins, etc.), avoid complex
> > uses
> > > > of
> > > > > Maven toolchains, the annoying `-bootclasspath`, or the
> > > > > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation
> > > strict
> > > > > compatibility is now enforced as a built in feature of newer JDKs.
> > > > >
> > > > > Sorry I couldn't explain more briefly... it's a very tiny change,
> but
> > > > > there's a lot of backstory to communicate why it matters.
> > > > >
> > > > > Incidentally, you can probably get rid of a few build jobs in
> Jenkins
> > > if
> > > > > you were to use the maven-enforcer-plugin to *only* support builds
> on
> > > > newer
> > > > > JDKs (but still using the release flag to target Java 8, of
> course).
> > > This
> > > > > is what Apache Accumulo does for the stuff that we want to be
> > > compatible
> > > > > with Java 8: we require the build to use at least 11, but
> > cross-compile
> > > > to
> > > > > 8, because some of the Maven plugins we use for quality control
> > checks
> > > > are
> > > > > now starting to require Java 11.
> > > > >
> > > > >
> > > > > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> > > > scritto:
> > > > > >
> > > > > > > I would very much like my pull request in
> > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be
> > merged
> > > in
> > > > > > 3.6.1
> > > > > > > and newer, because it helps ensure newer java APIs don't leak
> > into
> > > > pull
> > > > > > > requests/patches when developing stuff to contribute using a
> > newer
> > > > JDK,
> > 

Re: ZooKeeper 3.6.1...is it time ?

[Enrico Olivelli]

Patrick
We have this issue that you created

https://issues.apache.org/jira/browse/ZOOKEEPER-3696

I feel it is better to address the problem in 3.7 and not on 3.6.1

Do you agree?

Enrico


Il Ven 20 Mar 2020, 11:11 Norbert Kalmar  ha
scritto:

> I just saw the discussion here on ZOOKEEPER-3739, which I committed about a
> minute ago. But please do check if anyone is interested in it (we can
> always revert).
> My very short take on the issue: Removed an unsupported api class that use
> was *supposedly* to speed up file modification watch event on OSX systems.
>
> - Norbert
>
> On Fri, Mar 20, 2020 at 8:07 AM Christopher  wrote:
>
> > Ted, please see the JIRA issue and linked PR for detailed answers to
> those
> > questions. My side issue has already hijacked this thread enough :) We
> can
> > continue in the forked thread that Enrico created, if you wish.
> >
> > On Fri, Mar 20, 2020 at 1:32 AM Ted Dunning 
> wrote:
> >
> > > What internal class?
> > >
> > > Where is it used?
> > >
> > > Why are others using JDK 13 not seeing the problem that you described?
> > >
> > >
> > >
> > > On Thu, Mar 19, 2020 at 8:58 PM Christopher 
> wrote:
> > >
> > > > Apologies, but it's a bit hard for me to explain briefly. Here's an
> > > attempt
> > > > to be as brief as I can:
> > > >
> > > > Newer JDKs add a `-release` option to cross-compile with strict
> > > > compatibility enforced. When this flag is enabled, as my PR does
> > > > automatically with the profiles (when it detects building with a
> newer
> > > JDK
> > > > or in m2e Eclipse), the compiler identifies the use of a class that
> is
> > > not
> > > > strictly compatible with Java 8. My PR removes this not strictly
> > > compatible
> > > > class. The use of this class should be considered a bug because it
> will
> > > > cause a failure in some JDKs (those without the com.sun class)... a
> bug
> > > > that is caught by the strict compatibility checks of the `-release`
> > flag
> > > in
> > > > newer JDKs.
> > > >
> > > > The problem with the class is that it only is available on some
> > versions
> > > of
> > > > Java (those with com.sun internals). It is not only not strictly
> > > compatible
> > > > with Java 8 ("not guaranteed to work"), it is specifically
> "guaranteed
> > to
> > > > not work" in future Java versions, because it is planned for explicit
> > > > removal. It already causes problems in current versions if you have
> an
> > > > application that uses modules and depends on ZooKeeper, because the
> > > > unsupported class is hidden away in an internal module.
> > > >
> > > > What you say about ensuring that releases are built with JDK8 is
> > actually
> > > > not necessary any longer with the use of the `-release` flag. Newer
> > JDKs
> > > > enforce cross-compilation compatibility better. This release flag
> > option
> > > > eliminates the need to hold yourself back, doing release builds on an
> > > older
> > > > end-of-life JDK to retain compatibility. Now, you can build with a
> > newer
> > > > JDK (taking advantage of newer Maven plugins, etc.), avoid complex
> uses
> > > of
> > > > Maven toolchains, the annoying `-bootclasspath`, or the
> > > > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation
> > strict
> > > > compatibility is now enforced as a built in feature of newer JDKs.
> > > >
> > > > Sorry I couldn't explain more briefly... it's a very tiny change, but
> > > > there's a lot of backstory to communicate why it matters.
> > > >
> > > > Incidentally, you can probably get rid of a few build jobs in Jenkins
> > if
> > > > you were to use the maven-enforcer-plugin to *only* support builds on
> > > newer
> > > > JDKs (but still using the release flag to target Java 8, of course).
> > This
> > > > is what Apache Accumulo does for the stuff that we want to be
> > compatible
> > > > with Java 8: we require the build to use at least 11, but
> cross-compile
> > > to
> > > > 8, because some of the Maven plugins we use for quality control
> checks
> > > are
> > > > now starting to require Java 11.
> > > >
> > > >
> > > > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli  >
> > > > wrote:
> > > >
> > > > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> > > scritto:
> > > > >
> > > > > > I would very much like my pull request in
> > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be
> merged
> > in
> > > > > 3.6.1
> > > > > > and newer, because it helps ensure newer java APIs don't leak
> into
> > > pull
> > > > > > requests/patches when developing stuff to contribute using a
> newer
> > > JDK,
> > > > > by
> > > > > > leveraging the "release" flag of newer JDKs. However, in order to
> > do
> > > > so,
> > > > > it
> > > > > > needs to remove the use of an internal com.sun API which is not
> > > > supported
> > > > > > in newer JDKs. I would like it included, because it makes it
> easier
> > > for
> > > > > me
> > > > > > (and others using newer JDKs) to contribute.
> > > > > >
> > > > >
> > > > > I usually use jdk13 

Re: [PROPOSAL] Send build notifications to notifications list

[Christopher]

Okay, I cleaned up the views. They are now under the "Z" top-level group
at: https://builds.apache.org/view/Z/view/ZooKeeper/
(I'm probably going to try to help other projects organize theirs under
their corresponding top-level single-character groups as well, but that
will take time)
I updated the destination email for all the jobs to go to the notifications
list now. It may be necessary for a moderator to whitelist Jenkins there.
I'm not sure.

The following jobs did not have email notifications set up, so I left them
alone:
PreCommit-ZOOKEEPER-github-pr-build (might be a duplicate of
PreCommit-ZOOKEEPER-github-pr-build-maven ?)
ZooKeeper-Find-Flaky-Tests

Additionally, I think the JDK testing can be limited to just the JDK for
the target version (8), the latest (14), and the intermediate LTS ones
(11). I don't think 9, 10, 12, and 13 need to be tested as well. I'm not
trying to argue for *less* testing; rather, I'm trying to argue for more
*strategic* testing that 1) better utilizes the shared build server
resources, and 2) to increase the signal-to-noise ratio for developers
trying to pay attention to the test results.
Another way to increase the signal-to-noise ratio around testing is to
avoid duplication of effort between Travis CI and Jenkins, but I haven't
looked into this much yet.


On Fri, Mar 20, 2020 at 10:05 AM Andor Molnar  wrote:

> Hi Christopher,
>
> I’m happy to use the new, top-level view from now. Additionally I deleted
> all the disabled projects, as you said, we don’t need them anymore. I’m not
> sure about the rest, need to take a closer look.
>
> Thanks for your efforts.
>
> Andor
>
>
>
>
> > On 2020. Mar 19., at 19:50, Christopher  wrote:
> >
> > That view only shows 23 of ZK's total jobs. I think that view should also
> > be deleted in favor of the one at
> > https://builds.apache.org/view/Z/view/ZooKeeper/
> > Part of the impetus for creating individual tabs, one for each letter, is
> > for stability, because when we group letters into groups, people feel the
> > need to adjust them when one group gets bigger than others. This happened
> > twice, and broke links. Having the top-level views of a single letter,
> and
> > subviews under that creates stable, predictable links to job views.
> >
> > On Thu, Mar 19, 2020 at 2:38 PM Patrick Hunt  wrote:
> >
> >> afaik this is the view (at least the one I use):
> >> https://builds.apache.org/view/S-Z/view/ZooKeeper/
> >> it's curated to maintain the current/active jobs for the current/active
> >> branches and smattering of jdks for coverage.
> >>
> >> Patrick
> >>
> >> On Thu, Mar 19, 2020 at 11:30 AM Christopher 
> wrote:
> >>
> >>> It sounds like there's at least the beginnings of consensus here.
> >>>
> >>> So, I took a look at how ZK's jobs are configured in builds.apache.org
> ,
> >> in
> >>> order to prepare to help make the change, and saw some stuff that was
> >>> slightly upsetting (), or at least daunting for whoever makes the
> >> change:
> >>>
> >>> 1. There are 45 Jenkins jobs for ZK. That seems insane to me. Is there
> >> any
> >>> old ones that can be removed? I'm not sure which the community finds
> >>> valuable and which are excess, but all the disabled ones should
> probably
> >> be
> >>> deleted, because they are definitely not in use. It would be best if a
> >>> committer could identify and delete the ones that aren't needed. That
> >> will
> >>> save some time in editing the configs to change the target email.
> >>>
> >>> 2. I also noticed there's a top-level view at
> >>> https://builds.apache.org/view/ZK%20All/ ; several years ago, I tried
> to
> >>> get people to do a better job of keeping Jenkins views organized, so I
> >>> created a new tab for each letter of the Latin alphabet to encourage
> >>> projects to create subviews under that. ZooKeeper didn't have one, and
> >> had
> >>> the top-level view instead. So, I went ahead and created a new view at
> >>> https://builds.apache.org/view/Z/view/ZooKeeper/ ; I think the
> top-level
> >>> view at https://builds.apache.org/view/ZK%20All/should be deleted.
> >>>
> >>> Once the extras repos are deleted, I'd be happy to be the one to
> >> tediously
> >>> update each job to point to the notifications list, but a moderator
> might
> >>> need to approve the Jenkins sender. 
> >>>
> >>>
> >>> On Thu, Mar 19, 2020 at 11:18 AM Szalay-Bekő Máté <
> >>> szalay.beko.m...@gmail.com> wrote:
> >>>
>  I also like it! It will make it easier for new community members to
> >> joint
>  to discussions.
> 
>  although it also means I will have to setup new filters now for
>  the notifications@ list ;)
> 
>  On Thu, Mar 19, 2020 at 1:52 PM RABI K.C.  wrote:
> 
> > Hi Christopher,
> >
> > I am +1 on this.
> >
> > Regards,
> > Rabi Kumar K C
> >
> > On Wed, Mar 18, 2020 at 10:25 PM Christopher 
>  wrote:
> >
> >> Hi ZK Devs,
> >>
> >> I have been trying to watch the ZooKeeper developer list lately, to
> 

Build failed in Jenkins: zookeeper-branch36-java8 #99

[Apache Jenkins Server]

See 

Changes:


--
[...truncated 657.99 KB...]
Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Generating 

Re: [PROPOSAL] Send build notifications to notifications list

[Andor Molnar]

Hi Christopher,

I’m happy to use the new, top-level view from now. Additionally I deleted all 
the disabled projects, as you said, we don’t need them anymore. I’m not sure 
about the rest, need to take a closer look.

Thanks for your efforts.

Andor




> On 2020. Mar 19., at 19:50, Christopher  wrote:
> 
> That view only shows 23 of ZK's total jobs. I think that view should also
> be deleted in favor of the one at
> https://builds.apache.org/view/Z/view/ZooKeeper/
> Part of the impetus for creating individual tabs, one for each letter, is
> for stability, because when we group letters into groups, people feel the
> need to adjust them when one group gets bigger than others. This happened
> twice, and broke links. Having the top-level views of a single letter, and
> subviews under that creates stable, predictable links to job views.
> 
> On Thu, Mar 19, 2020 at 2:38 PM Patrick Hunt  wrote:
> 
>> afaik this is the view (at least the one I use):
>> https://builds.apache.org/view/S-Z/view/ZooKeeper/
>> it's curated to maintain the current/active jobs for the current/active
>> branches and smattering of jdks for coverage.
>> 
>> Patrick
>> 
>> On Thu, Mar 19, 2020 at 11:30 AM Christopher  wrote:
>> 
>>> It sounds like there's at least the beginnings of consensus here.
>>> 
>>> So, I took a look at how ZK's jobs are configured in builds.apache.org,
>> in
>>> order to prepare to help make the change, and saw some stuff that was
>>> slightly upsetting (), or at least daunting for whoever makes the
>> change:
>>> 
>>> 1. There are 45 Jenkins jobs for ZK. That seems insane to me. Is there
>> any
>>> old ones that can be removed? I'm not sure which the community finds
>>> valuable and which are excess, but all the disabled ones should probably
>> be
>>> deleted, because they are definitely not in use. It would be best if a
>>> committer could identify and delete the ones that aren't needed. That
>> will
>>> save some time in editing the configs to change the target email.
>>> 
>>> 2. I also noticed there's a top-level view at
>>> https://builds.apache.org/view/ZK%20All/ ; several years ago, I tried to
>>> get people to do a better job of keeping Jenkins views organized, so I
>>> created a new tab for each letter of the Latin alphabet to encourage
>>> projects to create subviews under that. ZooKeeper didn't have one, and
>> had
>>> the top-level view instead. So, I went ahead and created a new view at
>>> https://builds.apache.org/view/Z/view/ZooKeeper/ ; I think the top-level
>>> view at https://builds.apache.org/view/ZK%20All/should be deleted.
>>> 
>>> Once the extras repos are deleted, I'd be happy to be the one to
>> tediously
>>> update each job to point to the notifications list, but a moderator might
>>> need to approve the Jenkins sender. 
>>> 
>>> 
>>> On Thu, Mar 19, 2020 at 11:18 AM Szalay-Bekő Máté <
>>> szalay.beko.m...@gmail.com> wrote:
>>> 
 I also like it! It will make it easier for new community members to
>> joint
 to discussions.
 
 although it also means I will have to setup new filters now for
 the notifications@ list ;)
 
 On Thu, Mar 19, 2020 at 1:52 PM RABI K.C.  wrote:
 
> Hi Christopher,
> 
> I am +1 on this.
> 
> Regards,
> Rabi Kumar K C
> 
> On Wed, Mar 18, 2020 at 10:25 PM Christopher 
 wrote:
> 
>> Hi ZK Devs,
>> 
>> I have been trying to watch the ZooKeeper developer list lately, to
>> stay on top of latest issues, and to contribute in whatever small
>> way
>> I can. However, I noticed the list gets a lot of spam from
>> automated
>> notifications (especially Jenkins).
>> 
>> This has the effect of drowning out conversation between actual
>> humans, and can discourage participation, because it requires new
>> subscribers to set up a bunch of filters to suppress these, if they
>> don't want to see them (and they may unintentionally suppress human
>> conversation, if they don't set up their filters perfectly).
>> 
>> What do the ZK committers/PMC members think about having Jenkins
>> send
>> build notifications to the notifications@ list instead of dev@?
>> 
>> Thanks,
>> Christopher
>> 
> 
 
>>> 
>> 

Jenkins build is back to stable : zookeeper-master-maven-jdk14 #6

[Apache Jenkins Server]

See 

Jenkins build is back to stable : zookeeper-master-maven-jdk11 #423

[Apache Jenkins Server]

See 

Jenkins build is still unstable: zookeeper-master-maven-jdk14 #5

[Apache Jenkins Server]

See 

Jenkins build is back to normal : zookeeper-master-maven-jdk12 #422

[Apache Jenkins Server]

See 

Jenkins build is back to normal : zookeeper-master-maven #716

[Apache Jenkins Server]

See 

Re: ZooKeeper 3.6.1...is it time ?

[Norbert Kalmar]

I just saw the discussion here on ZOOKEEPER-3739, which I committed about a
minute ago. But please do check if anyone is interested in it (we can
always revert).
My very short take on the issue: Removed an unsupported api class that use
was *supposedly* to speed up file modification watch event on OSX systems.

- Norbert

On Fri, Mar 20, 2020 at 8:07 AM Christopher  wrote:

> Ted, please see the JIRA issue and linked PR for detailed answers to those
> questions. My side issue has already hijacked this thread enough :) We can
> continue in the forked thread that Enrico created, if you wish.
>
> On Fri, Mar 20, 2020 at 1:32 AM Ted Dunning  wrote:
>
> > What internal class?
> >
> > Where is it used?
> >
> > Why are others using JDK 13 not seeing the problem that you described?
> >
> >
> >
> > On Thu, Mar 19, 2020 at 8:58 PM Christopher  wrote:
> >
> > > Apologies, but it's a bit hard for me to explain briefly. Here's an
> > attempt
> > > to be as brief as I can:
> > >
> > > Newer JDKs add a `-release` option to cross-compile with strict
> > > compatibility enforced. When this flag is enabled, as my PR does
> > > automatically with the profiles (when it detects building with a newer
> > JDK
> > > or in m2e Eclipse), the compiler identifies the use of a class that is
> > not
> > > strictly compatible with Java 8. My PR removes this not strictly
> > compatible
> > > class. The use of this class should be considered a bug because it will
> > > cause a failure in some JDKs (those without the com.sun class)... a bug
> > > that is caught by the strict compatibility checks of the `-release`
> flag
> > in
> > > newer JDKs.
> > >
> > > The problem with the class is that it only is available on some
> versions
> > of
> > > Java (those with com.sun internals). It is not only not strictly
> > compatible
> > > with Java 8 ("not guaranteed to work"), it is specifically "guaranteed
> to
> > > not work" in future Java versions, because it is planned for explicit
> > > removal. It already causes problems in current versions if you have an
> > > application that uses modules and depends on ZooKeeper, because the
> > > unsupported class is hidden away in an internal module.
> > >
> > > What you say about ensuring that releases are built with JDK8 is
> actually
> > > not necessary any longer with the use of the `-release` flag. Newer
> JDKs
> > > enforce cross-compilation compatibility better. This release flag
> option
> > > eliminates the need to hold yourself back, doing release builds on an
> > older
> > > end-of-life JDK to retain compatibility. Now, you can build with a
> newer
> > > JDK (taking advantage of newer Maven plugins, etc.), avoid complex uses
> > of
> > > Maven toolchains, the annoying `-bootclasspath`, or the
> > > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation
> strict
> > > compatibility is now enforced as a built in feature of newer JDKs.
> > >
> > > Sorry I couldn't explain more briefly... it's a very tiny change, but
> > > there's a lot of backstory to communicate why it matters.
> > >
> > > Incidentally, you can probably get rid of a few build jobs in Jenkins
> if
> > > you were to use the maven-enforcer-plugin to *only* support builds on
> > newer
> > > JDKs (but still using the release flag to target Java 8, of course).
> This
> > > is what Apache Accumulo does for the stuff that we want to be
> compatible
> > > with Java 8: we require the build to use at least 11, but cross-compile
> > to
> > > 8, because some of the Maven plugins we use for quality control checks
> > are
> > > now starting to require Java 11.
> > >
> > >
> > > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> > scritto:
> > > >
> > > > > I would very much like my pull request in
> > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be merged
> in
> > > > 3.6.1
> > > > > and newer, because it helps ensure newer java APIs don't leak into
> > pull
> > > > > requests/patches when developing stuff to contribute using a newer
> > JDK,
> > > > by
> > > > > leveraging the "release" flag of newer JDKs. However, in order to
> do
> > > so,
> > > > it
> > > > > needs to remove the use of an internal com.sun API which is not
> > > supported
> > > > > in newer JDKs. I would like it included, because it makes it easier
> > for
> > > > me
> > > > > (and others using newer JDKs) to contribute.
> > > > >
> > > >
> > > > I usually use jdk13 for my applications and while I work on ZK code.
> > > > What problems are you facing?
> > > >
> > > > It is important to remember to build with jdk8 while making releases.
> > But
> > > > for dev it is not needed.
> > > >
> > > > Maybe we can merge that patch on master but I don't feel it is so
> > > important
> > > > at the moment. (With jdk13)
> > > >
> > > > Enrico
> > > >
> > > >
> > > > > The patch is currently pending additional reviewers.
> > > > >
> > > > > On Thu, Mar 19, 2020 at 11:17 AM Patrick Hunt 
> > 

Re: Which JDK? (Was Re: ZooKeeper 3.6.1...is it time ?)

[Christopher]

On Fri, Mar 20, 2020 at 2:30 AM Enrico Olivelli  wrote:
>
> Yes
> Do you have problems with a specific JDK?

Every one I've used with the `-release` flag set for the compiler.
This is expected, because that's what the `-release` flag is supposed
to do: reveal and prevent non-compliant APIs from being used. This
affects OpenJDK11, 13, Eclipse (with compiler-compliance settings set
correctly, which performs a similar function to `-release`).

Although I haven't tried on any other JVMs, I would it expect it to
outright fail on any Java (regardless of compiler compliance settings)
that isn't derived from OpenJDK/OracleJDK. Such JVMs would likely not
have the class present, because the class is not part of any Java spec
and cannot be assumed to be present.

Eventually, those internal classes will be encapsulated or removed, so
that they are inaccessible to user code (see
https://openjdk.java.net/jeps/260). I would strongly prefer to address
it sooner, rather than later, so people aren't forced to upgrade
ZooKeeper at the same time as they upgrade Java, in order to avoid
runtime failures. Upgrading multiple components at the same time on a
large cluster can be very disruptive to devops teams trying to isolate
problems to a specific upgrade.

Also, the use of this internal API blocks the ability to have
automatic module names for ZooKeeper in the jar manifests (an
intermediate step to full module use within ZooKeeper, which is
probably a long way off, and far less important that cross-compilation
strict compatibility, which is why I didn't mention it before now).

>
> Btw I had already approved that patch days ago. We need another sponsor. I
> am ok with committing it to 3.6.1

Technically, you did not mark it as approved. It's still marked as
"Changes requested". But, you did review it, and I appreciate that. I
brought it up again mainly because I thought the 3.6.1 conversation
would be a good chance to get another person's approval on it. :)

>
> Regarding CI: we have different jobs with different JDKs not just for byte
> code and API compatibility but because internal behaviour of JDK is really
> different from version to version, think about networking (they recently
> rewritten all the blocking IO system),  security (Kerberos stuff is the
> most tricky)...

That's fair. Those are good reasons to test on various platforms. I
would probably approach it as a: "build once, test many", rather than
have multiple build jobs on various platforms (since the byte code
itself shouldn't be platform dependent). However, I'm not in a
position to question the PMC's testing strategies. I trust the
community's judgment. I only have casual observations... I haven't
looked into it deeply. :)

>
> We have old jobs that we can drop and they are not running anymore.

Yeah, I'm pursuing that clean up in the thread I started regarding
build notifications to the separate notifications list, and hoping to
clean some of that up, once I'm certain there's clear consensus, and
no objections. :)

>
>
> Enrico
>
> Il Ven 20 Mar 2020, 06:32 Ted Dunning  ha scritto:
>
> > What internal class?
> >
> > Where is it used?
> >
> > Why are others using JDK 13 not seeing the problem that you described?
> >
> >
> >
> > On Thu, Mar 19, 2020 at 8:58 PM Christopher  wrote:
> >
> > > Apologies, but it's a bit hard for me to explain briefly. Here's an
> > attempt
> > > to be as brief as I can:
> > >
> > > Newer JDKs add a `-release` option to cross-compile with strict
> > > compatibility enforced. When this flag is enabled, as my PR does
> > > automatically with the profiles (when it detects building with a newer
> > JDK
> > > or in m2e Eclipse), the compiler identifies the use of a class that is
> > not
> > > strictly compatible with Java 8. My PR removes this not strictly
> > compatible
> > > class. The use of this class should be considered a bug because it will
> > > cause a failure in some JDKs (those without the com.sun class)... a bug
> > > that is caught by the strict compatibility checks of the `-release` flag
> > in
> > > newer JDKs.
> > >
> > > The problem with the class is that it only is available on some versions
> > of
> > > Java (those with com.sun internals). It is not only not strictly
> > compatible
> > > with Java 8 ("not guaranteed to work"), it is specifically "guaranteed to
> > > not work" in future Java versions, because it is planned for explicit
> > > removal. It already causes problems in current versions if you have an
> > > application that uses modules and depends on ZooKeeper, because the
> > > unsupported class is hidden away in an internal module.
> > >
> > > What you say about ensuring that releases are built with JDK8 is actually
> > > not necessary any longer with the use of the `-release` flag. Newer JDKs
> > > enforce cross-compilation compatibility better. This release flag option
> > > eliminates the need to hold yourself back, doing release builds on an
> > older
> > > end-of-life JDK to retain compatibility. 

Re: ZooKeeper 3.6.1...is it time ?

[Christopher]

Ted, please see the JIRA issue and linked PR for detailed answers to those
questions. My side issue has already hijacked this thread enough :) We can
continue in the forked thread that Enrico created, if you wish.

On Fri, Mar 20, 2020 at 1:32 AM Ted Dunning  wrote:

> What internal class?
>
> Where is it used?
>
> Why are others using JDK 13 not seeing the problem that you described?
>
>
>
> On Thu, Mar 19, 2020 at 8:58 PM Christopher  wrote:
>
> > Apologies, but it's a bit hard for me to explain briefly. Here's an
> attempt
> > to be as brief as I can:
> >
> > Newer JDKs add a `-release` option to cross-compile with strict
> > compatibility enforced. When this flag is enabled, as my PR does
> > automatically with the profiles (when it detects building with a newer
> JDK
> > or in m2e Eclipse), the compiler identifies the use of a class that is
> not
> > strictly compatible with Java 8. My PR removes this not strictly
> compatible
> > class. The use of this class should be considered a bug because it will
> > cause a failure in some JDKs (those without the com.sun class)... a bug
> > that is caught by the strict compatibility checks of the `-release` flag
> in
> > newer JDKs.
> >
> > The problem with the class is that it only is available on some versions
> of
> > Java (those with com.sun internals). It is not only not strictly
> compatible
> > with Java 8 ("not guaranteed to work"), it is specifically "guaranteed to
> > not work" in future Java versions, because it is planned for explicit
> > removal. It already causes problems in current versions if you have an
> > application that uses modules and depends on ZooKeeper, because the
> > unsupported class is hidden away in an internal module.
> >
> > What you say about ensuring that releases are built with JDK8 is actually
> > not necessary any longer with the use of the `-release` flag. Newer JDKs
> > enforce cross-compilation compatibility better. This release flag option
> > eliminates the need to hold yourself back, doing release builds on an
> older
> > end-of-life JDK to retain compatibility. Now, you can build with a newer
> > JDK (taking advantage of newer Maven plugins, etc.), avoid complex uses
> of
> > Maven toolchains, the annoying `-bootclasspath`, or the
> > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation strict
> > compatibility is now enforced as a built in feature of newer JDKs.
> >
> > Sorry I couldn't explain more briefly... it's a very tiny change, but
> > there's a lot of backstory to communicate why it matters.
> >
> > Incidentally, you can probably get rid of a few build jobs in Jenkins if
> > you were to use the maven-enforcer-plugin to *only* support builds on
> newer
> > JDKs (but still using the release flag to target Java 8, of course). This
> > is what Apache Accumulo does for the stuff that we want to be compatible
> > with Java 8: we require the build to use at least 11, but cross-compile
> to
> > 8, because some of the Maven plugins we use for quality control checks
> are
> > now starting to require Java 11.
> >
> >
> > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli 
> > wrote:
> >
> > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> scritto:
> > >
> > > > I would very much like my pull request in
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be merged in
> > > 3.6.1
> > > > and newer, because it helps ensure newer java APIs don't leak into
> pull
> > > > requests/patches when developing stuff to contribute using a newer
> JDK,
> > > by
> > > > leveraging the "release" flag of newer JDKs. However, in order to do
> > so,
> > > it
> > > > needs to remove the use of an internal com.sun API which is not
> > supported
> > > > in newer JDKs. I would like it included, because it makes it easier
> for
> > > me
> > > > (and others using newer JDKs) to contribute.
> > > >
> > >
> > > I usually use jdk13 for my applications and while I work on ZK code.
> > > What problems are you facing?
> > >
> > > It is important to remember to build with jdk8 while making releases.
> But
> > > for dev it is not needed.
> > >
> > > Maybe we can merge that patch on master but I don't feel it is so
> > important
> > > at the moment. (With jdk13)
> > >
> > > Enrico
> > >
> > >
> > > > The patch is currently pending additional reviewers.
> > > >
> > > > On Thu, Mar 19, 2020 at 11:17 AM Patrick Hunt 
> > wrote:
> > > >
> > > > > Great idea, thanks Enrico!
> > > > >
> > > > > Patrick
> > > > >
> > > > > On Thu, Mar 19, 2020 at 4:58 AM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > > after a couple of weeks after the release of 3.6.0 we are seeing
> a
> > > few
> > > > > > blocker issues related to the upgrade from 3.5.
> > > > > >
> > > > > > There is already a good list of issues already ported to 3.6 and
> a
> > > few
> > > > > > pending PRs that fix problems reported by users.
> > > > > >
> > > > > > You can find the list here, I have already did some clean up 

Which JDK? (Was Re: ZooKeeper 3.6.1...is it time ?)

[Enrico Olivelli]

Yes
Do you have problems with a specific JDK?

Btw I had already approved that patch days ago. We need another sponsor. I
am ok with committing it to 3.6.1

Regarding CI: we have different jobs with different JDKs not just for byte
code and API compatibility but because internal behaviour of JDK is really
different from version to version, think about networking (they recently
rewritten all the blocking IO system),  security (Kerberos stuff is the
most tricky)...

We have old jobs that we can drop and they are not running anymore.


Enrico

Il Ven 20 Mar 2020, 06:32 Ted Dunning  ha scritto:

> What internal class?
>
> Where is it used?
>
> Why are others using JDK 13 not seeing the problem that you described?
>
>
>
> On Thu, Mar 19, 2020 at 8:58 PM Christopher  wrote:
>
> > Apologies, but it's a bit hard for me to explain briefly. Here's an
> attempt
> > to be as brief as I can:
> >
> > Newer JDKs add a `-release` option to cross-compile with strict
> > compatibility enforced. When this flag is enabled, as my PR does
> > automatically with the profiles (when it detects building with a newer
> JDK
> > or in m2e Eclipse), the compiler identifies the use of a class that is
> not
> > strictly compatible with Java 8. My PR removes this not strictly
> compatible
> > class. The use of this class should be considered a bug because it will
> > cause a failure in some JDKs (those without the com.sun class)... a bug
> > that is caught by the strict compatibility checks of the `-release` flag
> in
> > newer JDKs.
> >
> > The problem with the class is that it only is available on some versions
> of
> > Java (those with com.sun internals). It is not only not strictly
> compatible
> > with Java 8 ("not guaranteed to work"), it is specifically "guaranteed to
> > not work" in future Java versions, because it is planned for explicit
> > removal. It already causes problems in current versions if you have an
> > application that uses modules and depends on ZooKeeper, because the
> > unsupported class is hidden away in an internal module.
> >
> > What you say about ensuring that releases are built with JDK8 is actually
> > not necessary any longer with the use of the `-release` flag. Newer JDKs
> > enforce cross-compilation compatibility better. This release flag option
> > eliminates the need to hold yourself back, doing release builds on an
> older
> > end-of-life JDK to retain compatibility. Now, you can build with a newer
> > JDK (taking advantage of newer Maven plugins, etc.), avoid complex uses
> of
> > Maven toolchains, the annoying `-bootclasspath`, or the
> > maven-enforcer-plugin's animal-sniffer rule. The cross-compilation strict
> > compatibility is now enforced as a built in feature of newer JDKs.
> >
> > Sorry I couldn't explain more briefly... it's a very tiny change, but
> > there's a lot of backstory to communicate why it matters.
> >
> > Incidentally, you can probably get rid of a few build jobs in Jenkins if
> > you were to use the maven-enforcer-plugin to *only* support builds on
> newer
> > JDKs (but still using the release flag to target Java 8, of course). This
> > is what Apache Accumulo does for the stuff that we want to be compatible
> > with Java 8: we require the build to use at least 11, but cross-compile
> to
> > 8, because some of the Maven plugins we use for quality control checks
> are
> > now starting to require Java 11.
> >
> >
> > On Thu, Mar 19, 2020 at 3:32 PM Enrico Olivelli 
> > wrote:
> >
> > > Il Gio 19 Mar 2020, 20:11 Christopher  ha
> scritto:
> > >
> > > > I would very much like my pull request in
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3739 to be merged in
> > > 3.6.1
> > > > and newer, because it helps ensure newer java APIs don't leak into
> pull
> > > > requests/patches when developing stuff to contribute using a newer
> JDK,
> > > by
> > > > leveraging the "release" flag of newer JDKs. However, in order to do
> > so,
> > > it
> > > > needs to remove the use of an internal com.sun API which is not
> > supported
> > > > in newer JDKs. I would like it included, because it makes it easier
> for
> > > me
> > > > (and others using newer JDKs) to contribute.
> > > >
> > >
> > > I usually use jdk13 for my applications and while I work on ZK code.
> > > What problems are you facing?
> > >
> > > It is important to remember to build with jdk8 while making releases.
> But
> > > for dev it is not needed.
> > >
> > > Maybe we can merge that patch on master but I don't feel it is so
> > important
> > > at the moment. (With jdk13)
> > >
> > > Enrico
> > >
> > >
> > > > The patch is currently pending additional reviewers.
> > > >
> > > > On Thu, Mar 19, 2020 at 11:17 AM Patrick Hunt 
> > wrote:
> > > >
> > > > > Great idea, thanks Enrico!
> > > > >
> > > > > Patrick
> > > > >
> > > > > On Thu, Mar 19, 2020 at 4:58 AM Enrico Olivelli <
> eolive...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > > after a couple of weeks after the release of 3.6.0 we 

[jira] [Created] (ZOOKEEPER-3766) zkServer and other scripts should export CLASSPATH rather than use -cp

[Christopher Tubbs (Jira)]

Christopher Tubbs created ZOOKEEPER-3766:


 Summary: zkServer and other scripts should export CLASSPATH rather 
than use -cp
 Key: ZOOKEEPER-3766
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3766
 Project: ZooKeeper
  Issue Type: Improvement
  Components: scripts
Affects Versions: 3.5.7
Reporter: Christopher Tubbs


ZooKeeper's launch scripts use `-cp` to pass the class path to Java when 
launching. This creates insanely large command-lines which are completely 
unnecessary.

Java respects the CLASSPATH environment variable, and this is how the class 
path should be passed to Java when the process launches.

So, instead of doing {{java -cp $CLASSPATH ...}}, the scripts should be doing 
{{export CLASSPATH; java ...}}.

The long command-lines make it difficult to troubleshoot, or view running 
process lists in tools like top, htop, ps, but also make it impossible to 
search and manage using tools like pgrep and pkill, which can only search for 
the first 4096 characters in the command-line. (See 
https://github.com/apache/fluo-uno/issues/243#issuecomment-601553260 for a 
specific issue involving this limit caused by ZK's scripts.)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)