Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

On Mon, Sep 23, 2019 at 11:08 PM David Holmes 
wrote:

> Except when I run it through our test system ReservedStackTest is still
> failing :(
>
> I tested it initially when Fred proposed it and that went fine. It also
> passes for me locally on Linux.
>

I committed the changes other than for @ReservedStackAccess, which remains
in limbo.

Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

Frederic, could you figure out how to resolve

8231031: runtime/ReservedStack/ReservedStackTest.java fails after jsr166
refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ReservedStackTest/index.html
https://bugs.openjdk.java.net/browse/JDK-8231031

On Mon, Sep 23, 2019 at 11:08 PM David Holmes 
wrote:

> Except when I run it through our test system ReservedStackTest is still
> failing :(
>
> I tested it initially when Fred proposed it and that went fine. It also
> passes for me locally on Linux.
>
> David
>
> On 24/09/2019 12:20 pm, David Holmes wrote:
> > Hi Martin,
> >
> > That all seems fine to me.
> >
> > Thanks,
> > David
> >
> > On 24/09/2019 3:43 am, Martin Buchholz wrote:
> >> We now have a fix-up integration that removes all the previously
> >> excluded tests from their exclude lists.
> >>
> >>
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html
> >>
> >>
> >> 8231031: runtime/ReservedStack/ReservedStackTest.java fails after
> >> jsr166 refresh
> >>
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ReservedStackTest/index.html
> >>
> >> https://bugs.openjdk.java.net/browse/JDK-8231031
> >>
> >> LockInfo objects now restored to their previous values (although David
> >> was looking at future improvements).
> >>
> >> 8231032: ThreadMXBean locking tests fail after JSR 166 refresh
> >>
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/wrong-blocker/index.html
> >>
> >> https://bugs.openjdk.java.net/browse/JDK-8231032
> >>
> >> 8231036: vmTestbase monitoring tests fail after JSR 166 refresh
> >>
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SynchronizerLockingThreads/index.html
> >>
> >> https://bugs.openjdk.java.net/browse/JDK-8231036
>

Re: RFR: jsr166 integration 2019-09

[David Holmes]

Except when I run it through our test system ReservedStackTest is still 
failing :(


I tested it initially when Fred proposed it and that went fine. It also 
passes for me locally on Linux.


David

On 24/09/2019 12:20 pm, David Holmes wrote:

Hi Martin,

That all seems fine to me.

Thanks,
David

On 24/09/2019 3:43 am, Martin Buchholz wrote:
We now have a fix-up integration that removes all the previously 
excluded tests from their exclude lists.


https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 



8231031: runtime/ReservedStack/ReservedStackTest.java fails after 
jsr166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ReservedStackTest/index.html 


https://bugs.openjdk.java.net/browse/JDK-8231031

LockInfo objects now restored to their previous values (although David 
was looking at future improvements).


8231032: ThreadMXBean locking tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/wrong-blocker/index.html 


https://bugs.openjdk.java.net/browse/JDK-8231032

8231036: vmTestbase monitoring tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SynchronizerLockingThreads/index.html 


https://bugs.openjdk.java.net/browse/JDK-8231036

Re: RFR: jsr166 integration 2019-09

[David Holmes]

Hi Martin,

That all seems fine to me.

Thanks,
David

On 24/09/2019 3:43 am, Martin Buchholz wrote:
We now have a fix-up integration that removes all the previously 
excluded tests from their exclude lists.


https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html

8231031: runtime/ReservedStack/ReservedStackTest.java fails after jsr166 
refresh

https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ReservedStackTest/index.html
https://bugs.openjdk.java.net/browse/JDK-8231031

LockInfo objects now restored to their previous values (although David 
was looking at future improvements).


8231032: ThreadMXBean locking tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/wrong-blocker/index.html
https://bugs.openjdk.java.net/browse/JDK-8231032

8231036: vmTestbase monitoring tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SynchronizerLockingThreads/index.html
https://bugs.openjdk.java.net/browse/JDK-8231036

Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

We now have a fix-up integration that removes all the previously excluded
tests from their exclude lists.

https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html

8231031: runtime/ReservedStack/ReservedStackTest.java fails after jsr166
refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ReservedStackTest/index.html
https://bugs.openjdk.java.net/browse/JDK-8231031

LockInfo objects now restored to their previous values (although David was
looking at future improvements).

8231032: ThreadMXBean locking tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/wrong-blocker/index.html
https://bugs.openjdk.java.net/browse/JDK-8231032

8231036: vmTestbase monitoring tests fail after JSR 166 refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/SynchronizerLockingThreads/index.html
https://bugs.openjdk.java.net/browse/JDK-8231036

Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

Sorry about the breakage.  In retrospect, this is one jsr166 integration
where we should have run more tests from other areas of the jdk.
Especially monitoring tests are sensitive to implementation changes in
j.u.c.locks.
(but we've never run nsk tests)


On Sun, Sep 15, 2019 at 3:14 PM David Holmes 
wrote:

> Hi Martin,
>
> These changes have caused a number of tests to break and wreaked havoc
> in our CI system (as those tests get executed a lot in different
> configs). The problem seems to be changes to stack use and contents:
>
> -
>
> vmTestbase/nsk/monitoring/ThreadMXBean/ThreadInfo/SynchronizerLockingThreads/SynchronizerLockingThreads*
> - vmTestbase/nsk/monitoring/ThreadMXBean/ThreadInfo/Multi/Multi*
> - runtime/ReservedStack/ReservedStackTest.java
> - java/lang/management/ThreadMXBean/LockedSynchronizers.java
>
> I'm filing bugs for each test group, but we may need to problem-list
> them in the meantime.
>
> David
> -
>
> On 14/09/2019 3:23 am, Martin Buchholz wrote:
> > Thank you!
> >
> > On Fri, Sep 13, 2019 at 9:44 AM  > > wrote:
> >
> >
> > Android (d8 in fact) is able to desugar private access with
> > nestmates since last April,
> > your latest backport target 9 which is not supported anymore, so
> > when you will move the backport to support Java 11,
> > this tool can become handy.
> >
> >
> > In fact, jsr166 CVS has already abandoned support for jdk9 (and jdk10)
> > so there's absolutely no reason not to apply the suggestions made
> > by  Rémi's program.
> >
> > https://bugs.openjdk.java.net/browse/JDK-8230966
>

Re: RFR: jsr166 integration 2019-09

[David Holmes]

Hi Martin,

These changes have caused a number of tests to break and wreaked havoc 
in our CI system (as those tests get executed a lot in different 
configs). The problem seems to be changes to stack use and contents:


- 
vmTestbase/nsk/monitoring/ThreadMXBean/ThreadInfo/SynchronizerLockingThreads/SynchronizerLockingThreads*

- vmTestbase/nsk/monitoring/ThreadMXBean/ThreadInfo/Multi/Multi*
- runtime/ReservedStack/ReservedStackTest.java
- java/lang/management/ThreadMXBean/LockedSynchronizers.java

I'm filing bugs for each test group, but we may need to problem-list 
them in the meantime.


David
-

On 14/09/2019 3:23 am, Martin Buchholz wrote:

Thank you!

On Fri, Sep 13, 2019 at 9:44 AM > wrote:



Android (d8 in fact) is able to desugar private access with
nestmates since last April,
your latest backport target 9 which is not supported anymore, so
when you will move the backport to support Java 11,
this tool can become handy.


In fact, jsr166 CVS has already abandoned support for jdk9 (and jdk10) 
so there's absolutely no reason not to apply the suggestions made 
by  Rémi's program.


https://bugs.openjdk.java.net/browse/JDK-8230966

Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

Thank you!

On Fri, Sep 13, 2019 at 9:44 AM  wrote:

>
> Android (d8 in fact) is able to desugar private access with nestmates
> since last April,
> your latest backport target 9 which is not supported anymore, so when you
> will move the backport to support Java 11,
> this tool can become handy.
>

In fact, jsr166 CVS has already abandoned support for jdk9 (and jdk10) so
there's absolutely no reason not to apply the suggestions made by  Rémi's
program.

https://bugs.openjdk.java.net/browse/JDK-8230966

Re: RFR: jsr166 integration 2019-09

[forax]

- Mail original -
> De: "Doug Lea" 
> À: "Remi Forax" , "Martin Buchholz" 
> Cc: "core-libs-dev" , "loom-dev" 
> , "David Holmes"
> 
> Envoyé: Vendredi 13 Septembre 2019 17:19:21
> Objet: Re: RFR: jsr166 integration 2019-09

> Would doing this make anything better enough to outweigh Martin and I
> needing to deal with one more incompatible codebase ?

I believe the answer is the question :)

Android (d8 in fact) is able to desugar private access with nestmates since 
last April,
your latest backport target 9 which is not supported anymore, so when you will 
move the backport to support Java 11,
this tool can become handy. 

> 
> -Doug

Rémi

> 
> On 9/13/19 11:10 AM, fo...@univ-mlv.fr wrote:
>> 
>> 
>> 
>> 
>> *De: *"Martin Buchholz" 
>> *À: *"Remi Forax" 
>> *Cc: *"core-libs-dev" , "loom-dev"
>> , "David Holmes"
>> , "Doug Lea" 
>> *Envoyé: *Jeudi 12 Septembre 2019 16:20:12
>> *Objet: *Re: RFR: jsr166 integration 2019-09
>> 
>> 
>> 
>> On Thu, Sep 12, 2019 at 1:48 AM Remi Forax > <mailto:fo...@univ-mlv.fr>> wrote:
>> 
>> This remember me something,
>> we should refactor most of the the package-private final methods
>> (and the corresponding constructors) at least inside
>> java.lang/java.util to make them private, there is no need to
>> make them package-private anymore given that since Java 11 the
>> compiler emits nestmate attributes instead of generating the
>> method access$XXX.
>> 
>> Maybe i should write a bytecode analyzer for that ?
>> 
>> 
>> Right! Going the other way it was fairly easy to trawl through the
>> generated bytecode using javap looking for "access$".
>> I don't think the nestmates feature is really complete until there
>> is an easy tool to find all the package-private elements accessed
>> only within their nest.
>> 
>> 
>> https://github.com/forax/nestmate-tidyup/blob/master/NestMateTidyUp.java
>> 
>> To reduce the number of dependencies to 0, it uses the jdk internal
>> version of ASM, so the command line is
>>    java --add-exports
>> java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED NestMateTidyUp.java
>> 
>> It crawles java.base to find the fields/methods/constructors that should
>> be declared private.
>> 
>> weirdly, the compiler (javac) also generates some package private fields
>> (those pesky this$0) if the class itself is package private (i believe).
>> 
>> Rémi

Re: RFR: jsr166 integration 2019-09

[Doug Lea]

Would doing this make anything better enough to outweigh Martin and I
needing to deal with one more incompatible codebase?

-Doug

On 9/13/19 11:10 AM, fo...@univ-mlv.fr wrote:
> 
> 
> 
> 
> *De: *"Martin Buchholz" 
> *À: *"Remi Forax" 
> *Cc: *"core-libs-dev" , "loom-dev"
> , "David Holmes"
> , "Doug Lea" 
> *Envoyé: *Jeudi 12 Septembre 2019 16:20:12
> *Objet: *Re: RFR: jsr166 integration 2019-09
> 
> 
> 
> On Thu, Sep 12, 2019 at 1:48 AM Remi Forax  <mailto:fo...@univ-mlv.fr>> wrote:
> 
> This remember me something,
> we should refactor most of the the package-private final methods
> (and the corresponding constructors) at least inside
> java.lang/java.util to make them private, there is no need to
> make them package-private anymore given that since Java 11 the
> compiler emits nestmate attributes instead of generating the
> method access$XXX.
> 
> Maybe i should write a bytecode analyzer for that ?
> 
> 
> Right! Going the other way it was fairly easy to trawl through the
> generated bytecode using javap looking for "access$".
> I don't think the nestmates feature is really complete until there
> is an easy tool to find all the package-private elements accessed
> only within their nest.
> 
> 
> https://github.com/forax/nestmate-tidyup/blob/master/NestMateTidyUp.java
> 
> To reduce the number of dependencies to 0, it uses the jdk internal
> version of ASM, so the command line is
>    java --add-exports
> java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED NestMateTidyUp.java
> 
> It crawles java.base to find the fields/methods/constructors that should
> be declared private.
> 
> weirdly, the compiler (javac) also generates some package private fields
> (those pesky this$0) if the class itself is package private (i believe).
> 
> Rémi
> 

Re: RFR: jsr166 integration 2019-09

[forax]

> De: "Martin Buchholz" 
> À: "Remi Forax" 
> Cc: "core-libs-dev" , "loom-dev"
> , "David Holmes" , "Doug
> Lea" 
> Envoyé: Jeudi 12 Septembre 2019 16:20:12
> Objet: Re: RFR: jsr166 integration 2019-09

> On Thu, Sep 12, 2019 at 1:48 AM Remi Forax < [ mailto:fo...@univ-mlv.fr |
> fo...@univ-mlv.fr ] > wrote:

>> This remember me something,
>> we should refactor most of the the package-private final methods (and the
>> corresponding constructors) at least inside java.lang/java.util to make them
>> private, there is no need to make them package-private anymore given that 
>> since
>> Java 11 the compiler emits nestmate attributes instead of generating the 
>> method
>> access$XXX.

>> Maybe i should write a bytecode analyzer for that ?

> Right! Going the other way it was fairly easy to trawl through the generated
> bytecode using javap looking for "access$".
> I don't think the nestmates feature is really complete until there is an easy
> tool to find all the package-private elements accessed only within their nest.

[ https://github.com/forax/nestmate-tidyup/blob/master/NestMateTidyUp.java | 
https://github.com/forax/nestmate-tidyup/blob/master/NestMateTidyUp.java ] 

To reduce the number of dependencies to 0, it uses the jdk internal version of 
ASM, so the command line is 
java --add-exports java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED 
NestMateTidyUp.java 

It crawles java.base to find the fields/methods/constructors that should be 
declared private. 

weirdly, the compiler (javac) also generates some package private fields (those 
pesky this$0) if the class itself is package private (i believe). 

Rémi 

Re: RFR: jsr166 integration 2019-09

[Martin Buchholz]

On Thu, Sep 12, 2019 at 1:48 AM Remi Forax  wrote:

> This remember me something,
> we should refactor most of the the package-private final methods (and the
> corresponding constructors) at least inside java.lang/java.util to make
> them private, there is no need to make them package-private anymore given
> that since Java 11 the compiler emits nestmate attributes instead of
> generating the method access$XXX.
>
> Maybe i should write a bytecode analyzer for that ?
>

Right! Going the other way it was fairly easy to trawl through the
generated bytecode using javap looking for "access$".
I don't think the nestmates feature is really complete until there is an
easy tool to find all the package-private elements accessed only within
their nest.

Re: RFR: jsr166 integration 2019-09

[Remi Forax]

This remember me something,
we should refactor most of the the package-private final methods (and the 
corresponding constructors) at least inside java.lang/java.util to make them 
private, there is no need to make them package-private anymore given that since 
Java 11 the compiler emits nestmate attributes instead of generating the method 
access$XXX.

Maybe i should write a bytecode analyzer for that ?

Rémi

- Mail original -
> De: "Martin Buchholz" 
> À: "core-libs-dev" , "loom-dev" 
> , "David Holmes"
> 
> Envoyé: Lundi 9 Septembre 2019 17:06:28
> Objet: RFR: jsr166 integration 2019-09

> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html
> 
> 8229442: AQS and lock classes refresh
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/lock-classes-refresh/index.html
> https://bugs.openjdk.java.net/browse/JDK-8229442
> 
> Lock classes refresh is a small step towards loom.
> Loom folk, how about adding Doug and I as Loom Project Committers, even
> though we have no concrete plans to submit to the loom repo yet?
> 
> Another round of stress testing at Google allowed us to fix a handful of
> very rare test failures.
> 
> 8227235: rare failures in testForkHelpQuiesce tck tests
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ForkJoin-quiesce/index.html
> https://bugs.openjdk.java.net/browse/JDK-8227235
> 
> 8221168: java/util/concurrent/CountDownLatch/Basic.java fails
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CountDownLatch-Basic/index.html
> https://bugs.openjdk.java.net/browse/JDK-8221168
> 
> 8145138: CyclicBarrier/Basic.java failed with "3 not equal to 4"
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CyclicBarrier-Basic/index.html
> https://bugs.openjdk.java.net/browse/JDK-8145138
> 
> 8225490: Miscellaneous changes imported from jsr166 CVS 2019-09
> https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html
> https://bugs.openjdk.java.net/browse/JDK-8225490

Re: RFR: jsr166 integration 2019-09

[Alan Bateman]

On 09/09/2019 16:06, Martin Buchholz wrote:

:

Another round of stress testing at Google allowed us to fix a handful of
very rare test failures.

8227235: rare failures in testForkHelpQuiesce tck tests
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ForkJoin-quiesce/index.html
https://bugs.openjdk.java.net/browse/JDK-8227235

8221168: java/util/concurrent/CountDownLatch/Basic.java fails
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CountDownLatch-Basic/index.html
https://bugs.openjdk.java.net/browse/JDK-8221168

8145138: CyclicBarrier/Basic.java failed with "3 not equal to 4"
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CyclicBarrier-Basic/index.html
https://bugs.openjdk.java.net/browse/JDK-8145138

8225490: Miscellaneous changes imported from jsr166 CVS 2019-09
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html
https://bugs.openjdk.java.net/browse/JDK-8225490
I read through the test updates and other misc. changes and don't see 
anything obviously wrong (the discussion in JIRA on the test failures 
was useful). At some point I could imagine Joe suggesting add "@key 
randomness" to more of these tests.


-Alan.

RFR: jsr166 integration 2019-09

[Martin Buchholz]

https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html

8229442: AQS and lock classes refresh
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/lock-classes-refresh/index.html
https://bugs.openjdk.java.net/browse/JDK-8229442

Lock classes refresh is a small step towards loom.
Loom folk, how about adding Doug and I as Loom Project Committers, even
though we have no concrete plans to submit to the loom repo yet?

Another round of stress testing at Google allowed us to fix a handful of
very rare test failures.

8227235: rare failures in testForkHelpQuiesce tck tests
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ForkJoin-quiesce/index.html
https://bugs.openjdk.java.net/browse/JDK-8227235

8221168: java/util/concurrent/CountDownLatch/Basic.java fails
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CountDownLatch-Basic/index.html
https://bugs.openjdk.java.net/browse/JDK-8221168

8145138: CyclicBarrier/Basic.java failed with "3 not equal to 4"
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/CyclicBarrier-Basic/index.html
https://bugs.openjdk.java.net/browse/JDK-8145138

8225490: Miscellaneous changes imported from jsr166 CVS 2019-09
https://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html
https://bugs.openjdk.java.net/browse/JDK-8225490