Improve timezone mapping for AIX platform

[Jonathan Lu]

Hi ppc-aix-port-dev, core-libs-dev,

Here's a patch for JDK-8034220,

http://cr.openjdk.java.net/~luchsh/JDK-8034220/

It is trying to add the a more complete timezone mapping mechanism for AIX
platform.
the changes are primarily based on IBM's commercial JDK code, which
includes:

- A new timezone mapping file added under directory jdk/src/aix/lib/
- Code to parse above config file, changed file is
src/solaris/native/java/util/TimeZone_md.c
- And also makefile change in make/CopyFiles.gmk to copy the config file

Could you pls help to review and give comments ?

Cheers
- Jonathan

Re: Implicit 'this' return for void methods

[David Holmes]

On 26/03/2014 6:19 AM, Victor Polischuk wrote:

Good day, everyone,

I have a question regarding a proposal I found some time ago: 
http://tech.puredanger.com/java7#chained.


More direct original link:

http://www.mernst.org/blog/rss.xml#AModestLanguageProposal


In a word, it is all about replacing 'void' behavior to always return 'this' 
and I think for static methods it would be nice to return Class instance. I 
have not found any mentions in JEPs and just wonder if the idea is rejected or 
still somewhere in pipeline for Java 1.x. (or perhaps it is just waiting to be 
filed by someone in brand new JEP).


I don't think this has gone beyond a blog post from 2007. I don't recall 
any attempt at turning this into a JEP but perhaps it was discussed as 
part of Project Coin for Java 7?


David



---
Regards,
Victor Polischuk

Re: 8020860: cluster Hashtable/Vector field updates for better transactional memory behaviour

[David Holmes]

Mike,

On 26/03/2014 6:37 AM, Mike Duigou wrote:

Hello all;

Recently HotSpot gained additional support for transactional memory, 
. This patch is a libraries 
followon to that change. RTM and other transactional memory implementations benefit 
from clustering writes towards the end of the transaction whenever possible. This 
change optimizes the behaviour of two collection classes, Hashtable and Vector by 
moving several field updates to cluster them together near the end of the 
transaction. Yes, we know, these are obsolete collections but these two classes were 
used as the basis for the original benchmarking and evaluation during the development 
of the transactional memory JVM features. Future changes providing similar 
optimizations to other collections will be pursued when it can be shown they offer 
value and don't add a cost to non TM performance (TM is not yet a mainstream feature).

https://bugs.openjdk.java.net/browse/JDK-8020860
http://cr.openjdk.java.net/~mduigou/JDK-8020860/0/webrev/

It is not expected that this change will have any meaningful impact upon 
performance (either positive or negative) outside of TM-enabled configurations. 
The main change is to move existing field updates towards the end of the 
transaction and avoid conditionals between field updates.

There is a slight behaviour change introduced in this changeset. Previously 
some methods updated the modcount unconditionally updated even when an 
ArrayIndexOutOfBoundsException was subsequently thrown for an invalid index and 
the Vector was not modified. With this change the modcount will only be updated 
if the Vector is actually changed. It is not expected that applications will 
have relied or should have relied on this behaviour.


I could live with that change in behaviour, but this change completely 
breaks the fail-fast semantics of the iterators in some cases! If you 
don't update modCount until after the change is complete, the iterator 
may access the updated state and not throw CME!.


I think this change is misguided.

David
-



Mike

Re: Process API Updates (JEP 102)

[Roger Riggs]

Hi Florian,

On 3/25/14 5:18 AM, Florian Weimer wrote:

On 03/24/2014 10:11 PM, roger riggs wrote:


 From the scope of the JEP, a fairly simple API seems sufficient.
  - Enumerate the direct children
  - The rest of the functions are similar to Process
- to terminate a process, forcibly and normally
- to destroy a process and all of its children recursively
- to check if one is alive
- to waitFor for termination and retrieve the exit status


On Linux, I think you need to use obscure mechanisms to make this 
iteration reliable (cgroups perhaps).  There is no openpid() or 
similar mechanism that allows you to obtain a stable handle.
Given that processes are asynchronous in creation and termination, I 
don't think
'reliable' listing is possible.  In a pathological case, processes can 
be created

as fast/faster than they can be terminated.


Somewhat relatedly, it would make sense to make Process 
auto-closeable, so that you can make sure that no non-Java resources 
are retained. Right now, this seems to require calling 
destroyForcibly().waitFor().

That's worth looking at a close() method could have that compound behavior.


It would be nice if there was a way to directly pipe the output of one 
process to the input of another process.  I think that even with Java 
8, this still needs an explicit copy loop.
That's similar to other requests to directly feed or consume from the 
subprocess from

a file or pipe.

Thanks, Roger

Re: Process API Updates (JEP 102)

[Roger Riggs]

Hi Peter,

On 3/25/14 2:50 AM, Peter Levart wrote:

On 03/24/2014 10:11 PM, roger riggs wrote:

Hi,

I'm starting to work on JEP 102, Process API Updates for JDK 9.
The use cases identified include test execution and build systems such
as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
to monitor the health of a more complex system where the processes
are not spawned by the same manager.

The current API of Process itself is pretty complete with the 
addition of a getPid
method to enable identification of subprocesses spawned by the 
application
and allow external identification.  It will not be possible to 
intercept the input,

output and error streams of an arbitrary process.

From the scope of the JEP, a fairly simple API seems sufficient.
 - Enumerate the direct children


What about self? An application might want to communicate it's own PID 
to some other process (write it into a file, for example). Other 
Process API methods are not sensible for "self" process so this might 
as well be just simple static getSelfPid() method.

yes, I assume getPid() for the current process and Process.getPid().



 - The rest of the functions are similar to Process
   - to terminate a process, forcibly and normally
   - to destroy a process and all of its children recursively
   - to check if one is alive
   - to waitFor for termination and retrieve the exit status

Are there use cases for which this is insufficient?  Please comment.


It would be nice if this new API interoperated with old Process API in 
a way that using both interchangeably for the same child process would 
be possible. For example: spawning a child with old Process API and 
enumerating/waiting/checking it with new API in some thread while 
waiting for termination using old API in another thread...

Why make it more complex.  What's the value?
The current Process object is a capability, no one can control it unless 
they have
the instance. Security is based on the permission to execute the 
binary.  SM.checkExec().


Having an interface with the common function may be useful but the 
usefulness

of the polymorphism needs to be validated.

I wonder if an approach to just add getPid() (and maybe some other 
method) to old Process class together with some static methods to 
enumerate child processes would be possible?
That makes sense to extend the current API but is not sufficient for 
processes
not spawned by Process.  The enumeration of the children would not be 
Processes,

a different type of process handle is needed.



It would just require the API specification to strengthen it's 
guarantees on some methods: for example destroyForcibly() would 
guarantee forcible termination. It might require an additional method 
like destroyNicely() or terminateNicely() which would guarantee a kind 
of termination where victim process could clean-up. The specification 
already allows getInputStream()/getOutputStream()/getErrorStream() to 
return special "null streams" when spawned process IO is redirected 
to/from file. The enumeration methods could simply return Process 
object which would:


- be same instances of Process objects created with ProcessBuilder API 
for processes spawned by the Process API. This would require an 
internal registry of live Process objects. We already have reaper 
threads which could be used to clean-up this registry on process 
termination.
- be new Process objects for child processes spawned in some other way 
(or by children) and which would return special "null streams" from 
their getInputStream()/getOutputStream()/getErrorStream() methods. 
These Process objects could be instantiated lazily on first request 
and then registered in the same internal registry. They might still 
require reaper threads to enable waiting on them from multiple Java 
threads and cleaning-up registry, but they could probably be created 
lazily.


The worst thing is to have an entirely separate API that does not 
interoperate with old API. I think the old API is not so bad. It just 
lacks some features and it's implementation could be revamped a bit.
I'm quite leery of just glombing on new functions to Process especially 
those forcing
the Process API semantics to be weakened and diluted; it is likely to 
make the implementation
even more complex and hard to maintain.  I think there are distinct 
functions for

managing non-spawned processes.

There are also security issues to consider,  an application that might 
have permission
to launch and control specific other programs should not be able 
(without other permissions)

to control those processes children or other processes.

Roger



What do you think?

Regards, Peter



Thanks, Roger

Re: ProcessReaper: single thread reaper

[Roger Riggs]

No, in the prototype only.

The current shipping implementation dedicates a thread to each Process 
and waits

only for a specific pid to exit.

Roger

On 3/25/14 3:50 PM, Martin Buchholz wrote:


Roger, are you saying there's a race in the currently shipping 
implementation?


---

I believe It's possible to use ManagedBlocker to run tasks on FJ threads.

But the current reaper thread pool is probably better because of the 
small stack sizes and there's no benefit running these reaper tasks 
that do nothing but wait on the common FJ pool.

ok, will look at that again.

Re: 8020860: cluster Hashtable/Vector field updates for better transactional memory behaviour

[Remi Forax]

On 03/25/2014 09:37 PM, Mike Duigou wrote:

Hello all;

Recently HotSpot gained additional support for transactional memory, 
. This patch is a libraries 
followon to that change. RTM and other transactional memory implementations benefit 
from clustering writes towards the end of the transaction whenever possible. This 
change optimizes the behaviour of two collection classes, Hashtable and Vector by 
moving several field updates to cluster them together near the end of the 
transaction. Yes, we know, these are obsolete collections but these two classes were 
used as the basis for the original benchmarking and evaluation during the development 
of the transactional memory JVM features. Future changes providing similar 
optimizations to other collections will be pursued when it can be shown they offer 
value and don't add a cost to non TM performance (TM is not yet a mainstream feature).

https://bugs.openjdk.java.net/browse/JDK-8020860
http://cr.openjdk.java.net/~mduigou/JDK-8020860/0/webrev/

It is not expected that this change will have any meaningful impact upon 
performance (either positive or negative) outside of TM-enabled configurations. 
The main change is to move existing field updates towards the end of the 
transaction and avoid conditionals between field updates.

There is a slight behaviour change introduced in this changeset. Previously 
some methods updated the modcount unconditionally updated even when an 
ArrayIndexOutOfBoundsException was subsequently thrown for an invalid index and 
the Vector was not modified. With this change the modcount will only be updated 
if the Vector is actually changed. It is not expected that applications will 
have relied or should have relied on this behaviour.

Mike


Hi Mike,
I don't like the principle behind this patch.
Your changing the behavior because it's better with the current state of 
the implementation of the RTM.
First, as Martin said, you need a very good reason to do that and it's a 
kind of black magic change, when the RTM code will change you will have 
to update the JDK code too.


It's better that the VM cluster the write itself by pushing the writes 
at the end of every code path if RTM is enabled
(I don't say that this is easy to implement that but it will benefit to 
more code).


regards,
Rémi

Re: ProcessReaper: single thread reaper

[Roger Riggs]

Hi Peter,

The stream draining code was added in 2011 to address resource exhaustion
due to poor hygiene.

 * JDK-6944584 
   Improvements to subprocess handling on Unix

Moves the resource 'leak' from the OS kernel/process space to the heap
and allows the process to be cleaned up sooner.

Roger


On 3/25/14 9:50 AM, Peter Levart wrote:

On 03/25/2014 05:25 PM, Roger Riggs wrote:

Hi Peter,

Thanks for the comments, I'll fix the races conditions, I intended
to go back to the dedicated thread but wanted to check for scalability
in the case of a large number of threads and non-trivial termination 
overheads

where the streams had to be drained.



Why does stream need to be drained right away (in reaper thread)? 
Isn't it enough for child pid to be waited upon and let the client of 
the stream pull from the stream until it reaches the EOF or not if it 
doesn't need the data... Is this just to be able to reclaim the file 
descriptor of the pipe right away? Wouldn't FileInputStream's close() 
method do the job (or finalize() if user forgets to call close())? 
There would be a native resource leakage if user forgets to call 
close() AND retains the reference to InputStream, but this is the same 
with all normal streams like FileInputStream...


Regards, Peter


Thanks, Roger



On 3/25/14 7:05 AM, Peter Levart wrote:

On 03/24/2014 10:05 PM, roger riggs wrote:

Hi Rob, Martin, et.al.

I've prototyped (for 9) a thread reaper[1] that uses a single 
thread to wait for exiting

processes and calling back to the process with the exit status.
the interesting part is getting the exit status back to the Process 
that needs it

It needs more testing and hardening.

I had not considered using a signal handler for SIGCHLD but that's 
an option,

though we need to be very careful about thread usage.

Roger

p.s. comments on the single thread reaper appreciated (in a new thread)
[1] http://cr.openjdk.java.net/~rriggs/webrev-waitpid/



Hi Roger,

I think I found a little race. Suppose reaper task is still alive 
and that all consumers have been serviced (consumerCount is 0). The 
reaper task waits for 500 millis for next consumer to be registered, 
but times out. Before calling "reaperThread.release()", new consumer 
comes around and registers itself, also calling runReaper(), but 
since reaperThread.release() has not yet been called by old reaper 
task, new reaper task is not submitted to commonPool. The old reaper 
task finishes, leaving one consumer on the waitingList with no 
reaper task to service it. If no new consumers get registered, the 
waiting consumer will never be notified...


The simplest solution for this race, I think, would be to have a 
dedicated long-running thread. It could be spawned lazily, but then 
it would never finish.


Otherwise a nice solution with two lists (exitList/waitList) and 
avoidance of race with reversed orders between
- consumer registration: register on waitList 1st then check 
exitList, and

- exit event dispatch: register on exitList 1st then check waitList

...but the check you use with consumeCount local variable to detect 
processes spawned by other means (for purposes of logging only) has 
a race:


thread1: Suppose a new consumer is being registered with 
onExitCall(...), is added on the waitList, but before checking 
exitList().size() and iterating the exitList, ...
thread2: the reaper task detects that the very same process has 
finished (gets it's pid from waitpid) and adds it's pid to exitList. 
Then before iterating waitList, ...
thread1: iterates the exitList, finds a match and consumes the pid, 
removing the matching entries from both exitList and waitList. Then ...
thread2: the reaper task iterates waitList, doesn't find a matching 
entry for exitPid, doesn't increment consumeCount and voila: debug 
log("Unexpected process exit for pid:...").



That's enough races for today.

Regards, Peter




On 3/24/2014 12:38 AM, Rob McKenna wrote:

Hi folks,

Roger Riggs (cc'd) may want to chip in here as he's looking at the 
reaper thread arrangement in 9 at the moment.


On another note, I too support the merging of those files. I 
didn't think there was much appetite for it at the time so I must 
admit this fell down my todo list. Looking at this bug did remind 
me that its something worth trying though. As per Alan's mail, I'm 
going to tackle it separately if you folks don't mind. I'll have a 
look at Peter's changes (thanks Peter!) as soon as I can and see 
about getting them in.


-Rob

On 23/03/14 22:30, Martin Buchholz wrote:




On Sun, Mar 23, 2014 at 2:34 AM, Martin Buchholz 
mailto:marti...@google.com>> wrote:



We have also thought about whether having reaper threads is
necessary.  The Unix rule is that child processes should be
waited for, and some thread needs to do that.  There's no way
to wait for a set of child pids, or to specify a "completion
handler".  Well, you might be able to get

Re: ProcessReaper: single thread reaper

[Roger Riggs]

Hi Martin,

Two cases, one current and one future.

In the current case, Process can spawn a process and the process
can exit before Process can register the callback to get the exitValue.
Peter pointed out this race in his comments.
The exitValue needs to be saved (for some time yet to be determined)
to allow the Process to register and get its callback with the exitValue.

The second case is future looking to the case where a child process
not spawned by Process is exiting.  It might be due to a child being
inherited from a dieing child or due to some different subprocess launcher.

When the JEP 102 process work happens, it should be possible to
wait or get called back for those non-spawned processes.

The single (smaller) number of threads has been requested to handle
processes that control a large number of children.  It could be from
a thread pool, either dedicated or common.  The common threadpool
does not expect its tasks to hang for an indefinite period as might
occur when waiting for along running process to exit.

Thanks, Roger


On 3/25/14 2:39 PM, Martin Buchholz wrote:

What happens if the pid you get back is a subprocess not created by 
ProcessBuilder?
+pid = waitpid(-1, &exitValue, 0);
---

What is the advantage of having a single thread?  Are you just trying 
to save threads?  The cost of the reaper threads is much lower than 
usual because of the small stack size.




On Tue, Mar 25, 2014 at 7:05 AM, Peter Levart > wrote:


On 03/24/2014 10:05 PM, roger riggs wrote:

Hi Rob, Martin, et.al .

I've prototyped (for 9) a thread reaper[1] that uses a single
thread to wait for exiting
processes and calling back to the process with the exit status.
the interesting part is getting the exit status back to the
Process that needs it
It needs more testing and hardening.

I had not considered using a signal handler for SIGCHLD but
that's an option,
though we need to be very careful about thread usage.

Roger

p.s. comments on the single thread reaper appreciated (in a new
thread)
[1] http://cr.openjdk.java.net/~rriggs/webrev-waitpid/




Hi Roger,

I think I found a little race. Suppose reaper task is still alive
and that all consumers have been serviced (consumerCount is 0).
The reaper task waits for 500 millis for next consumer to be
registered, but times out. Before calling
"reaperThread.release()", new consumer comes around and registers
itself, also calling runReaper(), but since reaperThread.release()
has not yet been called by old reaper task, new reaper task is not
submitted to commonPool. The old reaper task finishes, leaving one
consumer on the waitingList with no reaper task to service it. If
no new consumers get registered, the waiting consumer will never
be notified...

The simplest solution for this race, I think, would be to have a
dedicated long-running thread. It could be spawned lazily, but
then it would never finish.

Otherwise a nice solution with two lists (exitList/waitList) and
avoidance of race with reversed orders between
- consumer registration: register on waitList 1st then check
exitList, and
- exit event dispatch: register on exitList 1st then check waitList

...but the check you use with consumeCount local variable to
detect processes spawned by other means (for purposes of logging
only) has a race:

thread1: Suppose a new consumer is being registered with
onExitCall(...), is added on the waitList, but before checking
exitList().size() and iterating the exitList, ...
thread2: the reaper task detects that the very same process has
finished (gets it's pid from waitpid) and adds it's pid to
exitList. Then before iterating waitList, ...
thread1: iterates the exitList, finds a match and consumes the
pid, removing the matching entries from both exitList and
waitList. Then ...
thread2: the reaper task iterates waitList, doesn't find a
matching entry for exitPid, doesn't increment consumeCount and
voila: debug log("Unexpected process exit for pid:...").


That's enough races for today.

Regards, Peter




On 3/24/2014 12:38 AM, Rob McKenna wrote:

Hi folks,

Roger Riggs (cc'd) may want to chip in here as he's looking at
the reaper thread arrangement in 9 at the moment.

On another note, I too support the merging of those files. I
didn't think there was much appetite for it at the time so I
must admit this fell down my todo list. Looking at this bug did
remind me that its something worth trying though. As per Alan's
mail, I'm going to tackle it separately if you folks don't mind.
I'll have a look at Peter's changes (thanks Peter!) as soon as I
can and see about getting them in.

-Rob

On 23/03/14 22:30, Martin Buchho

Re: Review request for 8038177 Eliminate unnecessary dependency to sun.security.action

[Alan Bateman]

On 25/03/2014 20:41, Mandy Chung wrote:

:

The ExtendedCharsets.init method has a check to ensure that the VM has 
been booted.  Sherman can confirm that.


1170 protected void init() {
1171 if (initialized)
1172 return;
1173 if (!sun.misc.VM.isBooted())
1174 return;

Thanks, that covers my concern.

-Alan.

Re: Review request for 8038177 Eliminate unnecessary dependency to sun.security.action

[Mandy Chung]

On 3/24/2014 8:15 AM, Alan Bateman wrote:

On 21/03/2014 21:28, Mandy Chung wrote:
This is the second patch to eliminate the dependencies to 
sun.security.action:

   https://bugs.openjdk.java.net/browse/JDK-8038177

Webrev at:
http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8038177/webrev.00/

It looks good to me too. The only one that I wonder about is 
ExtendedCharsets and whether this might run early in the startup on 
some locales.


The ExtendedCharsets.init method has a check to ensure that the VM has 
been booted.  Sherman can confirm that.


1170 protected void init() {
1171 if (initialized)
1172 return;
1173 if (!sun.misc.VM.isBooted())
1174 return;

Mandy

8020860: cluster Hashtable/Vector field updates for better transactional memory behaviour

[Mike Duigou]

Hello all;

Recently HotSpot gained additional support for transactional memory, 
. This patch is a libraries 
followon to that change. RTM and other transactional memory implementations 
benefit from clustering writes towards the end of the transaction whenever 
possible. This change optimizes the behaviour of two collection classes, 
Hashtable and Vector by moving several field updates to cluster them together 
near the end of the transaction. Yes, we know, these are obsolete collections 
but these two classes were used as the basis for the original benchmarking and 
evaluation during the development of the transactional memory JVM features. 
Future changes providing similar optimizations to other collections will be 
pursued when it can be shown they offer value and don't add a cost to non TM 
performance (TM is not yet a mainstream feature).

https://bugs.openjdk.java.net/browse/JDK-8020860
http://cr.openjdk.java.net/~mduigou/JDK-8020860/0/webrev/

It is not expected that this change will have any meaningful impact upon 
performance (either positive or negative) outside of TM-enabled configurations. 
The main change is to move existing field updates towards the end of the 
transaction and avoid conditionals between field updates.

There is a slight behaviour change introduced in this changeset. Previously 
some methods updated the modcount unconditionally updated even when an 
ArrayIndexOutOfBoundsException was subsequently thrown for an invalid index and 
the Vector was not modified. With this change the modcount will only be updated 
if the Vector is actually changed. It is not expected that applications will 
have relied or should have relied on this behaviour.

Mike

Re: RFR (L) 8037210: Get rid of char-based descriptions 'J' of basic types

[Christian Thalinger]

+ enum BasicType {
+ L_TYPE('L', Object.class, Wrapper.OBJECT),  // all reference types
+ I_TYPE('I', int.class,Wrapper.INT),
+ J_TYPE('J', long.class,   Wrapper.LONG),
+ F_TYPE('F', float.class,  Wrapper.FLOAT),
+ D_TYPE('D', double.class, Wrapper.DOUBLE),  // all primitive types
+ V_TYPE('V', void.class,   Wrapper.VOID);// not valid in all 
contexts

I would suggest to not name them X_TYPE but give them meaningful names like 
Int, Float, Void.  The enum BasicType already infers that it’s a type.  If you 
think it’s not clear that it’s a type just use BasicType.Double in that places.

On Mar 24, 2014, at 9:29 AM, Vladimir Ivanov  
wrote:

> Updated version:
> http://cr.openjdk.java.net/~vlivanov/8037210/webrev.03/
> 
> - changed the way how arrays of types are created:
>static final BasicType[] ALL_TYPES = BasicType.values();
>static final BasicType[] ARG_TYPES = Arrays.copyOf(ALL_TYPES, 
> ALL_TYPES.length-1);
> 
> - added a test for BasicType (LambdaFormTest.testBasicType).
> 
> Best regards,
> Vladimir Ivanov
> 
> On 3/22/14 2:08 AM, Vladimir Ivanov wrote:
>> John, thanks for the feedback.
>> 
>> Updated webrev:
>> http://cr.openjdk.java.net/~vlivanov/8037210/webrev.02
>> 
>> Also moved LambdaForm.testShortenSignature() into a stand-alone unit test.
>> 
>> Best regards,
>> Vladimir Ivanov
>> 
>> On 3/21/14 10:54 PM, John Rose wrote:
>>> On Mar 21, 2014, at 8:49 AM, Vladimir Ivanov
>>> mailto:vladimir.x.iva...@oracle.com>>
>>> wrote:
>>> 
 Thanks for the feedback.
 
 What do you think about the following:
 http://cr.openjdk.java.net/~vlivanov/8037210/webrev.01/
>>> 
>>> That looks nice.  Strong typing; who woulda' thunk it.  :-)
>>> 
>>> The uses of ".ordinal()" are the extra cost relative to using just small
>>> integers.  They seem totally reasonable in the code.
>>> 
>>> I suggest either wrapping "ordinal" as something like "id" or else
>>> changing temp names like "id" to "ord", to reinforce the use of a common
>>> name.
>>> 
>>> Don't make the enum class public.  And especially don't make the mutable
>>> arrays public:
>>> 
>>> +public static final BasicType[] ALL_TYPES = { L_TYPE, I_TYPE,
>>> J_TYPE, F_TYPE, D_TYPE, V_TYPE };
>>> +public static final BasicType[] ARG_TYPES = { L_TYPE, I_TYPE,
>>> J_TYPE, F_TYPE, D_TYPE };
>>> 
>>> — John
>>> 
>>> P.S.  That would only be safe if we had (what we don't yet) a notion of
>>> frozen arrays like:
>>> 
>>> +public static final BasicType final[] ALL_TYPES = { L_TYPE,
>>> I_TYPE, J_TYPE, F_TYPE, D_TYPE, V_TYPE };
>>> +public static final BasicType final[] ARG_TYPES = { L_TYPE,
>>> I_TYPE, J_TYPE, F_TYPE, D_TYPE };
>>> 
>>> 
>>> 
>>> ___
>>> mlvm-dev mailing list
>>> mlvm-...@openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> 
> ___
> mlvm-dev mailing list
> mlvm-...@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Implicit 'this' return for void methods

[Victor Polischuk]

Good day, everyone,

I have a question regarding a proposal I found some time ago: 
http://tech.puredanger.com/java7#chained.

In a word, it is all about replacing 'void' behavior to always return 'this' 
and I think for static methods it would be nice to return Class instance. I 
have not found any mentions in JEPs and just wonder if the idea is rejected or 
still somewhere in pipeline for Java 1.x. (or perhaps it is just waiting to be 
filed by someone in brand new JEP).

---
Regards, 
Victor Polischuk

Re: RFR(S): 8034960: Serviceability tests using @library failing with java.lang.NoClassDefFoundError

[Jonathan Gibbons]

On 03/25/2014 07:31 AM, Yekaterina Kantserova wrote:

On 03/25/2014 02:53 PM, Alan Bateman wrote:

On 25/03/2014 12:14, Staffan Larsen wrote:

I’ve looked at a random sample of these changes and they look ok.

Since some of the changes are in non-serviceability code I have also 
added core-libs to the review thread.


I’m sure you know this, but for the record: please don’t push this 
until jtreg with the fix has been promoted.


Thanks,
/Staffan
Katja - the changes look okay but I wonder about tests that didn't 
previously have @build or @compile tags. Take TestJcmdDefaults as an 
example, it now has:


@build jdk.testlibrary.*

and I just wonder if it should be:

@build TestJcmdDefaults jdk.testlibrary.*

so that it compiles the test in addition to the test library. This 
could be a misunderstanding on my part of course, maybe the source 
file with @test is always compiled if it's a java test?


My assumption is the test itself is always compiled. If it's false, 
I'm afraid we need to change a bunch of other tests.


Jon, could you please explain?

// Katja




-Alan




There is no such thing as "the source file with @test".   Those words 
don't make sense.


The class specified in an "@run main" directive is subject to an 
implicit "@build" meaning that it will be compiled if needed.


-- Jon

Re: Process API Updates (JEP 102)

[Mark Sheppard]

Hi Roger,
   I have a few questions for your consideration, hopefully they make 
sense!


Is this API to be available to the plug-in/applet? Even with appropriate 
permissions?

What permissions are required to invoke operations on the API?

Do you need to differentiate between "spawning" a Java process and a 
process that will execute

a non Java executable?

Is this part of the model?

If a Java subprocess is created should it be "instrumented" with a 
remote control interface object

to enable "fine grained" process control?

Is the current Process abstraction to remain or be depecrated?

If it remains why not extend its API rather introduce a second abtraction?

Will the Runtime and ProcessBuilder be adapted to new ProcessHandle 
abstraction?

i.e.they will return ProcessHandle?

Why not refactor the Process abstraction to be a final class rather than 
an abstract class

as part of this work?

ProcessHandle.children() returns a Stream, why not return 
a Collection and the Stream can be

obained from the Collection?

What will the children method return when invoked on the ProcessHandle 
of a subprocess?
  the subprocesses may, in turn, have subprocesses but they will not be 
known within

the context of the parent.

What should be the security and permission model for invoking a destroy 
operation?


What about encapsulating additional information in ProcessHandle, e.g. 
uid or owner,

start time, process name, target executable

If it is possible to invoke
ProcessHandle somePidPh = ProcessHandle.get(somePid), where pid is an 
arbitrary value

then what will happen
if an attempt is made to subsequently invoke 
somePidPh.destroyForcibly(). somePidPh is not

a sub-process of the invoker. should it just silently fail?
Or to re-phrase, how will the OS level restrictions on killing a process 
be encapsulated in the

ProcessHandle model?

Are the ProcessHandles of all processes created within a single JVM 
available to all threads with
that JVM, or is there to be any levels of isolation and visibility 
imposed within the calling JVM?


WRT waitFor() method could take "callback argument" which is invoked 
when the sub-processed is finally reaped?


regards
Mark
On 24/03/2014 21:11, roger riggs wrote:

Hi,

I'm starting to work on JEP 102, Process API Updates for JDK 9.
The use cases identified include test execution and build systems such
as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
to monitor the health of a more complex system where the processes
are not spawned by the same manager.

The current API of Process itself is pretty complete with the addition 
of a getPid
method to enable identification of subprocesses spawned by the 
application
and allow external identification.  It will not be possible to 
intercept the input,

output and error streams of an arbitrary process.

From the scope of the JEP, a fairly simple API seems sufficient.
 - Enumerate the direct children
 - The rest of the functions are similar to Process
   - to terminate a process, forcibly and normally
   - to destroy a process and all of its children recursively
   - to check if one is alive
   - to waitFor for termination and retrieve the exit status

Are there use cases for which this is insufficient?  Please comment.

Thanks, Roger

Re: Experimental patches for unsafe cleanup Re: The s.m.Unsafe representation in hotspot and method registration

[Paul Sandoz]

On Mar 25, 2014, at 2:37 PM, Staffan Larsen  wrote:

> (Adding hotspot-runtime-dev since some changes are in hotspot code).
> 
> 
> On 25 mar 2014, at 14:20, Paul Sandoz  wrote:
> 
>> 
>> On Mar 25, 2014, at 9:34 AM, Paul Sandoz  wrote:
>> 
>>> 
>>> On Mar 24, 2014, at 7:49 PM, Staffan Larsen  
>>> wrote:
>>> 
 We have abandoned the HSX model. From JDK 8 one version of Hotspot will be 
 tied to one version of the JDK. This looks like old code that has not been 
 cleaned up.
 
>>> 
>>> Thanks, yes, looks like we can clean this up and also remove the deprecated 
>>> methods at the same time.
>>> 
>> 
>> Here are some preliminary webrevs to clean up unsafe and also remove all 
>> old, deprecated and monitor-related methods:
>> 
>> http://cr.openjdk.java.net/~psandoz/jdk9/jdk-unsafe-cleanup/webrev/jdk.patch
>> 
>> http://cr.openjdk.java.net/~psandoz/jdk9/hotspot-unsafe-cleanup/webrev/
>> 
>> Passes all tests i have thrown at it locally.
>> 
>> There is some additional clean up that could be done to remove prefetch 
>> read/write intrinsics hooked up to older versions of Unsafe:
>> 
>> do_intrinsic(_prefetchRead, sun_misc_Unsafe,
>> prefetchRead_name, prefetch_signature, F_RN)  \
>>  do_name( prefetchRead_name,   
>> "prefetchRead")\
>> do_intrinsic(_prefetchWrite,sun_misc_Unsafe,
>> prefetchWrite_name, prefetch_signature,F_RN)  \
>>  do_name( prefetchWrite_name,  
>> "prefetchWrite")   \
>> do_intrinsic(_prefetchReadStatic,   sun_misc_Unsafe,
>> prefetchReadStatic_name, prefetch_signature,   F_SN)  \
>>  do_name( prefetchReadStatic_name, 
>> "prefetchReadStatic")  \
>> do_intrinsic(_prefetchWriteStatic,  sun_misc_Unsafe,
>> prefetchWriteStatic_name, prefetch_signature,  F_SN)  \
>>  do_name( prefetchWriteStatic_name,
>> "prefetchWriteStatic") \
>> 
>> That will also require updates in various other files. I presume these 
>> prefetch methods are never used throughout the runtime?
>> 

Here is that removal:

  
http://cr.openjdk.java.net/~psandoz/jdk9/hotspot-unsafe-prefetch-cleanup/webrev/

Again all tests i threw at it passed.

Paul.

Re: ProcessReaper: single thread reaper

[Peter Levart]

On 03/25/2014 05:25 PM, Roger Riggs wrote:

Hi Peter,

Thanks for the comments, I'll fix the races conditions, I intended
to go back to the dedicated thread but wanted to check for scalability
in the case of a large number of threads and non-trivial termination 
overheads

where the streams had to be drained.



Why does stream need to be drained right away (in reaper thread)? Isn't 
it enough for child pid to be waited upon and let the client of the 
stream pull from the stream until it reaches the EOF or not if it 
doesn't need the data... Is this just to be able to reclaim the file 
descriptor of the pipe right away? Wouldn't FileInputStream's close() 
method do the job (or finalize() if user forgets to call close())? There 
would be a native resource leakage if user forgets to call close() AND 
retains the reference to InputStream, but this is the same with all 
normal streams like FileInputStream...


Regards, Peter


Thanks, Roger



On 3/25/14 7:05 AM, Peter Levart wrote:

On 03/24/2014 10:05 PM, roger riggs wrote:

Hi Rob, Martin, et.al.

I've prototyped (for 9) a thread reaper[1] that uses a single thread 
to wait for exiting

processes and calling back to the process with the exit status.
the interesting part is getting the exit status back to the Process 
that needs it

It needs more testing and hardening.

I had not considered using a signal handler for SIGCHLD but that's 
an option,

though we need to be very careful about thread usage.

Roger

p.s. comments on the single thread reaper appreciated (in a new thread)
[1] http://cr.openjdk.java.net/~rriggs/webrev-waitpid/



Hi Roger,

I think I found a little race. Suppose reaper task is still alive and 
that all consumers have been serviced (consumerCount is 0). The 
reaper task waits for 500 millis for next consumer to be registered, 
but times out. Before calling "reaperThread.release()", new consumer 
comes around and registers itself, also calling runReaper(), but 
since reaperThread.release() has not yet been called by old reaper 
task, new reaper task is not submitted to commonPool. The old reaper 
task finishes, leaving one consumer on the waitingList with no reaper 
task to service it. If no new consumers get registered, the waiting 
consumer will never be notified...


The simplest solution for this race, I think, would be to have a 
dedicated long-running thread. It could be spawned lazily, but then 
it would never finish.


Otherwise a nice solution with two lists (exitList/waitList) and 
avoidance of race with reversed orders between
- consumer registration: register on waitList 1st then check 
exitList, and

- exit event dispatch: register on exitList 1st then check waitList

...but the check you use with consumeCount local variable to detect 
processes spawned by other means (for purposes of logging only) has a 
race:


thread1: Suppose a new consumer is being registered with 
onExitCall(...), is added on the waitList, but before checking 
exitList().size() and iterating the exitList, ...
thread2: the reaper task detects that the very same process has 
finished (gets it's pid from waitpid) and adds it's pid to exitList. 
Then before iterating waitList, ...
thread1: iterates the exitList, finds a match and consumes the pid, 
removing the matching entries from both exitList and waitList. Then ...
thread2: the reaper task iterates waitList, doesn't find a matching 
entry for exitPid, doesn't increment consumeCount and voila: debug 
log("Unexpected process exit for pid:...").



That's enough races for today.

Regards, Peter




On 3/24/2014 12:38 AM, Rob McKenna wrote:

Hi folks,

Roger Riggs (cc'd) may want to chip in here as he's looking at the 
reaper thread arrangement in 9 at the moment.


On another note, I too support the merging of those files. I didn't 
think there was much appetite for it at the time so I must admit 
this fell down my todo list. Looking at this bug did remind me that 
its something worth trying though. As per Alan's mail, I'm going to 
tackle it separately if you folks don't mind. I'll have a look at 
Peter's changes (thanks Peter!) as soon as I can and see about 
getting them in.


-Rob

On 23/03/14 22:30, Martin Buchholz wrote:




On Sun, Mar 23, 2014 at 2:34 AM, Martin Buchholz 
mailto:marti...@google.com>> wrote:



We have also thought about whether having reaper threads is
necessary.  The Unix rule is that child processes should be
waited for, and some thread needs to do that.  There's no way
to wait for a set of child pids, or to specify a "completion
handler".  Well, you might be able to get the newish waitid()
to do what you want, but it looks like it's not sufficient
when java is running inside a process that might do
independent subprocess creation outside of the JVM.


Actually, I take it back.  With sufficient work, it looks like you 
can get SIGCHLD to give you pid information in siginfo_t si_pid, 
and that can be used to trigger the reaping.  It looks lik

Re: ProcessReaper: single thread reaper

[Roger Riggs]

Hi Peter,

Thanks for the comments, I'll fix the races conditions, I intended
to go back to the dedicated thread but wanted to check for scalability
in the case of a large number of threads and non-trivial termination 
overheads

where the streams had to be drained.

Thanks, Roger



On 3/25/14 7:05 AM, Peter Levart wrote:

On 03/24/2014 10:05 PM, roger riggs wrote:

Hi Rob, Martin, et.al.

I've prototyped (for 9) a thread reaper[1] that uses a single thread 
to wait for exiting

processes and calling back to the process with the exit status.
the interesting part is getting the exit status back to the Process 
that needs it

It needs more testing and hardening.

I had not considered using a signal handler for SIGCHLD but that's an 
option,

though we need to be very careful about thread usage.

Roger

p.s. comments on the single thread reaper appreciated (in a new thread)
[1] http://cr.openjdk.java.net/~rriggs/webrev-waitpid/



Hi Roger,

I think I found a little race. Suppose reaper task is still alive and 
that all consumers have been serviced (consumerCount is 0). The reaper 
task waits for 500 millis for next consumer to be registered, but 
times out. Before calling "reaperThread.release()", new consumer comes 
around and registers itself, also calling runReaper(), but since 
reaperThread.release() has not yet been called by old reaper task, new 
reaper task is not submitted to commonPool. The old reaper task 
finishes, leaving one consumer on the waitingList with no reaper task 
to service it. If no new consumers get registered, the waiting 
consumer will never be notified...


The simplest solution for this race, I think, would be to have a 
dedicated long-running thread. It could be spawned lazily, but then it 
would never finish.


Otherwise a nice solution with two lists (exitList/waitList) and 
avoidance of race with reversed orders between
- consumer registration: register on waitList 1st then check exitList, 
and

- exit event dispatch: register on exitList 1st then check waitList

...but the check you use with consumeCount local variable to detect 
processes spawned by other means (for purposes of logging only) has a 
race:


thread1: Suppose a new consumer is being registered with 
onExitCall(...), is added on the waitList, but before checking 
exitList().size() and iterating the exitList, ...
thread2: the reaper task detects that the very same process has 
finished (gets it's pid from waitpid) and adds it's pid to exitList. 
Then before iterating waitList, ...
thread1: iterates the exitList, finds a match and consumes the pid, 
removing the matching entries from both exitList and waitList. Then ...
thread2: the reaper task iterates waitList, doesn't find a matching 
entry for exitPid, doesn't increment consumeCount and voila: debug 
log("Unexpected process exit for pid:...").



That's enough races for today.

Regards, Peter




On 3/24/2014 12:38 AM, Rob McKenna wrote:

Hi folks,

Roger Riggs (cc'd) may want to chip in here as he's looking at the 
reaper thread arrangement in 9 at the moment.


On another note, I too support the merging of those files. I didn't 
think there was much appetite for it at the time so I must admit 
this fell down my todo list. Looking at this bug did remind me that 
its something worth trying though. As per Alan's mail, I'm going to 
tackle it separately if you folks don't mind. I'll have a look at 
Peter's changes (thanks Peter!) as soon as I can and see about 
getting them in.


-Rob

On 23/03/14 22:30, Martin Buchholz wrote:




On Sun, Mar 23, 2014 at 2:34 AM, Martin Buchholz 
mailto:marti...@google.com>> wrote:



We have also thought about whether having reaper threads is
necessary.  The Unix rule is that child processes should be
waited for, and some thread needs to do that.  There's no way
to wait for a set of child pids, or to specify a "completion
handler".  Well, you might be able to get the newish waitid()
to do what you want, but it looks like it's not sufficient when
java is running inside a process that might do independent
subprocess creation outside of the JVM.


Actually, I take it back.  With sufficient work, it looks like you 
can get SIGCHLD to give you pid information in siginfo_t si_pid, 
and that can be used to trigger the reaping.  It looks like waitpid 
is "async-signal-safe", so we can call it from our signal handler.


While we're at it we can fix SIGCHLD handling to do signal 
chaining, as with other signals.

Re: ProcessReaper: single thread reaper

[Peter Levart]

On 03/25/2014 03:05 PM, Peter Levart wrote:

That's enough races for today.


Or just one more...

The debug log("Unexpected process exit for pid:...") could also be 
reached when new process is spawned by UNIXProcess constructor and 
before the constructor manages to register new consumer for it's pid, 
the child process finishes and gets processed by reaper task, which does 
not find a registration for the pid and reports unexpected process exit. 
But that's just logging. What's important is that when consumer finally 
manages to be registered, it will be called synchronously from the same 
thread that registers it (from UNIXProcess constructor)...


Regards, Peter

Re: JDK 9 RFR of 6375303: Review use of caching in BigDecimal

[Brian Burkhalter]

On Mar 25, 2014, at 1:58 AM, Paul Sandoz  wrote:

> This is another example of a stable variable.
> 
> I would like to re-iterate my scepticism that such changes are necessary in 
> this case (i am not sure if it is possible to create a benchmark that could 
> better exacerbate the concurrent overlap of calls to layoutChars). But, i do 
> agree the discussion has been useful and interesting.

I am happy either to leave the toString() code as it is or to change it to the 
variant with toStringSlow(). There is however other cleanup in the patch to 
consider. So it would be good to get consensus on the two points:

1) Change toString() to variant using toStringSlow() or leave it as-is.
2) Change non-toString() code as indicated in the patch or leave it as-is.

If “as-is” is the answer in both cases, then it’s simply a matter of resolving 
the enhancement as “not an issue.”

Thanks,

Brian

Re: RFR(S): 8034960: Serviceability tests using @library failing with java.lang.NoClassDefFoundError

[Yekaterina Kantserova]

On 03/25/2014 02:53 PM, Alan Bateman wrote:

On 25/03/2014 12:14, Staffan Larsen wrote:

I’ve looked at a random sample of these changes and they look ok.

Since some of the changes are in non-serviceability code I have also 
added core-libs to the review thread.


I’m sure you know this, but for the record: please don’t push this 
until jtreg with the fix has been promoted.


Thanks,
/Staffan
Katja - the changes look okay but I wonder about tests that didn't 
previously have @build or @compile tags. Take TestJcmdDefaults as an 
example, it now has:


@build jdk.testlibrary.*

and I just wonder if it should be:

@build TestJcmdDefaults jdk.testlibrary.*

so that it compiles the test in addition to the test library. This 
could be a misunderstanding on my part of course, maybe the source 
file with @test is always compiled if it's a java test?


My assumption is the test itself is always compiled. If it's false, I'm 
afraid we need to change a bunch of other tests.


Jon, could you please explain?

// Katja




-Alan

Re: RFR(S): 8034960: Serviceability tests using @library failing with java.lang.NoClassDefFoundError

[Yekaterina Kantserova]

On 03/25/2014 01:14 PM, Staffan Larsen wrote:

I’ve looked at a random sample of these changes and they look ok.

Since some of the changes are in non-serviceability code I have also added 
core-libs to the review thread.

I’m sure you know this, but for the record: please don’t push this until jtreg 
with the fix has been promoted.
I would not. My thought was to be done with review process and be ready 
to push when JTreg is promoted.


Thank you for adding other groups!
Katja



Thanks,
/Staffan

On 25 mar 2014, at 13:07, Yekaterina Kantserova 
 wrote:


Hi,

Could I please have a review of this fix.

webrev: http://cr.openjdk.java.net/~ykantser/8034960/webrev.00/
bug: https://bugs.openjdk.java.net/browse/JDK-8034960

When using @library in a JTreg test even @build need to be specify for all 
library files used by the test. If @build is not specified it can lead to 
intermittent failures when running tests concurrently, since javac implicit 
compilation and @library and -concurrency don't play well together.

Verified locally since no JTreg with fix has been promoted yet.


Thanks,
Katja

ProcessReaper: single thread reaper

[Peter Levart]

On 03/24/2014 10:05 PM, roger riggs wrote:

Hi Rob, Martin, et.al.

I've prototyped (for 9) a thread reaper[1] that uses a single thread 
to wait for exiting

processes and calling back to the process with the exit status.
the interesting part is getting the exit status back to the Process 
that needs it

It needs more testing and hardening.

I had not considered using a signal handler for SIGCHLD but that's an 
option,

though we need to be very careful about thread usage.

Roger

p.s. comments on the single thread reaper appreciated (in a new thread)
[1] http://cr.openjdk.java.net/~rriggs/webrev-waitpid/



Hi Roger,

I think I found a little race. Suppose reaper task is still alive and 
that all consumers have been serviced (consumerCount is 0). The reaper 
task waits for 500 millis for next consumer to be registered, but times 
out. Before calling "reaperThread.release()", new consumer comes around 
and registers itself, also calling runReaper(), but since 
reaperThread.release() has not yet been called by old reaper task, new 
reaper task is not submitted to commonPool. The old reaper task 
finishes, leaving one consumer on the waitingList with no reaper task to 
service it. If no new consumers get registered, the waiting consumer 
will never be notified...


The simplest solution for this race, I think, would be to have a 
dedicated long-running thread. It could be spawned lazily, but then it 
would never finish.


Otherwise a nice solution with two lists (exitList/waitList) and 
avoidance of race with reversed orders between

- consumer registration: register on waitList 1st then check exitList, and
- exit event dispatch: register on exitList 1st then check waitList

...but the check you use with consumeCount local variable to detect 
processes spawned by other means (for purposes of logging only) has a race:


thread1: Suppose a new consumer is being registered with 
onExitCall(...), is added on the waitList, but before checking 
exitList().size() and iterating the exitList, ...
thread2: the reaper task detects that the very same process has finished 
(gets it's pid from waitpid) and adds it's pid to exitList. Then before 
iterating waitList, ...
thread1: iterates the exitList, finds a match and consumes the pid, 
removing the matching entries from both exitList and waitList. Then ...
thread2: the reaper task iterates waitList, doesn't find a matching 
entry for exitPid, doesn't increment consumeCount and voila: debug 
log("Unexpected process exit for pid:...").



That's enough races for today.

Regards, Peter




On 3/24/2014 12:38 AM, Rob McKenna wrote:

Hi folks,

Roger Riggs (cc'd) may want to chip in here as he's looking at the 
reaper thread arrangement in 9 at the moment.


On another note, I too support the merging of those files. I didn't 
think there was much appetite for it at the time so I must admit this 
fell down my todo list. Looking at this bug did remind me that its 
something worth trying though. As per Alan's mail, I'm going to 
tackle it separately if you folks don't mind. I'll have a look at 
Peter's changes (thanks Peter!) as soon as I can and see about 
getting them in.


-Rob

On 23/03/14 22:30, Martin Buchholz wrote:




On Sun, Mar 23, 2014 at 2:34 AM, Martin Buchholz 
mailto:marti...@google.com>> wrote:



We have also thought about whether having reaper threads is
necessary.  The Unix rule is that child processes should be
waited for, and some thread needs to do that.  There's no way to
wait for a set of child pids, or to specify a "completion
handler".  Well, you might be able to get the newish waitid() to
do what you want, but it looks like it's not sufficient when
java is running inside a process that might do independent
subprocess creation outside of the JVM.


Actually, I take it back.  With sufficient work, it looks like you 
can get SIGCHLD to give you pid information in siginfo_t si_pid, and 
that can be used to trigger the reaping.  It looks like waitpid is 
"async-signal-safe", so we can call it from our signal handler.


While we're at it we can fix SIGCHLD handling to do signal chaining, 
as with other signals.

Re: RFR(S): 8034960: Serviceability tests using @library failing with java.lang.NoClassDefFoundError

[Alan Bateman]

On 25/03/2014 12:14, Staffan Larsen wrote:

I’ve looked at a random sample of these changes and they look ok.

Since some of the changes are in non-serviceability code I have also added 
core-libs to the review thread.

I’m sure you know this, but for the record: please don’t push this until jtreg 
with the fix has been promoted.

Thanks,
/Staffan
Katja - the changes look okay but I wonder about tests that didn't 
previously have @build or @compile tags. Take TestJcmdDefaults as an 
example, it now has:


@build jdk.testlibrary.*

and I just wonder if it should be:

@build TestJcmdDefaults jdk.testlibrary.*

so that it compiles the test in addition to the test library. This could 
be a misunderstanding on my part of course, maybe the source file with 
@test is always compiled if it's a java test?


-Alan

Re: Process API Updates (JEP 102)

[Alan Bateman]

On 25/03/2014 12:17, David M. Lloyd wrote:


There are two features we would definitely like to see added:

1) NIO Channel interface for process streams
2) The ability to pass additional channels to child processes

#1 seems relatively straightforward,
Assuming #1 means non-blocking and multiplexing then it has been looked 
into a few times. The main challenge is Windows where there are 
challenges multiplexing socket and handles to process streams at the 
same time. There have been a number of proposals and would need more 
investigating to come up with something that scales and is clean.


-Alan

Re: Experimental patches for unsafe cleanup Re: The s.m.Unsafe representation in hotspot and method registration

[Staffan Larsen]

(Adding hotspot-runtime-dev since some changes are in hotspot code).


On 25 mar 2014, at 14:20, Paul Sandoz  wrote:

> 
> On Mar 25, 2014, at 9:34 AM, Paul Sandoz  wrote:
> 
>> 
>> On Mar 24, 2014, at 7:49 PM, Staffan Larsen  
>> wrote:
>> 
>>> We have abandoned the HSX model. From JDK 8 one version of Hotspot will be 
>>> tied to one version of the JDK. This looks like old code that has not been 
>>> cleaned up.
>>> 
>> 
>> Thanks, yes, looks like we can clean this up and also remove the deprecated 
>> methods at the same time.
>> 
> 
> Here are some preliminary webrevs to clean up unsafe and also remove all old, 
> deprecated and monitor-related methods:
> 
>  http://cr.openjdk.java.net/~psandoz/jdk9/jdk-unsafe-cleanup/webrev/jdk.patch
> 
>  http://cr.openjdk.java.net/~psandoz/jdk9/hotspot-unsafe-cleanup/webrev/
> 
> Passes all tests i have thrown at it locally.
> 
> There is some additional clean up that could be done to remove prefetch 
> read/write intrinsics hooked up to older versions of Unsafe:
> 
>  do_intrinsic(_prefetchRead, sun_misc_Unsafe,
> prefetchRead_name, prefetch_signature, F_RN)  \
>   do_name( prefetchRead_name,   
> "prefetchRead")\
>  do_intrinsic(_prefetchWrite,sun_misc_Unsafe,
> prefetchWrite_name, prefetch_signature,F_RN)  \
>   do_name( prefetchWrite_name,  
> "prefetchWrite")   \
>  do_intrinsic(_prefetchReadStatic,   sun_misc_Unsafe,
> prefetchReadStatic_name, prefetch_signature,   F_SN)  \
>   do_name( prefetchReadStatic_name, 
> "prefetchReadStatic")  \
>  do_intrinsic(_prefetchWriteStatic,  sun_misc_Unsafe,
> prefetchWriteStatic_name, prefetch_signature,  F_SN)  \
>   do_name( prefetchWriteStatic_name,
> "prefetchWriteStatic") \
> 
> That will also require updates in various other files. I presume these 
> prefetch methods are never used throughout the runtime?
> 
> Paul.

Experimental patches for unsafe cleanup Re: The s.m.Unsafe representation in hotspot and method registration

[Paul Sandoz]

On Mar 25, 2014, at 9:34 AM, Paul Sandoz  wrote:

> 
> On Mar 24, 2014, at 7:49 PM, Staffan Larsen  wrote:
> 
>> We have abandoned the HSX model. From JDK 8 one version of Hotspot will be 
>> tied to one version of the JDK. This looks like old code that has not been 
>> cleaned up.
>> 
> 
> Thanks, yes, looks like we can clean this up and also remove the deprecated 
> methods at the same time.
> 

Here are some preliminary webrevs to clean up unsafe and also remove all old, 
deprecated and monitor-related methods:

  http://cr.openjdk.java.net/~psandoz/jdk9/jdk-unsafe-cleanup/webrev/jdk.patch

  http://cr.openjdk.java.net/~psandoz/jdk9/hotspot-unsafe-cleanup/webrev/

Passes all tests i have thrown at it locally.

There is some additional clean up that could be done to remove prefetch 
read/write intrinsics hooked up to older versions of Unsafe:

  do_intrinsic(_prefetchRead, sun_misc_Unsafe,
prefetchRead_name, prefetch_signature, F_RN)  \
   do_name( prefetchRead_name,   
"prefetchRead")\
  do_intrinsic(_prefetchWrite,sun_misc_Unsafe,
prefetchWrite_name, prefetch_signature,F_RN)  \
   do_name( prefetchWrite_name,  
"prefetchWrite")   \
  do_intrinsic(_prefetchReadStatic,   sun_misc_Unsafe,
prefetchReadStatic_name, prefetch_signature,   F_SN)  \
   do_name( prefetchReadStatic_name, 
"prefetchReadStatic")  \
  do_intrinsic(_prefetchWriteStatic,  sun_misc_Unsafe,
prefetchWriteStatic_name, prefetch_signature,  F_SN)  \
   do_name( prefetchWriteStatic_name,
"prefetchWriteStatic") \

That will also require updates in various other files. I presume these prefetch 
methods are never used throughout the runtime?

Paul.

Re: Process API Updates (JEP 102)

[Alan Bateman]

On 25/03/2014 09:50, Peter Levart wrote:

:

It would just require the API specification to strengthen it's 
guarantees on some methods: for example destroyForcibly() would 
guarantee forcible termination. It might require an additional method 
like destroyNicely() or terminateNicely() which would guarantee a kind 
of termination where victim process could clean-up. 
This is something that has been looked a few times. When we added 
destroyForcibly then we had to allow for the possibility that it might 
not be immediate so this is the reason for the current wording in the 
javadoc. It's not too bad when used with the timed waitFor. I had hoped 
we would also add a destroyGracefully but we didn't come up with a good 
solution on Windows at the time.




:

The worst thing is to have an entirely separate API that does not 
interoperate with old API. I think the old API is not so bad. It just 
lacks some features and it's implementation could be revamped a bit.
I agree but I know that Roger has been trying out a few API approaches 
with a view to coming up with a nice solution.


-Alan.

Re: Process API Updates (JEP 102)

[Roger Riggs]

Hi Martin,

Good idea,  I'll put something together in the next couple of weeks.

Roger


On 3/25/14 1:54 AM, Martijn Verburg wrote:

Hi Roger,

Is there a short survey we could send out to the wider dev community 
on this one?  I think just about every day to day Java developer has 
had to use Process at some stage in their career.


Cheers,
Martijn


On 24 March 2014 21:11, roger riggs > wrote:


Hi,

I'm starting to work on JEP 102, Process API Updates for JDK 9.
The use cases identified include test execution and build systems such
as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
to monitor the health of a more complex system where the processes
are not spawned by the same manager.

The current API of Process itself is pretty complete with the
addition of a getPid
method to enable identification of subprocesses spawned by the
application
and allow external identification.  It will not be possible to
intercept the input,
output and error streams of an arbitrary process.

From the scope of the JEP, a fairly simple API seems sufficient.
 - Enumerate the direct children
 - The rest of the functions are similar to Process
   - to terminate a process, forcibly and normally
   - to destroy a process and all of its children recursively
   - to check if one is alive
   - to waitFor for termination and retrieve the exit status

Are there use cases for which this is insufficient?  Please comment.

Thanks, Roger

Re: Process API Updates (JEP 102)

[Florian Weimer]

On 03/24/2014 10:11 PM, roger riggs wrote:


 From the scope of the JEP, a fairly simple API seems sufficient.
  - Enumerate the direct children
  - The rest of the functions are similar to Process
- to terminate a process, forcibly and normally
- to destroy a process and all of its children recursively
- to check if one is alive
- to waitFor for termination and retrieve the exit status


On Linux, I think you need to use obscure mechanisms to make this 
iteration reliable (cgroups perhaps).  There is no openpid() or similar 
mechanism that allows you to obtain a stable handle.


Somewhat relatedly, it would make sense to make Process auto-closeable, 
so that you can make sure that no non-Java resources are retained. 
Right now, this seems to require calling destroyForcibly().waitFor().


It would be nice if there was a way to directly pipe the output of one 
process to the input of another process.  I think that even with Java 8, 
this still needs an explicit copy loop.


--
Florian Weimer / Red Hat Product Security Team

Re: Process API Updates (JEP 102)

[David M. Lloyd]

On 03/24/2014 04:11 PM, roger riggs wrote:

Hi,

I'm starting to work on JEP 102, Process API Updates for JDK 9.
The use cases identified include test execution and build systems such
as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
to monitor the health of a more complex system where the processes
are not spawned by the same manager.

The current API of Process itself is pretty complete with the addition
of a getPid
method to enable identification of subprocesses spawned by the application
and allow external identification.  It will not be possible to intercept
the input,
output and error streams of an arbitrary process.

 From the scope of the JEP, a fairly simple API seems sufficient.
  - Enumerate the direct children
  - The rest of the functions are similar to Process
- to terminate a process, forcibly and normally
- to destroy a process and all of its children recursively
- to check if one is alive
- to waitFor for termination and retrieve the exit status

Are there use cases for which this is insufficient?  Please comment.


There are two features we would definitely like to see added:

1) NIO Channel interface for process streams
2) The ability to pass additional channels to child processes

#1 seems relatively straightforward, but admittedly it's not clear how 
#2 might be possible from the child process side.



--
- DML

Re: RFR(S): 8034960: Serviceability tests using @library failing with java.lang.NoClassDefFoundError

[Staffan Larsen]

I’ve looked at a random sample of these changes and they look ok. 

Since some of the changes are in non-serviceability code I have also added 
core-libs to the review thread.

I’m sure you know this, but for the record: please don’t push this until jtreg 
with the fix has been promoted.

Thanks,
/Staffan

On 25 mar 2014, at 13:07, Yekaterina Kantserova 
 wrote:

> Hi,
> 
> Could I please have a review of this fix.
> 
> webrev: http://cr.openjdk.java.net/~ykantser/8034960/webrev.00/
> bug: https://bugs.openjdk.java.net/browse/JDK-8034960
> 
> When using @library in a JTreg test even @build need to be specify for all 
> library files used by the test. If @build is not specified it can lead to 
> intermittent failures when running tests concurrently, since javac implicit 
> compilation and @library and -concurrency don't play well together.
> 
> Verified locally since no JTreg with fix has been promoted yet.
> 
> 
> Thanks,
> Katja

Re: RFR: 8036786: Update jdk7 testlibrary to match jdk8

[Staffan Larsen]

I’ve looked at the changes in StreamPumper and ProcessTools: they look good.

Since the rest of the code is a straight backport, I didn’t look at it.

Reviewed.

Thanks,
/Staffan

On 6 mar 2014, at 16:34, Vladimir Kempik  wrote:

> Please review this change to update jdk part of testlibrary in jdk7.
> 
> It updates testlibrary with new apis from jdk8's testlibrary.
> 
> When porting jdk8's testlibrary to jdk7, I had to make few changes:
> 
> 1) rewrite one lambda usage in StreamPumper.java to non-lambda version.
> 
> 2) get rid of predicate functionality in ProcessTools.java: startProcess as 
> it wasn't used anyway and unsupported in jdk7.
> 
> Every test in open part of jdk that uses testlibrary still works after update 
> (in fact there are 3 of them).
> 
> Webrev:
> http://cr.openjdk.java.net/~vkempik/8036786/webrev.00/
> 
> Thanks, Vladimir.

Re: Process API Updates (JEP 102)

[Peter Levart]

On 03/24/2014 10:11 PM, roger riggs wrote:

Hi,

I'm starting to work on JEP 102, Process API Updates for JDK 9.
The use cases identified include test execution and build systems such
as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
to monitor the health of a more complex system where the processes
are not spawned by the same manager.

The current API of Process itself is pretty complete with the addition 
of a getPid
method to enable identification of subprocesses spawned by the 
application
and allow external identification.  It will not be possible to 
intercept the input,

output and error streams of an arbitrary process.

From the scope of the JEP, a fairly simple API seems sufficient.
 - Enumerate the direct children


What about self? An application might want to communicate it's own PID 
to some other process (write it into a file, for example). Other Process 
API methods are not sensible for "self" process so this might as well be 
just simple static getSelfPid() method.



 - The rest of the functions are similar to Process
   - to terminate a process, forcibly and normally
   - to destroy a process and all of its children recursively
   - to check if one is alive
   - to waitFor for termination and retrieve the exit status

Are there use cases for which this is insufficient?  Please comment.


It would be nice if this new API interoperated with old Process API in a 
way that using both interchangeably for the same child process would be 
possible. For example: spawning a child with old Process API and 
enumerating/waiting/checking it with new API in some thread while 
waiting for termination using old API in another thread...


I wonder if an approach to just add getPid() (and maybe some other 
method) to old Process class together with some static methods to 
enumerate child processes would be possible?


It would just require the API specification to strengthen it's 
guarantees on some methods: for example destroyForcibly() would 
guarantee forcible termination. It might require an additional method 
like destroyNicely() or terminateNicely() which would guarantee a kind 
of termination where victim process could clean-up. The specification 
already allows getInputStream()/getOutputStream()/getErrorStream() to 
return special "null streams" when spawned process IO is redirected 
to/from file. The enumeration methods could simply return Process object 
which would:


- be same instances of Process objects created with ProcessBuilder API 
for processes spawned by the Process API. This would require an internal 
registry of live Process objects. We already have reaper threads which 
could be used to clean-up this registry on process termination.
- be new Process objects for child processes spawned in some other way 
(or by children) and which would return special "null streams" from 
their getInputStream()/getOutputStream()/getErrorStream() methods. These 
Process objects could be instantiated lazily on first request and then 
registered in the same internal registry. They might still require 
reaper threads to enable waiting on them from multiple Java threads and 
cleaning-up registry, but they could probably be created lazily.


The worst thing is to have an entirely separate API that does not 
interoperate with old API. I think the old API is not so bad. It just 
lacks some features and it's implementation could be revamped a bit.


What do you think?

Regards, Peter



Thanks, Roger

RE: RFR: 8036786: Update jdk7 testlibrary to match jdk8

[Mattis Castegren]

This is becoming URGENT as we have other high priority cases pending the 
backport of these test libraries.

We could backport these fixes without the tests and then file a separate bug to 
backport the tests, but it would be great if we could just get a review of 
this, as it is a pretty straight forward backport

Kind Regards
/Mattis

-Original Message-
From: Alan Bateman 
Sent: den 14 mars 2014 15:42
To: Mattis Castegren
Cc: Vladimir Kempik; core-libs-dev@openjdk.java.net; Ekaterina Medvedeva; 
serviceability-...@openjdk.java.net
Subject: Re: RFR: 8036786: Update jdk7 testlibrary to match jdk8


Adding serviceability-dev to the CC list as this seems to be where this 
testlibrary has mostly been developed and maintained.

-Alan.

On 13/03/2014 14:02, Mattis Castegren wrote:
> Hi
>
> Could we have someone review this change. This is just a backport from JDK8, 
> but there were some small changes so we want to get a proper review here.
>
> We have several other backports waiting for this change (fixes using the new 
> test libraries), so it would be good to get this going.
>
> Kind Regards
> /Mattis
>
> -Original Message-
> From: Vladimir Kempik
> Sent: den 6 mars 2014 16:34
> To: core-libs-dev@openjdk.java.net
> Cc: Mattis Castegren
> Subject: RFR: 8036786: Update jdk7 testlibrary to match jdk8
>
> Please review this change to update jdk part of testlibrary in jdk7.
>
> It updates testlibrary with new apis from jdk8's testlibrary.
>
> When porting jdk8's testlibrary to jdk7, I had to make few changes:
>
> 1) rewrite one lambda usage in StreamPumper.java to non-lambda version.
>
> 2) get rid of predicate functionality in ProcessTools.java: startProcess
> as it wasn't used anyway and unsupported in jdk7.
>
> Every test in open part of jdk that uses testlibrary still works after
> update (in fact there are 3 of them).
>
> Webrev:
> http://cr.openjdk.java.net/~vkempik/8036786/webrev.00/
>
> Thanks, Vladimir.

Re: Process API Updates (JEP 102)

[Chris Hegarty]

On 25 Mar 2014, at 08:45, Kasper Nielsen  wrote:

> On Mon, Mar 24, 2014 at 10:11 PM, roger riggs wrote:
> 
>> Hi,
>> 
>> I'm starting to work on JEP 102, Process API Updates for JDK 9.
>> 
>> Are there use cases for which this is insufficient?  Please comment.
>> 
>> 
> What about a non-blocking waitFor mechanism such as
> 
> void onTerminate(IntConsumer consumer) or
> CompletionStage onTerminate();
> 
> Would be useful for spawning many small processes.
> 
> 
> Also a waitFor method taking a timeout parameter. So you, for example, can
> wait for the subprocess to exit within 10 seconds. And if not kill it.

Have you looked at these new methods in Java 8? Or are you looking for 
something different?

http://download.java.net/jdk8/docs/api/java/lang/Process.html#waitFor-long-java.util.concurrent.TimeUnit-
http://download.java.net/jdk8/docs/api/java/lang/Process.html#destroyForcibly--

-Chris.

> 
> - Kasper

Re: JDK 9 RFR of 6375303: Review use of caching in BigDecimal

[Paul Sandoz]

On Mar 24, 2014, at 8:30 PM, Mike Duigou  wrote:
> The other issue here, and why this thread has gone on so long, is that we've 
> been trying to establish what the best idiom is (for 2014-) for this lazy 
> initialization situation so that we can reuse it with less thought and 
> review. In this case the chances of race are fairly low and the cost of extra 
> layoutChars is bearable. In other cases where we are likely to use the same 
> idiom this is less likely true. Peter's current implementation has the 
> advantage that is the best-right answer for lazy initialization even in the 
> contended/expensive case. I look forward to revisiting this again when the 
> new JMM primitives are available. :-)
> 

This is another example of a stable variable.

I would like to re-iterate my scepticism that such changes are necessary in 
this case (i am not sure if it is possible to create a benchmark that could 
better exacerbate the concurrent overlap of calls to layoutChars). But, i do 
agree the discussion has been useful and interesting.

Paul.

Re: Process API Updates (JEP 102)

[Robert Stupp]

Would be nice to have some kind of "async" process I/O streaming.
And an event mechanism to receive notifications when the process terminates or 
input is available from stdout/stderr.

> Am 25.03.2014 um 09:45 schrieb Kasper Nielsen :
> 
> On Mon, Mar 24, 2014 at 10:11 PM, roger riggs wrote:
> 
>> Hi,
>> 
>> I'm starting to work on JEP 102, Process API Updates for JDK 9.
>> 
>> Are there use cases for which this is insufficient?  Please comment.
> What about a non-blocking waitFor mechanism such as
> 
> void onTerminate(IntConsumer consumer) or
> CompletionStage onTerminate();
> 
> Would be useful for spawning many small processes.
> 
> 
> Also a waitFor method taking a timeout parameter. So you, for example, can
> wait for the subprocess to exit within 10 seconds. And if not kill it.
> 
> - Kasper

Re: Process API Updates (JEP 102)

[Martijn Verburg]

Hi Roger,

Is there a short survey we could send out to the wider dev community on
this one?  I think just about every day to day Java developer has had to
use Process at some stage in their career.

Cheers,
Martijn


On 24 March 2014 21:11, roger riggs  wrote:

> Hi,
>
> I'm starting to work on JEP 102, Process API Updates for JDK 9.
> The use cases identified include test execution and build systems such
> as Jtreg and Hudson/Jenkins. And there is a use-case for using Java
> to monitor the health of a more complex system where the processes
> are not spawned by the same manager.
>
> The current API of Process itself is pretty complete with the addition of
> a getPid
> method to enable identification of subprocesses spawned by the application
> and allow external identification.  It will not be possible to intercept
> the input,
> output and error streams of an arbitrary process.
>
> From the scope of the JEP, a fairly simple API seems sufficient.
>  - Enumerate the direct children
>  - The rest of the functions are similar to Process
>- to terminate a process, forcibly and normally
>- to destroy a process and all of its children recursively
>- to check if one is alive
>- to waitFor for termination and retrieve the exit status
>
> Are there use cases for which this is insufficient?  Please comment.
>
> Thanks, Roger
>
>

Re: Process API Updates (JEP 102)

[Kasper Nielsen]

On Mon, Mar 24, 2014 at 10:11 PM, roger riggs wrote:

> Hi,
>
> I'm starting to work on JEP 102, Process API Updates for JDK 9.
>
> Are there use cases for which this is insufficient?  Please comment.
>
>
What about a non-blocking waitFor mechanism such as

void onTerminate(IntConsumer consumer) or
CompletionStage onTerminate();

Would be useful for spawning many small processes.


Also a waitFor method taking a timeout parameter. So you, for example, can
wait for the subprocess to exit within 10 seconds. And if not kill it.

- Kasper

Re: The s.m.Unsafe representation in hotspot and method registration

[Paul Sandoz]

On Mar 24, 2014, at 7:49 PM, Staffan Larsen  wrote:

> We have abandoned the HSX model. From JDK 8 one version of Hotspot will be 
> tied to one version of the JDK. This looks like old code that has not been 
> cleaned up.
> 

Thanks, yes, looks like we can clean this up and also remove the deprecated 
methods at the same time.

Paul.

Re: JDK 9 RFR of 6375303: Review use of caching in BigDecimal

[Peter Levart]

On 03/24/2014 06:48 PM, Brian Burkhalter wrote:

Hi Peter,

On Mar 24, 2014, at 2:21 AM, Peter Levart > wrote:


Thanks to Aleksey for re-establishing the access, I bring you results 
of the microbenchmark from his quad-core Cortex-A9:


Thanks to you and Aleksey for taking the initiative to test on that 
platform.


...as can be seen, the double-checked read-then-volatile-read+CAS 
trick is about 15% faster than classic volatile-read+CAS in this case.


Just to be sure, by “double-checked read-then-volaite-read+CAS trick” 
you intend "the variant with two methods toString/toStringSlow?”


That's right. I split the code into two methods in both cases to make 
sure it gets inlined for fast-path. The variant in your webrev with 
single method has been demonstrated to exceed the maximum inline 
bytecode size.


Regards, Peter



Thanks,

Brian