How to close a Process and having shutdown hooks called ?
Hello everyone !
I looked for the answer in this list and did not find anything relevant.
I don't know it it's possible to close a Process et call shutdown hooks
automatically. I looked into the code and found it was not possible (in jdk7).
When we call destroy() on a Process, it will call terminateProcess of
ProcessImpl class.
This link shows that a TerminateProcess kills the process and WH_CBT hooks
are NOT called.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms686722(v=vs.85).aspx
Calling exitProcess (instead of TeminateProcess) seems to be dangerous ... but
I don't understand why (ok, dlls could be locked, but when you do a CTRL +C it
calls an exitProcess... so what's the problem ???)
So I ask how can we close a process, with the hooks called ? . My knowledge
is way smaller than people on openjdk... so I hope someone has an answer or a
workaround.
Byte the way, I looked for sending signals (SIGINT for Windows, SIGKILL for
linux etc) to close a process.
Maybe I'm wrong, but a solution could be to add to ProcessImpl_md.c :
#include signal.h
JNIEXPORT jboolean JNICALL
Java_java_lang_ProcessImpl_CloseHandleWithHooks(JNIEnv *env, jclass ignored,
jlong handle)
{
raise(SIGINT) ;
}
I think,it will call an ExitProcess so ... dangerous or not ?
Of course, we have to add also a function in ProcessImpl and make call
available from Process through an abstract function like close()
Any clever thoughts on this subject (I'm sure it is a common problem...) ?
Thanks :)
Regards.
Nicolas Le Picard
Re: How to close a Process and having shutdown hooks called ?
Hi Nicolas,
Looks like a well discussed question. On Posix systems, SIGTERM should work
for you. That's the default signal sent by the 'kill' command on Linux.
e.g. please take a look here:
http://stackoverflow.com/questions/2541597/how-to-gracefully-handle-the-sigkill-signal-in-java
- Kris
On Tue, Mar 4, 2014 at 2:45 AM, LE PICARD NICOLAS
n.lepic...@chu-tours.frwrote:
Hello everyone !
I looked for the answer in this list and did not find anything relevant.
I don't know it it's possible to close a Process et call shutdown hooks
automatically. I looked into the code and found it was not possible (in
jdk7).
When we call destroy() on a Process, it will call terminateProcess of
ProcessImpl class.
This link shows that a TerminateProcess kills the process and WH_CBT
hooks are NOT called.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms686722(v=vs.85).aspx
Calling exitProcess (instead of TeminateProcess) seems to be dangerous ...
but I don't understand why (ok, dlls could be locked, but when you do a
CTRL +C it calls an exitProcess... so what's the problem ???)
So I ask how can we close a process, with the hooks called ? . My
knowledge is way smaller than people on openjdk... so I hope someone has an
answer or a workaround.
Byte the way, I looked for sending signals (SIGINT for Windows, SIGKILL
for linux etc) to close a process.
Maybe I'm wrong, but a solution could be to add to ProcessImpl_md.c :
#include signal.h
JNIEXPORT jboolean JNICALL
Java_java_lang_ProcessImpl_CloseHandleWithHooks(JNIEnv *env, jclass
ignored, jlong handle)
{
raise(SIGINT) ;
}
I think,it will call an ExitProcess so ... dangerous or not ?
Of course, we have to add also a function in ProcessImpl and make call
available from Process through an abstract function like close()
Any clever thoughts on this subject (I'm sure it is a common problem...) ?
Thanks :)
Regards.
Nicolas Le Picard
Re: JEP 193: Enhanced Volatiles
On 03/04/2014 02:41 AM, Jeroen Frijters wrote: Brian Goetz wrote: Embedded in this proposal is the desire to not provide a full-blown lvalue form for variables; supporting any form of pass-by-reference at the language level is a super-non-goal here. Why is this? It solves these problems in an extremely clean way and also provides lots of other value (for example, for JEP 191: Foreign Function Interface). I understand pass-by-reference is an expensive feature, but IMNSHO poluting Java with this proposal will prove to be more expensive in the long run. It's like erased generics all over again. The expensive version of pass-by-reference is already supported using java.lang.reflect.Field. You can think of .volatile as a version of such mechanics that only works within the context of a single expression, not across method calls. There is plenty of precedent for this. For example you cannot emulate the += operator as a standalone method without resorting to reflection. You can view this JEP as an extension mechanism for safely adding similar l-value-based operators. (Although only the JDK can implement these extensions.) -Doug
Re: How to close a Process and having shutdown hooks called ?
On 04/03/2014 11:51, Krystal Mok wrote: Hi Nicolas, Looks like a well discussed question. On Posix systems, SIGTERM should work for you. That's the default signal sent by the 'kill' command on Linux. e.g. please take a look here: http://stackoverflow.com/questions/2541597/how-to-gracefully-handle-the-sigkill-signal-in-java - Kris I think he's on Windows and is looking for destroy to use something other than TerminateProcess. As background, the destroy method was confusingly specified to kill the sub-process forcibly but it wasn't implemented this way everywhere. On Unix/Linux then the long standing implementation used SIGTERM and so no guarantee that it would cause the sub-process to terminate. This mismatch was examined in JDK 8 (you'll need to go through the archives of this mailing list to see the discussion) and the javadoc updated to make it clear that it is implementation specific as to whether it is done forcibly or not. In addition a new destroyForcibly was added to do the SIGKILL or TerminateProcess for cases where you really want to kill the child. There isn't a corresponding destroyGracefully but clearly this would be useful if were feasible to implement everywhere. -Alan.
RE: How to close a Process and having shutdown hooks called ?
I didn't know for jdk8 functions destroy and the new one destroyForcibly... And Jonathan was right in his previous answer, I was looking for a solution in Java, I mean a portable one like in Process class, like the destroyGracefully. I am looking to signals because it seems to be a simple way to achieve this portable solution, maybe I'm wrong. It would be useful (as you said) to have a function to kill gracefully through Process class, no ? Nicolas Le Picard -Message d'origine- De : Alan Bateman [mailto:alan.bate...@oracle.com] Envoyé : mardi 4 mars 2014 13:12 À : Krystal Mok Cc : LE PICARD NICOLAS; core-libs-dev@openjdk.java.net Objet : Re: How to close a Process and having shutdown hooks called ? On 04/03/2014 11:51, Krystal Mok wrote: Hi Nicolas, Looks like a well discussed question. On Posix systems, SIGTERM should work for you. That's the default signal sent by the 'kill' command on Linux. e.g. please take a look here: http://stackoverflow.com/questions/2541597/how-to-gracefully-handle-th e-sigkill-signal-in-java - Kris I think he's on Windows and is looking for destroy to use something other than TerminateProcess. As background, the destroy method was confusingly specified to kill the sub-process forcibly but it wasn't implemented this way everywhere. On Unix/Linux then the long standing implementation used SIGTERM and so no guarantee that it would cause the sub-process to terminate. This mismatch was examined in JDK 8 (you'll need to go through the archives of this mailing list to see the discussion) and the javadoc updated to make it clear that it is implementation specific as to whether it is done forcibly or not. In addition a new destroyForcibly was added to do the SIGKILL or TerminateProcess for cases where you really want to kill the child. There isn't a corresponding destroyGracefully but clearly this would be useful if were feasible to implement everywhere. -Alan.
Re: How to close a Process and having shutdown hooks called ?
On 03/04/2014 02:09 PM, LE PICARD NICOLAS wrote: I didn't know for jdk8 functions destroy and the new one destroyForcibly... And Jonathan was right in his previous answer, I was looking for a solution in Java, I mean a portable one like in Process class, like the destroyGracefully. I am looking to signals because it seems to be a simple way to achieve this portable solution, maybe I'm wrong. It would be useful (as you said) to have a function to kill gracefully through Process class, no ? Nicolas Le Picard Hi Nicolas, So you're spawning a Java sub-process from a Java master process using java.lang.Process, do I understand correctly? Are you in control of the code that represents the child sub-process? Is this code already using System.in (STDIN) stream for anything? If the answers are YES and NO respectively, then you may be able to use the System.in in the sub-process (spawn a special monitoring thread) and Process.getOutputStream() in the master process to establish a one way channel for passing commands in direction: master process - sub-process. One of those commands could be shutdown for example, that would trigger a System.exit() in the sub-process. This way you could make the shut-down hooks in sub-process be executed before exiting... Regards, Peter -Message d'origine- De : Alan Bateman [mailto:alan.bate...@oracle.com] Envoyé : mardi 4 mars 2014 13:12 À : Krystal Mok Cc : LE PICARD NICOLAS; core-libs-dev@openjdk.java.net Objet : Re: How to close a Process and having shutdown hooks called ? On 04/03/2014 11:51, Krystal Mok wrote: Hi Nicolas, Looks like a well discussed question. On Posix systems, SIGTERM should work for you. That's the default signal sent by the 'kill' command on Linux. e.g. please take a look here: http://stackoverflow.com/questions/2541597/how-to-gracefully-handle-th e-sigkill-signal-in-java - Kris I think he's on Windows and is looking for destroy to use something other than TerminateProcess. As background, the destroy method was confusingly specified to kill the sub-process forcibly but it wasn't implemented this way everywhere. On Unix/Linux then the long standing implementation used SIGTERM and so no guarantee that it would cause the sub-process to terminate. This mismatch was examined in JDK 8 (you'll need to go through the archives of this mailing list to see the discussion) and the javadoc updated to make it clear that it is implementation specific as to whether it is done forcibly or not. In addition a new destroyForcibly was added to do the SIGKILL or TerminateProcess for cases where you really want to kill the child. There isn't a corresponding destroyGracefully but clearly this would be useful if were feasible to implement everywhere. -Alan.
RE: How to close a Process and having shutdown hooks called ?
Exactly !
In fact, I'm in mastery of ... nothing ! But I tried to workaround this problem
and found that Java Process class lacks something ! I was working in Eclipse
and you can process while testing (the red button in console view) but shutdown
hooks are not called. So I dug a little to understand why Eclipse team said
that it is not possible (up to now). Indeed, Eclipse spawns a new Process
Object (a new JVM in fact) but can't close it gracefully. I think other
application / programs may need such functionnality so I post on this list to
have advices of specialists of openJdk.
You are right again on the second part ! Actually I did exactly that in my
applications, only for debug/test, very simple :
boolean loopz = true;
InputStreamReader isr = null;
BufferedReader br = null;
try {
isr = new InputStreamReader(System.in);
br = new BufferedReader(isr);
while (loopz) {
String userInput = br.readLine();
System.out.println(input = +userInput);
if (userInput.equalsIgnoreCase(quit) ||
userInput.equalsIgnoreCase(exit)) {
System.exit(0);
}
}
}
catch (Exception er) {
er.printStackTrace();
loopz = false;
}
finally {
try { br.close(); } catch (Exception e) {}
try { isr.close(); } catch (Exception e) {}
}
However I think it could be a good add to Java to make a Process having better
control on its own child termination.
Thanks !
Nicolas Le Picard
-Message d'origine-
De : Peter Levart [mailto:peter.lev...@gmail.com]
Envoyé : mardi 4 mars 2014 15:47
À : LE PICARD NICOLAS; Alan Bateman; Krystal Mok
Cc : core-libs-dev@openjdk.java.net
Objet : Re: How to close a Process and having shutdown hooks called ?
On 03/04/2014 02:09 PM, LE PICARD NICOLAS wrote:
I didn't know for jdk8 functions destroy and the new one destroyForcibly...
And Jonathan was right in his previous answer, I was looking for a solution
in Java, I mean a portable one like in Process class, like the
destroyGracefully.
I am looking to signals because it seems to be a simple way to achieve this
portable solution, maybe I'm wrong.
It would be useful (as you said) to have a function to kill gracefully
through Process class, no ?
Nicolas Le Picard
Hi Nicolas,
So you're spawning a Java sub-process from a Java master process using
java.lang.Process, do I understand correctly?
Are you in control of the code that represents the child sub-process? Is this
code already using System.in (STDIN) stream for anything? If the answers are
YES and NO respectively, then you may be able to use the System.in in the
sub-process (spawn a special monitoring thread) and
Process.getOutputStream() in the master process to establish a one way channel
for passing commands in direction: master process - sub-process. One of those
commands could be shutdown for example, that would trigger a System.exit() in
the sub-process. This way you could make the shut-down hooks in sub-process be
executed before exiting...
Regards, Peter
-Message d'origine-
De : Alan Bateman [mailto:alan.bate...@oracle.com] Envoyé : mardi 4
mars 2014 13:12 À : Krystal Mok Cc : LE PICARD NICOLAS;
core-libs-dev@openjdk.java.net Objet : Re: How to close a Process and
having shutdown hooks called ?
On 04/03/2014 11:51, Krystal Mok wrote:
Hi Nicolas,
Looks like a well discussed question. On Posix systems, SIGTERM
should work for you. That's the default signal sent by the 'kill' command on
Linux.
e.g. please take a look here:
http://stackoverflow.com/questions/2541597/how-to-gracefully-handle-t
h
e-sigkill-signal-in-java
- Kris
I think he's on Windows and is looking for destroy to use something other
than TerminateProcess.
As background, the destroy method was confusingly specified to kill the
sub-process forcibly but it wasn't implemented this way everywhere. On
Unix/Linux then the long standing implementation used SIGTERM and so no
guarantee that it would cause the sub-process to terminate. This mismatch was
examined in JDK 8 (you'll need to go through the archives of this mailing
list to see the discussion) and the javadoc updated to make it clear that it
is implementation specific as to whether it is done forcibly or not. In
addition a new destroyForcibly was added to do the SIGKILL or
TerminateProcess for cases where you really want to kill the child. There
isn't a corresponding destroyGracefully but clearly this would be useful if
were feasible to implement everywhere.
-Alan.
Re: JDK 9 RFR of 6375303: Review use of caching in BigDecimal
On 03/04/2014 01:14 AM, Brian Burkhalter wrote: - add AtomicReferenceFieldUpdater-type constant for stringCache initialization Hi Brian, By using volatile read and CAS, there's still a chance that multiple concurrent threads will be invoking the layoutChars(true) concurrently, but you guarantee that only a single instance of String will ever be returned as a result of the toString() method. Is that what you were pursuing? Regards, Peter
Re: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods
On 03/03/2014 09:45 PM, David Holmes wrote: On 3/03/2014 10:56 PM, David M. Lloyd wrote: Yes, that would necessarily be the contract of a Monitors class, just as it is part of the contract of Lock today. If your argument is that it shouldn't be allowed because it might be used wrong, we might as well just delete most of the JDK, ReentrantLock included, since it suffers from the exact same potential problem. The difference is that monitors have a simple API in the form of synchronized that people were in the past and would continue to be free (and recommended) to use. We should not introduce anything that allows something that was guaranteed to be safe by the language, to become unsafe. Define 'safe'. Because I don't think it's unsafe, any more than unpaired lock/unlock are. There is no uninitialized memory being accessed, no way to bypass security checks, no way to manipulate memory directly, etc. I find this argument to be particularly weak without specific cases. As far as I can tell, the language doesn't make it safe, it makes it *convenient*, and that's a big distinction. There are two cases I can see that are potentially problematic: extra acquires, and extra releases. The risk of extra acquires is that the thread might exit while still holding the lock, leaving it locked forever. However, just parking a thread indefinitely (via various means) within a synchronized block is equally unsafe. The risk of extra releases is that you'll hit IllegalMonitorState exceptions and thread-safety/publication problems. But a user can make this same mistake today with locks. The difference is, that in most cases people would choose the convenience of synchronized unless there is a specific reason for it, making this problem far less likely than the corresponding locks case. -- - DML
Re: JDK 9 RFR of 6375303: Review use of caching in BigDecimal
On Mar 4 2014, at 07:13 , Peter Levart peter.lev...@gmail.com wrote: On 03/04/2014 01:14 AM, Brian Burkhalter wrote: - add AtomicReferenceFieldUpdater-type constant for stringCache initialization Hi Brian, By using volatile read and CAS, there's still a chance that multiple concurrent threads will be invoking the layoutChars(true) concurrently, but you guarantee that only a single instance of String will ever be returned as a result of the toString() method. Is that what you were pursuing? Yes. (I provided the AtomicReferenceFieldUpdater code). The previous implementation had a benign data race that could result in a later layoutChars result replacing an earlier result and multiple string instances being returned. The new implementation, at small cost, prevents multiple different instances from being returned. Mike
Re: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger
On Mar 3, 2014, at 11:37 AM, Brian Burkhalter wrote: The fields bitCount, bitLength, and lowestSetBit appear in the serialized form only for backward compatibility and are otherwise ignored, so their @serialField entries should just say that instead of describing how they were formerly used. Also, firstNonzeroByteNum is missing a @serialField entry, and it should have the same description as the others. Corrected I found another small item here. Currently we have for serialPersistentFields the following: 4203 * @serialField magnitude int[] 4204 * magnitude array of this BigInteger and 4216 new ObjectStreamField(magnitude, byte[].class), In the @serialField annotation the type should also be byte[], no? Thanks, Brian
[9] RFR (S): 8036117: MethodHandles.catchException doesn't handle VarargsCollector right (8034120 failed)
http://cr.openjdk.java.net/~vlivanov/8036117/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8036117 84 lines changed: 74 ins; 3 del; 7 mod I have to revert a cleanup I did for 8027827. MethodHandle.invokeWithArguments (and generic invocation) has unpleasant peculiarity in behavior when used with VarargsCollector. So, unfortunately, invokeWithArguments is not an option there. Looking at the API (excerpts from javadoc [1] [2]), the following condition doesn't hold in that case: trailing parameter type of the caller is a reference type identical to or assignable to the trailing parameter type of the adapter. Example: target.invokeWithArguments((Object[])args) = target.invoke((Object)o1,(Object)o2,(Object)o3) =/ target.invokeExact((Object)o1, (Object)o2, (Object[])o3) because Object !: Object[]. The fix is to skip unnecessary conversion when invoking a method handle and just do a pairwise type conversion. Testing: failing test case, nashorn w/ experimental features (octane) Thanks! Best regards, Vladimir Ivanov [1] MethodHandle.invokeWithArguments Performs a variable arity invocation, ..., as if via an inexact invoke from a call site which mentions only the type Object, and whose arity is the length of the argument array. [2] MethodHandle.asVarargsCollector When called with plain, inexact invoke, if the caller type is the same as the adapter, the adapter invokes the target as with invokeExact. (This is the normal behavior for invoke when types match.) Otherwise, if the caller and adapter arity are the same, and the trailing parameter type of the caller is a reference type identical to or assignable to the trailing parameter type of the adapter, the arguments and return values are converted pairwise, as if by asType on a fixed arity method handle. Otherwise, the arities differ, or the adapter's trailing parameter type is not assignable from the corresponding caller type. In this case, the adapter replaces all trailing arguments from the original trailing argument position onward, by a new array of type arrayType, whose elements comprise (in order) the replaced arguments.
Fedora sun.misc.Unsafe statistics
Out of curiosity, I looked at JAR files in all RPMs in Fedora rawhide (the development version of Fedora that will turn into Fedora 21) and counted the number of RPM packages which had Java classes referencing individual methods in sun.misc.Unsafe. The attached table is sorted by decreasing package count, and grouped by method name and descriptor. These numbers do not include references through reflection, for obvious reasons. Java programming practices are sometimes at odds with distribution packaging, and due to licensing bias, the Fedora repository might not be representative of the general Java ecosystem. However, the statistics cover 1177 Java source RPMs containing 550829 classes in 570477 versions, so it's quite a bit of code, despite the fairly low number of references to sun.misc.Unsafe. -- Florian Weimer / Red Hat Product Security Team name | descriptor | count --++--- objectFieldOffset| (Ljava/lang/reflect/Field;)J |17 arrayBaseOffset | (Ljava/lang/Class;)I |14 putObject| (Ljava/lang/Object;JLjava/lang/Object;)V |12 arrayIndexScale | (Ljava/lang/Class;)I |11 getLong | (Ljava/lang/Object;J)J |11 compareAndSwapInt| (Ljava/lang/Object;JII)Z | 9 compareAndSwapLong | (Ljava/lang/Object;JJJ)Z | 9 getUnsafe| ()Lsun/misc/Unsafe; | 9 compareAndSwapObject | (Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z | 8 getObject| (Ljava/lang/Object;J)Ljava/lang/Object; | 8 getInt | (Ljava/lang/Object;J)I | 7 putInt | (Ljava/lang/Object;JI)V | 6 putLong | (Ljava/lang/Object;JJ)V | 6 getObjectVolatile| (Ljava/lang/Object;J)Ljava/lang/Object; | 5 putBoolean | (Ljava/lang/Object;JZ)V | 5 putByte | (Ljava/lang/Object;JB)V | 5 putChar | (Ljava/lang/Object;JC)V | 5 putDouble| (Ljava/lang/Object;JD)V | 5 putFloat | (Ljava/lang/Object;JF)V | 5 putObjectVolatile| (Ljava/lang/Object;JLjava/lang/Object;)V | 5 putOrderedObject | (Ljava/lang/Object;JLjava/lang/Object;)V | 5 putShort | (Ljava/lang/Object;JS)V | 5 getBoolean | (Ljava/lang/Object;J)Z | 4 getByte | (Ljava/lang/Object;J)B | 4 getChar | (Ljava/lang/Object;J)C | 4 getDouble| (Ljava/lang/Object;J)D | 4 getFloat | (Ljava/lang/Object;J)F | 4 getInt | (J)I | 4 getLong | (J)J | 4 getShort | (J)S | 4 getShort | (Ljava/lang/Object;J)S | 4 park | (ZJ)V | 4 putInt | (JI)V | 4 putLong | (JJ)V | 4 putShort | (JS)V | 4 throwException | (Ljava/lang/Throwable;)V | 4 unpark | (Ljava/lang/Object;)V | 4 copyMemory | (JJJ)V | 3 copyMemory | (Ljava/lang/Object;JLjava/lang/Object;JJ)V | 3 getByte | (J)B | 3 putByte | (JB)V | 3 putOrderedInt| (Ljava/lang/Object;JI)V | 3 allocateInstance | (Ljava/lang/Class;)Ljava/lang/Object;
Re: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger
Hi Brian, Just a couple small items. At line 4203, the type of magnitude should be byte[] instead of int[]. Whoops, I could have sworn I wrote that in my previous review, but it must have gotten dropped while I was editing. Sorry about that. For the four compatibility fields in the serial form, the comment is appears in the serialized for backward compatibility Something is missing here. Should it say appears in the serialized form for backward compatibility ? The comment block at lines 4300-4306 is good. I might also add a note to say these values are compatible with older implementations. There are some things in the serialization doc that ought to be brought up to date, though. Note that the docs for serialPersistentFields, readObject, and writeObject appear in the javadoc output, in the Serialized Form page, even though these members are private! Isn't this controlled by options passed to the javadoc tool as opposed to settings in the source code? No, serialization is special in that all information about the serialized form, including the docs these special private methods and fields, do appear in the Serialized Form output, regardless of the javadoc tool arguments. I think I'll need another thumbs up as this has changed since Paul's approval was posted. Paul is not available this week. If you want to make these corrections and then just push the changeset, it's fine by me; I think it's had enough review. s'marks On 3/3/14 11:37 AM, Brian Burkhalter wrote: Hi Stuart, Thanks for the detailed review! Please see the refreshed webrev http://cr.openjdk.java.net/~bpb/8035279/webrev.02/ and my comments inline, below. I think I'll need another thumbs up as this has changed since Paul's approval was posted. On Feb 28, 2014, at 5:35 PM, Stuart Marks wrote: Thanks, Paul. I refreshed the webrev http://cr.openjdk.java.net/~bpb/8035279/webrev.01/ with the agreed upon version. This is pretty good. After this long, strange trip through the JMM, restoring the sentinel values to zeroes and renaming the fields to be explicit about how they represent the actual values seems to be the best approach. Paul's suggestion about using the term stable value in comments is good too. I took a look at the serialization stuff. The actual serialized form hasn't changed, so there should be no compatibility here with previous versions. There are some things in the serialization doc that ought to be brought up to date, though. Note that the docs for serialPersistentFields, readObject, and writeObject appear in the javadoc output, in the Serialized Form page, even though these members are private! Isn't this controlled by options passed to the javadoc tool as opposed to settings in the source code? Per another of Paul's comments, the @serial tag should be removed from bitCountPlusOne, bitLengthPlusOne, and lowestSetBitPlusTwo, since these fields do not appear in the serialized representation. Corrected. The fields bitCount, bitLength, and lowestSetBit appear in the serialized form only for backward compatibility and are otherwise ignored, so their @serialField entries should just say that instead of describing how they were formerly used. Also, firstNonzeroByteNum is missing a @serialField entry, and it should have the same description as the others. Corrected. Typo at 4236-4237, it says be\ndefault instead of by\ndefault. Corrected. The comment at lines 4242-4246 should simply be removed. The first and third sentences are redundant with other docs. The second sentence, The magnitude field is used as a temporary store for the byte array that is deserialized is incorrect, as there is no longer a 'magnitude' field; a local is used instead. Corrected. The @serialData tag at line 4316 for writeObject is misused; this is really intended for *extra* serial data written by writeObject after the writeFields() or defaultWriteObject() call, which doesn't occur here. It might be worth being explicit in writeObject's doc comment about writing -1's and -2's as the values for bitCount, bitLength, lowestSetBit, and firstNonzeroByteNum for compatibility with older implementations, even though current implementations will ignore these values. Corrected. On Feb 28, 2014, at 5:54 PM, Stuart Marks wrote: Woops, I forgot a couple points. The @serial for the 'signum' field at line 130 should be removed. Note that even though this has the same name as the field that appears in the serialized form, the text from the @serialField tag for 'signum' that's part of the serialPersistentFields doc comment is the one that actually appears on the Serialized Form page. Corrected. It might be worthwhile copying the more verbose description from lines 125-128 to the @serialField tag text (lines 4206-4207) since this describes the requirements on the serialized form more precisely. I hate redundancy in documentation, though. I opted for not copying the verbiage in the interested of reduced
Re: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger
Hi Stuart, On Mar 4, 2014, at 1:09 PM, Stuart Marks wrote: Just a couple small items. At line 4203, the type of magnitude should be byte[] instead of int[]. Whoops, I could have sworn I wrote that in my previous review, but it must have gotten dropped while I was editing. Sorry about that. I caught that myself and fixed it. For the four compatibility fields in the serial form, the comment is appears in the serialized for backward compatibility Something is missing here. Should it say appears in the serialized form for backward compatibility ? I also caught and fixed that. The comment block at lines 4300-4306 is good. I might also add a note to say these values are compatible with older implementations. Will do (I'll refresh the updated webrev at the link below). There are some things in the serialization doc that ought to be brought up to date, though. Note that the docs for serialPersistentFields, readObject, and writeObject appear in the javadoc output, in the Serialized Form page, even though these members are private! Isn't this controlled by options passed to the javadoc tool as opposed to settings in the source code? No, serialization is special in that all information about the serialized form, including the docs these special private methods and fields, do appear in the Serialized Form output, regardless of the javadoc tool arguments. Thanks for the clarification. I think I'll need another thumbs up as this has changed since Paul's approval was posted. Paul is not available this week. If you want to make these corrections and then just push the changeset, it's fine by me; I think it's had enough review. Sounds good. http://cr.openjdk.java.net/~bpb/8035279/webrev.03/ I think I have to get a CCC request approved first however but if the approval is in place I can push immediately thereafter. Thanks, Brian
Re: JEP 193: Enhanced Volatiles
On 03/03/2014 04:53 PM, David M. Lloyd wrote:
On 03/03/2014 04:25 PM, Brian Goetz wrote:
Posted: http://openjdk.java.net/jeps/193
Some follow-up thoughts on teasing apart the issues covered by this JEP.
There are three main layers of questions to answer:
1. How do we surface the various pieces of this into the programming
model? This includes language syntax (e.g.,
x.volatile.compareAndSet()), library surface (e.g., the fictitious and
not-terribly-satisfying VolatileInt interface), and relevant
restrictions (e.g., can we do volatile operations on non-volatile fields
or array elements?)
[...and then...]
2. Translation to bytecodes. What bytecode should javac emit when it
encounters an volatile accessor or atomic update? We've identified a
handful of candidates:
2a: new bytecodes. This is the most direct and pure, but most intrusive
(and least extensible), means of delivering on the promise of it's time
to move this stuff into the programming model proper. The wide
bytecode offers a means to express fenced variants of
{get,put}{field,static} with only a single new bytecode, but this
doesn't scale well to CAS (explosion of data types and data locations
(static field, instance field, array element)), which is really the
important case.
Somewhere between new bytecodes and field handles, perhaps a single
bytecode could suffice - one which is similar to getfield except instead
of reading the field, it pushes a virtual object on to the stack which
can then only be operated upon by invokeinterface on the pertinent
VolatileXxx interface and otherwise doesn't have a real type per se.
I suppose you'd also need a second bytecode for doing the same thing
with array elements like *aload, if that branch of this idea is pursued.
--
- DML
Re: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger
OK, thanks for fixing up this stuff, even before I had mentioned it. As far as I'm concerned the change is ready to go in, but as you mention we still need to get (internal) approval for spec changes. s'marks On 3/4/14 1:12 PM, Brian Burkhalter wrote: Hi Stuart, On Mar 4, 2014, at 1:09 PM, Stuart Marks wrote: Just a couple small items. At line 4203, the type of magnitude should be byte[] instead of int[]. Whoops, I could have sworn I wrote that in my previous review, but it must have gotten dropped while I was editing. Sorry about that. I caught that myself and fixed it. For the four compatibility fields in the serial form, the comment is appears in the serialized for backward compatibility Something is missing here. Should it say appears in the serialized form for backward compatibility ? I also caught and fixed that. The comment block at lines 4300-4306 is good. I might also add a note to say these values are compatible with older implementations. Will do (I'll refresh the updated webrev at the link below). There are some things in the serialization doc that ought to be brought up to date, though. Note that the docs for serialPersistentFields, readObject, and writeObject appear in the javadoc output, in the Serialized Form page, even though these members are private! Isn't this controlled by options passed to the javadoc tool as opposed to settings in the source code? No, serialization is special in that all information about the serialized form, including the docs these special private methods and fields, do appear in the Serialized Form output, regardless of the javadoc tool arguments. Thanks for the clarification. I think I'll need another thumbs up as this has changed since Paul's approval was posted. Paul is not available this week. If you want to make these corrections and then just push the changeset, it's fine by me; I think it's had enough review. Sounds good. http://cr.openjdk.java.net/~bpb/8035279/webrev.03/ I think I have to get a CCC request approved first however but if the approval is in place I can push immediately thereafter. Thanks, Brian
Re: [REFRESH] JDK 9 RFR of JDK-8035279: Clean up internal deprecations in BigInteger
On Mar 4, 2014, at 1:38 PM, Stuart Marks wrote: OK, thanks for fixing up this stuff, even before I had mentioned it. This is a positive race condition! As far as I'm concerned the change is ready to go in, but as you mention we still need to get (internal) approval for spec changes. OK. I'll push it then once the spec change is approved (assuming it is). Thanks for multiple, careful reviews. Brian
Re: JEP 193: Enhanced Volatiles
On 03/04/2014 01:05 PM, Doug Lea wrote: On 03/04/2014 02:41 AM, Jeroen Frijters wrote: Brian Goetz wrote: Embedded in this proposal is the desire to not provide a full-blown lvalue form for variables; supporting any form of pass-by-reference at the language level is a super-non-goal here. Why is this? It solves these problems in an extremely clean way and also provides lots of other value (for example, for JEP 191: Foreign Function Interface). I understand pass-by-reference is an expensive feature, but IMNSHO poluting Java with this proposal will prove to be more expensive in the long run. It's like erased generics all over again. The expensive version of pass-by-reference is already supported using java.lang.reflect.Field. And per the statistics posted in http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-March/025531.html, the slightly faster pointer-to-field-member support is one of the prevalent use cases for sun.misc.Unsafe. That's why I share Jeroen's puzzlement. -- Florian Weimer / Red Hat Product Security Team
Re: JEP 193: Enhanced Volatiles
On 03/04/2014 05:12 PM, Florian Weimer wrote: On 03/04/2014 01:05 PM, Doug Lea wrote: On 03/04/2014 02:41 AM, Jeroen Frijters wrote: I understand pass-by-reference is an expensive feature, but IMNSHO poluting Java with this proposal will prove to be more expensive in the long run. It's like erased generics all over again. The expensive version of pass-by-reference is already supported using java.lang.reflect.Field. And per the statistics posted in http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-March/025531.html, the slightly faster pointer-to-field-member support is one of the prevalent use cases for sun.misc.Unsafe. That's why I share Jeroen's puzzlement. Sorry, I'm not sure what usages you have in mind, or what constructions and implementable JVM mechanics could be used to deal with them? -Doug
Re: JEP 193: Enhanced Volatiles
Embedded in this proposal is the desire to not provide a full-blown lvalue form for variables; supporting any form of pass-by-reference at the language level is a super-non-goal here. Why is this? It solves these problems in an extremely clean way and also provides lots of other value (for example, for JEP 191: Foreign Function Interface). I understand pass-by-reference is an expensive feature, but IMNSHO poluting Java with this proposal will prove to be more expensive in the long run. It's like erased generics all over again. It has nothing to do with it being expensive (though it is); it has to do with being an outright *bad idea* from a stewardship perspective. Right now, the semantics of method calls in Java are simple -- everything (primitives, object references) is passed by value. Adding pass-by-reference would add significant complexity. And method calls are not a niche feature; that added complexity will be borne by every developer, every day. On the other hand, these weird foo.volatile.xxx operations would be used by .0001% of Java developers (congratulations, you're special!) Doesn't it make sense to move the damage to where it only affects those who need it? Or, looking at it the other way, does it make any sense at all to make a key facet of the language dramatically more complex just so that a niche cadre of sewer-dwelling expert users can more easily write the code they want? Spending most of your time in the performance-critical coding sewers often has the unfortunate effect of anti-qualifying you for reasoning about what platform features would be good for the community as a whole. My goal here is to make sure that expert users can get their job done somehow, *without* making the job of mainstream developers harder. The add lvalues to Java so experts can write CAS-libraries fails that test miserably. Note that we're not adding any new functionality here; what we're doing is pulling some functionality from Unsafe into the public programming model. Making drastic changes to a key language feature to support that would be killing a fly with a bazooka. And, as I said at the top, its a bad idea anyway.
JDK 9 RFR of JDK-8036568: Serial incompatibility in java.util.TreeMap.NavigableSubMap
Hello,
Following up from the recent discussions around
JDK-8035452: Fix serial lint warnings in core libs
it turns out that java.util.TreeMap.NavigableSubMap, a package private
nested class used in the TreeMap implementation does indeed need to have
a serialVersionUID defined to allow for easy cross-release serialization.
An API change made to TreeMap during JDK 8 changed the serialver hash of
the class. This patch
--- a/src/share/classes/java/util/TreeMap.javaMon Mar 03 18:17:00
2014 +0400
+++ b/src/share/classes/java/util/TreeMap.javaTue Mar 04 17:41:09
2014 -0800
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 1997, 2013, Oracle and/or its affiliates. All rights
reserved.
+ * Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights
reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -1335,6 +1335,7 @@
*/
abstract static class NavigableSubMapK,V extends AbstractMapK,V
implements NavigableMapK,V, java.io.Serializable {
+private static final long serialVersionUID = -2102997345730753016L;
/**
* The backing map.
*/
restores the serial hash value present in JDK 6 and 7.
This particular review request is for JDK 9; we are evaluating options
on how best to fix this issue in the JDK 8 train.
Thanks,
-Joe
Re: JDK 9 RFR of JDK-8035452: Fix serial lint warnings in core libs
Hi Joe,
These changes are fine. You might want to update the copyright year of
ExceptionProxy.java though.
s'marks
On 3/3/14 10:05 PM, Joe Darcy wrote:
Hi Stuart,
Thanks for the careful review. I agree the case of TreeMap.NavigableSbugMap
requires some more investigation. In the meantime, I'd like to proceed with the
revised patch below for EnumSet and ExceptionProxy.
Thanks,
-Joe
diff -r 6cfedc362f48 src/share/classes/java/util/EnumSet.java
--- a/src/share/classes/java/util/EnumSet.javaMon Mar 03 18:17:00 2014 +0400
+++ b/src/share/classes/java/util/EnumSet.javaMon Mar 03 22:02:06 2014 -0800
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2003, 2014, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -77,6 +77,8 @@
* @see EnumMap
* @serial exclude
*/
+@SuppressWarnings(serial) // No serialVersionUID due to usage of
+// serial proxy pattern
public abstract class EnumSetE extends EnumE extends AbstractSetE
implements Cloneable, java.io.Serializable
{
diff -r 6cfedc362f48
src/share/classes/sun/reflect/annotation/ExceptionProxy.java
--- a/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Mon Mar 03
18:17:00 2014 +0400
+++ b/src/share/classes/sun/reflect/annotation/ExceptionProxy.java Mon Mar 03
22:02:06 2014 -0800
@@ -37,5 +37,6 @@
* @since 1.5
*/
public abstract class ExceptionProxy implements java.io.Serializable {
+private static final long serialVersionUID = 7241930048386631401L;
protected abstract RuntimeException generateException();
}
On 02/28/2014 08:58 AM, Stuart Marks wrote:
On 2/27/14 12:11 PM, Joe Darcy wrote:
I am trying hard to remain blissfully ignorant of any more low-level details of
the serialization format; however, I might not be successful on that goal much
longer ;-)
I believe your latter statement is correct. :-)
My preference in a case like this is to add the svuid if for no other reason
that is is simple to explain and understand, even if it is not strictly
required.
In general, it does seem reasonable to add a svuid in cases where it's
difficult or impractical to prove that the lack of a svuid causes no
compatibility issues.
There is a difference in character between a serializable class in Java SE
(java.* and javax.*) and the jdk.Exported(true) types in the JDK and a
serializable class that lives in sun.* or some other jdk.Exported(false) area.
For that latter, the serialization contract has to be different, with fewer
guarantees, just as the general usage contract for those types has fewer
guarantees. I think this is analogous to putting non-serializable classes into
collections; the collection itself is serializable, but it won't be anymore if
you put non-serializable objects into it.
If a user happens to have a direct or indirect reference to an object of a JDK
implementation type, the compatibility contract is weaker than if an object with
a public Java SE type were being dealt with.
I'm not sure I buy this. Unfortunately, serialization differs from the general
usage contract in that serialization exposes internals. Just like it can
expose private (non-transient) fields, serialization can also can expose what
might otherwise look like purely internal implementation types.
The canonical example of how an application can get a reference to an
apparently internal class is java.util.TimeZone. If an app calls
TimeZone.getDefault(), it gets back an instance of sun.util.calendar.ZoneInfo
without ever mentioning this class by name. Furthermore, TimeZone and ZoneInfo
are serializable, so they can be serialized directly or as part of another
serializable object graph, and an instance of s.u.c.ZoneInfo does appear in
the serialized byte stream. If s.u.c.ZoneInfo were not to have a svuid, an
apparently innocuous change to it would clearly cause application
incompatibilities.
As it happens, s.u.c.ZoneInfo does have a svuid. I also note in passing that
TimeZone, an abstract class, also has a svuid.
Finally, EnumSet doesn't need a serial version UID. It's serialized using a
proxy class, so EnumSet never appears in a serialized byte stream. (Note, its
readObject throws an exception unconditionally.) So it's probably safe to
suppress its serialization warning.
Yes, EnumSet was a bit tricky, it is serializable itself, but uses a proxy
internally. (Effective Java, 2nd edition both recommends the proxy pattern and
recommends adding a svuid to all serializable classes, but doesn't explicitly
give guidance to this combination of features.)
To avoid adding a long comment explaining the proxy pattern and why a svuid on
EnumSet isn't really required, my preference would just be to add the svuid if
it doesn't cause any harm.
In this case I think it's possible by
Re: JDK 9 RFR of JDK-8036568: Serial incompatibility in java.util.TreeMap.NavigableSubMap
Hi Joe,
Fix looks good. I confirmed that the svuid value in your patch is the right one
to be compatible with 6 and 7, at least for all the versions of those releases
that I have at my fingertips (which is several). And yes, we need to figure out
how best to fix this in the 8 release family.
s'marks
On 3/4/14 5:44 PM, Joe Darcy wrote:
Hello,
Following up from the recent discussions around
JDK-8035452: Fix serial lint warnings in core libs
it turns out that java.util.TreeMap.NavigableSubMap, a package private nested
class used in the TreeMap implementation does indeed need to have a
serialVersionUID defined to allow for easy cross-release serialization.
An API change made to TreeMap during JDK 8 changed the serialver hash of the
class. This patch
--- a/src/share/classes/java/util/TreeMap.javaMon Mar 03 18:17:00 2014 +0400
+++ b/src/share/classes/java/util/TreeMap.javaTue Mar 04 17:41:09 2014 -0800
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 1997, 2013, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 1997, 2014, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@@ -1335,6 +1335,7 @@
*/
abstract static class NavigableSubMapK,V extends AbstractMapK,V
implements NavigableMapK,V, java.io.Serializable {
+private static final long serialVersionUID = -2102997345730753016L;
/**
* The backing map.
*/
restores the serial hash value present in JDK 6 and 7.
This particular review request is for JDK 9; we are evaluating options on how
best to fix this issue in the JDK 8 train.
Thanks,
-Joe
Re: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods
On 5/03/2014 1:32 AM, David M. Lloyd wrote: On 03/03/2014 09:45 PM, David Holmes wrote: On 3/03/2014 10:56 PM, David M. Lloyd wrote: Yes, that would necessarily be the contract of a Monitors class, just as it is part of the contract of Lock today. If your argument is that it shouldn't be allowed because it might be used wrong, we might as well just delete most of the JDK, ReentrantLock included, since it suffers from the exact same potential problem. The difference is that monitors have a simple API in the form of synchronized that people were in the past and would continue to be free (and recommended) to use. We should not introduce anything that allows something that was guaranteed to be safe by the language, to become unsafe. Define 'safe'. Because I don't think it's unsafe, any more than unpaired lock/unlock are. There is no uninitialized memory being accessed, no way to bypass security checks, no way to manipulate memory directly, etc. I find this argument to be particularly weak without specific cases. As far as I can tell, the language doesn't make it safe, it makes it *convenient*, and that's a big distinction. I agree it is as unsafe as using Lock. I think not being able to forget to unlock a monitor is more than just convenient. YMMV and obviously does. David - There are two cases I can see that are potentially problematic: extra acquires, and extra releases. The risk of extra acquires is that the thread might exit while still holding the lock, leaving it locked forever. However, just parking a thread indefinitely (via various means) within a synchronized block is equally unsafe. The risk of extra releases is that you'll hit IllegalMonitorState exceptions and thread-safety/publication problems. But a user can make this same mistake today with locks. The difference is, that in most cases people would choose the convenience of synchronized unless there is a specific reason for it, making this problem far less likely than the corresponding locks case.
Re: Unsafe: removing the monitorEnter/monitorExit/tryMonitorEnter methods
On Mar 4, 2014, at 7:41 PM, David Holmes david.hol...@oracle.com wrote: I agree it is as unsafe as using Lock. I think not being able to forget to unlock a monitor is more than just convenient. YMMV and obviously does. That's why that unsafe kind of stuff currently lives in class unsafe. Next there are safe ways to build on those primitives to create safe APIs, using encapsulation and access restriction. The Runnable trick for matching monitors is (probably) compatible with low-level unsafe unmatched monitor operations. Another metaphor: It's OK to build an office building using tools that the office workers won't have daily access to. Construction sites have different safety rules than office environments. Some folks get to use nailguns at work; I don't, but that doesn't make me want my building to be constructed using staplers only. — John
RE: JEP 193: Enhanced Volatiles
Brian Goetz wrote: Right now, the semantics of method calls in Java are simple -- everything (primitives, object references) is passed by value. Adding pass-by-reference would add significant complexity. And method calls are not a niche feature; that added complexity will be borne by every developer, every day. We'll just have to disagree on this. VB developers are not more sophisticated than Java developers and they've dealt with ByRef just fine (even though many don't understand it). In practice they don't run into it that often. My goal here is to make sure that expert users can get their job done somehow, *without* making the job of mainstream developers harder. The add lvalues to Java so experts can write CAS-libraries fails that test miserably. Why not go for something far less intrusive then? Here's my straw man proposal: Add an annotation that can be placed on native methods to synthesize atomic accessor methods. Example usage: @AtomicField(next) private native boolean compareAndSet(Node expected, Node newValue); @AtomicArray private static native boolean compareAndSet(Node[] array, Node expected, Node newValue); (Note that the method name is not significant, the operation can be derived from the signature, or explicit in the annotation, if necessary.) This requires no changes to the language and adds only a slight burden on the developer (but it's very easy to add tooling support). Regards, Jeroen
Re: JEP 193: Enhanced Volatiles
Am 05.03.2014 um 08:40 schrieb Jeroen Frijters jer...@sumatra.nl: My goal here is to make sure that expert users can get their job done somehow, *without* making the job of mainstream developers harder. The add lvalues to Java so experts can write CAS-libraries fails that test miserably. Why not go for something far less intrusive then? Here's my straw man proposal: Add an annotation that can be placed on native methods to synthesize atomic accessor methods. Example usage: @AtomicField(next) private native boolean compareAndSet(Node expected, Node newValue); @AtomicArray private static native boolean compareAndSet(Node[] array, Node expected, Node newValue); (Note that the method name is not significant, the operation can be derived from the signature, or explicit in the annotation, if necessary.) This requires no changes to the language and adds only a slight burden on the developer (but it's very easy to add tooling support). Regards, Jeroen That looks like a good fit towards what was mentioned earlier using method handles. The JVM or compiler could just look out for those annotations and generate corresponding method / field handles to execute it. Thanks, Chris
