Re: Floating Point Repair?
On 27.04.22 12:47, sminervini.prism wrote: To the Java OpenJDK Java Community Process, [...] You could propose a new data type that follows BCDs from IEE754-2008. You would have to define conversions of course. You would have to of course define type conversions, find somebody doing whatever a JSR is today, extend hotspot and the java compiler, or prototype with a software emulation. bye Jochen
Re: Getting a live view of environment variables (Gradle and JDK 9)
On 19.05.2017 08:08, Noctarius wrote: Hey Jochen, Couldn't the gradle call transfer his view of the environment? I know that's a workaround but it would solve the issue, wouldn't it? IMHO it could so by calling native code using posix functions. But what does it matter if getEnv is not updated? So you need to bytecode-rewrite that... and frankly, then you will no longer need the native code approach. bye Jochen
Re: Getting a live view of environment variables (Gradle and JDK 9)
On 16.05.2017 19:46, Uwe Schindler wrote: Hi, I still don't understand why you need to change the environment variables of the actual process. I was talking with Rémi about that last week, and it's not obvious to me why you need that. Sure, it is easier to change the environment of the actual process and then spawn a child process for some non-java build tool like GCC or even another standalone java program with option fork=true. But: Why not use the ProcessBuilder API when spawning those childs? So you just add an "official" build API inside Gradle (an official one, documented that allows Gradle plugins to modify the environment variables for the current build running). If you missed to add such an API from the beginning (IMHO its essential for a build system like Gradle), then you now have to tell your users: "Sorry we did something wrong and all our (bad) hacks to allow you to change environment variables are no longer working in the future, so please change your code. We are so sorry!" the daemon outlives the gradle call, and the next gradle call can be done in a changed environment, thus the need to update the daemon. Spawning a child process each time would be very bad for performance. bye Jochen
Re: RFR: 8177086: java.lang.reflect.AccessibleObject::canAccess should share access cache with internal method ::checkAccess
frankly I would migrate more from the normal reflection API to the MethodHandles API instead of extending the Reflection API further, but I see the use case and reasoning here. thanks. bye Jochen On 20.03.2017 10:04, Alan Bateman wrote: On 19/03/2017 22:47, Jochen Theodorou wrote: not wanting to hijack the thread, but why is there no canAccess method that takes a class argument to check if that class can access? Why always depending so much on caller sensitive methods? This method is intended to be used in conjunction with Constructor.newInstance, Method.invoke, Field.get, ... The idiom in Peter's mail combines this with trySetAccessible and makes it easy to gracefully handle cases where access is not allowed. JEP 274 added Lookup.accessClass and may be closer to what you need. -Alan
Re: RFR: 8177086: java.lang.reflect.AccessibleObject::canAccess should share access cache with internal method ::checkAccess
On 19.03.2017 18:52, Peter Levart wrote: Hi, Module system implementation refresh 2017/2 (JDK-8173393) introduced new API method AccessibleObject::canAccess which can be used to test if the caller has access to the reflected member (with possible target object argument for instance members). not wanting to hijack the thread, but why is there no canAccess method that takes a class argument to check if that class can access? Why always depending so much on caller sensitive methods? bye Jochen
Re: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch
Hi, I strongly hope Paul and Cedric will be able to start the release process next week, if not we will have to do it the old way I think. what would help us a lot would be you testing the GROOVY_2_4_X branch with your build system to see if it really does solve your problem. Even if it is only locally on your computer bye Jochen On 16.12.2016 10:58, Uwe Schindler wrote: Hi Jochen, thank you for the information! Is there any plan about a release? I also found no JIRA issue about this issue to link it against our JIRA: https://issues.apache.org/jira/browse/LUCENE-7596 The problem makes our build system unusable, so it would be very important to have a fix quite soon! As our Ant/Ivy-based build relies on Maven Central, it would be good to have the bugfix release available there, which requires a release. I think the same applies for Gradle users (Elasticsearch). As a temporary workaround we might be able to use the Apache Snapshot repository, but this is not allowed if we do a release of Lucene. Uwe - Uwe Schindler uschind...@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ -Original Message- From: Jochen Theodorou [mailto:blackd...@gmx.org] Sent: Saturday, December 10, 2016 9:23 AM To: Uwe Schindler <uschind...@apache.org>; jigsaw-...@openjdk.java.net; Core-Libs-Dev <core-libs-dev@openjdk.java.net> Subject: Re: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch On 09.12.2016 23:32, Uwe Schindler wrote: Hi, I updated our Jenkins server for the JDK 9 preview testing to use build 148. Previously we had build 140 and build 147, which both worked without any issues. But after the update the following stuff goes wrong: (1) Unmapping of direct buffers no longer works, although this API was marked as critical because there is no replacement up to now, so code can unmap memory mapped files, which is one of the most important things Apache Lucene needs to use to access huge random access files while reading the index. Without memory mapping, the slowdown for Lucene users will be huge This is caused by the recent Jigsaw changes, published in build 148. Unfortunately we did not test the Jigsaw builds, so we would have noticed that earlier. Basically the following peace of code fails now (with or without doPrivileged and with/without security manager): final Class directBufferClass = Class.forName("java.nio.DirectByteBuffer"); final Method m = directBufferClass.getMethod("cleaner"); m.setAccessible(true); MethodHandle directBufferCleanerMethod = lookup.unreflect(m); Class cleanerClass = directBufferCleanerMethod.type().returnType(); // build method handle for unmapping, full code is here: https://goo.gl/TfQWl6 I guess that is the effect of #AwkwardStrongEncapsulation. I would advise doing regular checks against the jigsaw builds to know about such problems in the future earlier... but seeing your code break without an obvious good solution sure is stressful. I feel with you. [...] (2) A second thing we noticed is that Groovy no longer works and dies with strange error messages. That is because versions including Groovy 2.4.7 are using setAccessible(AccessibleObject[] array, true), and the array will also include private methods or fields. This worked till #AwkwardStrongEncapsulation because will then a class was either exported and its method can all be made accessible or not. For example on GAE or earlier versions of the module system. Now an exported class may break this, since its private methods can no longer be made accessible using setAccessible. A fix for this is already committed, we are only waiting for release of Groovy 2.4.8. Of course even with the fix Groovy code can possibly break... for example if you did the direct buffer access in Groovy. Btw, do not hesitate to ask about such problems on groovy-user, please. bye Jochen
Re: AccessbileObject setAccessible array version vs non-array version
On 10.12.2016 13:08, Alan Bateman wrote: On 10/12/2016 09:13, Jochen Theodorou wrote: Hi all, motivated by the recent "Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch" thead, I thought I ask... there is AcccessibleObject#setAccessible(boolean), which will ask the SecurityManager for permissions and then make itself accessible according to the boolean flag. the array version takes an array of AccessibleObjects, asks the security manager *once* and then makes all of them accessible. So if you are in need to make a lot of objects accessible the array version is superior in performance. Now with jigsaw it is no longer a all or nothing for the class, now single methods or fields may no longer be made accessible, even without security manager. That means that even without a security manager set using the array version on File for example will fail with an exception (unless the module is opened from the command line, but we should leave that out for now) My question now basically is the following... why is this method not made deprecated? It is kind of useless now, even misleading I would say. I'm not sure that I understand your mail. The permission check when running with a security manager has not changed. If you use the array version then there is one permission check. Maybe you mean that the array version will fail when the array contains at least one element where the access check cannot be suppressed? That is possible of course. You mentioned File and maybe you mean you the array has a mix of public methods and non-public members and fields? yes. You will not need to set them to accessible for public members after all bye Jochen
AccessbileObject setAccessible array version vs non-array version
Hi all, motivated by the recent "Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch" thead, I thought I ask... there is AcccessibleObject#setAccessible(boolean), which will ask the SecurityManager for permissions and then make itself accessible according to the boolean flag. the array version takes an array of AccessibleObjects, asks the security manager *once* and then makes all of them accessible. So if you are in need to make a lot of objects accessible the array version is superior in performance. Now with jigsaw it is no longer a all or nothing for the class, now single methods or fields may no longer be made accessible, even without security manager. That means that even without a security manager set using the array version on File for example will fail with an exception (unless the module is opened from the command line, but we should leave that out for now) My question now basically is the following... why is this method not made deprecated? It is kind of useless now, even misleading I would say. bye Jochen
Re: Java 9 build 148 causes trouble in Apache Lucene/Solr/Elasticsearch
On 09.12.2016 23:32, Uwe Schindler wrote: Hi, I updated our Jenkins server for the JDK 9 preview testing to use build 148. Previously we had build 140 and build 147, which both worked without any issues. But after the update the following stuff goes wrong: (1) Unmapping of direct buffers no longer works, although this API was marked as critical because there is no replacement up to now, so code can unmap memory mapped files, which is one of the most important things Apache Lucene needs to use to access huge random access files while reading the index. Without memory mapping, the slowdown for Lucene users will be huge This is caused by the recent Jigsaw changes, published in build 148. Unfortunately we did not test the Jigsaw builds, so we would have noticed that earlier. Basically the following peace of code fails now (with or without doPrivileged and with/without security manager): final Class directBufferClass = Class.forName("java.nio.DirectByteBuffer"); final Method m = directBufferClass.getMethod("cleaner"); m.setAccessible(true); MethodHandle directBufferCleanerMethod = lookup.unreflect(m); Class cleanerClass = directBufferCleanerMethod.type().returnType(); // build method handle for unmapping, full code is here: https://goo.gl/TfQWl6 I guess that is the effect of #AwkwardStrongEncapsulation. I would advise doing regular checks against the jigsaw builds to know about such problems in the future earlier... but seeing your code break without an obvious good solution sure is stressful. I feel with you. [...] (2) A second thing we noticed is that Groovy no longer works and dies with strange error messages. That is because versions including Groovy 2.4.7 are using setAccessible(AccessibleObject[] array, true), and the array will also include private methods or fields. This worked till #AwkwardStrongEncapsulation because will then a class was either exported and its method can all be made accessible or not. For example on GAE or earlier versions of the module system. Now an exported class may break this, since its private methods can no longer be made accessible using setAccessible. A fix for this is already committed, we are only waiting for release of Groovy 2.4.8. Of course even with the fix Groovy code can possibly break... for example if you did the direct buffer access in Groovy. Btw, do not hesitate to ask about such problems on groovy-user, please. bye Jochen
Re: Alternatives for Unsafe field access
On 09.12.2016 12:59, Alan Bateman wrote: On 09/12/2016 07:55, Jochen Theodorou wrote: Maybe we have luck here and it does not apply, but if the field comes from a class of a module and is private, my current level knowledge of jigsaw says, that setAccessible will fail with an exception. It does not matter at all if the class with the field is exported or not. That is because just before the JavaOne #AwkwardStrongEncapsulation made it into jigsaw. Only way around this is to use the Lookup object that has the correct rights from the beginning. How to get it... well... there will be only solutions for certain types of classes. MethodHandles.privateLookupIn [1] was recently added to get a Lookup with private access, it may be useful here. I think the documentation needs some improvement to make it clear what lookup modes are set when. And by that I mean really using the mode names in the description. Just from a short lookup for example I have no idea when the MODULE mode is set for a Lookup and when not. [...] [1] http://download.java.net/java/jdk9/docs/api/java/lang/invoke/MethodHandles.html#privateLookupIn-java.lang.Class-java.lang.invoke.MethodHandles.Lookup- bye Jochen
Re: Alternatives for Unsafe field access
On 09.12.2016 08:55, Jochen Theodorou wrote: On 08.12.2016 22:10, Uwe Schindler wrote: Hi, You can first do standard reflection, then use setAccessible Maybe we have luck here and it does not apply, but if the field comes from a class of a module and is private, my current level knowledge of jigsaw says, that setAccessible will fail with an exception. It does not matter at all if the class with the field is exported or not. That is because just before the JavaOne #AwkwardStrongEncapsulation made it into jigsaw. Only way around this is to use the Lookup object that has the correct rights from the beginning. How to get it... well... there will be only solutions for certain types of classes. oh, forgot... it works if the module is made open from the command line or declared as such as a work around bye Jcohen
Re: Alternatives for Unsafe field access
On 08.12.2016 22:10, Uwe Schindler wrote: Hi, You can first do standard reflection, then use setAccessible Maybe we have luck here and it does not apply, but if the field comes from a class of a module and is private, my current level knowledge of jigsaw says, that setAccessible will fail with an exception. It does not matter at all if the class with the field is exported or not. That is because just before the JavaOne #AwkwardStrongEncapsulation made it into jigsaw. Only way around this is to use the Lookup object that has the correct rights from the beginning. How to get it... well... there will be only solutions for certain types of classes. bye Jochen
Re: 8169425: Values computed by a ClassValue should not strongly reference the ClassValue
resend after I did accidentally send it to Paul directly instead of the list On 10.11.2016 21:24, Paul Sandoz wrote: Hi Jochen, Can you confirm if my analysis of Groovy using ClassValue was correct: https://bugs.openjdk.java.net/browse/JDK-8136353 AFAICT in this case the issue was not with ClassValue itself but the storing of computed values in a global set. Our last "solution" was not to use ClassValue at all and restore the faulty pre-ClassValue logic. Because not even making those weak references solved the problem. If i understand correctly you are you raising a wider issue with the function of ClassValue itself, it may be insufficient for your use-case, and not specifically with computed values strongly referring the associated ClassValue? Specifically, I am struggling to unpack this bit: But if you will need at the same time a ClassValue to not to prevent a class we computed the value for from unloading and have the computed value alive for min(lifespan class, lifespan runtime), then you get a real big problem in realizing this. Computed values can strongly refer to the associated class, it becomes problematic when computed values strongly refer to the associated ClassValue. Is that something you require? The problem is more how to avoid that, but still get the semantics I need. Example... have the computed value be the meta class of a class and let there be a class X with a static field representing the ClassValue. Now create the meta class of X and let say each meta class references also the meta class of the super class. And of course I do want to be able to init and destroy the runtime thousands of times without leaving garbage in memory. By doing this I have a computed value strongly referencing the ClassValue and I do have ClassValue pinned for VM lifetime because I made a metaclass of Object. A weak reference for the computed value is no solution, since we then would loose modifications done to the metaclasses. And this is the reason why we wanted ClassValue in the first place. I can also not just reference the ClassValue or the class containing the field, because that would still keep the strong reference alive. What is for example totally unclear to me is, what happens if the ClassValue itself is weakly referenced and a computed value references the ClassValue through that weak reference. Does ClassValue then possibly get collected before the computed values? If "yes" it is no solution again, as we would again loose meta class information. And a "no" most likely means another memory leak. Even if I lift the constraint of a meta class referencing its super class meta class - I will have meta classes of classes from java.lang.* and then a memory leak So how to solve the problem? bye Jochen
Re: 8169425: Values computed by a ClassValue should not strongly reference the ClassValue
On 09.11.2016 19:18, Doug Lea wrote: [...] And: Similar cases can indeed occur with ThreadLocal, but users seem to mostly avoid them, sometimes after painful experience. ThreadLocal is already a bad choice once you subclass it. bye Jochen
Re: 8169425: Values computed by a ClassValue should not strongly reference the ClassValue
On 09.11.2016 17:47, Paul Sandoz wrote: Hi Peter, Good point about if such support was added it would break the API and also (with Remi) about Ephemerons. You are correct in stating the constraints in more detail regarding classes in the same loader. However, i think that is going into more detail than I would prefer for what i think is an edge case. I would not regard that an edge case. For a dynamic language using ClassValue to store information, it is very easy to produce a memory leak with ClassValue. So I want in general to warn developers away from strongly referencing this ClassValue in the computed value for any associated class. If we do get strong feedback that this is a real problem we could consider adding a clever little static method like you suggest, with caveats that the computing Function might go away. for us it is such a strong problem, that we are unable to transfer the old structure to use ClassValue without a major rewrite of large parts. Just imagine a runtime that uses ClassValue and that your application, let it be a web application, will spawn many runtimes over time. You do want the runtimes and all computed values be able to be garbage collected. But if you will need at the same time a ClassValue to not to prevent a class we computed the value for from unloading and have the computed value alive for min(lifespan class, lifespan runtime), then you get a real big problem in realizing this. As it stands for me ClassValue is only usable as a class associated cache with values you can recreate at any moment. That is not good enough for us in the general case. If that is an edge case I still miss a major part in the documentation... and that is what a ClassValue should be used for instead of a cryptic and incomplete description of what a ClassValue is. And then we could talk about if the intended use and the actual usability fit together bye Jcohen
Re: RFC: System.console().encoding()
On 15.09.2016 09:21, Dawid Weiss wrote: Console is supposed to be a "char/String" based class, "encoding" really should have no business here in its api. While I agree with your concerns about the functional side of the API, I disagree about this method having no practical use. I can give you a concrete example. The use case that we had was to check whether the "terminal" (console) would be able to handle non-ASCII characters. A Writer doesn't tell you anything. An encoding does provide at least some confidence that certain characters will be translated properly -- if your encoding is US-ASCII or ISO8859-1 then Polish diacritics won't get displayed for sure. This doesn't mean 100% confidence in actual glyph rendering of course, but it's a cheap and safe sanity check of the terminal's capabilities. out of curiosity... what will you do if you find the encoding lacking what you need? bye Jochen
Re: JEP 276: Dynamic Linking of Language-Defined Object Models
On 19.10.2015 12:56, Attila Szegedi wrote: On Oct 19, 2015, at 10:46 AM, Jochen Theodorou <blackd...@gmx.org <mailto:blackd...@gmx.org>> wrote: since it is dynalink there is I guess only one master linker in the end. Can you point me to some code showing how the composition of linkers is done to refresh my memory on that? Sure, here’s how Nashorn does it: http://hg.openjdk.java.net/jdk9/dev/nashorn/file/f93753325c7b/src/jdk.scripting.nashorn/share/classes/jdk/nashorn/internal/runtime/linker/Bootstrap.java#l92 You’ll notice that Nashorn has a total of 10 linkers - they’re modularized per purpose. [...] interesting... I will take a closer look at those some time later this week So am I right in assuming that if for example there is the public nashorn linker and the linker from Groovy, that we simply go through the list of linkers (from a service based lookup I guess) and the first linker claiming to be able to link that gets to do it? bye Jochen
Re: JEP 276: Dynamic Linking of Language-Defined Object Models
On 18.10.2015 10:55, Andrew Haley wrote: On 10/18/2015 09:49 AM, Jochen Theodorou wrote: * Invokedynamic (like invokeinterface and invokevirtual) does not like calls with null as receiver, quitting the call right away with a NPE. But languages may allow for calls on null. That again means some kind of dummy receiver is required, if this is supposed to work. Wouldn't you just insert a null pointer check? That's surely going to be faster (if it's used) than the runtime taking a NullPointerException. And you don't want the runtime to have to trap for null when many languages don't need it. Let's scrap that, outdated information on my side
Re: JEP 276: Dynamic Linking of Language-Defined Object Models
On 19.10.2015 03:09, John Rose wrote: On Oct 18, 2015, at 1:49 AM, Jochen Theodorou <blackd...@gmx.org <mailto:blackd...@gmx.org>> wrote: * Invokedynamic (like invokeinterface and invokevirtual) does not like calls with null as receiver :-) Jochen, you are one of the few people on the planet who was around JSR 292 when this *was* true. A very early draft of indy piggy-backed on top of the invokeinterface instruction. Eventually we made it symmetrical in all its arguments, like invokestatic, as Attila pointed out. ups... looks like I missed that. Well, then sorry for the fuzz ;) Can you tell me since what version this has been changed? Because I thought I had this problem with 1.8 still, but I could be wrong and it was only 1.7.. Was it done along with the lambda forms rewrite? bye blackdrag
Re: JEP 276: Dynamic Linking of Language-Defined Object Models
On 17.10.2015 13:30, Martijn Verburg wrote: This looks very, very promising. Would it help to get the language maintainers of the most popular scripting/dynamic JVM languages involved ASAP? Happy to contact Groovy, Clojure, Scala, JRuby folks (although I suspect many of them are on this list). still easy to miss something like this ;) But I can give some comments for Groovy on this already. As I understand it, this JEP is mostly for calls from Java on objects from other languages. I have basically only main points: * Invokedynamic (like invokeinterface and invokevirtual) does not like calls with null as receiver, quitting the call right away with a NPE. But languages may allow for calls on null. That again means some kind of dummy receiver is required, if this is supposed to work. Then of course, if you want to allow calls on null, every call would have to go through the runtime in the end, thus I suspect this will be out of scope for this JEP. * Another part is the lookup of calls itself. To me the JEP is not fully clear here. Sure, there is a service for this, but is every call checked? That may slow down normal compilation for Java. The JEP speaks of methods provided beyond what Java offers - but what about methods replacing normal Java methods? Btw, one part that would make me very happy, is an official API, that allows some control of phases of the Java compiler as well as make missing classes programatically known to the compiler without requiring the java compiler to write those out as well. And that especially because I assume that jigsaw will put all the api needed for this into hiding by modules. bye Jochen Theodorou
Re: JAVADOC clarification needed in Class.getMethod
yeah, you are right, I made a mistake. supplying a null argument t a method call to call a Class varargs taking method is ambiguous. The compiler will want you to cast to either Class, in which case the null would be wrapped, or to Class[], in which case there will be no wrapping. Example: public class Foo { static void test(Class... params) { System.out.println(params==null);} public static void main(String[] args) {test((Class[])null);} } javap -c will show for the main method: public static void main(java.lang.String[]); Code: 0: aconst_null 1: checkcast #4 // class [Ljava/lang/Class; 4: invokestatic #5 // Method test:([Ljava/lang/Class;)V 7: return which means no wrapping in an array. But you can bypass the cast through a different method as well: public class Foo { static void test(Class... params) { System.out.println(params==null);} static void bar(Class[] params) {test(params);} public static void main(String[] args){bar(null);} } since bar already supplies a Class[], it will be directly taken. But maybe I did misunderstand the intention of your post. In that cae, sorry for the noise Am 29.08.2015 15:40, schrieb Kyung Koo Yoon: Hi, Jochen. -- Software Innovation Driver yoonforh at gmail dot com Analyze, Reason, Reshape the Question PGP http://www.javadom.com/personal/yoonforhatgmaildotcom.asc 2015. 8. 29., 오후 10:01, Jochen Theodorou blackd...@gmx.org mailto:blackd...@gmx.org 작성: Am 29.08.2015 10:56, schrieb Kyung Koo Yoon: Hi, all. The javadoc comment of java.lang.Class.getMethod API has following clause. @CallerSensitive public Method getMethod(String name, Class?... parameterTypes) throws NoSuchMethodException, SecurityException The {@code parameterTypes} parameter is an array of {@code Class} objects that identify the method's formal parameter types, in declared order. If {@code parameterTypes} is {@code null}, it is treated as if it were an empty array. As you know the method signature changed to use varargs since JDK 1.5, if parameterTypes is given null, the compiler will interpret the parameter with “new Class[] { null }”. you are making a wrong assumption here. If null is given there will be no wrapping in an array. varargs are defined that there is no wrapping if the given argument type is compatible. For Class[] this is Class[] and of course null. Even if you had been right, you could still give produce a call with null by reflection or using the MethodHandles API You can check by simply decompiling the generated byte codes. vararg is not a JVM feature so the compiler pre-processes and the compiler handles the given null as array’s first element. Originally class.getMethod(“methodName”, null) should have meant a static method, but with vararg, it’s not. bye blackdrag -- Jochen blackdrag Theodorou blog:http://blackdragsview.blogspot.com/ -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
JDK9 changes to Date?
Hi, after fixing some JDK9 related bugs in our build for Groovy we stumbled over new SimpleDateFormat(EEE MMM dd HH:mm:ss zzz , Locale.US).parse(Thu Jan 01 01:00:00 CET 1970) failing to parse with java.text.ParseException: Unparseable date: Thu Jan 01 01:00:00 CET 1970 at java.text.DateFormat.parse(DateFormat.java:366) This happens not with my local machine using jdk1.9.0-ea-b68, but it does fail on our CI server used revision 0906b79c9df4 for this. A second case is this one: Locale locale = Locale.UK Locale.setDefault locale TimeZone.setDefault TimeZone.getTimeZone('Etc/GMT') Date d = new Date(0) assertEquals '01/01/70', DateFormat.getDateInstance(DateFormat.SHORT).format(d); the assert used to work, but now the format changed to print 01/01/1970 instead of 01/01/70. With the same constellation regarding jdk versions. Are those expected to fail? Can someone verify this? bye Jochen Theodorou -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: JDK9 changes to Date?
That means the US locale cannot understand CET as timezone then? And... it assuming that TimeZone.getTimeZone('Etc/GMT') returns the GMT timezone, it means the short format for a date changed from using 70 for the year to 1970? Or is that CLDR related as well? bye blackdrag Am 09.07.2015 21:27, schrieb Naoto Sato: I meant the root cause seems to be the same between your first issue and 8130845, where parsing short time zone names has some problem regardless of it's the ROOT locale or not. Naoto On 7/9/15 12:23 PM, Jochen Theodorou wrote: but both examples set a locale I thought Am 09.07.2015 21:06, schrieb Naoto Sato: Hi Jochen, It is likely that these are caused by the default locale data change to CLDR with 8008577. On 7/9/15 10:12 AM, Jochen Theodorou wrote: Hi, after fixing some JDK9 related bugs in our build for Groovy we stumbled over new SimpleDateFormat(EEE MMM dd HH:mm:ss zzz , Locale.US).parse(Thu Jan 01 01:00:00 CET 1970) failing to parse with java.text.ParseException: Unparseable date: Thu Jan 01 01:00:00 CET 1970 at java.text.DateFormat.parse(DateFormat.java:366) This happens not with my local machine using jdk1.9.0-ea-b68, but it does fail on our CI server used revision 0906b79c9df4 for this. This seems to be the same issue which has already been reported in 8130845. A second case is this one: Locale locale = Locale.UK Locale.setDefault locale TimeZone.setDefault TimeZone.getTimeZone('Etc/GMT') Date d = new Date(0) assertEquals '01/01/70', DateFormat.getDateInstance(DateFormat.SHORT).format(d); the assert used to work, but now the format changed to print 01/01/1970 instead of 01/01/70. With the same constellation regarding jdk versions. This is expected, as the short date format for UK in CLDR is dd/MM/y. Naoto -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: JDK9 changes to Date?
ok, so https://bugs.openjdk.java.net/browse/JDK-6712094 tells me that the long year format is the right one for Locale.UK, and it mentions also CLDR. So I guess the change to CLDR by default made this change happen as well. Am 09.07.2015 21:39, schrieb Jochen Theodorou: That means the US locale cannot understand CET as timezone then? And... it assuming that TimeZone.getTimeZone('Etc/GMT') returns the GMT timezone, it means the short format for a date changed from using 70 for the year to 1970? Or is that CLDR related as well? bye blackdrag Am 09.07.2015 21:27, schrieb Naoto Sato: I meant the root cause seems to be the same between your first issue and 8130845, where parsing short time zone names has some problem regardless of it's the ROOT locale or not. Naoto On 7/9/15 12:23 PM, Jochen Theodorou wrote: but both examples set a locale I thought Am 09.07.2015 21:06, schrieb Naoto Sato: Hi Jochen, It is likely that these are caused by the default locale data change to CLDR with 8008577. On 7/9/15 10:12 AM, Jochen Theodorou wrote: Hi, after fixing some JDK9 related bugs in our build for Groovy we stumbled over new SimpleDateFormat(EEE MMM dd HH:mm:ss zzz , Locale.US).parse(Thu Jan 01 01:00:00 CET 1970) failing to parse with java.text.ParseException: Unparseable date: Thu Jan 01 01:00:00 CET 1970 at java.text.DateFormat.parse(DateFormat.java:366) This happens not with my local machine using jdk1.9.0-ea-b68, but it does fail on our CI server used revision 0906b79c9df4 for this. This seems to be the same issue which has already been reported in 8130845. A second case is this one: Locale locale = Locale.UK Locale.setDefault locale TimeZone.setDefault TimeZone.getTimeZone('Etc/GMT') Date d = new Date(0) assertEquals '01/01/70', DateFormat.getDateInstance(DateFormat.SHORT).format(d); the assert used to work, but now the format changed to print 01/01/1970 instead of 01/01/70. With the same constellation regarding jdk versions. This is expected, as the short date format for UK in CLDR is dd/MM/y. Naoto -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: JDK9 changes to Date?
but both examples set a locale I thought Am 09.07.2015 21:06, schrieb Naoto Sato: Hi Jochen, It is likely that these are caused by the default locale data change to CLDR with 8008577. On 7/9/15 10:12 AM, Jochen Theodorou wrote: Hi, after fixing some JDK9 related bugs in our build for Groovy we stumbled over new SimpleDateFormat(EEE MMM dd HH:mm:ss zzz , Locale.US).parse(Thu Jan 01 01:00:00 CET 1970) failing to parse with java.text.ParseException: Unparseable date: Thu Jan 01 01:00:00 CET 1970 at java.text.DateFormat.parse(DateFormat.java:366) This happens not with my local machine using jdk1.9.0-ea-b68, but it does fail on our CI server used revision 0906b79c9df4 for this. This seems to be the same issue which has already been reported in 8130845. A second case is this one: Locale locale = Locale.UK Locale.setDefault locale TimeZone.setDefault TimeZone.getTimeZone('Etc/GMT') Date d = new Date(0) assertEquals '01/01/70', DateFormat.getDateInstance(DateFormat.SHORT).format(d); the assert used to work, but now the format changed to print 01/01/1970 instead of 01/01/70. With the same constellation regarding jdk versions. This is expected, as the short date format for UK in CLDR is dd/MM/y. Naoto -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names
Am 16.06.2015 10:40, schrieb Vladimir Ivanov: [...] Example: class TopLevel { static class Nested {} classInner {} void f() { class Local {} } Object o = new TopLevel() {}; // anonymous } And here's how they look like on bytecode level. I'll use both javap and ASM to dump class structure: $ java jdk.internal.org.objectweb.asm.util.ASMifier class_file Nested: javap: static #11= #10 of #5; //Nested=class TopLevel$Nested of class TopLevel asm: cw.visitInnerClass(TopLevel$Nested, TopLevel, Nested, ACC_STATIC); Inner: javap: #8= #7 of #5; //Inner=class TopLevel$Inner of class TopLevel asm: cw.visitInnerClass(TopLevel$Inner, TopLevel, Inner, 0); that nested and inner differ only by the modifier is ok Local: javap: #13= #12; //Local=class TopLevel$1Local asm: cw.visitInnerClass(TopLevel$1Local, null, Local, 0); Anonymous: javap: #2; //class TopLevel$1 asm: cw.visitInnerClass(TopLevel$1, null, null, 0); I did not expect that the outer class is not set for Local and Anonymous classes. I would have thought that the enclosing method is the only difference. That's surprising to me but actually there is even code for this. And I found the bug as well... So thanks for the help bye Jochen -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names
Am 15.06.2015 18:04, schrieb Vladimir Ivanov: [...] In order to make the class non-anonymous again, you have to specify inner_name_index and, optionally, outer_class_info_index. ok... let me try to understand this better... taking this Java source public class Test { public static void main(String[] args) { class X{} Runnable foo = new Runnable(){public void run(){}}; }} I get for Test InnerClasses: static #2; //class Test$1 #12= #11; //X=class Test$1X Is it the #12=#11 part which tells me X is no anonymous class? Is #11 the inner_name_index? bye Jochen -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
JDK-8057919 Class.getSimpleName() should work for non-JLS compliant class names
Hi all, this mail is refering to http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-April/032733.html and JDK-8057919 In a groovy program def cl = { - '' } assert cl.class.anonymousClass == false the open block {-''} is compiled into an inner class. We had problems with this in the past, because for several reasons we did not follow the JLS naming conventions here. The change in JDK-8057919 was supposed to make things better, but the change also means that our open block class will now have an empty simple name. At the same time reflection tells me the class is not an anonymous class. If I see this right, then regardless of if we set the attributes right or not, reflection and getSimpleName have a differing understanding about the nature of that class. I consider that a bug. They should have a common understanding about what a anonymous inner class is, since getSimpleName is supposed to return an empty string for that case. plus... it would be really really nice if someone could tell me how a non-anonymous class in a method has to look like with javap, so that I can know from there if it is no anonymous and finally fix he flags for Groovy's class generation. Because javap and the asm lib really don't help much with getting inner classes and their flags right. bye jochen -- Jochen blackdrag Theodorou blog: http://blackdragsview.blogspot.com/
Re: A List from Iterator
Am 28.08.2014 18:13, schrieb Pavel Rappo: Hi everyone, Is there any particular reason why there's no convenience method for iterators similar to j.u.Collections.list for enumerations? Or at least the one that adapts Iterator to Enumeration and vice versa. Thanks. my opinion: Enumeration is pre java 1.2. Back then java we to get away from that to collections already. Now it is probably streams. Besides supporting old code, there is rarely a good reason to support Enumeration in anything. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Implicit 'this' return for void methods
Am 26.03.2014 16:51, schrieb Guy Steele: [...] I am wholeheartedly in favor of allowing “chaining” of dotted expressions such as CharBuffer.allocate(26).position(2).put(C).position(25).put(Z”) this also shows a potential point of critic this proposal will have to find arguments against. This style highly encourages mutation of the object, which is not very functional. Since java8 java tries to be more functional, thus this idea would be totally against this new line. I leave it to you guys if this is even a valid argument ;) bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Implicit 'this' return for void methods
Am 27.03.2014 20:22, schrieb Steven Schlansker: On Mar 27, 2014, at 9:35 AM, Jochen Theodorou blackd...@gmx.org wrote: Am 26.03.2014 16:51, schrieb Guy Steele: [...] I am wholeheartedly in favor of allowing “chaining” of dotted expressions such as CharBuffer.allocate(26).position(2).put(C).position(25).put(Z”) this also shows a potential point of critic this proposal will have to find arguments against. This style highly encourages mutation of the object, which is not very functional. Since java8 java tries to be more functional, thus this idea would be totally against this new line. I leave it to you guys if this is even a valid argument ;) It would be much easier to convince people to do less mutation if the language had support for the Builder pattern or freezing objects. I wanted to counter that people would probably complain about the huge amount of objects you create, but actually you can do this with a helper class producing a dummy, that then becomes the final object of the real class once you freeze or otherwise fixate it. Well, the details don't really matter. But yes, I agree... some language support here would help, though not sure how that would look like... Currently working with nontrivial immutable objects quickly leads to a lot of boilerplate code that requires frustrating maintenance every time the object definition changes. yeah, with the approach I sketched I can totally see that. Adding both the implicit ‘this’ proposal as well as some syntax enhancements for immutable objects would strengthen the Java language quite a bit, in my opinion :) And here I fail. If you need special support for the builder and freezer to produce some helper or something already, then you don't need the return 'this' for void at all. The helper would be generated and thus already have all the bits you want. Right now I am actually wondering if I should add that as a transform to Groovy. But well, this is Java. For the part with immutable I agree again of course. byeJochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Implicit 'this' return for void methods
Am 27.03.2014 21:52, schrieb Eirik Lygre: [...] The JavaBean specification, with it's void setSomething() functions are fundamental to so many things Java that they will never go away (good thing, too!).The suggested language change builds on top of that, is beneficial to a large body of existing code and does not invalidate any existing practices. It is, I think, a great example of bang for the buck -- low cost, high return. No changes needed in the VM; everything can be done by the compiler. Almost too sweet. Hmm... isn't it a problem if I change void setSomething() to MyClass setSomething(). In example, is that still a valid setter in terms of the JavaBean Spec? I might be wrong, but afaik there are bean based tools out there, not recognizing setters like that. At least I remember having such cases in the past. If I am right, then there is quite a chance of invalidating working code. I should have paid more attention to the dragon book, so that I could have taken a shot at the compiler myself :-/ I think you won't need it for that kind of change. You don't change the grammar, or any first level rules. You would change the internal AST, or even the bytecode generation part for this. You need to know javac internals instead... bad enough imho bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
clarification on docs of Class#getInterfaces() and Class#getGenericInterfaces() needed?
Hi, hoping I will not annoy anyway asking this here I was wondering about how to interpret the javadoc of these methods correctly. You basically find this sentence for Class#getInterfaces() and Class#getGenericInterfaces() If this object represents a class, the return value is an array containing objects representing all interfaces implemented by the class. The problem is that wording of all. When I look at for example AbstractList, then all means to me more than just List (which is what is returned for AbstractList.class.getInterfaces()). Furthermore, since the wording is so equal you would assume that getInterfaces() and getGenericInterfaces() will always return the same interfaces, in the same order, same length, only one generified, the other not. But this is only true for classes produced by the java compiler. getInterfaces seems to read what is produced from implements part in bytecode, while getGenericinterfaces uses the signature. There seems to be nothing forbidding a different order or even a different number. The example, that did bring me to this is scala.runtime.AbstractPartialFunction from Scala 2.10.2 (other versions I did not check). While the implements part says only PartialFunction, the generics signature contains Function1 as well. Since PartialFunction implements Function1 there should be no real problem with that, unless you wrote code like me, that did depend on the two methods returning more or less the same. While much shorter the javadoc used in Android is for me much more clear: http://developer.android.com/reference/java/lang/Class.html#getInterfaces%28%29 Returns an array of Class objects that match the interfaces in the implements declaration of the class represented by this Class. The order of the elements in the array is identical to the order in the original class declaration. If the class does not implement any interfaces, an empty array is returned. I won't say this will prevent the mistake I made, but at least it will prevent the assumption *all* interfaces are to be shown here. All in all I find it quite difficult to phrase this to not awake assumptions like that, without especially mentioning it. But anyway... should the javadoc be changed? bye Jochen
Re: rationale for swallowing exceptions in Class#getEnumConstantsShared
Am 28.02.2014 17:36, schrieb Peter Levart: [...] The other use of getEnumConstantsShared() is to support the Enum.valueOf(Class, String) method which already throws IllegalArgumentException if the 1st parameter does not represent an enum class, so there's no compatibility concerns about that use. Enum.valueOf is the method through which I discovered this. I have exactly this case, an enum, which seems not to fit getEnumConstantsShared for an unknown reason. My additional problem though is, that sometimes the call works and sometimes not. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [JDK8] RFR (XS): JSR292: IncompatibleClassChangeError in LambdaForm for CharSequence.toString() method handle type converter
Hi, since I am indirectly the reporter of this bug I have one remark for the test. The error happens only for compiled lambda forms. The given test does imho not use a compiled lambda form. In other words, afaik the test would pass without the fix. As such it would be useless as regression test. Am 15.01.2014 16:31, schrieb Vladimir Ivanov: http://cr.openjdk.java.net/~vlivanov/8031502/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8031502 InvokeBytecodeGenerator can produce incorrect bytecode for a LambdaForm when invoking a method from Object declared in an interface. The problem is the following: (1) java.lang.CharSequence interface declares abstract method String toString(); (2) after 8014013 fix, VM resolves CharSequence::toString()/invokeInterface to CharSequence::toString()/invokeVirtual; (3) during LambdaForm compilation, CharSequence is considered statically invocable (see InvokeBytecodeGenerator::isStaticallyInvocable) and invokevirtual for CharSequence::toString() is issued, which is wrong (invokevirtual throws ICCE if it references an interface); The fix is straightforward: during LambdaForm compilation, switch back from invokevirtual to invokeinterface instruction when invoking a method on an interface. The fix is targeted for 8. Will be also integrated into 9. Testing: regression test, jdk/test/java/lang/invoke, vm.mlvm.testlist, nashorn, jruby. Thanks! Best regards, Vladimir Ivanov ___ mlvm-dev mailing list mlvm-...@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [JDK8] RFR (XS): JSR292: IncompatibleClassChangeError in LambdaForm for CharSequence.toString() method handle type converter
Am 16.01.2014 09:48, schrieb A. Sundararajan: The test sets compile threshold to be zero (-Djava.lang.invoke.MethodHandle.COMPILE_THRESHOLD=0 ). I think compilation occurs on the first invoke. Also, I ran the test on a jdk8 build without Vladimir's fix - I saw the exception being thrown. I ran it by passing the above option in the command line (outside jtreg). ah, ok, thank you very much -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
CallerSensitive access rights problems
Hi, java.lang.Class has multiple methods annotated with CallerSensitive (see http://hg.openjdk.java.net/jdk8/jdk8-gate/jdk/file/tip/src/share/classes/java/lang/Class.java). Now if we in Groovy here want to build our runtime structure for this class, and the security manager is not allowing access to sun.reflect, then we get into trouble. https://jira.codehaus.org/browse/GROOVY-6405 is caused by this. What do you suggest people with this problem, if adding accessClassInPackage.sun.reflect is no option? bye Jochen
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 09:28, schrieb Nick Williams: [...] This is all well and good, but some of us just need a simple array. This seems like over-engineering. I just want an array of StackFrameInfos/StackTraceFrames. if you need the full stack, then misusing a Predicate and a dummy Consumer to collect everything looks a lot like not intended use. The intended inability of lambdas to write to local variables outside the lambda is not helping here to avoid anonymous inner classes. So what was a simple method call, will look really awful - I agree. But if you would have designed the API that uses the information now, rather than fitting the old API from the frame work and the new API here together, then this problem would most probably not have appeared, since the structure would have been very different from the beginning. I have modified java.util.logging.LogRecord to infer the caller using the new API (much simpler): StackTraceElement frame = Thread.firstCaller(e - {return !isLoggerImplFrame(e.getClassName()); }, StackFrameInfo::stackTraceElement); Replacement for getCallerClass() Class? c = Thread.getCaller(StackFrameInfo::getDeclaringClass); Replacement for traversing the stack with getCallerClass(int depth) Thread.walkStack(e - doSomething(e)); This looks beautiful for Java 8, sure. Now try doing these same things in a library compiled for Java 6 but made to be compatible with Java 8 and so all of this has to happen via reflection. Suddenly it's nightmarish. With a new class to contain the information it will look like a nightmare even if you get an array and do not have that callback style. What we will probably do is a kind of plugin that gets enabled if jdk8 is used. That class will have to be compiled using jdk8 though. And in case of an older jdk the old implementation will be used bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 09:09, schrieb Mandy Chung: [...] Stack is a stream that allows you to walk partial stack (e.g. find caller) or full stack trace (e.g. throwable). The filtering and mapping operations are lazy to avoid having the VM eagerly copying the entire stack trace data even for the short reach case (like Groovy and Log4j). and is there a link to StackStram too? The Thread#walkStack methods don't really describe the behaviour of the consumer. Will the consumer applied only once, or multiple times? If only once, then to replace getCallerClass(int depth), you will need the version with the predicate, which will be difficult to realize with a simple lambda, since you will need to count as well. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 10:50, schrieb Jörn Huxhorn: [...] But I have serious doubts that *this* is going to be more efficient than just iterating over an array. +1 Feel free to add all that functionality but all we are asking for right now is access to the call stack array in a way that is as fast, or, ideally, even faster than using the current getCallerClass(int) method. Performance is the main reason for this whole discussion. when it comes to getCallerClass(int) I see need for improvement in the API. But I see the patch as a draft only And it is very, very crucial that all of this can be done via (Java 6) reflection so that code will be able to use sun.reflect.Reflection up to J7 and the new API after that... you mean you want to use the new API via Java6 reflection? Then there should be no new class to be used. Otherwise you will get crazy with writing out all the reflective calls. I think it is better to go with a plugin-like structure and write code for the new API with JDK8. I am of course aware, that this makes the build more complicated bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 11:34, schrieb Peter Levart: [...] ListStackFrameInfo frames = new ArrayList(); Thread.walkStack(frames::add); No so awfull. as I said, it is unclear to me as of if walkStack walks the whole stack or not. Your code implies it does. If It does, I don't see the advantage of suing a Stream here or does the predicate version not? bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 12:05, schrieb Peter Levart: [...] The use-cases described used getCallerClass(int depth) repeatedly to find a caller by iterating over a range of depths and calling getCallerClass(depth). You can use either Thread.walkStack or Thread.firstCaller for implementing those use-cases. first Caller is only going back one, or not? Not useful for us in the cases we need this. I wouldn't say the API through getCallerClass(int) was optimal for us, I think the new one can be better... I only miss (or didn't get how) the ability not to have to walk the entire stack, without knowing before how deep I have to go Maybe the following method would be handy to optimize search when we know that we want to skip 1st N frames before starting testing with predicate: public static T T firstCaller(int startDepth, PredicateStackFrameInfo predicate, FunctionStackFrameInfo,T function) { Reflection.getCallerClass(depth) then becomes: Thread.firstCaller(depth, f - true, StackFrameInfo::getDeclaringClass); Hm... that is I think a usecase for some... as I said, getCallerClass(int) is not really ideal for us either. More ideal in this style would be for us public static T T findCaller(PredicateStackFrameInfo predicate, FunctionStackFrameInfo,T function) with the predicate indicating when to stop.. though the usage of this is not that nice: Class getCallerClass(final int nonSkippedFramesDepth) { return findCaller(new PredicateStackFrameInfo() { int depth = 0; boolean test(StackFrameInfo info) { if (haveToSkip(info.getDeclaringClass())) return false; depth++; if (depth=nonSkippedFramesDepth) return info.getDeclaringClass(); } }, StackFrameInfo::getDeclaringClass()); } Have you guys though about exposing the StackStream instead? Then you could use all the existing JDK method for streams on that, which gives you a much more flexible API. I could then for example change the Stream of StackFrameInfo into one of Class. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 20:46, schrieb Mandy Chung: [...] Exposing a StackStream API means that you need to eagerly walk the stack and copy the stack frames to it before it returns. I agree it is a much more flexible API. On the other hand, stack walking is sequential and ordered and a stack stream will be traversed as in an iterator. not the StackStream API, I did mean a Stream as in http://download.java.net/lambda/b78/docs/api/java/util/stream/Stream.html But I must say, I did not realize Stream has so little to offer in terms of default implementations. I am not saying using this class is a good idea, but if you already go on lambdas, then this should be at least something to consider... and to underline it: Stream does not require you to eagerly read all values What about a findCaller method that takes a parameter to indicate how many times you skip over the matching elements before applying the function: Thread.findCaller(info - {return !haveToSkip(info.getDeclaringClass());}, nonSkippedFramesDepth, StackFrameInfo::getDeclaringClass()); just to show you how this could work with the Stream Thread.getStackStream(). filter(info - {return !haveToSkip(info.getDeclaringClass());}. subStream(nonSkippedFramesDepth). map(StackFrameInfo::getDeclaringClass).findFirst() I am looking at those things since only recently, so there is surely a better way, but have an impression it should be enough. Especially I show this to you, so you can see that with a Stream no new method is needed. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 17:45, schrieb Peter Levart: [...] I think the reasoning behind call-back API is that it moves the logic to construct a suitable data structure to the Java side, skipping intermediary representations and conversions. StackFrameInfo is already a conversion for me. I see some value in the additional ability to impose security checks, but nothing for ease of use. I somehow cannot believe that StackFrameInfo is a structure the VM uses internally to manage stack frames I don't know what the overhead of call-backs from native code to Java is though. For constructing and returning an array of StackFrameInfo objects, the native code has to collect the objects into a GrowableArray (a native equivalent of ArrayList). Before returning, it has to copy elements into the Java array. And this Java array is usually later used just to iterate it's elements... Imagine a situation where GrowableArray and Java array could be skipped and StackFrameInfo objects directly formatted into a StringBuilder, creating the final logger message. This assumes that call-backs from native code are cheap. Are they? Can native method be intrinsified together with call-back invocations? You won't need a GrowableArray if you know the stack length. Then you can just create the target StackFrameInfo[] bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 18:38, schrieb Peter Levart: [...] As I understand the Thread.firstCaller() does exactly that. findFirstCallerMatchingPredicate. Choosen name abbreviation is maybe not making the semantic immediately obvious. then sorry, I overlooked that in the link though the usage of this is not that nice: Class getCallerClass(final int nonSkippedFramesDepth) { return findCaller(new PredicateStackFrameInfo() { int depth = 0; boolean test(StackFrameInfo info) { if (haveToSkip(info.getDeclaringClass())) return false; depth++; if (depth=nonSkippedFramesDepth) return info.getDeclaringClass(); } }, StackFrameInfo::getDeclaringClass()); } But the API is making it possible without walking entire stack or making N partial walks with lengths 1..N, which is O(N^2)... You know the trouble with the O-notation. For once, there are constants in this, that are not mentioned and for second O(N^2) is only always better than any O(N^3) if N is big enough. Otherwise even an O(2^N) can be less effort. And the lengths we are talking here about are not big Ns More compact, using lambda: Class getCallerClass(int nonSkippedFramesDepth) { int[] depth = new int[1]; return Thread.fistCaller( info - !haveToSkip(info.getDeclaringClass()) (++depth[0] nonSkippedFramesDepth), StackFrameInfo::getDeclaringClass ); } silly me... of course... why do I return the class when I need to return a boolean.. its been a long day already... ok, your version is accepted ;) using an array instead is quite the hack for me, but we are used to that for Java AIC bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: [PATCH] 4851444: Exposing sun.reflect.Reflection#getCallerClass as a public API in Java 8
Am 20.09.2013 20:46, schrieb Mandy Chung: [...] I have some trouble in expressing the skipToSecondPredicate without side effect in lambda. With the help from Paul Sandoz (thanks Paul), if I had a stack stream, Thread.getCaller() method would be like this: stream.filter(e - return REFLECTION_CLASSES.contains(e-getClassName())) .skip(2).findFirst(); and here I was and tried to show you what a Stream really is... I should have read that mail before the other ;) Only I was irritated not to find skip in http://download.java.net/lambda/b78/docs/api/java/util/stream/Stream.html I have seen skip in several examples. Anyway... has nothing to do with the matter here bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Replacement of sun.reflect.Reflection#getCallerClass
Am 04.09.2013 02:33, schrieb Mandy Chung: On 9/3/13 1:17 AM, Jochen Theodorou wrote: Defining a SE supported @CallerSensitive and also getCallerClass API poses the risk of encouraging developers to implement more @CS methods while not fully understand its implication. Again I am curious... what is such an implication? That your method will behave different depending on from where it got called and, depending on if that is from an expected or unexpected source, you may get an expected or unexpected result here? *and* this could lead to surprises and hard-to-diagnose issue to any framework/library or even another API provided in the same library calling a caller-sensitive method that can't tell if they get the true caller or not. Caller-sensitivity is used in getting the right visibility (e.g. Resource.getBundle and Class.forName) and for security checks. The platform's caller-sensitive methods involve both visibility and security. For the former (visiability): the use cases of getCallerClass are mostly for gaining visibility to avoid having the client code to supply the appropriate ClassLoader or Class. Such convenience has led to the existing stack walking code to search for a ClassLoader that it can find what the client code is asking for - it's not impossible that it might find the wrong ClassLoader but that happens to find a resource with the same name. For the latter (security), caller sensitive methods have kept us having so much fun in the past CPU releases. If you want to avoid such surprises and hard-to-diagnose issues, then you have to deprecate the convinience versions of Resource.getBundle and Class.forName. The new @CS will not change a bit of that misleading behaviour. As for the security... the whole ProtectionDomain and permissions system is highly caller sensitive. Again, that will not be changed, unless you want a completely new security model. So what is it you want me to tell? That making a public API for this will make things even worse? Then we don't need this discussion here. Even if you for example did the following and let's say introduce an annotation to mark a method that it should appear when I call getCallerClass, and everything else will not. Even then you might not get the expected caller, since a framework may use that in between somehow or by accident. And even if you gave the annotation for example a class as kind of key and use that key for getCallerClass to return only the last frame with that annotation, that used the specified key... I guess that would ensure predictable behaviour, but is far from convenient... Anyway, if the key is known somehow, you can still gain access Groovy/log4j and other existing framework has already dealt with such problem and written some non-trivial code to solve this problem. My point is more about the general app developers that need education and big warning in the documentation before they attempt learning this API to get the immediate caller. Sure, I am all for big warnings... Couple months ago a few of us discussed some security issues we had fixed in the past and evaluated what we could do about them - one thing we want to do is to remove the caller sensitive-ness. This is certainly a big big change and high compatibility risk - some future work. Just to give you a sense how serious we are the security issue around caller-sensitive methods. So Java will maybe get a new security model? I'm happy to know that you will build into Groovy 3.x the ability to transport the caller class from the Groovy class. I strongly believe this is the right thing to do and use the ResourceBundle.getBundle and Class.forName with the ClassLoader parameter. No filtering and no magic caller class API as you said. Well, Groovy 3.x will be a breaking change - one we intended to do anyway. So we are free to incorporate a fix for this with our abilities and independent of the VM. We don't really like to depend on sun.* packages, but sometimes there is just no way around. And this is not about the big breaking change, it is about how an existing framework can release a new version of its framework, that is compatible to the old versions, but has the replacement for getCallerClass. Such existing usages constrain the solution space quite a lot. I have replied to Nick's patch with the focus on the ability to get Class instance from stack trace for diagnosability purpose. I will soon bring up the discussion on the ability to get caller's Class / ClassLoader use case. I am looking forward to that bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Replacement of sun.reflect.Reflection#getCallerClass
Am 03.09.2013 05:04, schrieb Mandy Chung: Hi Nick, Thanks for the patch. JEP 176 [1] describes the caller-sensitive method and the need for a mechanical checking of caller-sensitive methods. Also Peter Levart in [2] explained the change in MethodHandles.Lookup related to @CS. I assume you understand the rationale behind and the potential security issues. In general defining caller-sensitive API is discouraged. The problem is, and the JDK itself shows that, that you sometimes cannot avoid caller-sensitive API. I thought that was clear by now. I am not the patch author, so understanding the potential security issues was not addressed to me, but I would like to understand them. So far I did hear only that if you have the Class object, you can do something, you normally cannot do. I would be interested in an example. Java security is really complicated to understand, and this would help to shed some light on the issue for others. Also I am wondering, if the Class object is a no-go, then how about class name and class loader? For Groovy we need the loader, but since we need to filter stack elements ourselves too, we also need the class names. A Class-object provides both, but I am sure almost all usages out there could work with the name and the loader only and if needed loading the class from the loader, with permissions set accordingly. Not sure how it is with internal class names here of course. AnonymousClassLoader would probably cause trouble here, but afaik that did not make it out of JSR292. A different way to solve the issue was suggested as using an annotation to define frames which are not looked up by getCallerClass. But it did not get any interest. So I am really curious how you want to solve this. Defining a SE supported @CallerSensitive and also getCallerClass API poses the risk of encouraging developers to implement more @CS methods while not fully understand its implication. Again I am curious... what is such an implication? That your method will behave different depending on from where it got called and, depending on if that is from an expected or unexpected source, you may get an expected or unexpected result here? [...] 1. Groovy 1.x and 2.x use the sun.reflect.Reflection.getCallerClass(int depth) method to: * emulates the way ResourceBundle.getBundle(String, Locale) works. Groovy runtime introduces intermediary stack frame between a caller and the call to getBundle, these frames needs to be filtered out; when looking for the caller classloader. * support the annotation @Grab, @Grab allows to specify a dependency between a code and a module (using apache ivy under the hood). The resolution is done at runtime and require a specific Classloader (the GroovyClassLoader), getCallerClass is used to find the class loader of the caller, again filtering out the intermediary stack frame. Groovy 3.x has a different implementation that doesn't need to do stack walk to filter its runtime frames and find the caller. To add here, I will build into Groovy 3.x the ability to transport the caller class from the Groovy class, that makes the call. So there will be no filtering and no magic caller class API at all. Still there will be several years we will have at least a Groovy 2.x out there. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Replacement of sun.reflect.Reflection#getCallerClass
Am 03.09.2013 11:04, schrieb Remi Forax: [...] So there is a need for an API to get the caller classes which is not getCallerClass (or any methods in a package sun.*) that will hide you the method handle API impementation and allow you choose to filter caller methods using a user defined API. not sure how filtering the caller methods would work (by annotation check?), but it is probably good enough for us, yes. In the end we need the class loader bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Replacement of sun.reflect.Reflection#getCallerClass
Am 03.09.2013 19:16, schrieb Peter Levart: [...] What about a simple restriction on methods returning such instances that Class objects are only returned when they are resolvable from the ClassLoader of client code. If they are not resolvable, null is returned. For example, the equivalent of: public class StackTraceFrame { private final Class? declaringClass; @CallerSensitive public Class? getDeclaringClass() { try { return Class.forName(declaringClass.getName(), false, Reflection.getCallerClass().getClassLoader()) == declaringClass ? declaringClass : null; } } catch (ClassNotFoundException ignore) {} return null; } // the name can be exposed without fear... public String getDeclaringClassName() { return declaringClass.getName(); } This example could be implemented more efficiently then above code (using private Class/ClassLoader API). for us it is the standard case, that the client code is not able to resolve the caller class. The code making the call to the method, that needs the information in the end is usually in a sub loader of the loader in which the declaring class of the method making use of the information is. Not always, but usually. Thus this would be useless to us bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Replacement of sun.reflect.Reflection#getCallerClass
Am 03.09.2013 16:12, schrieb A. Sundararajan: [...] If Groovy or any third-party framework gets away with that -- that is because you need to use modified security policy that gives those necessary permissions to groovy.jar or whatever third-party jar in question. just think of us needing to build a runtime structure copying what is in a normal class (plus some changes) available in terms of fields and methods. If you don't generate that information (and you cannot for unknown classes), then how can you get that without using reflection and things like getDeclaredMethods. (not to mention several properties and many other things). In other words: it is imho impossible to run even a single Groovy program without giving it some permissions. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Classes on the stack trace (was: getElementClass/StackTraceElement, was: @CallerSensitive public API, was: sun.reflect.Reflection.getCallerClass)
Am 30.07.2013 14:17, schrieb Peter Levart: [...] So what would give Groovy or other language runtimes headaches when all there was was a parameter-less getCallerClass() API? Aren't the intermediate frames inserted by those runtimes controlled by the runtimes? Couldn't the surface runtime-inserted methods capture the caller and pass it down? I guess the problem is supporting calling the caller-sensitive methods like Class.forName(String) and such which don't have the overloaded variant taking caller Class or ClassLoader as an argument... Speaking for Groovy... those intermediate frames are runtime controlled, yes, but passing down the caller class is exactly the problem. Imagine I would suggest that each and every method definition in Java automatically gets an additional parameter for the caller class, just to have access to it inside the method. You would not accept that for Java, would you? And so we cannot accept that for Groovy if we want to keep integration with Java... and the good integration with Java is one of the key points of Groovy. Even if we make something like that @CallerSensitive and add the parameter only in those cases, we break being able to override methods. Plus, before Groovy3 is not done we have to support several call paths. And the oldest one, which is still a fallback, does not support transporting the caller class through the runtime layers at all. Changing here is a breaking change. John Rose suggested to capture the caller in the surface method and bind it with a MethodHandle and then pass such MH down the runtime API and finally call that method via MH. Only that you then need a java7+ only version, plus the already mentioned problem, that not all of the three general call paths support that or can be changed in a way to enable that without breaking user code. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
Re: Classes on the stack trace (was: getElementClass/StackTraceElement, was: @CallerSensitive public API, was: sun.reflect.Reflection.getCallerClass)
Am 30.07.2013 16:16, schrieb Peter Levart: On 07/30/2013 03:19 PM, Jochen Theodorou wrote: Am 30.07.2013 14:17, schrieb Peter Levart: [...] So what would give Groovy or other language runtimes headaches when all there was was a parameter-less getCallerClass() API? Aren't the intermediate frames inserted by those runtimes controlled by the runtimes? Couldn't the surface runtime-inserted methods capture the caller and pass it down? I guess the problem is supporting calling the caller-sensitive methods like Class.forName(String) and such which don't have the overloaded variant taking caller Class or ClassLoader as an argument... Speaking for Groovy... those intermediate frames are runtime controlled, yes, but passing down the caller class is exactly the problem. Imagine I would suggest that each and every method definition in Java automatically gets an additional parameter for the caller class, just to have access to it inside the method. You would not accept that for Java, would you? And so we cannot accept that for Groovy if we want to keep integration with Java... Are you talking about internal Groovy implementation (the runtime-inserted methods) or the publicly visible API? that's the problem, it is a mix, some internal, other not. We are going to change that in Groovy 3 One solution for internal implementation of Groovy could be (speaking by heart since I don't know the internals of Groovy) for the surface public API method which doesn't have to have the special caller parameter, to capture the caller with getCallerClass() parameterless API (possibly enclosed with a quick check confirming that it might actually be needed) and bind it to a ThreadLocal, then use this ThreadLocal down at the end... confirming that it might actually be needed is a problem. In the old fallback path we don't know what we call until after we are deep in runtime code, and there it is too late. In the other paths we could mark those methods in a @CallerSensitive style and do it in that case only. and the good integration with Java is one of the key points of Groovy. Even if we make something like that @CallerSensitive and add the parameter only in those cases, we break being able to override methods. I guess I don't know every Groovy need to obtain the caller class. I thought the problem was to support calling caller-sensitive methods in Java API (like Class.forName(String)) from within Groovy code, where there are runtime-inserted frames between the call-site and the target method. Are there any other needs? ok, there is a misunderstanding... if we call a Java implemented method from Groovy, which is using getCallerClass() it may or may not work. In general this does not work and our problem is not about that at all. With the change to let getCallerClass() ignore some reflective frames it will work actually better as long as we use our custom callsite caching implementation, it will not work if indy is used or the fallback path. To be able to call a method Class#forName(String), we need to replace it with an implementation of our own, which we do with an approach similar to extension methods (only that ours can hide existing implementation methods for groovy). And in there we need to get to the caller class Our problem though is @Grab which is an annotation to add elements to the classpath while running a script. Plus, before Groovy3 is not done we have to support several call paths. And the oldest one, which is still a fallback, does not support transporting the caller class through the runtime layers at all. Changing here is a breaking change. Could you describe those call-paths? Examples of Groovy code and to what it gets translated (equivalent Java code at call site) with a brief description of what each intermediate layer (between the call-site and the target method) does and at which point the caller class is extracted... the code generated at the call site depends on several factors actually... The call site code itself is usually not very informative I start with Groovy 1.0, since that is basically the fallback path. Here this.foo() translates more or less to ScriptBytecodeAdapter.invokeMethod0(staticCallerClass, this,foo) which basically does this.getMetaClass().invokeMethod(staticCallerClass, this, foo). The problem is that the meta class might be user supplied and the code executed in invokeMethod as well. The invocation is then finally done by reflection. That means we have frames from ScriptBytecodeAdapter, from the meta class, as well as maybe frames from a custom meta class and reflection frames. At the level of ScriptBytecodeAdapter there is a means of transporting the caller class, but that is the static one. Once there is a subclass, this information is different from what is needed here and it cannot simply be exchanged. Even if the bytecode adapter is changed, we cannot change the public API for MetaClass#invokeMethod now
Re: problems with sun.reflect.Reflection.getCallerClass(int)
Am 20.07.2013 03:41, schrieb Mandy Chung: Hi Jochen, I read through the thread in mlvm-dev [1] that has a good discussion there. I have filed a RFE: 8020785: caller-sensitive methods to skip dynamic generated frames and look up the true caller This seems that java.lang.instrument might be an appropriate place for this support. This certainly requires further investigation. Mandy [1] http://mail.openjdk.java.net/pipermail/mlvm-dev/2013-July/005387.html [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8020785 can you explain a little how instrumentation could help here? Of course I don't expect a full solution, but the idea would be helping. bye blackdrag -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org
problems with sun.reflect.Reflection.getCallerClass(int)
Hi all, I started a thread about this already on the mlvm list, but since I get no responses there I am going to ask here as well. Plus I have an actual problem with openjdk 7u25 related to this it seems. First the u25 problem. According to https://jira.codehaus.org/browse/GROOVY-6246 openjdk 1.7.0_25 freezes with 100% CPU load in this code: https://github.com/groovy/groovy-core/blob/master/src/main/org/codehaus/groovy/reflection/ReflectionUtils.java#L105 This is surely not the nicest piece of code and all that, but that is not the point. The point is that this worked in openjdk 1.7.0_9 and does not in openjdk 1.7.0_25. Where it does work is in the oracle jdk 1.7.0_25. Btw: it is too bad there are no .tar.gz packages of the openjdk builds available. Assuming the diagnosis is right, this should help pinpointing the problem. Has there been any changes already to that method? Of course Reflection.getCallerClass(int) is a bigger problem for us. It was announced, that in u40 of jdk 1.7.0 this method will be no longer operational, unless you use a jvm switch. This is quite troublesome for us, but at least there is some way. In jdk 1.8 this method is supposed to disappear without replacement. There seems then to be the getCallerClass() method, which ignores frames from reflection (I strongly hope for you guys that lambda and indy are included here), but this will not rescue us at all. We have almost always a frame of our own appearing in a first call in Groovy. And there is no way to avoid that (not even in indy). That means for us, that any functionality depending on this cannot be called from a Groovy program anymore. We already replace RessourceBundle.getBundle(String) to let it discover the right loader. But after the change this will not help. Transporting the not always available caller class from the callsite into the method is for current versions of Groovy (and all versions of Groovy are influenced here) We are currently investigating if we can find a workaround for @Grab (http://groovy.codehaus.org/Grape) in some cases. There won't be one for all cases of course. What I would like to hear is a statement if this is taken as serious or if it will be ignored. From the mlvm list I have so far the impression, that it will be ignored. bye Jochen -- Jochen blackdrag Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org