Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
On 01/07/2016 02:18 AM, William ML Leslie wrote: On 7 January 2016 at 07:34, Peter Levart wrote: ClassValue API could use a different strategy and reference the cached instance of type T from ClassValue instance instead, but then you would have a problem with ClassValue instances reachable from other class loaders than the cached instances of type T. Regards, Peter Is the problem with this strategy [together with a java.util.WeakHashMap] that without this strong reference to the Class object, the Class itself could be collected? Hi William, ClassValue API maps a pair (Class, ClassValue) -> cached instance. The cached instance is "retrievable" if and only if both parts of the pair (Class, ClassValue) can be produced. As soon as either Class key or ClassValue instance becomes unreachable, the cached instance could become unreachable too as it is clearly not "retrievable" any more. But the cached instance must be kept reachable while it is still possible to produce both parts of the pair (Class, ClassValue). So ClassValue API chooses to make a strong link from Class key to cached instance. Alternative strategy would be for API to make a strong link from ClassValue instance to cached instance. Either case has it's drawback. The current ClassValue strategy has drawback when the cached instance directly or indirectly references the ClassValue instance it was constructed in (the case of the failing test in this thread). The cached instance lifetime is then bound to the lifetime of the Class key. The alternative strategy has drawback when the cached instance directly of indirectly references the Class key it was constructed for. The cached instance lifetime is then bound to the lifetime of the ClassValue instance. There's no way out short of choosing the right strategy for the task at hand. And I showed in the last post how to effectively transform current strategy to the alternative strategy by introducing an indirection to the cached value through a WeakReference and making a strong link from ClassValue instance to cache value at the same time. Regards, Peter P.S. With Ephemerons, I think the general solution would be possible. ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
Hi Jochen,
On 01/07/2016 03:05 PM, Jochen Theodorou wrote:
Hi Peter,
Am 06.01.2016 um 21:34 schrieb Peter Levart:
[...]
ClassValue API could use a different strategy and reference the cached
instance of type T from ClassValue instance instead, but then you
would have a problem with ClassValue instances reachable from other
class loaders than the cached instances of type T.
yes, that is what I tried as well... I used my own map directly in
ClassValue and avoided the map ClassValue uses natively.
So the solution for this case is basically using a Soft- or
Weak-Reference in Dummy, right?
The solution to use any Class key and not be dependent on it's origin
(i.e. the class loader) and therefore on it's lifetime, is to decouple
the cached value from the Class key by an indirection through a
WeakReference, but then you should make sure the cached value doesn't go
away while it is still possible to retrieve it - the trick is to hook it
on a list that is reachable from the ClassValue instance that
constructed it:
import java.io.File;
import java.io.UncheckedIOException;
import java.lang.ref.WeakReference;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Paths;
import java.util.LinkedList;
import java.util.List;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class CVTest {
public static class MyClassValue extends
ClassValue> {
private final List dummies = new ArrayList<>();
@Override
protected WeakReference computeValue(Class type) {
Dummy ret = new Dummy();
// make it reachable from MyClassValue instance that
constructed it
synchronized(dummies) {
dummies.add(ret);
}
Dummy.o = this;
// decouple it from 'type' - make it only weakly reachable
from 'type'
return new WeakReference<>(ret);
}
}
public static class Dummy {
static Object o;
@Override
protected void finalize() throws Throwable {
System.out.println(this + " finalized");
}
}
public static void main(String[] args) throws Exception {
String[] cp = System.getProperty("java.class.path")
.split(Pattern.quote(File.pathSeparator));
URL[] urls = Stream.of(cp).map(f -> {
try {
return Paths.get(f).toUri().toURL();
} catch (MalformedURLException e) {
throw new UncheckedIOException(e);
}
}).collect(Collectors.toList()).toArray(new URL[0]);
for (long i = 0; i < 10; i++) {
URLClassLoader classLoader = new URLClassLoader(
urls, ClassLoader.getSystemClassLoader().getParent());
@SuppressWarnings("unchecked")
ClassValue> cv =
(ClassValue>)
classLoader.loadClass("CVTest$MyClassValue").newInstance();
WeakReference valueRef = cv.get(Integer.TYPE);
// while 'cv' is alive, 'value' is alive too since 'cv' keeps a
// strong reference to 'value' in its 'dummies' list...
Object value = valueRef.get();
assert value != null;
assert value.getClass().getClassLoader() == classLoader;
System.out.println(i);
classLoader.close();
classLoader = null;
cv = null;
value = null;
System.gc();
Thread.sleep(200L);
}
}
}
...running this example produces:
0
CVTest$Dummy@6b027549 finalized
1
CVTest$Dummy@65c59659 finalized
2
CVTest$Dummy@61fa1e39 finalized
...
...
Regards, Peter
bye Jochen
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
On 01/07/2016 03:05 PM, Jochen Theodorou wrote:
Hi Peter,
Am 06.01.2016 um 21:34 schrieb Peter Levart:
[...]
Currently the cached instance of type T is reachable from Class
instance and you are using Integer.TYPE in your example. Which means
that the cached instance of T is never going to be released if
ClassValue instance is reachable from cached instance of T. In your
example, it is: (Dummy instance -> Dummy.class -> MyClassValue
instance). So you get this reachability chain: Integer.TYPE ->
ClassValueMap instance -> ClassValueMap.Entry[] -> ClassValueMap.Entry
-> Dummy instance -> Dummy.class -> MyClassValue instance.
>
Integer.TYPE is a Class instance representing int type which is
loaded by bootstrap class loader, which never goes away.
So in other words, this is supposed to work if I used for example
Dummy instead of Integer.TYPE? Because this is not the case (just
tested with 1.8.0_66). Which means we are not talking only about
bootstrap classes, but about basically any class.
Hi Jochen,
I suspect you specified MyClassValue.Dummy.class instead of Integer.TYPE
? Was Dummy.class loaded by same class loader as the MyClassValue for
which the ClassValue.get() was executed. I have tried to execute the
following variation of the test (no need to pack a separate jar file -
you can execute if directly from within an IDE):
import java.io.File;
import java.io.UncheckedIOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Paths;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class CVTest {
public static class MyClassValue extends ClassValue {
protected Object computeValue(Class type) {
Dummy ret = new Dummy();
Dummy.o = this;
return ret;
}
}
public static class Dummy {
static Object o;
@Override
protected void finalize() throws Throwable {
System.out.println(this + " finalized");
}
}
public static void main(String[] args) throws Exception {
String[] cp = System.getProperty("java.class.path")
.split(Pattern.quote(File.pathSeparator));
URL[] urls = Stream.of(cp).map(f -> {
try {
return Paths.get(f).toUri().toURL();
} catch (MalformedURLException e) {
throw new UncheckedIOException(e);
}
}).collect(Collectors.toList()).toArray(new URL[0]);
for (long i = 0; i < 10; i++) {
URLClassLoader classLoader = new URLClassLoader(
urls, ClassLoader.getSystemClassLoader().getParent());
ClassValue cv = (ClassValue)
classLoader.loadClass("CVTest$MyClassValue").newInstance();
Class key = classLoader.loadClass("CVTest$Dummy");
Object value = cv.get(key);
assert value.getClass().getClassLoader() == classLoader;
assert key.getClassLoader() == classLoader;
System.out.println(i);
classLoader.close();
classLoader = null;
cv = null;
key = null;
value = null;
System.gc();
Thread.sleep(200L);
}
}
}
...and it prints:
0
CVTest$Dummy@6b027549 finalized
1
CVTest$Dummy@65c59659 finalized
2
CVTest$Dummy@61fa1e39 finalized
3
CVTest$Dummy@24f89c62 finalized
4
CVTest$Dummy@5ca80d89 finalized
5
CVTest$Dummy@15963f1b finalized
6
CVTest$Dummy@242cf046 finalized
7
CVTest$Dummy@44d7632d finalized
8
CVTest$Dummy@588ffa7b finalized
9
CVTest$Dummy@7cc57896 finalized
...indicating that objects do get released in this case.
Regards, Peter
ClassValue API could use a different strategy and reference the cached
instance of type T from ClassValue instance instead, but then you
would have a problem with ClassValue instances reachable from other
class loaders than the cached instances of type T.
yes, that is what I tried as well... I used my own map directly in
ClassValue and avoided the map ClassValue uses natively.
So the solution for this case is basically using a Soft- or
Weak-Reference in Dummy, right?
bye Jochen
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
Hi Peter, Am 06.01.2016 um 21:34 schrieb Peter Levart: [...] Currently the cached instance of type T is reachable from Class instance and you are using Integer.TYPE in your example. Which means that the cached instance of T is never going to be released if ClassValue instance is reachable from cached instance of T. In your example, it is: (Dummy instance -> Dummy.class -> MyClassValue instance). So you get this reachability chain: Integer.TYPE -> ClassValueMap instance -> ClassValueMap.Entry[] -> ClassValueMap.Entry -> Dummy instance -> Dummy.class -> MyClassValue instance. > Integer.TYPE is a Class instance representing int type which is loaded by bootstrap class loader, which never goes away. So in other words, this is supposed to work if I used for example Dummy instead of Integer.TYPE? Because this is not the case (just tested with 1.8.0_66). Which means we are not talking only about bootstrap classes, but about basically any class. ClassValue API could use a different strategy and reference the cached instance of type T from ClassValue instance instead, but then you would have a problem with ClassValue instances reachable from other class loaders than the cached instances of type T. yes, that is what I tried as well... I used my own map directly in ClassValue and avoided the map ClassValue uses natively. So the solution for this case is basically using a Soft- or Weak-Reference in Dummy, right? bye Jochen ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
On 7 January 2016 at 07:34, Peter Levart wrote: > ClassValue API could use a different strategy and reference the cached > instance of type T from ClassValue instance instead, but then you would > have a problem with ClassValue instances reachable from other class loaders > than the cached instances of type T. > > Regards, Peter Is the problem with this strategy [together with a java.util.WeakHashMap] that without this strong reference to the Class object, the Class itself could be collected? -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement. ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
On 01/06/2016 09:34 PM, Peter Levart wrote: Hi Craig, I think that what you are asking for is impossible to achieve in current Java. ClassValue API is basically a cache that maps a pair: (ClassValue, Class) -> cached instance of type T You would like the cached instance of type T to not be reachable from either ClassValue instance or from Class instance, but still be non-reclaimable while either of correction: while either of ... or ... is -> while both of ... and ... are ClassValue instance or Class instance is reachable in some other way that excludes being reachable from the cached instance of type T. For that to be achievable, Java would have to provide a feature called "Ephemeron". Currently the cached instance of type T is reachable from Class instance and you are using Integer.TYPE in your example. Which means that the cached instance of T is never going to be released if ClassValue instance is reachable from cached instance of T. In your example, it is: (Dummy instance -> Dummy.class -> MyClassValue instance). So you get this reachability chain: Integer.TYPE -> ClassValueMap instance -> ClassValueMap.Entry[] -> ClassValueMap.Entry -> Dummy instance -> Dummy.class -> MyClassValue instance. Integer.TYPE is a Class instance representing int type which is loaded by bootstrap class loader, which never goes away. ClassValue API could use a different strategy and reference the cached instance of type T from ClassValue instance instead, but then you would have a problem with ClassValue instances reachable from other class loaders than the cached instances of type T. Regards, Peter On 01/06/2016 06:14 PM, Craig Andrews wrote: I apologize for contacting this list - I'm sure this isn't the "right way" to contact the OpenJDK project, but I'm not clear what the "right way" is. I'm hoping to raise https://bugs.openjdk.java.net/browse/JDK-8136353 "ClassValue preventing class unloading" as I believe it's a significant issue in ClassValue which is a blocker for its use in dynamic languages (which is primiarly why ClassValue was introduced). I think the P5 priority set on the bug now is way too low, perhaps you could consider raising the priority? The source code in the issue description is incorrect; it doesn't compile. Could you please attach the working test cases to the issue, so future testers and developers can reproduce the problem? Here are links to the 2 source code files: https://issues.apache.org/jira/secure/attachment/12765900/CVTest.java https://issues.apache.org/jira/secure/attachment/12765901/MyClassValue.java (which of course should be directly attached to the openjdk issue tracker issue, and not hyperlinked to the Groovy tracker) And to reproduce the issue, run: $ javac MyClassValue.java && javac CVTest.java && mkdir -p t && jar cvf t/t.jar MyClassValue*.class && rm MyClassValue*.class && JAVA_OPTS=-Xmx4m java CVTest and wait for the an error to occur, which is: Exception in thread "main" java.lang.OutOfMemoryError: Compressed class space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:759) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:152) at java.net.URLClassLoader.defineClass(URLClassLoader.java:470) at java.net.URLClassLoader.access$100(URLClassLoader.java:76) at java.net.URLClassLoader$1.run(URLClassLoader.java:371) at java.net.URLClassLoader$1.run(URLClassLoader.java:365) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:364) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at CVTest.main(CVTest.java:12) The bug is reproducible on all JDK 8 and 9 builds (I tested up to JDK 9 build 99). Based on my understanding of the situation from my research in the Groovy bug that discovered the issue ( https://issues.apache.org/jira/browse/GROOVY-6704 ) and some work another person did with the YourKit profiler ( https://www.yourkit.com/forum/viewtopic.php?f=3&t=12658 ), I suspect that the fix for https://bugs.openjdk.java.net/browse/JDK-8032606 "ClassValue.ClassValueMap.type is unused" is relevant; I think the problem lies in the java.lang.Class.classValueMap implementation. Thank you, ~Craig Andrews ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
Re: Questions about JDK-8136353 "ClassValue preventing class unloading"
Hi Craig, I think that what you are asking for is impossible to achieve in current Java. ClassValue API is basically a cache that maps a pair: (ClassValue, Class) -> cached instance of type T You would like the cached instance of type T to not be reachable from either ClassValue instance or from Class instance, but still be non-reclaimable while either of ClassValue instance or Class instance is reachable in some other way that excludes being reachable from the cached instance of type T. For that to be achievable, Java would have to provide a feature called "Ephemeron". Currently the cached instance of type T is reachable from Class instance and you are using Integer.TYPE in your example. Which means that the cached instance of T is never going to be released if ClassValue instance is reachable from cached instance of T. In your example, it is: (Dummy instance -> Dummy.class -> MyClassValue instance). So you get this reachability chain: Integer.TYPE -> ClassValueMap instance -> ClassValueMap.Entry[] -> ClassValueMap.Entry -> Dummy instance -> Dummy.class -> MyClassValue instance. Integer.TYPE is a Class instance representing int type which is loaded by bootstrap class loader, which never goes away. ClassValue API could use a different strategy and reference the cached instance of type T from ClassValue instance instead, but then you would have a problem with ClassValue instances reachable from other class loaders than the cached instances of type T. Regards, Peter On 01/06/2016 06:14 PM, Craig Andrews wrote: I apologize for contacting this list - I'm sure this isn't the "right way" to contact the OpenJDK project, but I'm not clear what the "right way" is. I'm hoping to raise https://bugs.openjdk.java.net/browse/JDK-8136353 "ClassValue preventing class unloading" as I believe it's a significant issue in ClassValue which is a blocker for its use in dynamic languages (which is primiarly why ClassValue was introduced). I think the P5 priority set on the bug now is way too low, perhaps you could consider raising the priority? The source code in the issue description is incorrect; it doesn't compile. Could you please attach the working test cases to the issue, so future testers and developers can reproduce the problem? Here are links to the 2 source code files: https://issues.apache.org/jira/secure/attachment/12765900/CVTest.java https://issues.apache.org/jira/secure/attachment/12765901/MyClassValue.java (which of course should be directly attached to the openjdk issue tracker issue, and not hyperlinked to the Groovy tracker) And to reproduce the issue, run: $ javac MyClassValue.java && javac CVTest.java && mkdir -p t && jar cvf t/t.jar MyClassValue*.class && rm MyClassValue*.class && JAVA_OPTS=-Xmx4m java CVTest and wait for the an error to occur, which is: Exception in thread "main" java.lang.OutOfMemoryError: Compressed class space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:759) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:152) at java.net.URLClassLoader.defineClass(URLClassLoader.java:470) at java.net.URLClassLoader.access$100(URLClassLoader.java:76) at java.net.URLClassLoader$1.run(URLClassLoader.java:371) at java.net.URLClassLoader$1.run(URLClassLoader.java:365) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:364) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at CVTest.main(CVTest.java:12) The bug is reproducible on all JDK 8 and 9 builds (I tested up to JDK 9 build 99). Based on my understanding of the situation from my research in the Groovy bug that discovered the issue ( https://issues.apache.org/jira/browse/GROOVY-6704 ) and some work another person did with the YourKit profiler ( https://www.yourkit.com/forum/viewtopic.php?f=3&t=12658 ), I suspect that the fix for https://bugs.openjdk.java.net/browse/JDK-8032606 "ClassValue.ClassValueMap.type is unused" is relevant; I think the problem lies in the java.lang.Class.classValueMap implementation. Thank you, ~Craig Andrews ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev ___ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
