Re: ClassValue rooting objects after it goes away?

[Attila Szegedi]

I had some similar concerns with some data structures I maintain in Dynalink; I 
specifically have a utility method to check if classes from two class loaders 
are allowed to keep strong references to each other 
(jdk.dynalink.internal.InternalTypeUtilities.canReferenceDirectly if you want 
to look it up). It’s used in few places to either choose between strong 
references and weak references, or to forgo some caching for unrelated classes 
entirely lest it creates an unwanted strong reference between class loaders.

Attila.

> On 2018. Mar 28., at 14:42, Peter Levart  wrote:
> 
> Hi Charles,
> 
> On 03/02/2018 09:22 PM, Charles Oliver Nutter wrote:
>> Put it another way: does a static reference from a class to itself prevent 
>> that class from being garbage collected? Of course not. ClassValue is 
>> intended to be a way to inject pseudo-static data into either a class or a 
>> Class. Injecting that data, even if it has a reference back to the class, 
>> should not prevent the class from being collected.
> 
> And it should not with ClassValue either. The problem is when you associate 
> values with "system" classes (such as Object, String, etc.). Such values 
> become strongly reachable from Object.class, String.class, etc., 
> respectively. It's like injecting an instance field into Class class and 
> assigning Object.class.myField = myValue, String.class.myField = myValue, ... 
> or equivalently injecting static fields into Object, String, etc class(es) 
> and assigning Object.myField = myValue, String.myField = myValue, ...
> 
> This would not in itself be a problem if myValue was composed only of objects 
> of system classes (like String, Integer, HashMap, ArrayList, etc..), but you 
> probably have your own Ruby runtime classes to maintain the data structure 
> behind myValue. And each of those custom objects maintains an implicit 
> reference to its runtime class, which is a Ruby runtime class, loaded by 
> whichever ClassLoader loaded the Ruby runtime, so the chain of strong 
> references extends from system class loader to Ruby runtime class loader, 
> effectively preventing it from being unloaded.
> 
> If you don't need to use ClassValue.remove() in your code, you could try 
> using the following wrapper above ClassValue. The overhead of its get() 
> method compared to ClassValue's should not be to big - just another lookup 
> into a ThreadLocal and a reference comparison with null (in the general 
> fast-path case):
> 
> 
> /**
>  * {@link ClassValue} makes mapped values strongly reachable from {@link 
> Class}(es)
>  * they are associated with while they are not reachable directly from {@link 
> ClassValue}(s).
>  * This is some times desirable, but other times not. This wrapper provides a
>  * ClassValue-like API where the roles are reversed - associated values are 
> not strongly
>  * reachable from Class(es) but from WeakClassValue(s) they are associated 
> with.
>  *
>  * @param  they type of associated values
>  */
> public abstract class WeakClassValue {
> 
> // temporary holder of a Node computed in embedded cv.computeValue() so we
> // can later find out whether the embedded WeakReference has been 
> installed into
> // ClassValueMap or not by comparing it to instance returned from cv.get()
> private static final ThreadLocal NODE_TL = new ThreadLocal<>();
> 
> // head of doubly-linked list of nodes, keeping associated values 
> reachable from
> // WeakClassValue instance
> private final Node head = new Node(null, null, null, null, null);
> 
> // a single reference queue for all Node(s) to get enqueued when their 
> referents
> // (Class instances) get GCed...
> private static final ReferenceQueue queue = new 
> ReferenceQueue<>();
> 
> // Node is a PhantomReference tracking reachability of associated Class.
> // it also holds the associated value, WeakReference to it, and prev/next 
> pointers into the list
> private static final class Node extends PhantomReference {
> private final Node head;
> private Node prev, next;
> private Object value;
> WeakReference ref;
> 
> Node(Class type, ReferenceQueue queue,
>  Node head, Object value, WeakReference ref) {
> super(type, queue);
> this.value = value;
> this.ref = ref;
> if (head == null) {
> // constructing head node
> this.prev = this.next = this;
> this.head = null;
> } else {
> // link node into list anchored by head
> synchronized (head) {
> this.prev = head;
> this.next = head.next;
> this.head = head; // remember where we belong to
> head.next.prev = this;
> head.next = this;
> }
> }
> }
> 
> // unlink the 

Re: ClassValue rooting objects after it goes away?

[Peter Levart]

Hi Charles,

On 03/02/2018 09:22 PM, Charles Oliver Nutter wrote:
Put it another way: does a static reference from a class to itself 
prevent that class from being garbage collected? Of course not. 
ClassValue is intended to be a way to inject pseudo-static data into 
either a class or a Class. Injecting that data, even if it has a 
reference back to the class, should not prevent the class from being 
collected.


And it should not with ClassValue either. The problem is when you 
associate values with "system" classes (such as Object, String, etc.). 
Such values become strongly reachable from Object.class, String.class, 
etc., respectively. It's like injecting an instance field into Class 
class and assigning Object.class.myField = myValue, String.class.myField 
= myValue, ... or equivalently injecting static fields into Object, 
String, etc class(es) and assigning Object.myField = myValue, 
String.myField = myValue, ...


This would not in itself be a problem if myValue was composed only of 
objects of system classes (like String, Integer, HashMap, ArrayList, 
etc..), but you probably have your own Ruby runtime classes to maintain 
the data structure behind myValue. And each of those custom objects 
maintains an implicit reference to its runtime class, which is a Ruby 
runtime class, loaded by whichever ClassLoader loaded the Ruby runtime, 
so the chain of strong references extends from system class loader to 
Ruby runtime class loader, effectively preventing it from being unloaded.


If you don't need to use ClassValue.remove() in your code, you could try 
using the following wrapper above ClassValue. The overhead of its get() 
method compared to ClassValue's should not be to big - just another 
lookup into a ThreadLocal and a reference comparison with null (in the 
general fast-path case):



/**
 * {@link ClassValue} makes mapped values strongly reachable from 
{@link Class}(es)
 * they are associated with while they are not reachable directly from 
{@link ClassValue}(s).
 * This is some times desirable, but other times not. This wrapper 
provides a
 * ClassValue-like API where the roles are reversed - associated values 
are not strongly
 * reachable from Class(es) but from WeakClassValue(s) they are 
associated with.

 *
 * @param  they type of associated values
 */
public abstract class WeakClassValue {

    // temporary holder of a Node computed in embedded 
cv.computeValue() so we
    // can later find out whether the embedded WeakReference has been 
installed into
    // ClassValueMap or not by comparing it to instance returned from 
cv.get()

    private static final ThreadLocal NODE_TL = new ThreadLocal<>();

    // head of doubly-linked list of nodes, keeping associated values 
reachable from

    // WeakClassValue instance
    private final Node head = new Node(null, null, null, null, null);

    // a single reference queue for all Node(s) to get enqueued when 
their referents

    // (Class instances) get GCed...
    private static final ReferenceQueue queue = new 
ReferenceQueue<>();


    // Node is a PhantomReference tracking reachability of associated 
Class.
    // it also holds the associated value, WeakReference to it, and 
prev/next pointers into the list

    private static final class Node extends PhantomReference {
    private final Node head;
    private Node prev, next;
    private Object value;
    WeakReference ref;

    Node(Class type, ReferenceQueue queue,
 Node head, Object value, WeakReference ref) {
    super(type, queue);
    this.value = value;
    this.ref = ref;
    if (head == null) {
    // constructing head node
    this.prev = this.next = this;
    this.head = null;
    } else {
    // link node into list anchored by head
    synchronized (head) {
    this.prev = head;
    this.next = head.next;
    this.head = head; // remember where we belong to
    head.next.prev = this;
    head.next = this;
    }
    }
    }

    // unlink the node from the list using right head as a lock
    void unlink() {
    synchronized (head) {
    prev.next = next;
    next.prev = prev;
    next = prev = null;
    value = null;
    ref.clear();
    ref = null;
    }
    }
    }

    // work is delegated to embedded ClassValue instance
    private final ClassValue cv = new ClassValue<>() {
    @SuppressWarnings("unchecked")
    @Override
    protected WeakReference computeValue(Class type) {
    T value = WeakClassValue.this.computeValue(type);
    WeakReference ref = new WeakReference<>(value);
    // link the value into a doubly-linked list rooted at
    // WeakClassValue 

Re: ClassValue rooting objects after it goes away?

[Charles Oliver Nutter]

Put it another way: does a static reference from a class to itself prevent
that class from being garbage collected? Of course not. ClassValue is
intended to be a way to inject pseudo-static data into either a class or a
Class. Injecting that data, even if it has a reference back to the class,
should not prevent the class from being collected.

On Fri, Mar 2, 2018 at 2:19 PM Charles Oliver Nutter 
wrote:

> I have posted a modified version of my description to the main bug report.
>
> TLDR: ClassValue should not root objects.
>
> - Charlie
>
> On Fri, Mar 2, 2018 at 2:13 PM Charles Oliver Nutter 
> wrote:
>
>> Yes, it may be the same bug.
>>
>> In my case, the ClassValue is held by a utility object used for our Java
>> integration. That utility object has to live somewhere, so it's held by the
>> JRuby runtime instance. There's a strong reference chain leading to the
>> ClassValue.
>>
>> The value is a Ruby representation of the class, with reflected methods
>> parsed out and turned into Ruby endpoints. Obviously, the value also
>> references the class, either directly or indirectly through reflected
>> members.
>>
>> The Ruby class wrapper is only hard referenced directly if there's an
>> instance of the object live and moving through JRuby. It may be referenced
>> indirectly through inline caches.
>>
>> However...I do not believe this should prevent collection of the class
>> associated with the ClassValue.
>>
>> The value referenced in the ClassValue should not constitute a hard
>> reference. If it is alive *only* because of its associate with a given
>> class, that should not be enough to root either the object or the class.
>>
>> ClassValue should work like ThreadLocal. If the Thread associated with a
>> value goes away, the value reference goes away. ThreadLocal does nothing
>> prevent it from being collected. If the Class associated with a Value goes
>> away, the same should happen to that Value and it should be collectable
>> once all other hard references are gone.
>>
>> Perhaps I've misunderstood?
>>
>> - Charlie
>>
>> On Fri, Mar 2, 2018 at 12:16 PM Vladimir Ivanov <
>> vladimir.x.iva...@oracle.com> wrote:
>>
>>> Charlie,
>>>
>>> Does it look similar to the following bugs?
>>>https://bugs.openjdk.java.net/browse/JDK-8136353
>>>https://bugs.openjdk.java.net/browse/JDK-8169425
>>>
>>> If that's the same (and it seems so to me [1]), then speak up and
>>> persuade Paul it's an important edge case (as stated in JDK-8169425).
>>>
>>> Best regards,
>>> Vladimir Ivanov
>>>
>>> [1] new RubyClass(Ruby.this) in
>>>
>>>  public static class Ruby {
>>>  private ClassValue cache = new
>>> ClassValue() {
>>>  protected RubyClass computeValue(Class type) {
>>>  return new RubyClass(Ruby.this);
>>>  }
>>>  };
>>>
>>> On 3/1/18 2:25 AM, Charles Oliver Nutter wrote:
>>> > So I don't think we ever closed the loop here. Did anyone on the JDK
>>> > side confirm this, file an issue, or fix it?
>>> >
>>> > We still have ClassValue disabled in JRuby because of the rooting
>>> issues
>>> > described here and in https://github.com/jruby/jruby/pull/3228.
>>> >
>>> > - Charlie
>>> >
>>> > On Thu, Aug 27, 2015 at 7:04 AM Jochen Theodorou >> > > wrote:
>>> >
>>> > One more thing...
>>> >
>>> > Remi, I tried your link with my simplified scenario and it does
>>> there
>>> > not stop the collection of the classloader
>>> >
>>> > Am 27.08.2015 11:54, schrieb Jochen Theodorou:
>>> >  > Hi,
>>> >  >
>>> >  > In trying to reproduce the problem outside of Groovy I stumbled
>>> > over a
>>> >  > case case which I think should work
>>> >  >
>>> >  > public class MyClassValue extends ClassValue {
>>> >  >  protected Object computeValue(Class type) {
>>> >  >  Dummy ret = new Dummy();
>>> >  >  Dummy.l.add (this);
>>> >  >  return ret;
>>> >  >  }
>>> >  > }
>>> >  >
>>> >  >   class Dummy {
>>> >  >   static final ArrayList l = new ArrayList();
>>> >  >   }
>>> >  >
>>> >  > basically this means there will be a hard reference on the
>>> ClassValue
>>> >  > somewhere. It can be in a static or non-static field, direct or
>>> >  > indirect. But this won't collect. If I put for example a
>>> > WeakReference
>>> >  > in between it works again.
>>> >  >
>>> >  > Finally I also tested to put the hard reference in a third class
>>> >  > instead, to avoid this self reference. But it can still not
>>> collect.
>>> >  >
>>> >  > So I currently have the impression that if anything holds a hard
>>> >  > reference on the class value that the classloader cannot be
>>> collected
>>> >  > anymore.
>>> >  >
>>> >  > Unless I misunderstand something here I see that as a bug
>>> >  >
>>> >  > bye blackdrag
>>> >  >

Re: ClassValue rooting objects after it goes away?

[Charles Oliver Nutter]

I have posted a modified version of my description to the main bug report.

TLDR: ClassValue should not root objects.

- Charlie

On Fri, Mar 2, 2018 at 2:13 PM Charles Oliver Nutter 
wrote:

> Yes, it may be the same bug.
>
> In my case, the ClassValue is held by a utility object used for our Java
> integration. That utility object has to live somewhere, so it's held by the
> JRuby runtime instance. There's a strong reference chain leading to the
> ClassValue.
>
> The value is a Ruby representation of the class, with reflected methods
> parsed out and turned into Ruby endpoints. Obviously, the value also
> references the class, either directly or indirectly through reflected
> members.
>
> The Ruby class wrapper is only hard referenced directly if there's an
> instance of the object live and moving through JRuby. It may be referenced
> indirectly through inline caches.
>
> However...I do not believe this should prevent collection of the class
> associated with the ClassValue.
>
> The value referenced in the ClassValue should not constitute a hard
> reference. If it is alive *only* because of its associate with a given
> class, that should not be enough to root either the object or the class.
>
> ClassValue should work like ThreadLocal. If the Thread associated with a
> value goes away, the value reference goes away. ThreadLocal does nothing
> prevent it from being collected. If the Class associated with a Value goes
> away, the same should happen to that Value and it should be collectable
> once all other hard references are gone.
>
> Perhaps I've misunderstood?
>
> - Charlie
>
> On Fri, Mar 2, 2018 at 12:16 PM Vladimir Ivanov <
> vladimir.x.iva...@oracle.com> wrote:
>
>> Charlie,
>>
>> Does it look similar to the following bugs?
>>https://bugs.openjdk.java.net/browse/JDK-8136353
>>https://bugs.openjdk.java.net/browse/JDK-8169425
>>
>> If that's the same (and it seems so to me [1]), then speak up and
>> persuade Paul it's an important edge case (as stated in JDK-8169425).
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> [1] new RubyClass(Ruby.this) in
>>
>>  public static class Ruby {
>>  private ClassValue cache = new
>> ClassValue() {
>>  protected RubyClass computeValue(Class type) {
>>  return new RubyClass(Ruby.this);
>>  }
>>  };
>>
>> On 3/1/18 2:25 AM, Charles Oliver Nutter wrote:
>> > So I don't think we ever closed the loop here. Did anyone on the JDK
>> > side confirm this, file an issue, or fix it?
>> >
>> > We still have ClassValue disabled in JRuby because of the rooting issues
>> > described here and in https://github.com/jruby/jruby/pull/3228.
>> >
>> > - Charlie
>> >
>> > On Thu, Aug 27, 2015 at 7:04 AM Jochen Theodorou > > > wrote:
>> >
>> > One more thing...
>> >
>> > Remi, I tried your link with my simplified scenario and it does
>> there
>> > not stop the collection of the classloader
>> >
>> > Am 27.08.2015 11:54, schrieb Jochen Theodorou:
>> >  > Hi,
>> >  >
>> >  > In trying to reproduce the problem outside of Groovy I stumbled
>> > over a
>> >  > case case which I think should work
>> >  >
>> >  > public class MyClassValue extends ClassValue {
>> >  >  protected Object computeValue(Class type) {
>> >  >  Dummy ret = new Dummy();
>> >  >  Dummy.l.add (this);
>> >  >  return ret;
>> >  >  }
>> >  > }
>> >  >
>> >  >   class Dummy {
>> >  >   static final ArrayList l = new ArrayList();
>> >  >   }
>> >  >
>> >  > basically this means there will be a hard reference on the
>> ClassValue
>> >  > somewhere. It can be in a static or non-static field, direct or
>> >  > indirect. But this won't collect. If I put for example a
>> > WeakReference
>> >  > in between it works again.
>> >  >
>> >  > Finally I also tested to put the hard reference in a third class
>> >  > instead, to avoid this self reference. But it can still not
>> collect.
>> >  >
>> >  > So I currently have the impression that if anything holds a hard
>> >  > reference on the class value that the classloader cannot be
>> collected
>> >  > anymore.
>> >  >
>> >  > Unless I misunderstand something here I see that as a bug
>> >  >
>> >  > bye blackdrag
>> >  >
>> >
>> >
>> > --
>> > Jochen "blackdrag" Theodorou
>> > blog: http://blackdragsview.blogspot.com/
>> >
>> > ___
>> > mlvm-dev mailing list
>> > mlvm-dev@openjdk.java.net 
>> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >
>> > --
>> >
>> > - Charlie (mobile)
>> >
>> >
>> >
>> > ___
>> > mlvm-dev mailing list
>> > mlvm-dev@openjdk.java.net
>> > 

Re: ClassValue rooting objects after it goes away?

[Charles Oliver Nutter]

Yes, it may be the same bug.

In my case, the ClassValue is held by a utility object used for our Java
integration. That utility object has to live somewhere, so it's held by the
JRuby runtime instance. There's a strong reference chain leading to the
ClassValue.

The value is a Ruby representation of the class, with reflected methods
parsed out and turned into Ruby endpoints. Obviously, the value also
references the class, either directly or indirectly through reflected
members.

The Ruby class wrapper is only hard referenced directly if there's an
instance of the object live and moving through JRuby. It may be referenced
indirectly through inline caches.

However...I do not believe this should prevent collection of the class
associated with the ClassValue.

The value referenced in the ClassValue should not constitute a hard
reference. If it is alive *only* because of its associate with a given
class, that should not be enough to root either the object or the class.

ClassValue should work like ThreadLocal. If the Thread associated with a
value goes away, the value reference goes away. ThreadLocal does nothing
prevent it from being collected. If the Class associated with a Value goes
away, the same should happen to that Value and it should be collectable
once all other hard references are gone.

Perhaps I've misunderstood?

- Charlie

On Fri, Mar 2, 2018 at 12:16 PM Vladimir Ivanov <
vladimir.x.iva...@oracle.com> wrote:

> Charlie,
>
> Does it look similar to the following bugs?
>https://bugs.openjdk.java.net/browse/JDK-8136353
>https://bugs.openjdk.java.net/browse/JDK-8169425
>
> If that's the same (and it seems so to me [1]), then speak up and
> persuade Paul it's an important edge case (as stated in JDK-8169425).
>
> Best regards,
> Vladimir Ivanov
>
> [1] new RubyClass(Ruby.this) in
>
>  public static class Ruby {
>  private ClassValue cache = new ClassValue()
> {
>  protected RubyClass computeValue(Class type) {
>  return new RubyClass(Ruby.this);
>  }
>  };
>
> On 3/1/18 2:25 AM, Charles Oliver Nutter wrote:
> > So I don't think we ever closed the loop here. Did anyone on the JDK
> > side confirm this, file an issue, or fix it?
> >
> > We still have ClassValue disabled in JRuby because of the rooting issues
> > described here and in https://github.com/jruby/jruby/pull/3228.
> >
> > - Charlie
> >
> > On Thu, Aug 27, 2015 at 7:04 AM Jochen Theodorou  > > wrote:
> >
> > One more thing...
> >
> > Remi, I tried your link with my simplified scenario and it does there
> > not stop the collection of the classloader
> >
> > Am 27.08.2015 11:54, schrieb Jochen Theodorou:
> >  > Hi,
> >  >
> >  > In trying to reproduce the problem outside of Groovy I stumbled
> > over a
> >  > case case which I think should work
> >  >
> >  > public class MyClassValue extends ClassValue {
> >  >  protected Object computeValue(Class type) {
> >  >  Dummy ret = new Dummy();
> >  >  Dummy.l.add (this);
> >  >  return ret;
> >  >  }
> >  > }
> >  >
> >  >   class Dummy {
> >  >   static final ArrayList l = new ArrayList();
> >  >   }
> >  >
> >  > basically this means there will be a hard reference on the
> ClassValue
> >  > somewhere. It can be in a static or non-static field, direct or
> >  > indirect. But this won't collect. If I put for example a
> > WeakReference
> >  > in between it works again.
> >  >
> >  > Finally I also tested to put the hard reference in a third class
> >  > instead, to avoid this self reference. But it can still not
> collect.
> >  >
> >  > So I currently have the impression that if anything holds a hard
> >  > reference on the class value that the classloader cannot be
> collected
> >  > anymore.
> >  >
> >  > Unless I misunderstand something here I see that as a bug
> >  >
> >  > bye blackdrag
> >  >
> >
> >
> > --
> > Jochen "blackdrag" Theodorou
> > blog: http://blackdragsview.blogspot.com/
> >
> > ___
> > mlvm-dev mailing list
> > mlvm-dev@openjdk.java.net 
> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >
> > --
> >
> > - Charlie (mobile)
> >
> >
> >
> > ___
> > mlvm-dev mailing list
> > mlvm-dev@openjdk.java.net
> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >
>
-- 

- Charlie (mobile)
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Vladimir Ivanov]

Charlie,

Does it look similar to the following bugs?
  https://bugs.openjdk.java.net/browse/JDK-8136353
  https://bugs.openjdk.java.net/browse/JDK-8169425

If that's the same (and it seems so to me [1]), then speak up and 
persuade Paul it's an important edge case (as stated in JDK-8169425).


Best regards,
Vladimir Ivanov

[1] new RubyClass(Ruby.this) in

public static class Ruby {
private ClassValue cache = new ClassValue() {
protected RubyClass computeValue(Class type) {
return new RubyClass(Ruby.this);
}
};

On 3/1/18 2:25 AM, Charles Oliver Nutter wrote:
So I don't think we ever closed the loop here. Did anyone on the JDK 
side confirm this, file an issue, or fix it?


We still have ClassValue disabled in JRuby because of the rooting issues 
described here and in https://github.com/jruby/jruby/pull/3228.


- Charlie

On Thu, Aug 27, 2015 at 7:04 AM Jochen Theodorou > wrote:


One more thing...

Remi, I tried your link with my simplified scenario and it does there
not stop the collection of the classloader

Am 27.08.2015 11:54, schrieb Jochen Theodorou:
 > Hi,
 >
 > In trying to reproduce the problem outside of Groovy I stumbled
over a
 > case case which I think should work
 >
 > public class MyClassValue extends ClassValue {
 >      protected Object computeValue(Class type) {
 >          Dummy ret = new Dummy();
 >          Dummy.l.add (this);
 >          return ret;
 >      }
 > }
 >
 >   class Dummy {
 >       static final ArrayList l = new ArrayList();
 >   }
 >
 > basically this means there will be a hard reference on the ClassValue
 > somewhere. It can be in a static or non-static field, direct or
 > indirect. But this won't collect. If I put for example a
WeakReference
 > in between it works again.
 >
 > Finally I also tested to put the hard reference in a third class
 > instead, to avoid this self reference. But it can still not collect.
 >
 > So I currently have the impression that if anything holds a hard
 > reference on the class value that the classloader cannot be collected
 > anymore.
 >
 > Unless I misunderstand something here I see that as a bug
 >
 > bye blackdrag
 >


--
Jochen "blackdrag" Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net 
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

--

- Charlie (mobile)



___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Charles Oliver Nutter]

So I don't think we ever closed the loop here. Did anyone on the JDK side
confirm this, file an issue, or fix it?

We still have ClassValue disabled in JRuby because of the rooting issues
described here and in https://github.com/jruby/jruby/pull/3228.

- Charlie

On Thu, Aug 27, 2015 at 7:04 AM Jochen Theodorou  wrote:

> One more thing...
>
> Remi, I tried your link with my simplified scenario and it does there
> not stop the collection of the classloader
>
> Am 27.08.2015 11:54, schrieb Jochen Theodorou:
> > Hi,
> >
> > In trying to reproduce the problem outside of Groovy I stumbled over a
> > case case which I think should work
> >
> > public class MyClassValue extends ClassValue {
> >  protected Object computeValue(Class type) {
> >  Dummy ret = new Dummy();
> >  Dummy.l.add (this);
> >  return ret;
> >  }
> > }
> >
> >   class Dummy {
> >   static final ArrayList l = new ArrayList();
> >   }
> >
> > basically this means there will be a hard reference on the ClassValue
> > somewhere. It can be in a static or non-static field, direct or
> > indirect. But this won't collect. If I put for example a WeakReference
> > in between it works again.
> >
> > Finally I also tested to put the hard reference in a third class
> > instead, to avoid this self reference. But it can still not collect.
> >
> > So I currently have the impression that if anything holds a hard
> > reference on the class value that the classloader cannot be collected
> > anymore.
> >
> > Unless I misunderstand something here I see that as a bug
> >
> > bye blackdrag
> >
>
>
> --
> Jochen "blackdrag" Theodorou
> blog: http://blackdragsview.blogspot.com/
>
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
-- 

- Charlie (mobile)
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Jochen Theodorou]

Hi,

In trying to reproduce the problem outside of Groovy I stumbled over a 
case case which I think should work


public class MyClassValue extends ClassValue {
protected Object computeValue(Class type) {
Dummy ret = new Dummy();
Dummy.l.add (this);
return ret;
}
}

 class Dummy {
 static final ArrayList l = new ArrayList();
 }

basically this means there will be a hard reference on the ClassValue 
somewhere. It can be in a static or non-static field, direct or 
indirect. But this won't collect. If I put for example a WeakReference 
in between it works again.


Finally I also tested to put the hard reference in a third class 
instead, to avoid this self reference. But it can still not collect.


So I currently have the impression that if anything holds a hard 
reference on the class value that the classloader cannot be collected 
anymore.


Unless I misunderstand something here I see that as a bug

bye blackdrag

--
Jochen blackdrag Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Jochen Theodorou]

One more thing...

Remi, I tried your link with my simplified scenario and it does there 
not stop the collection of the classloader


Am 27.08.2015 11:54, schrieb Jochen Theodorou:

Hi,

In trying to reproduce the problem outside of Groovy I stumbled over a
case case which I think should work

public class MyClassValue extends ClassValue {
 protected Object computeValue(Class type) {
 Dummy ret = new Dummy();
 Dummy.l.add (this);
 return ret;
 }
}

  class Dummy {
  static final ArrayList l = new ArrayList();
  }

basically this means there will be a hard reference on the ClassValue
somewhere. It can be in a static or non-static field, direct or
indirect. But this won't collect. If I put for example a WeakReference
in between it works again.

Finally I also tested to put the hard reference in a third class
instead, to avoid this self reference. But it can still not collect.

So I currently have the impression that if anything holds a hard
reference on the class value that the classloader cannot be collected
anymore.

Unless I misunderstand something here I see that as a bug

bye blackdrag




--
Jochen blackdrag Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Jochen Theodorou]

Am 27.08.2015 00:54, schrieb Remi Forax:

Hi Jochen,
what you can try is to use this code written by Jerome Pilliet as replacement 
of ClassValue (using -Xbootclasspath/p)
   
https://bitbucket.org/jpilliet/libcore-292/src/tip/libdvm/src/main/java/java/lang/ClassValue.java?at=4.4.3.292
to see if you can reproduce your bug or not with this other implementation of 
ClassValue (under Apache Licence).

Note that i hope that this implementation works but it's clearly haven't been 
battletested,
nevertheless if you can not reproduce your issue, it means that this is clearly 
a bug in the OpenJDK implementation.


I have not tried that yet. But Henry Tremblay came up with a more simple 
solution for the testing scenario. Our old ClassValue class was this:


public class GroovyClassValueJava7T extends ClassValueT implements 
GroovyClassValueT {

   private final ComputeValueT computeValue;
   private MapClass?, T cache = new HashMapClass?, T();

   public GroovyClassValueJava7(ComputeValueT computeValue){
  this.computeValue = computeValue;
   }
   @Override
   protected T computeValue(Class? type) {
  return computeValue.computeValue(type);
   }
}

and the suggestion was to move the caching out of ClassValue internals 
and into a simple map:


public class GroovyClassValueJava7T extends ClassValueT implements 
GroovyClassValueT {

   private final ComputeValueT computeValue;
   private MapClass?, T cache = new HashMapClass?, T();

   public GroovyClassValueJava7(ComputeValueT computeValue){
  this.computeValue = computeValue;
   }
   @Override
   protected T computeValue(Class? type) {
  return computeValue.computeValue(type);
   }

@Override public T get(Class? type) {
T t = cache.get(type);
if(t == null) {
t = computeValue(type);
cache.put(type, t);
}
return t;
}

@Override public void remove(Class? type) {
cache.remove(type);
}
}

Of course this is no proper replacement, since it is far from threadsafe 
and all, but it makes the test scenario work.


anyway... doesn't this also mean a OpenJDK bug?

bye blackdrag

--
Jochen blackdrag Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Remi Forax]

Hi Jochen,
what you can try is to use this code written by Jerome Pilliet as replacement 
of ClassValue (using -Xbootclasspath/p)
  
https://bitbucket.org/jpilliet/libcore-292/src/tip/libdvm/src/main/java/java/lang/ClassValue.java?at=4.4.3.292
to see if you can reproduce your bug or not with this other implementation of 
ClassValue (under Apache Licence).

Note that i hope that this implementation works but it's clearly haven't been 
battletested,
nevertheless if you can not reproduce your issue, it means that this is clearly 
a bug in the OpenJDK implementation.

regards,
Rémi

- Mail original -
De: Jochen Theodorou blackd...@gmx.org
À: Da Vinci Machine Project mlvm-dev@openjdk.java.net
Envoyé: Mercredi 26 Août 2015 22:40:53
Objet: Re: ClassValue rooting objects after it goes away?

Am 07.08.2015 13:04, schrieb Michael Haupt:
 Hi Charles,

 the pull request mentions a class named Java7ClassValue - does this mean
 it's a Java 7 issue or is it reproducible on 8 and maybe even 9? Names
 can be misleading. ;-)

 To clarify: by what means is the ClassValue removed in the JRuby
 implementation?

 I agree this should not happen.

coming back on this and after spending the whole day on debugging a 
permgen issue (and Cedric actually 3 days) that seems to be related to this.

I am not from the Ruby guess, so I will show the Groovy part. We 
discovered a permgen issue, that prevents the groovy runtime from 
unloading and the reason turned out to be ClassValue. We get a similar 
graphic Charles got, only a bit more complex. In my test scenario we 
load the Groovy runtime in an URLClassLoader, call a method to cause the 
runtime to initialize and then forget about it, just to do this again in 
a loop. What happens is that we get into a permgen problem, since the 
loader cannot be unloaded because ClassValue seems to keep classes from 
that loader to unload.

I tested this with 7_60, 7_65, 8_25, and jdk1.9.0-ea-b68. All have this 
problem.

As a special bonus we have a lot of system classes with added class 
values... for example all the primitive types. I suspected ClassValue 
earlier already and made myself a simple test but it seems the simple 
version is not going to show me what I want. So it is not as simple as 
having a classloader load a custom ClassValue class and have the 
classloader not unloading.

Guess we will try the WeakReference vesion next as well... not that is a 
solution for us.

bye blackdrag

-- 
Jochen blackdrag Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Jochen Theodorou]

Am 07.08.2015 13:04, schrieb Michael Haupt:

Hi Charles,

the pull request mentions a class named Java7ClassValue - does this mean
it's a Java 7 issue or is it reproducible on 8 and maybe even 9? Names
can be misleading. ;-)

To clarify: by what means is the ClassValue removed in the JRuby
implementation?

I agree this should not happen.


coming back on this and after spending the whole day on debugging a 
permgen issue (and Cedric actually 3 days) that seems to be related to this.


I am not from the Ruby guess, so I will show the Groovy part. We 
discovered a permgen issue, that prevents the groovy runtime from 
unloading and the reason turned out to be ClassValue. We get a similar 
graphic Charles got, only a bit more complex. In my test scenario we 
load the Groovy runtime in an URLClassLoader, call a method to cause the 
runtime to initialize and then forget about it, just to do this again in 
a loop. What happens is that we get into a permgen problem, since the 
loader cannot be unloaded because ClassValue seems to keep classes from 
that loader to unload.


I tested this with 7_60, 7_65, 8_25, and jdk1.9.0-ea-b68. All have this 
problem.


As a special bonus we have a lot of system classes with added class 
values... for example all the primitive types. I suspected ClassValue 
earlier already and made myself a simple test but it seems the simple 
version is not going to show me what I want. So it is not as simple as 
having a classloader load a custom ClassValue class and have the 
classloader not unloading.


Guess we will try the WeakReference vesion next as well... not that is a 
solution for us.


bye blackdrag

--
Jochen blackdrag Theodorou
blog: http://blackdragsview.blogspot.com/

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Michael Haupt]

Hi Charles,

the pull request mentions a class named Java7ClassValue - does this mean it's a 
Java 7 issue or is it reproducible on 8 and maybe even 9? Names can be 
misleading. ;-)

To clarify: by what means is the ClassValue removed in the JRuby implementation?

I agree this should not happen.

Best,

Michael

 Am 06.08.2015 um 20:26 schrieb Charles Oliver Nutter head...@headius.com:
 
 Pardon me if this has been discussed before, but we had a bug (with
 fix) reported today that seems to indicate that the JVM is rooting
 objects put into a ClassValue even if the ClassValue goes away.
 
 Here's the pull request: https://github.com/jruby/jruby/pull/3228
 
 And here's one example of the root trace leading back to our JRuby
 runtime. All the roots appear to be VM-level code:
 
 https://dl.dropboxusercontent.com/u/9213410/class-values-leak.png
 
 Is this expected? If we have to stuff a WeakReference into the
 ClassValue it seriously diminishes its utility to us.
 
 - Charlie

-- 

 http://www.oracle.com/
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V.  Co. KG, Schiffbauergasse 14 | 14467 Potsdam, Germany
 http://www.oracle.com/commitment Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: ClassValue rooting objects after it goes away?

[Rémi Forax]

No,
it's not expected !

Rémi 

Le 6 août 2015 20:26:43 CEST, Charles Oliver Nutter head...@headius.com a 
écrit :
Pardon me if this has been discussed before, but we had a bug (with
fix) reported today that seems to indicate that the JVM is rooting
objects put into a ClassValue even if the ClassValue goes away.

Here's the pull request: https://github.com/jruby/jruby/pull/3228

And here's one example of the root trace leading back to our JRuby
runtime. All the roots appear to be VM-level code:

https://dl.dropboxusercontent.com/u/9213410/class-values-leak.png

Is this expected? If we have to stuff a WeakReference into the
ClassValue it seriously diminishes its utility to us.

- Charlie
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

ClassValue rooting objects after it goes away?

[Charles Oliver Nutter]

Pardon me if this has been discussed before, but we had a bug (with
fix) reported today that seems to indicate that the JVM is rooting
objects put into a ClassValue even if the ClassValue goes away.

Here's the pull request: https://github.com/jruby/jruby/pull/3228

And here's one example of the root trace leading back to our JRuby
runtime. All the roots appear to be VM-level code:

https://dl.dropboxusercontent.com/u/9213410/class-values-leak.png

Is this expected? If we have to stuff a WeakReference into the
ClassValue it seriously diminishes its utility to us.

- Charlie
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev