[Everything I say below is about Sun's JDK/JRE implementation, although I
believe that IBM's inherits the same serialization code.]
Since the fix for 5056445/6232010, in JDK 6 and 5.0u7, the static class
descriptor cache in java.io.ObjectStreamClass does not maintain strong
references to serializable classes that have been used. A potential confusion,
though, is that it does maintain "soft" reachability to them, which is stronger
than "weak" (see the java.lang.ref package doc), and which means that such
classes (and thus their defining class loaders) might only be garbage collected
if there is sufficient pressure on the JVM heap that their storage is needed to
satisfy other allocations. In other words, the collector merely noticing that
the classes are not strongly reachable would not not sufficient for them to be
collected.
See below an edit of that SerializationTest.java, updated to use a reference
queue to monitor the garbage collection of the class in a non-IBM-specific way.
It should pass, with JDK 6 and 5.0u7, but note the invocation of flushSoftRefs
necessary to force sufficient heap pressure to cause soft references to be
cleared-- System.gc alone is not sufficient.
Some background: The ObjectStreamClass implementation faces a dilemma about
whether to maintain soft or weak references to the cached class descriptors
(which in turn reference their corresponding classes). It could use weak
references instead, but then cache entries would likely be cleared very soon
after a given serialization activity has completed, going against the purpose
of caching these expensive-to-compute descriptors. What it would really like
to do is make its referencing of a class descriptor conditional on the
reachability of the corresponding class itself, but that is not doable with the
current java.lang.ref API. Doing that sort of thing is the subject of this
RFE (which is probably my personal favorite open Java libraries RFE):
http://bugs.sun.com/view_bug.do?bug_id=4630118
Even before the 5056445/6232010 fix, though, the situation wasn't as bad as
suggested by the article you quoted. The serializable classes were indeed
strongly referenced by the static cache, as keys, but the corresponding class
descriptor values were (again) softly referenced, and the cache would clear out
entries whose soft references had been cleared. The problem was that this
clearing did not happen in the background, rather it required some cache usage
to occur, so garbage collection of unused serializable classes required ongoing
serialization activity. Also, this clearing was not in a closed loop with the
garbage collector, thus risking OutOfMemoryError in tight situations.
-- Peter
On Apr 10, 2010, at 9:24 PM, Peter Firmstone wrote:
> I have stumbled across a troubling problem with Serialization relating to
> Garbage Collection of Classes and ClassLoaders and was hoping someone might
> be able to shed some light on the issue.
>
> Is it really true that the more objects you distribute, the greater your
> memory consumption because Class files and ClassLoaders cannot be garbage
> collected?
>
> Regards,
>
> Peter.
>
> The issue can be found here:
> http://www.ibm.com/developerworks/java/library/j-dclp3/index.html
>
> And here's the relevant information, pasted from the link:
>
>
> Problems related to garbage collection and serialization
>
> The garbage collector interacts closely with the class loader. Among other
> things, the collector examines the class loader data structures to determine
> which classes are /live/ -- that is, are not garbage collectable. This can
> often lead to some unexpected problems.
>
> Figure 2 illustrates a situation where serialization affects the garbage
> collection (GC) of classes and a class loader in an unexpected way:
>
>
> *Figure 2. Serialization example*
>
> Serialization example
>
> In this example, |SerializationTest| instantiates a |URLClassLoader|, called
> |loader|. After loading |SerializationClass|, the class loader is
> dereferenced. The expectation is that this will allow the classes loaded by
> it to be garbage collected. The code for these classes is illustrated in
> Listings 9 and 10:
>
>
> *Listing 9. SerializationTest.java*
>
> import java.net.MalformedURLException;
> import java.net.URL;
> import java.net.URLClassLoader;
>
> public class SerializationTest extends ClassLoader {
>
> public static void main(String args[]) {
> try {
> URLClassLoader loader = new URLClassLoader(new URL[] { new URL(
> "file://C:/CL_Article/Serialization/dir1/") });
> System.out.println("Loading SerializationClass");
> Class c = loader.loadClass("SerializationClass");
> System.out.println("Creating an instance of SerializationClass");
> c.newInstance();
> System.out.println("Dereferencing the class loader");
> c = null;
> loader = null;
> System.out.println("Running GC...");
> System.gc();
> System.out.println("Triggering a Javadump");
> com.ibm.jvm.Dump.JavaDump();
> } catch (MalformedURLException e) {
> e.printStackTrace();
> } catch (InstantiationException e) {
> e.printStackTrace();
> } catch (IllegalAccessException e) {
> e.printStackTrace();
> } catch (ClassNotFoundException e) {
> e.printStackTrace();
> }
> }
> }
>
>
>
> *Listing 10. SerializationClass.java*
>
> import java.io.File;
> import java.io.FileOutputStream;
> import java.io.ObjectOutputStream;
> import java.io.Serializable;
>
> public class SerializationClass implements Serializable {
>
> private static final long serialVersionUID = 5024741671582526226L;
>
> public SerializationClass() {
> try {
> File file = new File("C:/CL_Article/Serialization/test.txt");
> FileOutputStream fos = new FileOutputStream(file);
> ObjectOutputStream oos = new ObjectOutputStream(fos);
> oos.writeObject(this);
> oos.reset();
> oos.close();
> fos.close();
> oos = null;
> fos = null;
> file = null;
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>
>
> Using a Javadump, it is possible to discover whether the class loader has
> been garbage collected. (See the first article in this series for more on
> using Javadump.) If the following section appears in the list of class
> loaders, then it has not been collected:
>
> ------a- Loader java/net/URLClassLoader(0x44DC6DE0), Shadow 0x00ADB6D8,
> Parent sun/misc/Launcher$AppClassLoader(0x00ADB7B0) Number of
> loaded classes 1 Number of cached classes 11 Allocation
> used for loaded classes 1 Package owner 0x00ADB6D8
>
>
> Though dereferencing a user-defined class loader seems like a way to ensure
> that the classes are garbage collected, this is not actually the case. In the
> previous example, the problem stems from the use of
> |java.io.ObjectOutputStream.writeObject(Object obj)| and its implications on
> GC.
>
> When |writeObject()| is invoked (to serialize |SerializationClass|), a
> reference to this class object is passed internally to |ObjectStreamClass|
> and stored in a lookup table (that is, in an internal cache). This reference
> is kept to speed up future serialization of the same class.
>
> When the class loader is dereferenced, the classes that it loaded are not
> garbage collectable. This is because there is a live reference to the
> |SerializationClass| class from the |ObjectStreamClass| lookup table.
> |ObjectStreamClass| is a primordial class and therefore is never garbage
> collected. The lookup table is referenced from a static field in
> |ObjectStreamClass| and is kept in the class itself rather than in an
> instance of it. As a result, the reference to |SerializationClass| exists for
> the lifetime of the JVM, and the class thus cannot be garbage collected.
> Importantly, the |SerializationClass| class has a reference to its defining
> class loader, and so it cannot be completely dereferenced either.
>
> To avoid this problem, any classes that are to be serialized should be loaded
> by a class loader that does not need to be garbage collected -- by the system
> class loader, for example.
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.List;
public class SerializationTest extends ClassLoader {
public static void main(String args[]) throws Exception {
URLClassLoader loader = new URLClassLoader(new URL[] {
new URL("file:dir1/")
});
System.out.println("Loading SerializationClass");
Class c = loader.loadClass("SerializationClass");
System.out.println("Creating an instance of SerializationClass");
c.newInstance();
ReferenceQueue<Class<?>> queue = new ReferenceQueue<Class<?>>();
Reference<Class<?>> ref = new WeakReference<Class<?>>(c, queue);
System.out.println("Dereferencing the class loader");
c = null;
loader = null;
System.out.println("Running GC...");
System.gc();
//System.out.println("Triggering a Javadump");
//com.ibm.jvm.Dump.JavaDump();
flushSoftRefs();
Reference<? extends Class<?>> dequeued = queue.remove(1000);
if (dequeued == ref) {
System.out.println("SerializationClass garbage collected");
} else {
throw new Error("SerializationClass not garbage collected");
}
}
private static void flushSoftRefs() {
try {
List<Object> l = new ArrayList<Object>();
while (true) l.add(new byte[10000]);
} catch (OutOfMemoryError e) {
System.out.println("memory exhausted");
}
}
}
