[Everything I say below is about Sun's JDK/JRE implementation, although I
believe that IBM's inherits the same serialization code.]
Since the fix for 5056445/6232010, in JDK 6 and 5.0u7, the static class descriptor cache in java.io.ObjectStreamClass does not maintain strong references to serializable classes that have been used. A potential confusion, though, is that it does maintain "soft" reachability to them, which is stronger than "weak" (see the java.lang.ref package doc), and which means that such classes (and thus their defining class loaders) might only be garbage collected if there is sufficient pressure on the JVM heap that their storage is needed to satisfy other allocations. In other words, the collector merely noticing that the classes are not strongly reachable would not not sufficient for them to be collected.
See below an edit of that SerializationTest.java, updated to use a reference
queue to monitor the garbage collection of the class in a non-IBM-specific way.
It should pass, with JDK 6 and 5.0u7, but note the invocation of flushSoftRefs
necessary to force sufficient heap pressure to cause soft references to be
cleared-- System.gc alone is not sufficient.
Some background: The ObjectStreamClass implementation faces a dilemma about
whether to maintain soft or weak references to the cached class descriptors
(which in turn reference their corresponding classes). It could use weak
references instead, but then cache entries would likely be cleared very soon
after a given serialization activity has completed, going against the purpose
of caching these expensive-to-compute descriptors. What it would really like
to do is make its referencing of a class descriptor conditional on the
reachability of the corresponding class itself, but that is not doable with the
current java.lang.ref API. Doing that sort of thing is the subject of this
RFE (which is probably my personal favorite open Java libraries RFE):
http://bugs.sun.com/view_bug.do?bug_id=4630118
Even before the 5056445/6232010 fix, though, the situation wasn't as bad as
suggested by the article you quoted. The serializable classes were indeed
strongly referenced by the static cache, as keys, but the corresponding class
descriptor values were (again) softly referenced, and the cache would clear out
entries whose soft references had been cleared. The problem was that this
clearing did not happen in the background, rather it required some cache usage
to occur, so garbage collection of unused serializable classes required ongoing
serialization activity. Also, this clearing was not in a closed loop with the
garbage collector, thus risking OutOfMemoryError in tight situations.
-- Peter
On Apr 10, 2010, at 9:24 PM, Peter Firmstone wrote:
I have stumbled across a troubling problem with Serialization relating to
Garbage Collection of Classes and ClassLoaders and was hoping someone might be
able to shed some light on the issue.
Is it really true that the more objects you distribute, the greater your memory
consumption because Class files and ClassLoaders cannot be garbage collected?
Regards,
Peter.
The issue can be found here:
http://www.ibm.com/developerworks/java/library/j-dclp3/index.html
And here's the relevant information, pasted from the link:
Problems related to garbage collection and serialization
The garbage collector interacts closely with the class loader. Among other
things, the collector examines the class loader data structures to determine
which classes are /live/ -- that is, are not garbage collectable. This can
often lead to some unexpected problems.
Figure 2 illustrates a situation where serialization affects the garbage
collection (GC) of classes and a class loader in an unexpected way:
*Figure 2. Serialization example*
Serialization example
In this example, |SerializationTest| instantiates a |URLClassLoader|, called
|loader|. After loading |SerializationClass|, the class loader is dereferenced.
The expectation is that this will allow the classes loaded by it to be garbage
collected. The code for these classes is illustrated in Listings 9 and 10:
*Listing 9. SerializationTest.java*
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
public class SerializationTest extends ClassLoader {
public static void main(String args[]) {
try {
URLClassLoader loader = new URLClassLoader(new URL[] { new URL(
"file://C:/CL_Article/Serialization/dir1/") });
System.out.println("Loading SerializationClass");
Class c = loader.loadClass("SerializationClass");
System.out.println("Creating an instance of SerializationClass");
c.newInstance();
System.out.println("Dereferencing the class loader");
c = null;
loader = null;
System.out.println("Running GC...");
System.gc();
System.out.println("Triggering a Javadump");
com.ibm.jvm.Dump.JavaDump();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
}
*Listing 10. SerializationClass.java*
import java.io.File;
import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
public class SerializationClass implements Serializable {
private static final long serialVersionUID = 5024741671582526226L;
public SerializationClass() {
try {
File file = new File("C:/CL_Article/Serialization/test.txt");
FileOutputStream fos = new FileOutputStream(file);
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(this);
oos.reset();
oos.close();
fos.close();
oos = null;
fos = null;
file = null;
} catch (Exception e) {
e.printStackTrace();
}
}
}
Using a Javadump, it is possible to discover whether the class loader has been
garbage collected. (See the first article in this series for more on using
Javadump.) If the following section appears in the list of class loaders, then
it has not been collected:
------a- Loader java/net/URLClassLoader(0x44DC6DE0), Shadow 0x00ADB6D8,
Parent sun/misc/Launcher$AppClassLoader(0x00ADB7B0) Number of
loaded classes 1 Number of cached classes 11 Allocation used
for loaded classes 1 Package owner 0x00ADB6D8
Though dereferencing a user-defined class loader seems like a way to ensure
that the classes are garbage collected, this is not actually the case. In the
previous example, the problem stems from the use of
|java.io.ObjectOutputStream.writeObject(Object obj)| and its implications on GC.
When |writeObject()| is invoked (to serialize |SerializationClass|), a
reference to this class object is passed internally to |ObjectStreamClass| and
stored in a lookup table (that is, in an internal cache). This reference is
kept to speed up future serialization of the same class.
When the class loader is dereferenced, the classes that it loaded are not
garbage collectable. This is because there is a live reference to the
|SerializationClass| class from the |ObjectStreamClass| lookup table.
|ObjectStreamClass| is a primordial class and therefore is never garbage
collected. The lookup table is referenced from a static field in
|ObjectStreamClass| and is kept in the class itself rather than in an instance
of it. As a result, the reference to |SerializationClass| exists for the
lifetime of the JVM, and the class thus cannot be garbage collected.
Importantly, the |SerializationClass| class has a reference to its defining
class loader, and so it cannot be completely dereferenced either.
To avoid this problem, any classes that are to be serialized should be loaded
by a class loader that does not need to be garbage collected -- by the system
class loader, for example.
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.List;
public class SerializationTest extends ClassLoader {
public static void main(String args[]) throws Exception {
URLClassLoader loader = new URLClassLoader(new URL[] {
new URL("file:dir1/")
});
System.out.println("Loading SerializationClass");
Class c = loader.loadClass("SerializationClass");
System.out.println("Creating an instance of SerializationClass");
c.newInstance();
ReferenceQueue<Class<?>> queue = new ReferenceQueue<Class<?>>();
Reference<Class<?>> ref = new WeakReference<Class<?>>(c, queue);
System.out.println("Dereferencing the class loader");
c = null;
loader = null;
System.out.println("Running GC...");
System.gc();
//System.out.println("Triggering a Javadump");
//com.ibm.jvm.Dump.JavaDump();
flushSoftRefs();
Reference<? extends Class<?>> dequeued = queue.remove(1000);
if (dequeued == ref) {
System.out.println("SerializationClass garbage collected");
} else {
throw new Error("SerializationClass not garbage collected");
}
}
private static void flushSoftRefs() {
try {
List<Object> l = new ArrayList<Object>();
while (true) l.add(new byte[10000]);
} catch (OutOfMemoryError e) {
System.out.println("memory exhausted");
}
}
}
