I've managed to wire the AnonymousClassLoader into JRuby. A patch is
attached, that will probably be rather confusing to anyone unfamiliar
with JRuby.
ClassCache is basically a smart class-loading utility that can be used
to share loaded classes across JRuby instances without keeping hard
references to them and preventing them from unloading.
AnonymousClassLoader is an all-static utility class that wraps the logic
of instantiating "one-shot" classloaders for loading bodies of code. In
this case, it either uses java.dyn.AnonymousClassLoader or our own
"OneShotClassLoader" which is not anonymous but which is used for only a
single class.
The code generally appears to work about the same with
java.dyn.AnonymousClassLoader available, which is good. But I'm having
trouble quantifying the benefit to JRuby. I'd like to be able to show
how it helps, but memory profiles look practically the same between the
version using an OneShotClassLoader per method body and the version
using AnonymousClassLoader per method body. I have not done a lot of
work to isolate the cost of loading, but it seems to be unnoticeable in
my simple benchmark which defines 100_000 Ruby methods and forces them
to JIT.
So the runtime benefits may not be so great. The practical benefits,
such as being able to chuck byte[] into AnonymousClassLoader without
decorating class names and ensuring uniqueness, I have not yet utilized
in this code. Those benefits may show how writing class caches like the
one in JRuby are made a lot easier. For the moment, in order to allow
JRuby to support both MLVM work and pre-JDK7 JVMs, this patch still does
name-mangling to ensure methods are unique.
I think part of my confusion is that originally I desired a class loader
for which I could have a single instance I would throw *many* byte[] at,
and they'd all be loaded using that classloader but without hard
references and without the overhead of a classloader-per-method. But the
AnonymousClassLoader interface appears to one exactly one byte[] per
instance, though it does have considerably reduced in-memory cost per
AnonymousClassLoader instance.
Basically, what I've been looking for to make my life easier and memory
costs lower is:
ClassLoader cl = someClassLoader....;
Class first = cl.defineClass(firstClass);
Class second = cl.defineClass(secondClass);
first = null; // and at some point I would expect the class to GC
Is this the purpose of AnonymousClassLoader? Am I doing it wrong? I'll
be poking around for example code now.
- Charlie
Index: src/org/jruby/runtime/MethodFactory.java
===================================================================
--- src/org/jruby/runtime/MethodFactory.java (revision 6575)
+++ src/org/jruby/runtime/MethodFactory.java (working copy)
@@ -74,13 +74,33 @@
* @param classLoader The classloader to use for searching for and
* dynamically loading code.
* @return A new MethodFactory.
+ * @deprecated Not compatible with DVM/JDK7 AnonymousClassLoader
*/
public static MethodFactory createFactory(ClassLoader classLoader) {
if (reflection) return new ReflectionMethodFactory();
- if (dumping) return new DumpingInvocationMethodFactory(dumpingPath,
classLoader);
return new InvocationMethodFactory(classLoader);
}
+
+ /**
+ * Based on optional properties, create a new MethodFactory. By default,
+ * this will create a code-generation-based InvocationMethodFactory. If
+ * security restricts code generation, ReflectionMethodFactory will be
used.
+ * If we are dumping class definitions, DumpingInvocationMethodFactory will
+ * be used. See MethodFactory's static initializer for more details.
+ *
+ * This version
+ *
+ * @param hostClass The class to use as "host" for the new invoker, or the
+ * class from which to get a classloader to use for searching for and
+ * dynamically loading code.
+ * @return A new MethodFactory.
+ */
+ public static MethodFactory createFactory(Class hostClass) {
+ if (reflection) return new ReflectionMethodFactory();
+
+ return new InvocationMethodFactory(hostClass);
+ }
/**
* Get a new method handle based on the target JRuby-compiled method.
Index: src/org/jruby/internal/runtime/methods/InvocationMethodFactory.java
===================================================================
--- src/org/jruby/internal/runtime/methods/InvocationMethodFactory.java
(revision 6575)
+++ src/org/jruby/internal/runtime/methods/InvocationMethodFactory.java
(working copy)
@@ -54,6 +54,7 @@
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.Visibility;
import org.jruby.runtime.builtin.IRubyObject;
+import org.jruby.util.AnonymousClassLoader;
import org.jruby.util.CodegenUtils;
import static org.jruby.util.CodegenUtils.*;
import static java.lang.System.*;
@@ -152,8 +153,8 @@
/** The lvar index of the passed-in Block on the call */
public static final int BLOCK_INDEX = 6;
- /** The classloader to use for code loading */
- protected JRubyClassLoader classLoader;
+ /** The class to use as host for anonymous classloading */
+ protected Class hostClass;
/**
* Whether this factory has seen undefined methods already. This is used to
@@ -169,14 +170,26 @@
*
* @param classLoader The classloader to use, or to wrap if it is not a
* JRubyClassLoader instance.
+ * @deprecated Do not use this version anymore, since it is not compatible
+ * with DVM/JDK7 anonymous classloading.
*/
public InvocationMethodFactory(ClassLoader classLoader) {
- if (classLoader instanceof JRubyClassLoader) {
- this.classLoader = (JRubyClassLoader)classLoader;
- } else {
- this.classLoader = new JRubyClassLoader(classLoader);
- }
+ hostClass = InvocationMethodFactory.class;
}
+
+ /**
+ * Construct a new InvocationMethodFactory using the class to host loaded
+ * code. Created classes are either constructed using the DVM/JDK7
+ * AnonymousClassLoader or using a "one shot" classloader for all
+ * construction.
+ *
+ * @param classLoader The classloader to use, or to wrap if it is not a
+ * JRubyClassLoader instance.
+ * @see AnonymousClassLoader
+ */
+ public InvocationMethodFactory(Class hostClass) {
+ this.hostClass = hostClass;
+ }
/**
* Use code generation to provide a method handle for a compiled Ruby
method.
@@ -189,7 +202,7 @@
String sup = COMPILED_SUPER_CLASS;
Class scriptClass = scriptObject.getClass();
String mname = scriptClass.getName() + "Invoker" + method + arity;
- synchronized (classLoader) {
+ synchronized (hostClass) {
Class generatedClass = tryClass(mname);
try {
@@ -291,10 +304,7 @@
if (DEBUG) out.println("Binding multiple: " + desc1.declaringClassName
+ "." + javaMethodName);
- String generatedClassName =
CodegenUtils.getAnnotatedBindingClassName(javaMethodName,
desc1.declaringClassName, desc1.isStatic, desc1.actualRequired, desc1.optional,
true);
- String generatedClassPath = generatedClassName.replace('.', '/');
-
- synchronized (classLoader) {
+ synchronized (hostClass) {
try {
Class c = getAnnotatedMethodClass(descs);
@@ -366,7 +376,7 @@
String generatedClassName =
CodegenUtils.getAnnotatedBindingClassName(javaMethodName,
desc1.declaringClassName, desc1.isStatic, desc1.actualRequired, desc1.optional,
true);
String generatedClassPath = generatedClassName.replace('.', '/');
- synchronized (classLoader) {
+ synchronized (hostClass) {
Class c = tryClass(generatedClassName);
int min = Integer.MAX_VALUE;
@@ -485,10 +495,7 @@
public DynamicMethod getAnnotatedMethod(RubyModule implementationClass,
JavaMethodDescriptor desc) {
String javaMethodName = desc.name;
- String generatedClassName =
CodegenUtils.getAnnotatedBindingClassName(javaMethodName,
desc.declaringClassName, desc.isStatic, desc.actualRequired, desc.optional,
false);
- String generatedClassPath = generatedClassName.replace('.', '/');
-
- synchronized (classLoader) {
+ synchronized (hostClass) {
try {
Class c = getAnnotatedMethodClass(desc);
@@ -518,7 +525,7 @@
String generatedClassName =
CodegenUtils.getAnnotatedBindingClassName(javaMethodName,
desc.declaringClassName, desc.isStatic, desc.actualRequired, desc.optional,
false);
String generatedClassPath = generatedClassName.replace('.', '/');
- synchronized (classLoader) {
+ synchronized (hostClass) {
Class c = tryClass(generatedClassName);
if (c == null) {
@@ -586,7 +593,7 @@
String generatedClassName = type.getName() + "Invoker";
String generatedClassPath = typePath + "Invoker";
- synchronized (classLoader) {
+ synchronized (hostClass) {
Class c = tryClass(generatedClassName);
try {
@@ -990,10 +997,10 @@
private Class tryClass(String name) {
try {
Class c = null;
- if (classLoader == null) {
- c = Class.forName(name, true, classLoader);
+ if (hostClass == null) {
+ c = Class.forName(name, true, null);
} else {
- c = classLoader.loadClass(name);
+ c = hostClass.getClassLoader().loadClass(name);
}
if (c != null && seenUndefinedClasses) {
@@ -1008,7 +1015,7 @@
}
}
- protected Class endCall(ClassWriter cw, MethodVisitor mv, String name) {
+ protected Class endCall(ClassWriter cw, MethodVisitor mv, String name)
throws ClassNotFoundException {
endMethod(mv);
return endClass(cw, name);
}
@@ -1018,12 +1025,12 @@
mv.visitEnd();
}
- protected Class endClass(ClassWriter cw, String name) {
+ protected Class endClass(ClassWriter cw, String name) throws
ClassNotFoundException {
cw.visitEnd();
byte[] code = cw.toByteArray();
CheckClassAdapter.verify(new ClassReader(code), false, new
PrintWriter(System.err));
- return classLoader.defineClass(name, code);
+ return AnonymousClassLoader.loadClass(name, code, hostClass,
JRubyClassLoader.class.getProtectionDomain());
}
private void loadArgument(MethodVisitor mv, int argsIndex, int argIndex) {
Index:
src/org/jruby/internal/runtime/methods/DumpingInvocationMethodFactory.java
===================================================================
--- src/org/jruby/internal/runtime/methods/DumpingInvocationMethodFactory.java
(revision 6575)
+++ src/org/jruby/internal/runtime/methods/DumpingInvocationMethodFactory.java
(working copy)
@@ -30,7 +30,6 @@
import java.io.File;
import java.io.FileOutputStream;
-import org.jruby.Ruby;
import org.objectweb.asm.ClassWriter;
/**
@@ -44,10 +43,16 @@
private String dumpPath;
+ @Deprecated
public DumpingInvocationMethodFactory(String path, ClassLoader
classLoader) {
super(classLoader);
this.dumpPath = path;
}
+
+ public DumpingInvocationMethodFactory(String path, Class hostClass) {
+ super(hostClass);
+ this.dumpPath = path;
+ }
@Override
protected Class endClass(ClassWriter cw, String name) {
@@ -62,6 +67,6 @@
fos.close();
} catch(Exception e) {
}
- return classLoader.defineClass(name, code);
+ return null; //classLoader.defineClass(name, code);
}
}// DumpingInvocationMethodFactory
Index: src/org/jruby/javasupport/util/RuntimeHelpers.java
===================================================================
--- src/org/jruby/javasupport/util/RuntimeHelpers.java (revision 6575)
+++ src/org/jruby/javasupport/util/RuntimeHelpers.java (working copy)
@@ -134,7 +134,7 @@
scope.determineModule();
scope.setArities(required, optional, rest);
- MethodFactory factory =
MethodFactory.createFactory(compiledClass.getClassLoader());
+ MethodFactory factory = MethodFactory.createFactory(compiledClass);
DynamicMethod method;
if (name.equals("initialize") || visibility ==
Visibility.MODULE_FUNCTION) {
@@ -191,7 +191,7 @@
scope.determineModule();
scope.setArities(required, optional, rest);
- MethodFactory factory =
MethodFactory.createFactory(compiledClass.getClassLoader());
+ MethodFactory factory = MethodFactory.createFactory(compiledClass);
DynamicMethod method;
method = factory.getCompiledMethod(rubyClass, javaName,
Index: src/org/jruby/RubyModule.java
===================================================================
--- src/org/jruby/RubyModule.java (revision 6575)
+++ src/org/jruby/RubyModule.java (working copy)
@@ -462,7 +462,7 @@
// FIXME: This is probably not very efficient, since it loads all
methods for each call
boolean foundMethod = false;
for (Method method : clazz.getDeclaredMethods()) {
- if (method.getName().equals(name) && defineAnnotatedMethod(method,
MethodFactory.createFactory(getRuntime().getJRubyClassLoader()))) {
+ if (method.getName().equals(name) && defineAnnotatedMethod(method,
MethodFactory.createFactory(Ruby.class))) {
foundMethod = true;
}
}
@@ -605,7 +605,7 @@
} catch (Throwable t) {
if (DEBUG) System.out.println("Could not find it!");
// fallback on non-pregenerated logic
- MethodFactory methodFactory =
MethodFactory.createFactory(getRuntime().getJRubyClassLoader());
+ MethodFactory methodFactory =
MethodFactory.createFactory(Ruby.class);
MethodClumper clumper = new MethodClumper();
clumper.clump(clazz);
@@ -638,7 +638,7 @@
}
private void defineAnnotatedMethodsIndexed(Class clazz) {
- MethodFactory methodFactory =
MethodFactory.createFactory(getRuntime().getJRubyClassLoader());
+ MethodFactory methodFactory = MethodFactory.createFactory(Ruby.class);
methodFactory.defineIndexedAnnotatedMethods(this, clazz,
methodDefiningCallback);
}
Index: src/org/jruby/util/ClassCache.java
===================================================================
--- src/org/jruby/util/ClassCache.java (revision 6575)
+++ src/org/jruby/util/ClassCache.java (working copy)
@@ -2,8 +2,6 @@
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
-import java.net.URL;
-import java.net.URLClassLoader;
import java.security.ProtectionDomain;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@@ -13,6 +11,13 @@
* multiple runtimes (or whole JVM).
*/
public class ClassCache<T> {
+ private final ReferenceQueue referenceQueue = new ReferenceQueue();
+ private final Map<Object, KeyedClassReference> cache =
+ new ConcurrentHashMap<Object, KeyedClassReference>();
+ private final ClassLoader classLoader;
+ private final int max;
+ private final ProtectionDomain defaultDomain;
+
/**
* The ClassLoader this class cache will use for any classes generated
through it. It is
* assumed that the classloader provided will be a parent loader of any
runtime using it.
@@ -21,6 +26,7 @@
public ClassCache(ClassLoader classLoader, int max) {
this.classLoader = classLoader;
this.max = max;
+ this.defaultDomain = classLoader.getClass().getProtectionDomain();
}
public ClassCache(ClassLoader classLoader) {
@@ -46,34 +52,6 @@
}
}
- private static class OneShotClassLoader extends URLClassLoader {
- private final static ProtectionDomain DEFAULT_DOMAIN =
- JRubyClassLoader.class.getProtectionDomain();
-
- public OneShotClassLoader(ClassLoader parent) {
- super(new URL[0], parent);
- }
-
- // Change visibility so others can see it
- public void addURL(URL url) {
- super.addURL(url);
- }
-
- public Class<?> defineClass(String name, byte[] bytes) {
- return super.defineClass(name, bytes, 0, bytes.length,
DEFAULT_DOMAIN);
- }
-
- public Class<?> defineClass(String name, byte[] bytes,
ProtectionDomain domain) {
- return super.defineClass(name, bytes, 0, bytes.length, domain);
- }
- }
-
- private ReferenceQueue referenceQueue = new ReferenceQueue();
- private Map<Object, KeyedClassReference> cache =
- new ConcurrentHashMap<Object, KeyedClassReference>();
- private ClassLoader classLoader;
- private int max;
-
public ClassLoader getClassLoader() {
return classLoader;
}
@@ -91,8 +69,7 @@
if (weakRef == null || contents == null) {
if (isFull()) return null;
- OneShotClassLoader oneShotCL = new
OneShotClassLoader(getClassLoader());
- contents = (Class<T>)oneShotCL.defineClass(classGenerator.name(),
classGenerator.bytecode());
+ contents =
(Class<T>)AnonymousClassLoader.loadClass(classGenerator.name(),
classGenerator.bytecode(), ClassCache.class, defaultDomain);
cache.put(key, new KeyedClassReference(key, contents,
referenceQueue));
}
Index: src/org/jruby/util/AnonymousClassLoader.java
===================================================================
--- src/org/jruby/util/AnonymousClassLoader.java (revision 0)
+++ src/org/jruby/util/AnonymousClassLoader.java (revision 0)
@@ -0,0 +1,84 @@
+package org.jruby.util;
+
+import java.lang.reflect.Constructor;
+import java.lang.reflect.Method;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.security.ProtectionDomain;
+
+public class AnonymousClassLoader {
+ private static final boolean DEBUG = true;
+
+ private static final Constructor ANON_CL_CONSTRUCTOR;
+ private static final Method ANON_CL_LOADCLASS;
+ private static final boolean ANONYMOUS_SUPPORTED;
+
+ static {
+ Exception exception = null;
+
+ Class cls = null;
+ Constructor constructor = null;
+ Method method = null;
+
+ try {
+ cls = Class.forName("java.dyn.AnonymousClassLoader");
+ constructor = cls.getConstructor(Class.class, byte[].class);
+ method = cls.getMethod("loadClass");
+ } catch (Exception e) {
+ exception = e;
+ }
+
+ if (method == null && DEBUG) {
+ System.err.println("Could not load AnonymousClassLoader:");
+ exception.printStackTrace();
+ ANON_CL_CONSTRUCTOR = null;
+ ANON_CL_LOADCLASS = null;
+ ANONYMOUS_SUPPORTED = false;
+ } else {
+ ANON_CL_CONSTRUCTOR = constructor;
+ ANON_CL_LOADCLASS = method;
+ ANONYMOUS_SUPPORTED = false;
+ }
+ }
+
+ public static Class loadClass(String name, byte[] bytecode, Class
hostClass, ProtectionDomain domain) throws ClassNotFoundException {
+ Class result;
+
+ if (ANONYMOUS_SUPPORTED) {
+ try {
+ Object anonCL = ANON_CL_CONSTRUCTOR.newInstance(hostClass,
bytecode);
+ result = (Class)ANON_CL_LOADCLASS.invoke(anonCL);
+ } catch (Exception e) {
+ throw new ClassNotFoundException("Could not instantiate
AnonymousClassLoader", e);
+ }
+ } else {
+ OneShotClassLoader oneShotCL = new
OneShotClassLoader(hostClass.getClassLoader(), domain);
+ result = (Class)oneShotCL.defineClass(name, bytecode);
+ }
+
+ return result;
+ }
+
+ private static class OneShotClassLoader extends URLClassLoader {
+ private final ProtectionDomain defaultDomain;
+
+ public OneShotClassLoader(ClassLoader parent, ProtectionDomain domain)
{
+ super(new URL[0], parent);
+ defaultDomain = domain;
+ }
+
+ // Change visibility so others can see it
+ public void addURL(URL url) {
+ super.addURL(url);
+ }
+
+ public Class<?> defineClass(String name, byte[] bytes) {
+ return super.defineClass(name, bytes, 0, bytes.length,
defaultDomain);
+ }
+
+ public Class<?> defineClass(String name, byte[] bytes,
ProtectionDomain domain) {
+ return super.defineClass(name, bytes, 0, bytes.length, domain);
+ }
+ }
+}
+
Index: src/org/jruby/anno/InvokerGenerator.java
===================================================================
--- src/org/jruby/anno/InvokerGenerator.java (revision 6575)
+++ src/org/jruby/anno/InvokerGenerator.java (working copy)
@@ -7,7 +7,6 @@
import java.util.Map;
import org.jruby.RubyModule.MethodClumper;
import org.jruby.internal.runtime.methods.DumpingInvocationMethodFactory;
-import org.jruby.util.JRubyClassLoader;
public class InvokerGenerator {
public static final boolean DEBUG = false;
@@ -26,7 +25,7 @@
br.close();
}
- DumpingInvocationMethodFactory dumper = new
DumpingInvocationMethodFactory(args[1], new
JRubyClassLoader(ClassLoader.getSystemClassLoader()));
+ DumpingInvocationMethodFactory dumper = new
DumpingInvocationMethodFactory(args[1], InvokerGenerator.class);
for (String name : classNames) {
MethodClumper clumper = new MethodClumper();
_______________________________________________
mlvm-dev mailing list
[email protected]
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
