Meanwhile I don't think that it is a bug, because the getStubWrapper()
method in TaskConfig uses the classloader. userCodeClass already
contains the loaded class, so no need for the classloader:
@Override
public T getUserCodeObject(Class<? super T> superClass, ClassLoader
cl) {
return InstantiationUtil.instantiate(userCodeClass, superClass);
}
Anyways, I'm still having the problem, that my InputFormat does not find
the user class HCatInputFormat. How can I access the user code class
loader from within an input format? Or do I have to serialize my class
within the InputFormat in my custom writeObject() method? Here is my
exception, maybe someone could give me a hint:
eu.stratosphere.client.program.ProgramInvocationException: The program
execution failed:
eu.stratosphere.nephele.executiongraph.GraphConversionException:
java.lang.RuntimeException: Unable to instantiate the hadoop input format
at
eu.stratosphere.hadoopcompatibility.mapreduce.HadoopInputFormat.readObject(HadoopInputFormat.java:315)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1001)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1892)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1797)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1349)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1914)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1797)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1349)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at
eu.stratosphere.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:239)
at
eu.stratosphere.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:227)
at
eu.stratosphere.pact.runtime.task.util.TaskConfig.getStubWrapper(TaskConfig.java:262)
at
eu.stratosphere.pact.runtime.task.RegularPactTask.instantiateUserCode(RegularPactTask.java:1487)
at
eu.stratosphere.pact.runtime.task.DataSourceTask.initInputFormat(DataSourceTask.java:313)
at
eu.stratosphere.pact.runtime.task.DataSourceTask.registerInputOutput(DataSourceTask.java:100)
at
eu.stratosphere.nephele.execution.RuntimeEnvironment.<init>(RuntimeEnvironment.java:192)
at
eu.stratosphere.nephele.executiongraph.ExecutionGroupVertex.<init>(ExecutionGroupVertex.java:229)
at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.createVertex(ExecutionGraph.java:493)
at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.constructExecutionGraph(ExecutionGraph.java:275)
at
eu.stratosphere.nephele.executiongraph.ExecutionGraph.<init>(ExecutionGraph.java:174)
at
eu.stratosphere.nephele.jobmanager.JobManager.submitJob(JobManager.java:499)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at eu.stratosphere.nephele.ipc.RPC$Server.call(RPC.java:417)
at eu.stratosphere.nephele.ipc.Server$Handler.run(Server.java:951)
Caused by: java.lang.ClassNotFoundException:
org.apache.hive.hcatalog.mapreduce.HCatInputFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at
eu.stratosphere.hadoopcompatibility.mapreduce.HadoopInputFormat.readObject(HadoopInputFormat.java:313)
... 31 more
On 17.06.2014 16:42, Stephan Ewen wrote:
Yep, this looks like a bug, I agree.
