Hi,
Thank you for the suggestion. Unfortunately, when I tried that I received the
same error. I've also tried copying the gson jar directly into $HADOOP_HOME/lib
(when I was running a single node pseudo-distributed) and get the same error
still.
Weirdly enough, if I try and run the Dirichlet example on the cluster I receive
another ClassNotFoundException:
09/07/16 10:27:54 INFO mapred.JobClient: Task Id :
attempt_200907161026_0002_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.mahout.clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
at
org.apache.mahout.clustering.dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:95)
at
org.apache.mahout.clustering.dirichlet.DirichletMapper.configure(DirichletMapper.java:60)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.apache.mahout.clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:288)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at
org.apache.mahout.clustering.dirichlet.DirichletDriver.createState(DirichletDriver.java:121)
at
org.apache.mahout.clustering.dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:71)
... 19 more
Hoping this sparks some other suggestions :)
Thanks,
Paul
On Wed Jul 15 22:08:09 UTC 2009, Adil Aijaz <[email protected]> wrote:
> try hadoop --config <hod-cluster-dir> jar -libjars <path to gson.jar>
> <your job/jar file> <your class> <arguments>
>
> Adil
>
> Paul Ingles wrote:
> > Hi,
> >
> > Apologies for the cross-posting (I also sent this to the Hadoop user
> > list) but I'm still getting errors if I try and run the KMeans
> > examples on a cluster, whether that be my single-node Mac Pro, or our
> > cluster. I've attached the stack trace at the bottom of the email.
> >
> > The gson jar is definitely included in the packaged .job, and is also
> > in the temporary directory when the task tracker picks up the work.
> > The gson jar also includes TypeToken.class in the expected path.
> >
> > Again, really appreciate people's help in getting this going!
> >
> > ----snip----
> > 09/07/15 17:06:38 INFO mapred.JobClient: Task Id :
> > attempt_200907151617_0010_m_000000_0, Status : FAILED
> > java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:703)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
> > at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
> > at
> > org.apache.mahout.matrix.AbstractVector.asFormatString(AbstractVector.java:374)
> >
> >
> > at
> > org.apache.mahout.clustering.kmeans.Cluster.outputPointWithClusterInfo(Cluster.java:198)
> >
> >
> > at
> > org.apache.mahout.clustering.kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:39)
> >
> >
> > at
> > org.apache.mahout.clustering.kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:32)
> >
> >
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:356)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> > at org.apache.hadoop.mapred.Child.main(Child.java:170)
> > Caused by: java.lang.ClassNotFoundException:
> > com.google.gson.reflect.TypeToken
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
> > ... 20 more
> > ----snip----
> >
> > Incidentally, as part of this work I've also implemented a Pearson
> > distance measure, if people think it would be useful to be folded in
> > I'd be happy to get the SVN patch with tests and implementation together.
> >
> > Thanks,
> > Paul
