Hi all, I am encountering a rather peculiar problem with Hadoop. First let me describe the setup. I have a 4 node cluster which I am using to run hadoop. In addition to hadoop, I have also installed Hbase and Hama (matrix multiplication package for hadoop). Now if I run a normal have run the sample map reduce jobs (not in the debug mode ) and everything is working out fine. Now I am made a program in HAMA which calls the map reduce function from hadoop. I am trying to run this code in the debug mode. Here I am getting a Java Heap space exception. I cannot understand why is there a heap space exception in the debug mode. Here is the complete output.
Grateful for any suggestions 03/05/03 05:16:09 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 03/05/03 05:16:09 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:, 03/05/03 05:16:09 INFO mapred.JobClient: Running job: job_local_0001 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:, 03/05/03 05:16:09 INFO mapred.MapTask: numReduceTasks: 1 03/05/03 05:16:09 INFO mapred.MapTask: io.sort.mb = 100 03/05/03 05:16:10 WARN mapred.LocalJobRunner: job_local_0001 java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:498) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138) Exception in thread "Thread-15" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196) Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.HttpMethod at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) ... 1 more Exception in thread "main" java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) at org.apache.hama.util.JobManager.execute(JobManager.java:52) at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:558) at Markowitz.MarkowitzMain.main(MarkowitzMain.java:33) Thank you Abhishek Agrawal SUNY- Buffalo (716-435-7122)
