You can change the heapsize for child jvm by setting mapred.child.java.opts to -Xmx512m
The default heapsize is 200m which may be not enough for you, since Hama is matrix computing framework which need lots of memory. Jeff Zhang On Tue, Nov 10, 2009 at 12:39 PM, <[email protected]> wrote: > Hi all, > > I am encountering a rather peculiar problem with Hadoop. First let me > describe the setup. I have a 4 node cluster which I am using to run hadoop. > In > addition to hadoop, I have also installed Hbase and Hama (matrix > multiplication > package for hadoop). Now if I run a normal have run the sample map reduce > jobs > (not in the debug mode ) and everything is working out fine. Now I am made > a > program in HAMA which calls the map reduce function from hadoop. I am > trying to > run this code in the debug mode. Here I am getting a Java Heap space > exception. I > cannot understand why is there a heap space exception in the debug mode. > Here is > the complete output. > > Grateful for any suggestions > > > 03/05/03 05:16:09 INFO jvm.JvmMetrics: Initializing JVM Metrics with > processName=JobTracker, sessionId= > 03/05/03 05:16:09 WARN mapred.JobClient: Use GenericOptionsParser for > parsing the > arguments. Applications should implement Tool for the same. > 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:, > 03/05/03 05:16:09 INFO mapred.JobClient: Running job: job_local_0001 > 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:, > 03/05/03 05:16:09 INFO mapred.MapTask: numReduceTasks: 1 > 03/05/03 05:16:09 INFO mapred.MapTask: io.sort.mb = 100 > 03/05/03 05:16:10 WARN mapred.LocalJobRunner: job_local_0001 > java.lang.OutOfMemoryError: Java heap space > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:498) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) > at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138) > Exception in thread "Thread-15" java.lang.NoClassDefFoundError: > org/apache/commons/httpclient/HttpMethod > at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196) > Caused by: java.lang.ClassNotFoundException: > org.apache.commons.httpclient.HttpMethod > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276) > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) > ... 1 more > Exception in thread "main" java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) > at org.apache.hama.util.JobManager.execute(JobManager.java:52) > at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:558) > at Markowitz.MarkowitzMain.main(MarkowitzMain.java:33) > > Thank you > > Abhishek Agrawal > > SUNY- Buffalo > (716-435-7122) > > > >
