Hi Jeff,
I am sorry but I do not have the file mapred-site.xml. So I made the
change in hadoop-site.xml. Also do we have to make the change only in the master
node or even in the slave nodes ?
Thank You
Abhishek Agrawal
SUNY- Buffalo
(716-435-7122)
On Tue 11/10/09 12:01 AM , Jeff Zhang [email protected] sent:
> You can change the heapsize for child jvm by setting
> mapred.child.java.opts to -Xmx512m
>
> The default heapsize is 200m which may be not enough for you, since
> Hama is matrix computing framework which need lots of memory.
> Jeff Zhang
>
> On Tue, Nov 10, 2009 at 12:39 PM, wrote:
> Hi all,
> I am encountering a rather peculiar problem with Hadoop. First let
> me
> describe the setup. I have a 4 node cluster which I am using to run
> hadoop. In
> addition to hadoop, I have also installed Hbase and Hama (matrix
> multiplication
> package for hadoop). Now if I run a normal have run the sample map
> reduce jobs
> (not in the debug mode ) and everything is working out fine. Now I
> am made a
> program in HAMA which calls the map reduce function from hadoop. I
> am trying to
> run this code in the debug mode. Here I am getting a Java Heap space
> exception. I
> cannot understand why is there a heap space exception in the debug
> mode. Here is
> the complete output.
> Grateful for any suggestions
> 03/05/03 05:16:09 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 03/05/03 05:16:09 WARN mapred.JobClient: Use GenericOptionsParser
> for parsing the
> arguments. Applications should implement Tool for the same.
> 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split:
> 0->aphrodite:,
> 03/05/03 05:16:09 INFO mapred.JobClient: Running job: job_local_0001
> 03/05/03 05:16:09 INFO mapred.HTableInputFormatBase: split:
> 0->aphrodite:,
> 03/05/03 05:16:09 INFO mapred.MapTask: numReduceTasks: 1
> 03/05/03 05:16:09 INFO mapred.MapTask: io.sort.mb = 100
> 03/05/03 05:16:10 WARN mapred.LocalJobRunner: job_local_0001
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
> Exception in thread "Thread-15" java.lang.NoClassDefFoundError:
> org/apache/commons/httpclient/HttpMethod
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.httpclient.HttpMethod
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> ... 1 more
> Exception in thread "main" java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
> at org.apache.hama.util.JobManager.execute(JobManager.java:52)
> at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:558)
> at Markowitz.MarkowitzMain.main(MarkowitzMain.java:33)
> Thank you
> Abhishek Agrawal
> SUNY- Buffalo
> (716-435-7122)
>
>
