Tim, That will allow you to increase the spawned vm's max heap size, which based on your trace looks like a good thing to try.
Adam On Tue, Feb 7, 2012 at 9:42 AM, Tim Broberg <[email protected]> wrote: > mapred.child.java.opts is not set. Should it be? > > ________________________________________ > From: Uma Maheswara Rao G [[email protected]] > Sent: Tuesday, February 07, 2012 2:27 AM > To: [email protected] > Subject: RE: out of memory running examples > > What is the Java heap space you configured? > for the property mapred.child.java.opts > ________________________________________ > From: Tim Broberg [[email protected]] > Sent: Tuesday, February 07, 2012 3:20 PM > To: [email protected] > Subject: out of memory running examples > > I'm trying to run the basic example from > hadoop/hadoop-1.0.0/docs/single_node_setup.html. > > I'm getting java.lang.OutOfMemoryError's when I run the grep example from > that page. > > Stackoverflow suggests various tweaks to the command line, > mapred-site.xml, or hadoop-env.sh, none of which seem to be helping in my > case. > > When I tweak hadoop-env.sh to echo text to a file, that file doesn't show > up, which suggests that hadoop-env.sh isn't even getting executed. > > Any hints on debugging this? > > - Tim. > > [tbroberg@san-mothra hadoop-1.0.0]$ bin/hadoop jar > hadoop-examples-1.0.0.jar grep input output 'dfs[a-z.]+' > Warning: $HADOOP_HOME is deprecated. > 12/02/07 01:39:35 INFO util.NativeCodeLoader: Loaded the native-hadoop > library > 12/02/07 01:39:35 INFO mapred.FileInputFormat: Total input paths to > process : 7 > 12/02/07 01:39:35 INFO mapred.JobClient: Running job: job_local_0001 > 12/02/07 01:39:35 INFO util.ProcessTree: setsid exited with exit code 0 > 12/02/07 01:39:35 INFO mapred.Task: Using ResourceCalculatorPlugin : > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471<mailto: > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471> > 12/02/07 01:39:35 INFO mapred.MapTask: numReduceTasks: 1 > 12/02/07 01:39:35 INFO mapred.MapTask: io.sort.mb = 100 > 12/02/07 01:39:35 WARN mapred.LocalJobRunner: job_local_0001 > java.lang.OutOfMemoryError: Java heap space > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) > 12/02/07 01:39:36 INFO mapred.JobClient: map 0% reduce 0% > 12/02/07 01:39:36 INFO mapred.JobClient: Job complete: job_local_0001 > 12/02/07 01:39:36 INFO mapred.JobClient: Counters: 0 > 12/02/07 01:39:36 INFO mapred.JobClient: Job Failed: NA > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265) > at org.apache.hadoop.examples.Grep.run(Grep.java:69) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.examples.Grep.main(Grep.java:93) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at > org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > [tbroberg@san-mothra hadoop-1.0.0]$ > > ________________________________ > The information and any attached documents contained in this message > may be confidential and/or legally privileged. The message is > intended solely for the addressee(s). If you are not the intended > recipient, you are hereby notified that any use, dissemination, or > reproduction is strictly prohibited and may be unlawful. If you are > not the intended recipient, please contact the sender immediately by > return e-mail and destroy all copies of the original message. > > The information and any attached documents contained in this message > may be confidential and/or legally privileged. The message is > intended solely for the addressee(s). If you are not the intended > recipient, you are hereby notified that any use, dissemination, or > reproduction is strictly prohibited and may be unlawful. If you are > not the intended recipient, please contact the sender immediately by > return e-mail and destroy all copies of the original message. > -- Adam Brown Enablement Engineer Hortonworks <http://www.hadoopsummit.org/>
