What is your io.sort.mb set to? Given the heap size, you may have to lower it down.
And did you intend to run the program in 'local' mode (i.e. non-distributed mode, no JobTracker but LocalJobRunner)? On Tue, Feb 7, 2012 at 3:20 PM, Tim Broberg <[email protected]> wrote: > I'm trying to run the basic example from > hadoop/hadoop-1.0.0/docs/single_node_setup.html. > > I'm getting java.lang.OutOfMemoryError's when I run the grep example from > that page. > > Stackoverflow suggests various tweaks to the command line, mapred-site.xml, > or hadoop-env.sh, none of which seem to be helping in my case. > > When I tweak hadoop-env.sh to echo text to a file, that file doesn't show up, > which suggests that hadoop-env.sh isn't even getting executed. > > Any hints on debugging this? > > - Tim. > > [tbroberg@san-mothra hadoop-1.0.0]$ bin/hadoop jar hadoop-examples-1.0.0.jar > grep input output 'dfs[a-z.]+' > Warning: $HADOOP_HOME is deprecated. > 12/02/07 01:39:35 INFO util.NativeCodeLoader: Loaded the native-hadoop library > 12/02/07 01:39:35 INFO mapred.FileInputFormat: Total input paths to process : > 7 > 12/02/07 01:39:35 INFO mapred.JobClient: Running job: job_local_0001 > 12/02/07 01:39:35 INFO util.ProcessTree: setsid exited with exit code 0 > 12/02/07 01:39:35 INFO mapred.Task: Using ResourceCalculatorPlugin : > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471> > 12/02/07 01:39:35 INFO mapred.MapTask: numReduceTasks: 1 > 12/02/07 01:39:35 INFO mapred.MapTask: io.sort.mb = 100 > 12/02/07 01:39:35 WARN mapred.LocalJobRunner: job_local_0001 > java.lang.OutOfMemoryError: Java heap space > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) > 12/02/07 01:39:36 INFO mapred.JobClient: map 0% reduce 0% > 12/02/07 01:39:36 INFO mapred.JobClient: Job complete: job_local_0001 > 12/02/07 01:39:36 INFO mapred.JobClient: Counters: 0 > 12/02/07 01:39:36 INFO mapred.JobClient: Job Failed: NA > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265) > at org.apache.hadoop.examples.Grep.run(Grep.java:69) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.examples.Grep.main(Grep.java:93) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > [tbroberg@san-mothra hadoop-1.0.0]$ > > ________________________________ > The information and any attached documents contained in this message > may be confidential and/or legally privileged. The message is > intended solely for the addressee(s). If you are not the intended > recipient, you are hereby notified that any use, dissemination, or > reproduction is strictly prohibited and may be unlawful. If you are > not the intended recipient, please contact the sender immediately by > return e-mail and destroy all copies of the original message. -- Harsh J Customer Ops. Engineer Cloudera | http://tiny.cloudera.com/about
