What is the heap size you are using for your tasks? Check
'mapred.child.java.opts' in your hadoop-default.xml. Try increasing it.
This will happen if you try running the random-writer + sort examples with
default parameters. The maps are not able to spill the data to the disk.
Btw what version of HADOOP are you using?
Amar
On Mon, 10 Mar 2008, Ved Prakash
wrote:
Hi friends,
I have made a cluster of 3 machines, one of them is master, and other 2
slaves. I executed a mapreduce job on master but after Map, the execution
terminates and Reduce doesn't happen. I have checked dfs and no output
folder gets created.
this is the error I see
08/03/10 10:35:21 INFO mapred.JobClient: Task Id :
task_200803101001_0001_m_000064_0, Status : FAILED
java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java
:95)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.write(Text.java:243)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(
MapTask.java:347)
at org.apache.hadoop.examples.WordCount$MapClass.map(WordCount.java
:72)
at org.apache.hadoop.examples.WordCount$MapClass.map(WordCount.java
:59)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java
:1787)
08/03/10 10:35:22 INFO mapred.JobClient: map 55% reduce 17%
08/03/10 10:35:31 INFO mapred.JobClient: map 56% reduce 17%
08/03/10 10:35:51 INFO mapred.JobClient: map 57% reduce 17%
08/03/10 10:36:04 INFO mapred.JobClient: map 58% reduce 17%
08/03/10 10:36:07 INFO mapred.JobClient: map 57% reduce 17%
08/03/10 10:36:07 INFO mapred.JobClient: Task Id :
task_200803101001_0001_m_000071_0, Status : FAILED
java.lang.OutOfMemoryError: Java heap space
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java
:95)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.write(Text.java:243)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(
MapTask.java:347)
at org.apache.hadoop.examples.WordCount$MapClass.map(WordCount.java
:72)
at org.apache.hadoop.examples.WordCount$MapClass.map(WordCount.java
:59)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java
:1787)
though it tries to overcome this problem but the mapreduce application
doesn't create output, can anyone tell me why is this happening?
Thanks
