Arun,
this is a Streaming job.
I should not even know what "max java heapsize" means...
-- ab
On Jan 22, 2008, at 6:40 AM, Arun C Murthy (JIRA) wrote:
[ https://issues.apache.org/jira/browse/HADOOP-2438?
page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
tabpanel&focusedCommentId=12561360#action_12561360 ]
Arun C Murthy commented on HADOOP-2438:
---------------------------------------
Arkady, what is the max java heapsize for the child when you notice
this OOM?
In streaming, jobs that used to work, crash in the map phase --
even if the mapper is /bin/cat
---------------------------------------------------------------------
-------------------------
Key: HADOOP-2438
URL: https://issues.apache.org/jira/browse/
HADOOP-2438
Project: Hadoop
Issue Type: Bug
Affects Versions: 0.15.1
Reporter: arkady borkovsky
The exception is either "out of memory" of or "broken pipe" -- see
both stack dumps bellow.
st Hadoop input: |null|
last tool output: |[EMAIL PROTECTED]|
Date: Sat Dec 15 21:02:18 UTC 2007
java.io.IOException: Broken pipe
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at java.io.BufferedOutputStream.flushBuffer
(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush
(BufferedOutputStream.java:123)
at java.io.BufferedOutputStream.flush
(BufferedOutputStream.java:124)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.hadoop.streaming.PipeMapper.map
(PipeMapper.java:96)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main
(TaskTracker.java:1760)
at org.apache.hadoop.streaming.PipeMapper.map
(PipeMapper.java:107)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main
(TaskTracker.java:1760)
-------------------------------------------------
java.io.IOException: MROutput/MRErrThread
failed:java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write
(ByteArrayOutputStream.java:94)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.write(Text.java:243)
at org.apache.hadoop.mapred.MapTask
$MapOutputBuffer.collect (MapTask.java:347)
at org.apache.hadoop.streaming.PipeMapRed
$MROutputThread.run (PipeMapRed.java:344)
at org.apache.hadoop.streaming.PipeMapper.map
(PipeMapper.java:76)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at org.apache.hadoop.mapred.TaskTracker$Child.main
(TaskTracker.java:
1760)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
