Hi,
Can someone help me asap? when i run my mapred job, it fails with this
error -
12/01/12 02:58:36 INFO mapred.JobClient: Task Id :
attempt_201112151554_0050_m_000071_0, Status : FAILED
Error: Java heap space
attempt_201112151554_0050_m_000071_0: log4j:ERROR Failed to flush writer,
attempt_201112151554_0050_m_000071_0: java.io.IOException: Stream closed
attempt_201112151554_0050_m_000071_0: at
sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:44)
attempt_201112151554_0050_m_000071_0: at
sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:139)
attempt_201112151554_0050_m_000071_0: at
java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
attempt_201112151554_0050_m_000071_0: at
org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:58)
attempt_201112151554_0050_m_000071_0: at
org.apache.hadoop.mapred.TaskLogAppender.flush(TaskLogAppender.java:94)
attempt_201112151554_0050_m_000071_0: at
org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:260)
attempt_201112151554_0050_m_000071_0: at
org.apache.hadoop.mapred.Child$2.run(Child.java:142)
so i updated my mapred-site.xml with these settings -
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx2048M</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx2048M</value>
</property>
also, when i run my jar, i provide -
"-Dmapred.map.child.java.opts="-Xmx4000m" at the end.
inspite of this, the task is not getting the max heap size im setting.
where did i go wrong?
after changing mapred-site.xml, i restarted jobtracker and tasktracker.. is
that not good enough?
thanks
