[
https://issues.apache.org/jira/browse/HADOOP-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12582909#action_12582909
]
Amar Kamat commented on HADOOP-2837:
------------------------------------
I suspect the *queue* in the task commit queue. Pre HADOOP-2119 the queue items
were processed one by one and hence the queue used to build up. Now with
HADOOP-2119, commit is done in batches of size 5000. Chances of this error
occurring with the trunk is less. We tried running 320,000 maps on 250 nodes
where each map generated 8mb of data (similar to RandomWriter) and we never saw
this error. Can someone confirm this?
> JobTracker got stuck
> ---------------------
>
> Key: HADOOP-2837
> URL: https://issues.apache.org/jira/browse/HADOOP-2837
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.15.3
> Reporter: Runping Qi
>
> When a job tracker ran a job with a lot of map tasks,
> the commit thread complained out of memory:
> 2008-02-14 02:52:13,809 ERROR org.apache.hadoop.mapred.JobTracker: Task
> Commit Thread got an exception: java.lang.OutOfMemoryError: Java heap space
> 2008-02-14 02:53:18,543 ERROR org.apache.hadoop.mapred.JobTracker: Task
> Commit Thread got an exception: java.lang.OutOfMemoryError: Java heap space
> 2008-02-14 02:53:26,965 ERROR org.apache.hadoop.mapred.JobTracker: Task
> Commit Thread got an exception: java.lang.OutOfMemoryError: Java heap space
> 2008-02-14 02:54:27,009 ERROR org.apache.hadoop.mapred.JobTracker: Task
> Commit Thread got an exception: java.lang.OutOfMemoryError: Java heap space
> then later, the job tracker stopped making any progress, and stopped
> responding web GUI.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
