[
https://issues.apache.org/jira/browse/HADOOP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462635
]
Owen O'Malley commented on HADOOP-815:
--------------------------------------
Here are the paths that don't lock the JobTracker as far as i can see...
ExpireLaunchingTask.run ->
JobInProgress.failedTask (line 670) ->
JobInProgress.updateTaskStatus ->
JobInProgress.failedTask (line 592)
JobTracker.markCompletedTaskAttempt
and:
ExpireTrackers.run ->
JobTracker.lostTaskTracker ->
JobInProgress.failedTask (line 670) -> from above
and a couple of similar variations. I think we are heading the wrong way with
this, because if it is this hard to verify that it is right, we will not be
able to keep it from breaking when we change the code later.
> Investigate and fix the extremely large memory-footprint of JobTracker
> ----------------------------------------------------------------------
>
> Key: HADOOP-815
> URL: https://issues.apache.org/jira/browse/HADOOP-815
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Fix For: 0.10.1
>
> Attachments: 150k_1199_774.nps, 75k_jobs.nps,
> HADOOP-815_20061220_1.patch, HADOOP-815_20061221_2.patch,
> HADOOP-815_20061222_3.patch, HADOOP-815_20061230_4.patch,
> HADOOP-815_20070105_5.patch, HADOOP-815_20070106_6.patch,
> jt_memory_profiles.tgz
>
>
> The JobTracker's memory footprint seems excessively large, especially when
> many jobs are submitted.
> Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus
> far:
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>
>
> 31877 arunc 19 0 2362m 261m 13m S 14.0 12.9 24:48.08 java
> Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic
> of this issue...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
