[
https://issues.apache.org/jira/browse/HADOOP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462418
]
Arun C Murthy commented on HADOOP-815:
--------------------------------------
> 1. The TaskInProgress.usableTaskIds should be removed and a new task id
> generated when needed. That list has been bothering me for a while. *smile*
Ok, coming up. :)
> 2. You replace a new style for loop with an old style for loop on
> TaskTracker.java line 597, which should stay a new style loop.
Subtle change due to the fact that
org.apache.hadoop.mapred.TaskTrackerStatus.getTaskReports returns an
'Iterator'. Shud i fix the (public) api to return 'List<TaskStatus>' instead?
> 3. The trackerToMarkedTaskMap isn't synchronized by a lock, so has race
> conditions.
The functions manipulating 'trackerToMarkedTaskMap' assume that the
JobTracker itself is locked on entry (much like existing 'removeTaskEntry'),
would it help if I put in a comment there? Or shud I explicitly mark the
function as 'synchronized' ?
>On formatting, indentation is also four-spaces per level rather than the
>preferred two.
Ok, I'll fix it. Spent time hitting the 'tab' key to ensure compliance with
existing indentation in some functions. *rueful smile*
Related thought: should we open a new issue to track and ensure 'all' code is
indented with 2 spaces instead of 4 (as in some places)? :)
> Investigate and fix the extremely large memory-footprint of JobTracker
> ----------------------------------------------------------------------
>
> Key: HADOOP-815
> URL: https://issues.apache.org/jira/browse/HADOOP-815
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Fix For: 0.10.0
>
> Attachments: 150k_1199_774.nps, 75k_jobs.nps,
> HADOOP-815_20061220_1.patch, HADOOP-815_20061221_2.patch,
> HADOOP-815_20061222_3.patch, HADOOP-815_20061230_4.patch,
> jt_memory_profiles.tgz
>
>
> The JobTracker's memory footprint seems excessively large, especially when
> many jobs are submitted.
> Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus
> far:
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>
>
> 31877 arunc 19 0 2362m 261m 13m S 14.0 12.9 24:48.08 java
> Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic
> of this issue...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
