[
https://issues.apache.org/jira/browse/HADOOP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462580
]
Arun C Murthy commented on HADOOP-815:
--------------------------------------
> Please change the TaskTrackerStatus.taskReports() method to be depricated and
> make a new:
> public List<TaskStatus> getTaskReports() { ... }
Ok, will do. Another patch coming up...
> I'm pretty sure there are some contexts where the JobTracker is not locked.
> I'll take a look.
I've run Devaraj through the whole synchronization logic here, basically the
functions assume that the JobTracker is locked on entry (as in the javadoc) and
happens so since most of the calls emanate from
public synchronized JobTracker.heartbeat() -> JobTracker.processHeartbeat() ->
JobTracker.updateTaskStatuses
I'd definitely appreciate another closer look at this... Thanks!
> Investigate and fix the extremely large memory-footprint of JobTracker
> ----------------------------------------------------------------------
>
> Key: HADOOP-815
> URL: https://issues.apache.org/jira/browse/HADOOP-815
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Fix For: 0.10.0
>
> Attachments: 150k_1199_774.nps, 75k_jobs.nps,
> HADOOP-815_20061220_1.patch, HADOOP-815_20061221_2.patch,
> HADOOP-815_20061222_3.patch, HADOOP-815_20061230_4.patch,
> HADOOP-815_20070105_5.patch, jt_memory_profiles.tgz
>
>
> The JobTracker's memory footprint seems excessively large, especially when
> many jobs are submitted.
> Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus
> far:
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>
>
> 31877 arunc 19 0 2362m 261m 13m S 14.0 12.9 24:48.08 java
> Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic
> of this issue...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
