[
https://issues.apache.org/jira/browse/MAPREDUCE-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226923#comment-13226923
]
Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3971:
-----------------------------------------------------------
Hi Robert,
Thanks for clarifying. Yes,
[MAPREDUCE-3944|https://issues.apache.org/jira/browse/MAPREDUCE-3944]
eliminated JobHistory.getJob(JobId) invocation for each job and replaced
CompletedJob with PartialJob.
As I observed in the JobHistory#getAllJobsInternal uses a TreeMap to store
jobId and its corresponding PartialJob. But, *intermediateListCache* &
*jobListCache* are implementations of SortedMap, so the iterator returns jobs
from most recent to least recent. Again we are storing these Jobs in a TreeMap.
So each put opearation takes O(lgn) time. Where as put operatin of
LinkedHashMap takes O(1) time and guareentes retrival of the elements as per
their insertion order so I *think* we can replace TreeMap with LinkedHashMap.
Or is there anything am I missing?
> Job History web services need to have limits on the number of itmes they can
> return.
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3971
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3971
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2
> Affects Versions: 0.23.2
> Reporter: Robert Joseph Evans
>
> The Job History web services canput a very large load on the job history
> server. We should put in a limit on the number of entries that can be
> returned by the web service, and also add in the ability to modify the
> starting location in the list, so that all entries can still be downlaoded.
> Just not all at once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
