[
https://issues.apache.org/jira/browse/MAPREDUCE-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307452#comment-14307452
]
Jason Lowe commented on MAPREDUCE-6059:
---------------------------------------
If you have no objections, I'd like to commit this to branch-2 as well. I'd
like to keep the trunk and branch-2 lines as reasonably close as we can to
minimize the pain of maintaining the two lines.
> Speed up history server startup time
> ------------------------------------
>
> Key: MAPREDUCE-6059
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6059
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.4.0
> Reporter: Siqi Li
> Assignee: Siqi Li
> Fix For: 3.0.0
>
> Attachments: YARN-2366.v1.patch
>
>
> When history server starts up, It scans every history directories and put all
> history files into a cache, whereas this cache only stores 20K recent history
> files. Therefore, it is wasting a large portion of time loading old history
> files into the cache, and the startup time will keep increasing if we don't
> trim the number of history files. For example, when history server starts up
> with 2.5M history files in HDFS, it took ~5 minutes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
