Siqi Li reassigned YARN-2366:

    Assignee: Siqi Li

> Speed up history server startup time
> ------------------------------------
>                 Key: YARN-2366
>                 URL: https://issues.apache.org/jira/browse/YARN-2366
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.4.0
>            Reporter: Siqi Li
>            Assignee: Siqi Li
>         Attachments: YARN-2366.v1.patch
> When history server starts up, It scans every history directories and put all 
> history files into a cache, whereas this cache only stores 20K recent history 
> files. Therefore, it is wasting a large portion of time loading old history 
> files into the cache, and the startup time will keep increasing if we don't 
> trim the number of history files. For example, when history server starts up 
> with 2.5M history files in HDFS, it took ~5 minutes.

This message was sent by Atlassian JIRA

Reply via email to