[
https://issues.apache.org/jira/browse/MAPREDUCE-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on MAPREDUCE-6283 started by Zhang Wei.
--------------------------------------------
> MRHistoryServer log files management optimization
> -------------------------------------------------
>
> Key: MAPREDUCE-6283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
> Reporter: Zhang Wei
> Assignee: Zhang Wei
> Priority: Minor
> Original Estimate: 2,016h
> Remaining Estimate: 2,016h
>
> In some heavy computation clusters, there might be a potential hdfs small
> files problem. The continually submitted MR jobs will create millions of log
> files (include the application master logs and application logs which are
> aggregated to hdfs by nodemanager). This optimization design helps to reduce
> the numbers of log files by merging them into bigger ones.
> Get the details from the design doc which I will upload later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
