Zhang Wei created MAPREDUCE-6283:
------------------------------------
Summary: MRHistoryServer log files management optimization
Key: MAPREDUCE-6283
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6283
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver
Reporter: Zhang Wei
Assignee: Zhang Wei
Priority: Minor
In some heavy computation clusters, there might be a potential hdfs small files
problem. The continually submitted MR jobs will create millions of log files
(include the application master logs and application logs which are aggregated
to hdfs by nodemanager). This optimization design helps to reduce the numbers
of log files by merging them into bigger ones.
Get the details from the design doc which I will upload later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
