Haibo Chen created YARN-4697:
--------------------------------
Summary: NM aggregation thread pool is not bound by limits
Key: YARN-4697
URL: https://issues.apache.org/jira/browse/YARN-4697
Project: Hadoop YARN
Issue Type: Improvement
Components: nodemanager
Reporter: Haibo Chen
Assignee: Haibo Chen
In the LogAggregationService.java we create a threadpool to upload logs from
the nodemanager to HDFS if log aggregation is turned on. This is a cached
threadpool which based on the javadoc is an ulimited pool of threads.
In the case that we have had a problem with log aggregation this could cause a
problem on restart. The number of threads created at that point could be huge
and will put a large load on the NameNode and in worse case could even bring it
down due to file descriptor issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
