Haibo Chen created YARN-4697: -------------------------------- Summary: NM aggregation thread pool is not bound by limits Key: YARN-4697 URL: https://issues.apache.org/jira/browse/YARN-4697 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Reporter: Haibo Chen Assignee: Haibo Chen
In the LogAggregationService.java we create a threadpool to upload logs from the nodemanager to HDFS if log aggregation is turned on. This is a cached threadpool which based on the javadoc is an ulimited pool of threads. In the case that we have had a problem with log aggregation this could cause a problem on restart. The number of threads created at that point could be huge and will put a large load on the NameNode and in worse case could even bring it down due to file descriptor issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)