[
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361748#comment-14361748
]
Hudson commented on MAPREDUCE-6265:
-----------------------------------
SUCCESS: Integrated in Hadoop-Yarn-trunk #866 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/866/])
MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to
better control to launch/kill containers. Contributed by Zhihai Xu (ozawa: rev
9d38520c8e42530a817a7f69c9aa73a9ad40639c)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control
> to launch/kill containers
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: zhihai xu
> Assignee: zhihai xu
> Fix For: 2.7.0
>
> Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch,
> MAPREDUCE-6265.002.patch
>
>
> make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better
> control the thread pool size to launch/kill containers
> Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at
> {code}
> protected static final int INITIAL_POOL_SIZE = 10;
> {code}
> We should make it configurable because the thread pool size will be decided
> by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
> Since we already made limitOnPoolSize configurable, it make senses to also
> make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
> We saw some issue due to the small thread pool size when some node is down.
> The recovery from a shutdown node take very long time due to all the
> ContainerLauncher threads are blocked by IPC client connection to the
> shutdown node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
