[
https://issues.apache.org/jira/browse/TEZ-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gurleen S Dhody updated TEZ-4198:
---------------------------------
Summary: TezContainerLauncherImpl launch and stop Container work (was:
ContainerLauncher launch and stop Container work)
> TezContainerLauncherImpl launch and stop Container work
> -------------------------------------------------------
>
> Key: TEZ-4198
> URL: https://issues.apache.org/jira/browse/TEZ-4198
> Project: Apache Tez
> Issue Type: Improvement
> Affects Versions: 0.9.1
> Reporter: Gurleen S Dhody
> Priority: Major
> Labels: refactor
>
> [https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L429]
> [https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L326]
> We are putting all the events (container events) in the event queue (Which is
> a blocking queue). And then on a separate thread we just take items from this
> queue and add it to a thread pool executor, which also has an internal
> blocking queue.
> This is a redundant creation of blocking queue and the thread to execute
> operations on the executor service.
> Also the executor service increases it's core pool size thread but never
> decreases it once the workload decreases. The core threads once started don't
> die unless we shutdown the service or we
> [specify|[https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html#allowsCoreThreadTimeOut()]]
> which we are also not doing. Also having the threads live for 1 hour is a
> stretch as we are potentially occupying resources that we don't use that
> often, especially when we are reusing containers ?
> We also don't need to specify INTEGER.MAX as max core pool size, but let it
> be the max value we want. The declaration of the thread pool service requires
> changes.
> [https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L283]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
