Gurleen S Dhody created TEZ-4198:
------------------------------------
Summary: ContainerLauncher launch and stop Container work
Key: TEZ-4198
URL: https://issues.apache.org/jira/browse/TEZ-4198
Project: Apache Tez
Issue Type: Improvement
Reporter: Gurleen S Dhody
[https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L429]
[https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L326]
We are putting all the events (container events) in the event queue (Which is a
blocking queue). And then on a separate thread we just take items from this
queue and add it to a thread pool executor, which also has an internal blocking
queue.
This is a redundant creation of blocking queue and the thread to execute
operations on the executor service.
Also the executor service increases it's core pool size thread but never
decreases it once the workload decreases. The core threads once started don't
die unless we shutdown the service or we
[specify|[https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html#allowsCoreThreadTimeOut()]]
which we are also not doing. Also having the threads live for 1 hour is a
stretch as we are potentially occupying resources that we don't use that often,
especially when we are reusing containers ?
We also don't need to specify INTEGER.MAX as max core pool size, but let it be
the max value we want. The declaration of the thread pool service requires
changes.
[https://github.com/apache/tez/blob/3f2373e2b2ab3825ef50e9f19b8704265542a8b2/tez-dag/src/main/java/org/apache/tez/dag/app/launcher/TezContainerLauncherImpl.java#L283]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
