Thomas Graves created SPARK-3607:
------------------------------------
Summary: ConnectionManager threads.max configs on the thread pools
don't work
Key: SPARK-3607
URL: https://issues.apache.org/jira/browse/SPARK-3607
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.1.0
Reporter: Thomas Graves
Priority: Minor
In the ConnectionManager we have a bunch of thread pools. They have settings
for the maximum number of threads for each Threadpool (like
spark.core.connection.handler.threads.max).
Those configs don't work because its using a unbounded queue. From the
threadpoolexecutor javadoc page: no more than corePoolSize threads will ever be
created. (And the value of the maximumPoolSize therefore doesn't have any
effect.)
luckily this doesn't matter to much as you can work around it by just
increasing the minimum like spark.core.connection.handler.threads.min.
These configs aren't documented either so its more of an internal thing when
someone is reading the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
