Imran Rashid created SPARK-24926:
------------------------------------
Summary: Ensure numCores is used consistently in all netty
configuration (driver and executors)
Key: SPARK-24926
URL: https://issues.apache.org/jira/browse/SPARK-24926
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.4.0
Reporter: Imran Rashid
I think there may be some places where we're not passing the right number of
configured cores to netty -- in particular in driver mode, we're not respecting
"spark.driver.cores". This means that spark will configure netty will be
configured based on the number of physical cores of the device, instead of
whatever resources spark requested from the cluster manager. It controls both
the number of threads netty uses *and* the number of arenas in its memory pools.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
