[
https://issues.apache.org/jira/browse/SPARK-41188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu updated SPARK-41188:
-------------------------------
Description: Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by
default for spark executor JVM processes, this is for limiting the thread
number for OpenBLAS routine to the number of cores assigned to this executor
because some spark ML algorithms calls OpenBlAS via netlib-java (was: Limit
the thread number for OpenBLAS routine to the number of cores assigned to this
executor because some spark ML algorithms calls OpenBlAS via netlib-java)
> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark
> executor JVM processes
> -------------------------------------------------------------------------------------------------
>
> Key: SPARK-41188
> URL: https://issues.apache.org/jira/browse/SPARK-41188
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 3.3.1
> Reporter: Weichen Xu
> Priority: Major
> Fix For: 3.4.0
>
>
> Set executorEnv OMP_NUM_THREADS to be spark.task.cpus by default for spark
> executor JVM processes, this is for limiting the thread number for OpenBLAS
> routine to the number of cores assigned to this executor because some spark
> ML algorithms calls OpenBlAS via netlib-java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
