Github user mpjlu commented on a diff in the pull request:
https://github.com/apache/spark/pull/18551#discussion_r126323336
--- Diff: docs/ml-guide.md ---
@@ -61,6 +61,11 @@ To configure `netlib-java` / Breeze to use system
optimised binaries, include
project and read the [netlib-java](https://github.com/fommil/netlib-java)
documentation for your
platform's additional installation instructions.
+The most popular native BLAS such as [Intel
MKL](https://software.intel.com/en-us/mkl),
[OpenBLAS](http://www.openblas.net), are based on multi-threading.
+For example, when OpenBLAS is loaded, it will create a thread pool with
`MAX_CPU_NUMBER` threads, and the threads are using spinlock by default, which
will conflict with Spark.
--- End diff --
Hi @srowen , it is not always worse, here is one test result:
https://issues.apache.org/jira/browse/SPARK-21305?focusedCommentId=16076152&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16076152
I will soften the language slightly. And give two links about recommended
settings for calling native blas. How do you think about it?
https://github.com/xianyi/OpenBLAS/wiki/faq#multi-threaded
https://software.intel.com/en-us/articles/recommended-settings-for-calling-intel-mkl-routines-from-multi-threaded-applications
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
