[
https://issues.apache.org/jira/browse/SPARK-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138057#comment-14138057
]
Xiangrui Meng commented on SPARK-3403:
--------------------------------------
1) Yes, I saw this issue in Linux.
2) In Spark, the most common setting is that each executor uses one thread. So
using multithreaded BLAS leads to competition between threads, which does slow
down the computation. However, on the driver node, maybe it is useful to use
multithreads.
3) It is hard to get performance gain on level-1 and level-2 BLAS operations.
You should see the difference on Level-3 BLAS, e.g, in ALS.
Could you try to submit this issue to OpenBLAS and let them check the JNI
compatibility with multithreading? I don't think this is caused by MLlib's
implementation.
> NaiveBayes crashes with blas/lapack native libraries for breeze (netlib-java)
> -----------------------------------------------------------------------------
>
> Key: SPARK-3403
> URL: https://issues.apache.org/jira/browse/SPARK-3403
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Affects Versions: 1.0.2
> Environment: Setup: Windows 7, x64 libraries for netlib-java (as
> described on https://github.com/fommil/netlib-java). I used OpenBlas x64 and
> MinGW64 precompiled dlls.
> Reporter: Alexander Ulanov
> Fix For: 1.2.0
>
> Attachments: NativeNN.scala
>
>
> Code:
> val model = NaiveBayes.train(train)
> val predictionAndLabels = test.map { point =>
> val score = model.predict(point.features)
> (score, point.label)
> }
> predictionAndLabels.foreach(println)
> Result:
> program crashes with: "Process finished with exit code -1073741819
> (0xC0000005)" after displaying the first prediction
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
