Vincent commented on SPARK-21688:

upload a data we collected before, uni-test on ddot, we can see for data size 
greater than 100, native blas normally has advantages. But if the size is 
smaller than 100, f2j would be a better choice.

> performance improvement in mllib SVM with native BLAS 
> ------------------------------------------------------
>                 Key: SPARK-21688
>                 URL: https://issues.apache.org/jira/browse/SPARK-21688
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 2.2.0
>         Environment: 4 nodes: 1 master node, 3 worker nodes
> model name      : Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz
> Memory : 180G
> num of core per node: 10
>            Reporter: Vincent
>         Attachments: mllib svm training.png, svm1.png, svm2.png, 
> svm-mkl-1.png, svm-mkl-2.png, uni-test on ddot.png
> in current mllib SVM implementation, we found that the CPU is not fully 
> utilized, one reason is that f2j blas is set to be used in the HingeGradient 
> computation. As we found out earlier 
> (https://issues.apache.org/jira/browse/SPARK-21305) that with proper 
> settings, native blas is generally better than f2j on the uni-test level, 
> here we make the blas operations in SVM go with MKL blas and get an end to 
> end performance report showing that in most cases native blas outperformance 
> f2j blas up to 50%.
> So, we suggest removing those f2j-fixed calling and going for native blas if 
> available. If this proposal is acceptable, we will move on to benchmark other 
> algorithms impacted. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to