yma11 commented on issue #27546: [SPARK-30773][ML]Support NativeBlas for level-1 routines URL: https://github.com/apache/spark/pull/27546#issuecomment-590792591 @srowen In my kmeans test, each executor is configured with 6 cores and I can confirm it's fully utilized through web UI. and under this condition, using default setting of MKL_NUM_THREADS is better than setting value to 1 but similar with setting value to 4. So I suggest to tune this env variable based on specific configuration of spark workloads. To determine the threshold for logic change in level-1 methods, I have collected perf data for more vector sizes. Thus based on the data of vector size(10, 100, 128, 150, 256, 1000), seems 128 is the point that nativeBlas has equal performance with java implementation and after that it shows obvious gain. Will it be reasonable to use this value as the logic change bar? Attached includes perf data for 128/150 elements vectors. [Level1-routines-perf - part-3.xlsx](https://github.com/apache/spark/files/4249399/Level1-routines-perf.-.part-3.xlsx)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
