Github user mpjlu commented on the issue:
https://github.com/apache/spark/pull/17742
F2Jblas is faster than MKL blas. The following test is based on F2jBLAS.
Method 1: BLAS 3 + quickselect on each row and select top k.
Method 2: this PR
BLOCK size: 256 512 1024 2048 4096 8192
Method 1: 48s 41s 36s 88s NA NA
Method 2: NA NA 34s 35s 34s 34s
3 workers: each worker 40 cores, each worker 120G memory, each worker 1
executor.
The Data: User 480,000, and Item 17,000
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
