Ohad Raviv created SPARK-16469:
----------------------------------
Summary: Long running Driver task while multiplying big matrices
Key: SPARK-16469
URL: https://issues.apache.org/jira/browse/SPARK-16469
Project: Spark
Issue Type: Improvement
Components: MLlib
Affects Versions: 2.0.0
Reporter: Ohad Raviv
Priority: Minor
Fix For: 2.0.0
We have a use case of multiplying very big sparse matrices. we have about
1000x1000 distributed block matrices multiplication and the simulate multiply
goes like O(n^4) (n being 1000). it takes about 1.5 hours. We modified it
slightly with classical hashmap and now run in about 30 seconds O(n^2).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
