[
https://issues.apache.org/jira/browse/SPARK-6442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627870#comment-14627870
]
Sean Owen commented on SPARK-6442:
----------------------------------
This is a follow-on from SPARK-9003, where the question was, what local linear
algebra library can we use instead of rebuilding yet another one?
[~josephkb] Commons Math? or Breeze? These are already in use in Spark even.
Commons Math is quite stable, still active, AL2, Java-friendly.
So, Spark needs to do some local math internally. User apps need to do some
local math too. The point of the light Vector API has been to shield callers
from implementation details and transfer between these two domains in terms of
a simple, generic representation. I can see the need to maybe add a few more
methods to Vector as convenience methods, like map.
Now, people can freely use, say, Commons Math locally in their app for any of
this. Spark could offer yet another local math library, but what's the
advantage?
More generally I find Spark spread waay to thin, and would like to push back a
lot more against yet more. I think it's always fun to look at slam-dunking
problems like this again, but is it a good use of time? There is so much else
that has been begun but not finished, or needs fixing. So I'm picking on this
one. Maybe you have something in mind much more modest?
> MLlib Local Linear Algebra Package
> ----------------------------------
>
> Key: SPARK-6442
> URL: https://issues.apache.org/jira/browse/SPARK-6442
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Reporter: Burak Yavuz
> Priority: Critical
>
> MLlib's local linear algebra package doesn't have any support for any type of
> matrix operations. With 1.5, we wish to add support to a complete package of
> optimized linear algebra operations for Scala/Java users.
> The main goal is to support lazy operations so that element-wise can be
> implemented in a single for-loop, and complex operations can be interfaced
> through BLAS.
> The design doc: http://goo.gl/sf5LCE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
