[
https://issues.apache.org/jira/browse/SPARK-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358820#comment-15358820
]
Nick Pentreath commented on SPARK-15575:
----------------------------------------
So it seems from the various discussion on SPARK-15581, SPARK-13944, and due to
the issue in SPARK-14438 that there is enough interest and immediacy around
this that it may make sense to review linear algebra within Spark and JVM in
general, to refresh the current state with respect to features, performance,
API / Java compat, and license.
I've created a (very initial and WIP) [Google
doc|https://docs.google.com/document/d/1UWhuP-YQXW6D-IZs2Dy9CrdxraUDMFG5DdX3eo6cuEs/edit?usp=sharing]
to begin discussion and collating the details. Please feel free to discuss
further here or on that doc.
I'm happy to move ahead with the evaluation and performance testing (also happy
to split up the work if desired).
> Remove breeze from dependencies?
> --------------------------------
>
> Key: SPARK-15575
> URL: https://issues.apache.org/jira/browse/SPARK-15575
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Joseph K. Bradley
>
> This JIRA is for discussing whether we should remove Breeze from the
> dependencies of MLlib. The main issues with Breeze are Scala 2.12 support
> and performance issues.
> There are a few paths:
> # Keep dependency. This could be OK, especially if the Scala version issues
> are fixed within Breeze.
> # Remove dependency
> ## Implement our own linear algebra operators as needed
> ## Design a way to build Spark using custom linalg libraries of the user's
> choice. E.g., you could build MLlib using Breeze, or any other library
> supporting the required operations. This might require significant work.
> See [SPARK-6442] for related discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
