[
https://issues.apache.org/jira/browse/SPARK-15575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358831#comment-15358831
]
Nick Pentreath commented on SPARK-15575:
----------------------------------------
Also related to the discussion on SPARK-15581 and here, replacing Breeze is not
just about replacing the core linalg operators. It also impacts L-BFGS and
OWL-QN optimizers, which in turn impacts linear/logistic regression and MLP. So
we would need to have a plan to replace that too.
cc [~avulanov] [~rxin] [~Xiangrui] [~josephkb] [~srowen] [~freiss] [~mboehm7]
> Remove breeze from dependencies?
> --------------------------------
>
> Key: SPARK-15575
> URL: https://issues.apache.org/jira/browse/SPARK-15575
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Joseph K. Bradley
>
> This JIRA is for discussing whether we should remove Breeze from the
> dependencies of MLlib. The main issues with Breeze are Scala 2.12 support
> and performance issues.
> There are a few paths:
> # Keep dependency. This could be OK, especially if the Scala version issues
> are fixed within Breeze.
> # Remove dependency
> ## Implement our own linear algebra operators as needed
> ## Design a way to build Spark using custom linalg libraries of the user's
> choice. E.g., you could build MLlib using Breeze, or any other library
> supporting the required operations. This might require significant work.
> See [SPARK-6442] for related discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
