GitHub user dbtsai opened a pull request:
https://github.com/apache/spark/pull/12172
[SPARK-13944][ML][WIP] Separate out local linear algebra as a standalone
module without Spark dependency
## What changes were proposed in this pull request?
Separate out linear algebra as a standalone module without Spark dependency
to simplify production deployment. We can call the new module
spark-mllib-local, which might contain local models in the future.
The major issue is to remove dependencies on user-defined types.
The package name will be changed from mllib to ml. For example, Vector will
be changed from `org.apache.spark.mllib.linalg.Vector` to
`org.apache.spark.ml.linalg.Vector`. The return vector type in the new ML
pipeline will be the one in ML package; however, the existing mllib code will
not be touched. As a result, this will potentially break the API. Also, when
the vector is loaded from mllib vector by Spark SQL, the vector will
automatically converted into the one in ml package.
## How was this patch tested?
WIP
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dbtsai/spark dbtsai-linear-algebra
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12172.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12172
----
commit 0cfc65d1aa4ab3c82459ab0cd3598fd2969387b6
Author: DB Tsai <[email protected]>
Date: 2016-03-22T23:54:11Z
dbtsai-linear-algebra
commit cb95b0c5194e0d53614c5ae9fd77f110bbd62826
Author: DB Tsai <[email protected]>
Date: 2016-04-05T07:32:11Z
more work
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
