[
https://issues.apache.org/jira/browse/SPARK-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477841#comment-15477841
]
Seth Hendrickson commented on SPARK-17471:
------------------------------------------
[~yanboliang] I guess it can be seen as a duplicate, but really there are two
separate tasks. 1.) Add a `compressed` method to the matrix library in spark,
which is non-trivial. 2.) Adding a mechanism inside of MLOR to use the
compressed method, and how to deal with flattening the sparse matrix into a
sparse vector when binomial family is used.
We can keep the JIRAs separate, or do them both together. I see them as
separate tasks.
> Add compressed method for Matrix class
> --------------------------------------
>
> Key: SPARK-17471
> URL: https://issues.apache.org/jira/browse/SPARK-17471
> Project: Spark
> Issue Type: New Feature
> Components: ML
> Reporter: Seth Hendrickson
>
> Vectors in Spark have a {{compressed}} method which selects either sparse or
> dense representation by minimizing storage requirements. Matrices should also
> have this method, which is now explicitly needed in {{LogisticRegression}}
> since we have implemented multiclass regression.
> The compressed method should also give the option to store row major or
> column major, and if nothing is specified should select the lower storage
> representation (for sparse).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
