[
https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589388#comment-14589388
]
Sebastian Alfers commented on SPARK-7334:
-----------------------------------------
I implemented RP as a transformer to be able to serialize the model and re-use
it later.
Also, the actual implementation of RP is separated and (theoretically) can be
used in LSH.
I implemented RP as a "stand alone" method as a replacement / comparison to PCA.
> Implement RandomProjection for Dimensionality Reduction
> -------------------------------------------------------
>
> Key: SPARK-7334
> URL: https://issues.apache.org/jira/browse/SPARK-7334
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Sebastian Alfers
> Priority: Minor
>
> Implement RandomProjection (RP) for dimensionality reduction
> RP is a popular approach to reduce the amount of data while preserving a
> reasonable amount of information (pairwise distance) of you data [1][2]
> - [1] http://www.yaroslavvb.com/papers/achlioptas-database.pdf
> - [2]
> http://people.inf.elte.hu/fekete/algoritmusok_msc/dimenzio_csokkentes/randon_projection_kdd.pdf
> I compared different implementations of that algorithm:
> - https://github.com/sebastian-alfers/random-projection-python
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
