[
https://issues.apache.org/jira/browse/FLINK-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116993#comment-15116993
]
Till Rohrmann commented on FLINK-1733:
--------------------------------------
Hi [~thang], I think for a first version your interface definition sounds fine.
The user provides the number of principal components he wants to obtain and he
receives a {{DataSet[Vector]}} or {{DataSet[DenseVector]}} which are the
principal components.
Your description of the standard PCA is also correct. However, I think for the
distributed execution it might be a bit different. Best you check out the
linked resources or google for papers describing a distributed PCA
implementation on MapReduce.
Be aware that if you want to order the vectors contained in the resulting
{{DataSet}} you have to give them IDs or assign them their eigenvalues because
the {{DataSet}} does not allow you to store the data in order.
If you're new to Scala, then I can recommend you reading
http://www.artima.com/pins1ed/. It's a good book even though it is getting a
bit long in the tooth.
> Add PCA to machine learning library
> -----------------------------------
>
> Key: FLINK-1733
> URL: https://issues.apache.org/jira/browse/FLINK-1733
> Project: Flink
> Issue Type: New Feature
> Components: Machine Learning Library
> Reporter: Till Rohrmann
> Assignee: Thang Nguyen
> Priority: Minor
> Labels: ML
>
> Dimension reduction is a crucial prerequisite for many data analysis tasks.
> Therefore, Flink's machine learning library should contain a principal
> components analysis (PCA) implementation. Maria-Florina Balcan et al. [1]
> proposes a distributed PCA. A more recent publication [2] describes another
> scalable PCA implementation.
> Resources:
> [1] [http://arxiv.org/pdf/1408.5823v5.pdf]
> [2] [http://ds.qcri.org/images/profile/tarek_elgamal/sigmod2015.pdf]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
