[
https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917392#action_12917392
]
Dmitriy Lyubimov commented on MAHOUT-376:
-----------------------------------------
Another detail in Q computation:
For now we assume that Q=(1/L)Q', where Q' is the output of block
orthonormalization, and L is the number of blocks we ended up with.
However, in case when A is rather sparse, some Y blocks may not end up spanning
R^(k+p) in which case orthogonalization would not produce normal columns. It
doesn't mean that we can't orthonormalize the entire Y though. It just means
that L is not for the entire Q but rather is individual for every column vector
of Q and our initial assumption Q=(1/L)Q' doesn't always hold.
Which is fine per se, but then it means that there are perhaps quite frequent
exceptions to assumption B =(1/L) Q'^transpose * A. It would be easy to compute
B^transpose in the same map-reduce pass as Q using multiple outputs , one for Q
and one for B-transpose (which is what i am doing right now). But unless
there's any workaround for the problem above, this B^transpose is incorrect for
some sparse cases.
Ted, any thoughts? Thank you.
-Dima
> Implement Map-reduce version of stochastic SVD
> ----------------------------------------------
>
> Key: MAHOUT-376
> URL: https://issues.apache.org/jira/browse/MAHOUT-376
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Assignee: Ted Dunning
> Fix For: 0.5
>
> Attachments: MAHOUT-376.patch, sd-bib.bib, sd.pdf, sd.tex, Stochastic
> SVD using eigensolver trick.pdf
>
>
> See attached pdf for outline of proposed method.
> All comments are welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
