[
https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12964475#action_12964475
]
Dmitriy Lyubimov edited comment on MAHOUT-376 at 11/27/10 7:27 PM:
-------------------------------------------------------------------
Added U and V computations. Labels of A rows are propagated to U rows as keys.
bumped up the test to A of dimensions of 100,000x100 (so it produces 3 A splts
now). Actually i just realize that technically U and V should be k columns
wide, and i am producing k+p. ok, who cares. Distinction between k and p
becomes really nominal now.
I'd appreciate if somebody reviews V computation (p.5.5.2 in working notes),
just in case, i already forget the derivation of V computation.
Another uncertainty i have is that i am not sure how to best construct tests
for U and V outputs, but i am not sure if i care that much since the
computation logic is really trivial (compared to everything else). Existing
tests verify singlular values against independent memory-only svd and assert
orthogonality of Q output only.
Stuff that is left is really minor and mahout-specific or engineering:
-- figure out how to integrate with mahout CLI
-- add minSplitSize parameter to CLI, and others (k,p, computeU, computeV, .. )
-- do we want to backport it to apache 0.20? multiple outputs are crippled in
0.20 and i don't know how one can live without multiple outputs in a job like
this.
was (Author: dlyubimov2):
Added U and V computations. Labels of A rows are propagated to U rows as
keys. bumped up the test to A of dimensions of 100,000x100 (so it produces 3 A
splts now). Actually i just realize that technically U and V should be k
columns wide, and i am producing k+p. ok, who cares. Distinction between k and
p becomes really nominal now.
I'd appreciate if somebody reviews V computation (p.5.5.2 in working notes),
just in case, i already forget the derivation of V computation.
Stuff that is left is really minor and mahout-specific or engineering:
-- figure out how to integrate with mahout CLI
-- add minSplitSize parameter to CLI, and others (k,p, computeU, computeV, .. )
-- do we want to backport it to apache 0.20? multiple outputs are crippled in
0.20 and i don't know how one can live without multiple outputs in a job like
this.
> Implement Map-reduce version of stochastic SVD
> ----------------------------------------------
>
> Key: MAHOUT-376
> URL: https://issues.apache.org/jira/browse/MAHOUT-376
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Assignee: Ted Dunning
> Fix For: 0.5
>
> Attachments: MAHOUT-376.patch, Modified stochastic svd algorithm for
> mapreduce.pdf, QR decomposition for Map.pdf, QR decomposition for Map.pdf, QR
> decomposition for Map.pdf, sd-bib.bib, sd.pdf, sd.pdf, sd.pdf, sd.pdf,
> sd.tex, sd.tex, sd.tex, sd.tex, SSVD working notes.pdf, SSVD working
> notes.pdf, ssvd-m1.patch.gz, ssvd-m2.patch.gz, Stochastic SVD using
> eigensolver trick.pdf
>
>
> See attached pdf for outline of proposed method.
> All comments are welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
