[
https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12967082#action_12967082
]
Dmitriy Lyubimov edited comment on MAHOUT-376 at 12/5/10 11:43 PM:
-------------------------------------------------------------------
Patch update.
* finalized mahout CLI integration & tested.
Tested on 10k x 10k dense matrix in distributed hadoop mode (compressed source
sequence file 743mb) on my 3 yo dual core. It is indeed, as i expected, quite
cpu-bound but good news is that it is so well parallelizable with most load on
map-only jobs that it should be no problem to redistribute and front end
doesn't require any or cpu capacity at all. Square symmetric matrix of 200x200
sizes computes instantaneously.
The command line i used was:
bin/mahout ssvd -i /mahout/ssvdtest/A -o /mahout/ssvd-out/1 -k 100 -p 100 -r
200 -t 2
I also was testing this with CDH3b3 setup.
was (Author: dlyubimov2):
Patch update.
* finalized mahout CLI integration & tested.
Tested on 10k x 10k dense matrix (compressed source sequence file 743mb) on my
3 yo dual core. It is indeed, as i expected, quite cpu-bound but good news is
that it is so well parallelizable with most load on map-only jobs that it
should be no problem to redistribute and front end doesn't require any or cpu
capacity at all. Square symmetric matrix of 200x200 sizes computes
instantaneously.
The command line i used was:
bin/mahout ssvd -i /mahout/ssvdtest/A -o /mahout/ssvd-out/1 -k 100 -p 100 -r
200 -t 2
I also was testing this with CDH3b3 setup.
> Implement Map-reduce version of stochastic SVD
> ----------------------------------------------
>
> Key: MAHOUT-376
> URL: https://issues.apache.org/jira/browse/MAHOUT-376
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Assignee: Ted Dunning
> Fix For: 0.5
>
> Attachments: MAHOUT-376.patch, Modified stochastic svd algorithm for
> mapreduce.pdf, QR decomposition for Map.pdf, QR decomposition for Map.pdf, QR
> decomposition for Map.pdf, sd-bib.bib, sd.pdf, sd.pdf, sd.pdf, sd.pdf,
> sd.tex, sd.tex, sd.tex, sd.tex, SSVD working notes.pdf, SSVD working
> notes.pdf, SSVD working notes.pdf, SSVD working notes.pdf,
> ssvd-CDH3-or-0.21.patch.gz, ssvd-CDH3-or-0.21.patch.gz,
> ssvd-CDH3-or-0.21.patch.gz, ssvd-CDH3-or-0.21.patch.gz, ssvd-m1.patch.gz,
> ssvd-m2.patch.gz, ssvd-m3.patch.gz, Stochastic SVD using eigensolver trick.pdf
>
>
> See attached pdf for outline of proposed method.
> All comments are welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
