not sure about numerical stability with 100% streaming though.
On Tue, Oct 26, 2010 at 12:22 PM, Dmitriy Lyubimov <[email protected]>wrote: > Just an update. > > I am still working out a version that would do 100% mapreduce version of Q > orhonormalization . I think i made some progress with the prototype, and > basically there seems also to be a way to drastically cut on the output of > the first map reduce (i.e. reduce amount of partial B^t outer products) > depending on how much one wants to compromise the stochastic effect produced > by Y. The algorithm also significantly simplifies the orthogonolization > process by basically getting rid of Y-blocking entirely inside the same > mapper (i.e. it's 100% streaming, 1 Q-block per mapper with memory > requirements still constant to size of a row of A). > > It's slow due to my family situtation (i had just several hours to work on > non-mr prototype so far, which passed Q orthogonality test, everything else > is just a thought experiment) > > > On Sun, Oct 24, 2010 at 12:11 AM, Ted Dunning (JIRA) <[email protected]>wrote: > >> >> [ >> https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel] >> >> Ted Dunning updated MAHOUT-376: >> ------------------------------- >> >> Attachment: sd.tex >> sd.pdf >> >> Updated version. >> >> I think that this is actually a feasible algorithm. >> >> > Implement Map-reduce version of stochastic SVD >> > ---------------------------------------------- >> > >> > Key: MAHOUT-376 >> > URL: https://issues.apache.org/jira/browse/MAHOUT-376 >> > Project: Mahout >> > Issue Type: Improvement >> > Components: Math >> > Reporter: Ted Dunning >> > Assignee: Ted Dunning >> > Fix For: 0.5 >> > >> > Attachments: MAHOUT-376.patch, Modified stochastic svd algorithm >> for mapreduce.pdf, sd-bib.bib, sd.pdf, sd.pdf, sd.pdf, sd.pdf, sd.tex, >> sd.tex, sd.tex, sd.tex, Stochastic SVD using eigensolver trick.pdf >> > >> > >> > See attached pdf for outline of proposed method. >> > All comments are welcome. >> >> -- >> This message is automatically generated by JIRA. >> - >> You can reply to this email to add a comment to the issue online. >> >> >
