Here you go, initial draft. It is basically done to your spec except i am using in-place amended gramSchmidt and correct for the number of blocks and make sure that at any time mapper can collect at least k+p rows for the next block.
-d On Sun, Oct 10, 2010 at 10:40 AM, Ted Dunning <[email protected]> wrote: > Nice work dima! > > Can you write a paragraph showing just what your current intended > algorithm. I can run a few checks in r. > > Sent from my iPhone > > On Oct 10, 2010, at 9:53 AM, "Dmitriy Lyubimov (JIRA)" <[email protected]> > wrote: > > > > > [ > https://issues.apache.org/jira/browse/MAHOUT-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919567#action_12919567] > > > > Dmitriy Lyubimov edited comment on MAHOUT-376 at 10/10/10 12:52 PM: > > -------------------------------------------------------------------- > > > > ok, i got to BBt normalized for rank deficiences in Q blocks. > > > > What eigensolver should i use in context of Mahout? I know there's > something in that Cern library but it is all deprecateed. There's also a > solver in apache commons, but it is not readily imported into the mahout > core. > > > > Thanks. > > > > PS. > > > > so i got to singular values now. I run a unit test so that k+p=n. When i > parameterize algorithm so that only one Q-block is produced , the > eigenvalues match the stock result at least as good as 10E-5. Which is > expected under the circumstances. however as soon as i increase number of > Q-blocks >1, the singular values go astray as much as 10%. Not good. In both > cases, the entire Q passes the orthonormality test. I guess it means that as > i thought before, doing block orthonormalization this way does result in a > subspace different from original span by Y.I need to research on doing > orthonormalization with blocks. I think that's the only showstopper here > that is still left. It may result in a rewrite that splits one job producing > both Q and Bt, into several though. > > > > was (Author: dlyubimov2): > > ok, i got to BBt normalized for rank deficiences in Q blocks. > > > > What eigensolver should i use in context of Mahout? I know there's > something in that Cern library but it is all deprecateed. There's also a > solver in apache commons, but it is not readily imported into the mahout > core. > > > > Thanks. > > > > PS. i tried EigenvalueDecomposition from cern library. I know i am quite > likely providing an ill-conditioned matrix in my unit test but strange thing > is that eigen values are not always in descending order. i thought they were > supposed to come out in descending order? > > > >> Implement Map-reduce version of stochastic SVD > >> ---------------------------------------------- > >> > >> Key: MAHOUT-376 > >> URL: https://issues.apache.org/jira/browse/MAHOUT-376 > >> Project: Mahout > >> Issue Type: Improvement > >> Components: Math > >> Reporter: Ted Dunning > >> Assignee: Ted Dunning > >> Fix For: 0.5 > >> > >> Attachments: MAHOUT-376.patch, sd-bib.bib, sd.pdf, sd.tex, > Stochastic SVD using eigensolver trick.pdf > >> > >> > >> See attached pdf for outline of proposed method. > >> All comments are welcome. > > > > -- > > This message is automatically generated by JIRA. > > - > > You can reply to this email to add a comment to the issue online. > > >
