Or incremental SVD provides orthogonality of the singular vectors while LLL does not? (my best guess why they do it incrementally).
On Tue, May 24, 2011 at 6:11 PM, Dmitriy Lyubimov <[email protected]> wrote: > Thanks, Ted. > > On Tue, May 24, 2011 at 5:44 PM, Ted Dunning <[email protected]> wrote: >> Heh? >> >> Are you referring to the Log linear latent factor code that I have in my >> mahout-525 github repo? >> > > I am referring to LatentLogLinear class in your repo under lll branch. > >> >> >>> However, i never understood why this factorization must come up with r >>> best factors. I understand incremental SVD approach >>> (essentially the same thing except learning factors iteratively >>> guarantees we capture the best ones) but if we do it all in parallel, >>> does it create any good in your trials? >>> >> >> I don't understand the question. >> >> Are you asking whether the random projection code finds the best (largest) >> singular >> values and corresponding vectors? If so, the answer is yes, it does with >> high probability >> of low error. >> > > Well you have alternating scheme there, right? you do learn left > singular vectors, then you switch, find the right singular vectors, > but as far as i can tell you are not doing it the same way as > incremental SVD does > > Incremental SVD goes thru the entire dataset the same way but only for > 1 factor first. then it frozes it once testing rmse curve is flat and > starts doing the same for the second one. Intuitively it's clear that > the first pass this way finds the largest factor and the next one > finds the next largest etc. Hence there's a 'step' curve on RMSE chart > for this process as it switches from factor to factor. > > But in your case, it looks like you are learning all the factors at > once. Is it going to result into the same result as incremental SVD > algorithm? if yes, why did they even do it incrementally, for it's > clear incremental approach would require more iterations? > > (there's a mahout issue for incremental svd implementation btw). > >> >> Regarding the side information, I thought it was there but may have made a >> mistake. >> > > which branch should i look at for the latest code? i looked at LLL branch. > thanks. >
