If you minimize squared error of a reconstruction, then you inherently have orthogonal vectors (for the first several, anyway).
It doesn't matter what order you do it in if you actually reach the quadratic optimum. The SGD approach will have issues with the order of the vectors produced. On Tue, May 24, 2011 at 6:13 PM, Dmitriy Lyubimov <[email protected]> wrote: > Or incremental SVD provides orthogonality of the singular vectors > while LLL does not? (my best guess why they do it incrementally). > > On Tue, May 24, 2011 at 6:11 PM, Dmitriy Lyubimov <[email protected]> > wrote: > > Thanks, Ted. > > > > On Tue, May 24, 2011 at 5:44 PM, Ted Dunning <[email protected]> > wrote: > >> Heh? > >> > >> Are you referring to the Log linear latent factor code that I have in my > >> mahout-525 github repo? > >> > > > > I am referring to LatentLogLinear class in your repo under lll branch. > > > >> > >> > >>> However, i never understood why this factorization must come up with r > >>> best factors. I understand incremental SVD approach > >>> (essentially the same thing except learning factors iteratively > >>> guarantees we capture the best ones) but if we do it all in parallel, > >>> does it create any good in your trials? > >>> > >> > >> I don't understand the question. > >> > >> Are you asking whether the random projection code finds the best > (largest) > >> singular > >> values and corresponding vectors? If so, the answer is yes, it does > with > >> high probability > >> of low error. > >> > > > > Well you have alternating scheme there, right? you do learn left > > singular vectors, then you switch, find the right singular vectors, > > but as far as i can tell you are not doing it the same way as > > incremental SVD does > > > > Incremental SVD goes thru the entire dataset the same way but only for > > 1 factor first. then it frozes it once testing rmse curve is flat and > > starts doing the same for the second one. Intuitively it's clear that > > the first pass this way finds the largest factor and the next one > > finds the next largest etc. Hence there's a 'step' curve on RMSE chart > > for this process as it switches from factor to factor. > > > > But in your case, it looks like you are learning all the factors at > > once. Is it going to result into the same result as incremental SVD > > algorithm? if yes, why did they even do it incrementally, for it's > > clear incremental approach would require more iterations? > > > > (there's a mahout issue for incremental svd implementation btw). > > > >> > >> Regarding the side information, I thought it was there but may have made > a > >> mistake. > >> > > > > which branch should i look at for the latest code? i looked at LLL > branch. > > thanks. > > >
