OK yes you're on to something here. I should clarify. Koobas you are right that the ALS algorithm itself is fine here as far as my knowledge takes me. The thing it inverts to solve for a row of X is something like (Y' * Cu * Y + lambda * I). No problem there, and indeed I see why the regularization term is part of that.
I'm talking about a later step, after the factorization. You get a new row in A and want to solve A = X*Y' for X, given the current Y. (And vice versa). I'm using a QR decomposition for that, but not to directly solve the system (and this may be the issue), but instead to compute and save off (Y' * Y)^-1 so that we can figure A * Y * (Y'*Y)^-1 very fast at runtime. That is to say the problem centers around the inverse of Y'*Y and in this example, it does not even exist. I am not sure it's just a numerical precision thing since using an SVD to get the inverse gives the same result. But I certainly have examples where the data (A) is most certainly rank >> k and get this bad behavior -- for example, when lambda is very *high*. On Fri, Apr 5, 2013 at 6:57 AM, Ted Dunning <[email protected]> wrote: > On Fri, Apr 5, 2013 at 2:40 AM, Koobas <[email protected]> wrote: > >> Anyways, I saw no particular reason for the method to fail with k >> approaching or exceeding m and n. >> It does if there is no regularization. >> But with regularization in place, k can be pretty much anything. >> > > Ahh... this is an important point and it should handle all of the issues of > poor conditioning. > > The regularizer takes the rank deficient A and makes it reasonably well > conditioned. How well conditioned depends on the choice of lambda, the > regularizing scale constant.
