Thanks for clarification, Owen! > ALS starts from a random solution and this will result in a different > solution. The overall problem is non-convex and the process will not > necessarily converge to the same solution. But doesn't alternation guarantee convexity? > Randomness is a common feature of machine learning: centroid selection > in k-means, the 'stochastic' in SGD, random forests, etc. I don't > think the question is why randomness is useful right? It isn't! :)
> For ALS... I don't quite understand the question, what's the > alternative? certainly I have always seen it formulated in terms of a > random initial solution. You don't want to always start from the same > point because of local minima. Ideally you start from many points and > take the best solution. Yeah, but then you start dealing with another problem, how to blend all results together and how doing this affects overall quality of results (in our case recommendations), right? But that's another story, I agree. In general I understood the reasons behind seeding the matrix values. > > On Mon, Jun 24, 2013 at 11:22 PM, Ted Dunning <[email protected]> wrote: > > This is a common chestnut that gets trotted out commonly, but I doubt that > > the effects that the OP was worried about where on the same scale. > > Non-commutativity of FP arithmetic on doubles rarely has a very large > > effect. > > > > > > On Mon, Jun 24, 2013 at 11:17 PM, Michael Kazekin > > <[email protected]>wrote: > > > >> Any algorithm is non-deterministic because of non-deterministic behavior > >> of underlying hardware, of course :) But that's an offtop. I'm talking > >> about specific implementation of specific algorithm, and in general I'd > >> like to know that at least some very general properties of the algorithm > >> implementation conserve (and why did authors added intentional > >> non-deterministic component to implementation).
