Re: mllib.recommendation Design

2015-03-30 Thread Xiangrui Meng
On Wed, Mar 25, 2015 at 7:59 AM, Debasish Das debasish.da...@gmail.com wrote: Hi Xiangrui, I am facing some minor issues in implementing Alternating Nonlinear Minimization as documented in this JIRA due to the ALS code being in ml package: https://issues.apache.org/jira/browse/SPARK-6323 I

Re: mllib.recommendation Design

2015-03-30 Thread Debasish Das
For alm I have started experimenting with the following: 1. rmse and map improvement from loglikelihood loss over least square loss. 2. Factorization for datasets that are not ratings (basically improvement over implicit ratings) 3. Sparse topic generation using plsa. We are directly optimizing

Re: mllib.recommendation Design

2015-03-25 Thread Debasish Das
Hi Xiangrui, I am facing some minor issues in implementing Alternating Nonlinear Minimization as documented in this JIRA due to the ALS code being in ml package: https://issues.apache.org/jira/browse/SPARK-6323 I need to use Vectors.fromBreeze / Vectors.toBreeze but they are package private on

Re: mllib.recommendation Design

2015-02-17 Thread Debasish Das
There is a usability difference...I am not sure if recommendation.ALS would like to add both userConstraint and productConstraint ? GraphLab CF for example has it and we are ready to support all the features for modest ranks where gram matrices can be made... For large ranks I am still working on

Re: mllib.recommendation Design

2015-02-17 Thread Xiangrui Meng
The current ALS implementation allow pluggable solvers for NormalEquation, where we put CholeskeySolver and NNLS solver. Please check the current implementation and let us know how your constraint solver would fit. For a general matrix factorization package, let's make a JIRA and move our