Re: MLLib: implementing ALS with distributed matrix

2014-08-17 Thread Wei Tan
Hi Deb, thanks for sharing your result. Please find my comments inline in blue. Best regards, Wei From: Debasish Das To: Wei Tan/Watson/IBM@IBMUS, Cc: Xiangrui Meng , "user@spark.apache.org" Date: 08/17/2014 08:15 PM Subject: Re: MLLib: implementing ALS with d

Re: MLLib: implementing ALS with distributed matrix

2014-08-17 Thread Debasish Das
rson/us-wtan* > <http://researcher.ibm.com/person/us-wtan> > > > > From:Xiangrui Meng > To: Wei Tan/Watson/IBM@IBMUS, > Cc: "user@spark.apache.org" > Date:08/04/2014 12:51 AM > Subject:Re: MLLib: implementing ALS with d

Re: MLLib: implementing ALS with distributed matrix

2014-08-17 Thread Wei Tan
uot; Date: 08/04/2014 12:51 AM Subject: Re: MLLib: implementing ALS with distributed matrix To be precise, the optimization is not `get all products that are related to this user` but `get all products that are related to users inside this block`. So a product factor won't be s

Re: MLLib: implementing ALS with distributed matrix

2014-08-03 Thread Xiangrui Meng
To be precise, the optimization is not `get all products that are related to this user` but `get all products that are related to users inside this block`. So a product factor won't be sent to the same block more than once. We considered using GraphX to implement ALS, which is much easier to unders

MLLib: implementing ALS with distributed matrix

2014-08-03 Thread Wei Tan
Hi, I wrote my centralized ALS implementation, and read the distributed implementation in MLlib. It uses InLink and OutLink to implement functions like "get all products which are related to this user", and ultimately achieves model distribution. If we have a distributed matrix lib, the c