Just a minor correction: The Sparkler paper was done by IBM. IIRC they did not only implement the algorithm but also modified Spark to tune it for that usecase.
--sebastian On 03.01.2014 00:16, Debasish Das wrote: > Hi, > > I am not noticing any DSGD implementation of ALS in Spark. > > There are two ALS implementations. > > org.apache.spark.examples.SparkALS does not run on large matrices and seems > more like a demo code. > > org.apache.spark.mllib.recommendation.ALS looks feels more robust version > and I am experimenting with it. > > References here are Jellyfish, Twitter's implementation of Jellyfish called > Scalafish, Google paper called Sparkler and similar idea put forward by IBM > paper by Gemulla et al. (large-scale matrix factorization with distributed > stochastic gradient descent) > > https://github.com/azymnis/scalafish > > Are there any plans of adding DSGD in Spark or there are any existing JIRA ? > > Thanks. > Deb >
