Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10411#discussion_r52728349
--- Diff: docs/ml-collaborative-filtering.md ---
@@ -0,0 +1,148 @@
+---
+layout: global
+title: Collaborative Filtering - spark.ml
+displayTitle: Collaborative Filtering - spark.ml
+---
+
+* Table of contents
+{:toc}
+
+## Collaborative filtering
+
+[Collaborative
filtering](http://en.wikipedia.org/wiki/Recommender_system#Collaborative_filtering)
+is commonly used for recommender systems. These techniques aim to fill in
the
+missing entries of a user-item association matrix. `spark.ml` currently
supports
+model-based collaborative filtering, in which users and products are
described
+by a small set of latent factors that can be used to predict missing
entries.
+`spark.ml` uses the [alternating least squares
+(ALS)](http://dl.acm.org/citation.cfm?id=1608614)
+algorithm to learn these latent factors. The implementation in `spark.ml`
has the
+following parameters:
+
+* *numBlocks* is the number of blocks the users and items will be
partitioned into in order to parallelize computation (defaults to 10).
+* *rank* is the number of latent factors in the model (defaults to 10).
+* *maxIter* is the maximum number of iterations to run (defaults to 10).
+* *regParam* specifies the regularization parameter in ALS (defaults to
1.0).
+* *implicitPrefs* specifies whether to use the *explicit feedback* ALS
variant or one adapted for
+ *implicit feedback* data (defaults to `false` which means using
*explicit feedback*).
+* *alpha* is a parameter applicable to the implicit feedback variant of
ALS that governs the
+ *baseline* confidence in preference observations (defaults to 1.0).
+* *nonnegative* specifies whether or not to use nonnegative constraints
for least squares (defaults to `false`).
+
+### Explicit vs. implicit feedback
+
+The standard approach to matrix factorization based collaborative
filtering treats
+the entries in the user-item matrix as *explicit* preferences given by the
user to the item.
+
+It is common in many real-world use cases to only have access to *implicit
feedback* (e.g. views,
+clicks, purchases, likes, shares etc.). The approach used in `spark.ml` to
deal with such data is taken
+from
+[Collaborative Filtering for Implicit Feedback
Datasets](http://dx.doi.org/10.1109/ICDM.2008.22).
+Essentially instead of trying to model the matrix of ratings directly,
this approach treats the data
+as a combination of binary preferences and *confidence values*. The
ratings are then related to the
--- End diff --
This might just be my own way of wording it, but the input is construed as
some kind of _strength_ value in implicit data. It's inherently count-like
(e.g. additive) which is how it differs from ratings. The idea of confidence is
pretty much an implementation detail. I would not say that "ratings are related
to.." anything in this model; there are no rating-like quantities. It's not
predicting the strength of a preference, really, but how much it's likely to
exist.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
