RowSimilarityJob computes the top-k similar rows to each row of the
input matrix. You can think of it as computing A'A and sparsifying the
result afterwards. Furthermore it allows to plug in a similarity measure
of your choice.
If you want to have a cooccurrence matrix, you can use
o.a.m.math.hadoop.similarity.cooccurrence.measures.CooccurrenceCountSimilarity
as similarity measure.
On 02.04.2013 23:43, Pat Ferrel wrote:
> Taking an idea from Ted, I'm working on a cross recommender starting from
> mahout's m/r implementation of an item-based recommender. We have purchases
> and views for items by user. It is straightforward to create a recommender on
> purchases but using views as a predictor of purchases does not work so
> well--giving us lower precision scores. This is, no doubt, because the events
> have a lot of noise, views that do not lead to purchases.
>
> To help solve this Ted suggests we think of a recommender in two parts:
>
> [B'B]h_p = r_p <== standard item-based recommender using purchases
> [B'A]h_v = r_v <== cross-recommender using views and purchases
> r = r_p + r_v <== linear combination of the two parts is the full
> recommendation vector
>
> These both make recommendations for purchases but method 2 makes cross
> recommendations based on views. [B'A] is the co-occurrence matrix of views
> with purchases.
>
> From RecommenderJob the 'similarity matrix' is created by:
>
> //calculate the co-occurrence matrix
> ToolRunner.run(getConf(), new RowSimilarityJob(), new String[]{
> "--input", new Path(prepPath,
> PreparePreferenceMatrixJob.RATING_MATRIX).toString(),
> "--output", similarityMatrixPath.toString(),
> "--similarityClassname", similarityClassname,
> …
>
> What is the role of RowSimilarityJob here and how does it lead to a
> co-occurrence matrix? I understand that in the general recommender the
> co-occurrence matrix is symmetric so columns = rows. Is the co-occurrence
> matrix actually calculated anywhere in the standard recommender?
>
> The output of PreparePreferenceMatrixJob is a DistributedRowMatrix. As a
> first cut it seems I can do the cross recommender part of the work by:
>
> //calculate the 'cross' co-occurrence matrix
> B = PreparePreferenceMatrixJob using user purchase prefs
> A = PreparePreferenceMatrixJob using user view prefs
> // note that users and items must be the same for A and B, their ids
> must map to the same things
> B' = TransposeJob on B
> [B'A] = MatrixMultJob on B', A
> [B'A]h_v by using the partial multiply process in the standard
> Recommender
> extract the needed recs
>
> Questions:
> * I need to get item similarities perhaps even more importantly than user
> history based recs. I use the [B'A] columns for this, right? Shouldn't I run
> RowSimilarityJob on [B'A]'?
> * There are assumptions in some code that the co-occurrence matrix is
> symmetric and so rows = columns. This is not true of the 'cross'
> co-occurrence matrix. Are there places I need to account for this?
>
