[
https://issues.apache.org/jira/browse/MAHOUT-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968664#action_12968664
]
Lance Norskog commented on MAHOUT-559:
--------------------------------------
Motivation:
I was hoping to find a "gold standard" recommender to compare with one I'm
writing.
The evaluator includes an option to print out all scores for every user in CSV
form. The output numbers are raw values. I have an Excel for those interested
in exploring. The bubble sort and statistical rank scores correlate quite well.
If the Hammiing had a variable-size sliding window it would probably correlate
with the other two.
I got all of the Recommenders in CompareRecommenders from examples. Their
outputs widely diverge. Usually they have two or three items in common among
the first 50 recommendations.
Also, it supports Recommender v.s. DataModel and DataModel v.s. DataModel.
> Compare Recommender output by order of recommendations.
> -------------------------------------------------------
>
> Key: MAHOUT-559
> URL: https://issues.apache.org/jira/browse/MAHOUT-559
> Project: Mahout
> Issue Type: New Feature
> Components: Collaborative Filtering
> Reporter: Lance Norskog
> Attachments: OrderBasedRecommenderEvaluator.patch
>
>
> The existing RecommenderEvaluator
> (AverageAbsoluteDifferenceRecommenderEvaluator.java) has a very limited API.
> It evaluates a Recommender's performance on a training v.s. test scenario. It
> does not allow comparing the outputs of different recommenders against the
> same data model. Also, I could not figure out how its comparison criteria.
> OrderBasedRecommenderEvaluator compares the output of two recommenders. It
> only checks the order of the items in the recommendations, ignoring the
> returned preference values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
