Speaking statistically, AAD has some interesting issues when you are trying
to estimate a boolean value.
In this framework, you produce an estimate in [0,1] of an actual value that
is in {0,1}. If you guess binary results, you will have an error of 0 or 1.
If you guess intermediate values, you are guaranteed to have a non-zero
error. If the actual probability of getting a 1 is p and you estimate x,
then the expected value of AAD is (1-p) x + p (1-x) =
x-px + p-px = p + x (1-2p). If p<0.5, this is minimized by setting x = 0
and if p>0.5, by setting x = 1. Thus, you minimize AAD by only guessing
binary values (which our recommenders never do, btw).
As such, if you really want to use AAD as a quality metric, you may want to
put in a step that clamps the output to 0 or 1 before evaluating.
On Tue, Oct 25, 2011 at 1:05 PM, Sean Owen <[email protected]> wrote:
> Well it is also a property of the recommender. If you throw a "normal"
> implementation at your data it will happily estimate, correctly, that
> all unknown ratings are 1. it's these other variants that do something
> different and meaningful.
>
> The reverse is fine -- you can use similarity metrics that don't
> assume ratings on data that does have ratings.
>
> No, you're welcome to make comparisons in these tables. It's valid.
>
> On Tue, Oct 25, 2011 at 9:02 PM, lee carroll
> <[email protected]> wrote:
> > Ah you did not say boolean / non boolean recommenders, you talking
> > about boolean preference ratings.
> >
> > Ok I think I have it.
> >
> > I'm up to chapter 5 in the mahout in action book (so please bare with
> me:-)
> > So is it fair to say table 5.1 and 5.2 should avoid the comparissons
> > between the top two ?
> >
>
