2009/4/6 Tom Steinberg <[email protected]>: > Harry has produced the distribution of ratings for images that have > been rated at least 3 times: > > 1 91 > 2 509 > 3 589 > 4 611 > 5 563 > 6 367 > 7 203 > 8 85 > 9 16 > 10 1 > > Add that to the debate! >
Cool. Does each picture occur multiple times in the above distribution? The usual objection to 1-10 ratings in this kind of work (with my stats consultant hat on) is that you can't meaningfully add the results of each rating (without some cross reference between a rater's different ratings). A picture rated 2,3,4 cannot meaningfully be assigned an "average" rating of "3" because there is no basis for summing the ratings (which are really best thought of as ordinal data). That doesn't make it useless, but it means that you have to use caution when processing the results. For what the site is due to be used for something like Maxdiff (best and worst) might actually produce the results you want faster, but I haven't though that through. The distribution you give suggests there will be a problem (assuming that the distribution continues) in producing a less usefully fine gradation of places than you might like. You really only have a scale of 2-7. Comparative rating systems might give you a much better scaling. I still think a factor analysis would be more useful/interesting/great and much easier to run on comparative rather than ordinal ratings, but that's just me dreaming of something wonderful as opposed to more straightforwardly usable. -- Francis Davey _______________________________________________ Mailing list [email protected] Archive, settings, or unsubscribe: https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public
