On 7/22/2004 11:41 PM, Eric Wilhelm wrote:
# The following was supposedly scribed by # Randy W. Sims # on Thursday 22 July 2004 07:56 pm:
A) Abuse
Authors abusing the system for political statements, to sabatoge authors of similars modules, etc.
1) The usuall solution is a Karma type system. Number of reviews contributed by a reviewer. Thumbs up/down for individual reviews by a reviewer ("Helpfulness ratings"). Thresholds on Karma that automatically invoke a moderator.
Okay, so if I go on a bashing-fest and then you come through and thumbs-down all of my reviews, I'll go through and thumbs-down your reviews too and then bash on your modules if I haven't made it there already. Does that trigger a karma threshold of some sort? Seems that it would be hard to detect.
How about peer-review of peer-review:
If I say that your review was bad, I think the next step is for you to defend your review (unless it has previously gotten a thumbs-up, in which case I must support my thumbs-down with a critique of your review.)
There may be a somewhat recursive process of attack and rebuttal here, but the point is that a mean review is likely not going to be defended, and even if it had received a spurious thumbs-up, a critical dismissal of said mean review is likely to be supported rather than dismissed (thus giving weight to the dismissal and counting further towards the thumbs-down.)
Recursion to level 3-or-so (pi) of the attack-rebuttal tree may invoke a moderator (or just a chanting, blood-lusting crowd/mob.)
Additional weight can be given to reviewers who have posted many reviews and received many thumbs-up, etc. But, the idea behind the tree is that it localizes the debate to the review in question (rather than risk weighting solely on what may have been karma generated by a flaming disagreement about a completely different module's merits.)
Absolute dead-beats can still be identified by their failure to provide a rebuttal or continually reaching level pi() with nonsensical or null arguments.
I don't think anything this complicated or involved is needed. Neither am I convinced that the Karma solution is the best or a complete solution. However, This is pretty much the system that Amazon.com uses and it works pretty well IMO. Also, for this to get into back and forth arguments/bashing, the reviewers would have to attempt to track all of their reviews. The system would not provide an easy way to do that-there is no legitimate need for such a feature since the system is about reviews not reviewers. That's not to say it's not possible, but it's not easy either. I think that will discourage a large percentage of abuse, and high percentages is the best you can shoot for.
Regards, Randy.
