On 8 May 2014 09:53, Andreas Kolbe <jayen...@gmail.com> wrote: > Yes Geni, absolutely. If I give Wikipedia's article on diabetes to three > acknowledged experts on diabetes for a detailed review, and they tell me at > the end of it that it is a wonderful, up-to-date and accurate article – or > they tell me that it contains numerous errors of fact – I won't have > learned anything. :) > > So you concede your approach has extremely limited resolution even under ideal conditions (that is where your testers actually completely agree with each other)?
Gets really fun when you discoverer that the numerous errors of fact are that it uses liters rather than decimeters cubed (in certian contexts liters is slightly ambiguous). > Incidentally, speaking of diabetes, one of the more striking hoaxes in > > https://en.wikipedia.org/wiki/Wikipedia:List_of_hoaxes_on_Wikipedia > > is "glucojasinogen". It lasted 4.5 years and entered several academic > sources that copied a section of the Wikipedia article, before someone > discovered that there was no such thing. > So what you are saying is that even experts aren't that great on raw fact checking. In fairness we know this. There is a reasons that when last june Organic Letters decided to do some serious fraud detection work on their spectra they went with a data analyst rather than a chemist, > One thing I would say is that if Wikipedia articles were to be compared > against articles from another source, they should have roughly the same > length. It's not fair to compare a 4,000-word article from Wikipedia > against a 500-word article from Britannica. That means you either end up artificially trimming articles (again with a significant risk of creating data anomalies) or have an even harder time getting your calibration curve in place. > Other than that, I think we > could leave the study design to those who do this sort of stuff for a > living. It's really not something you and I have to work out here on a > mailing list. You seem to think its straightforward. If you think that you should be able to propose a study design. _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
