I am pretty sure that a "90% wrong" figure would fail an elementary statistical test of significance...
On Sun, Jun 1, 2014 at 12:55 PM, James Heilman <jmh...@gmail.com> wrote: > The journal article by Hasty et al published on May 1st 2014 basically took > ten Wikipedia articles and ten “researchers” (either medical students or > residents). Each Wikipedia article was then assessed by two of these > researchers to try to determine how many statements of fact they contained. > The first issue was that the number of statements of fact each reviewer > found sometimes differed by nearly 100%. They than took these individual > facts and the “researchers” compared them with the peer reviewed literature > as found on pubmed or the medical website Uptodate. They did not check to > see if the sources Wikipedia was using were high quality or were accurately > reflected. Additionally medical students and residents are hardly experts > in medical research. > > No errors in Wikipedia are mentioned directly in the original journal > article. When I spoke with the lead author he declined to release the > underlying data for us at Wikipedia to correct the “errors” they had found > stating that he may 1) wish to publish more on the topic and 2) wished to > protect the researchers. So much for independent verifiability in science. > Hasty did make some claims to the popular press about errors on Wikipedia. > Some of the facts he mentioned however accurately reflected some of the > best available peer reviewed sources. For example he claimed that blood > pressure should only be checked twice to make the diagnosis of hypertension > and that when we state three times we are wrong. However look at the > National Institute for Health and Care Excellence (previous known as the > National Institute of Clinical Health / NICE) on page 7 in this document > http://www.nice.org.uk/nicemedia/pdf/CG18background.pdf It is thus a > little > ironic that the Telegraph, a UK paper, repeated this incorrect statement > and the BBC covered the story so uncritically. > > Wikipedia has strong recommendations for what counts as a suitable source. > We recommend the use of secondary sources published in well respected > journals from the last 3-5 years, position statements of national or > internationally recognized medical bodies or major textbooks. Is Wikipedia > a perfect source? No, but it is just as good as many and better than most > other sources out there. Or else why would the world be using it? Hasty's > work did not have a comparison group. Basically he invented a new method to > test the quality of medical content and then only applied this new method > to one source, Wikipedia. Without a comparator this single data point is > meaningless. I am curious what he would have found if he would have applied > this to a NICE guideline or emedicine? > > We recently surveyed our top contributors and asked about their > backgrounds. What we found was that 52% have either a masters, PhD, or MD. > Another 33% have a BSc. About half are health care providers. 82% are male, > 9% are female and 9% classified themselves as other or would rather not > say. This is very similar to results published by Nusa Faric in her > master's thesis. Additionally we are working with a number of organizations > including: the National Institute of Health, the Cochrane collaboration, > and the UCSF college of medicine among others to improve Wikipedia’s health > care content. > > What Hasty did show was 1) the peer reviewed literature does not agree with > itself (ie different peer reviewed sources come to different conclusions > which is no surprise to anyone that has read much of it) 2) the peer review > process is sometimes flawed as he was able to publish a "peer reviewed" > article whose data does not support its conclusions. As someone who has > read a lot of the peer reviewed literature this is also not surprising. > > > -- > James Heilman > MD, CCFP-EM, Wikipedian > > The Wikipedia Open Textbook of Medicine > www.opentextbookofmedicine.com > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > Wikimedia-l@lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
