I am pretty sure that a "90% wrong" figure would fail an elementary
statistical test of significance...

On Sun, Jun 1, 2014 at 12:55 PM, James Heilman <jmh...@gmail.com> wrote:

> The journal article by Hasty et al published on May 1st 2014 basically took
> ten Wikipedia articles and ten “researchers” (either medical students or
> residents). Each Wikipedia article was then assessed by two of these
> researchers to try to determine how many statements of fact they contained.
> The first issue was that the number of statements of fact each reviewer
> found sometimes differed by nearly 100%. They than took these individual
> facts and the “researchers” compared them with the peer reviewed literature
> as found on pubmed or the medical website Uptodate. They did not check to
> see if the sources Wikipedia was using were high quality or were accurately
> reflected. Additionally medical students and residents are hardly experts
> in medical research.
> No errors in Wikipedia are mentioned directly in the original journal
> article. When I spoke with the lead author he declined to release the
> underlying data for us at Wikipedia to correct the “errors” they had found
> stating that he may 1) wish to publish more on the topic and 2) wished to
> protect the researchers. So much for independent verifiability in science.
> Hasty did make some claims to the popular press about errors on Wikipedia.
> Some of the facts he mentioned however accurately reflected some of the
> best available peer reviewed sources. For example he claimed that blood
> pressure should only be checked twice to make the diagnosis of hypertension
> and that when we state three times we are wrong. However look at the
> National Institute for Health and Care Excellence (previous known as the
> National Institute of Clinical Health / NICE) on page 7 in this document
> http://www.nice.org.uk/nicemedia/pdf/CG18background.pdf It is thus a
> little
> ironic that the Telegraph, a UK paper, repeated this incorrect statement
> and the BBC covered the story so uncritically.
> Wikipedia has strong recommendations for what counts as a suitable source.
> We recommend the use of secondary sources published in well respected
> journals from the last 3-5 years, position statements of national or
> internationally recognized medical bodies or major textbooks. Is Wikipedia
> a perfect source? No, but it is just as good as many and better than most
> other sources out there. Or else why would the world be using it? Hasty's
> work did not have a comparison group. Basically he invented a new method to
> test the quality of medical content and then only applied this new method
> to one source, Wikipedia. Without a comparator this single data point is
> meaningless. I am curious what he would have found if he would have applied
> this to a NICE guideline or emedicine?
> We recently surveyed our top contributors and asked about their
> backgrounds. What we found was that 52% have either a masters, PhD, or MD.
> Another 33% have a BSc. About half are health care providers. 82% are male,
> 9% are female and 9% classified themselves as other or would rather not
> say. This is very similar to results published by Nusa Faric in her
> master's thesis. Additionally we are working with a number of organizations
> including: the National Institute of Health, the Cochrane collaboration,
> and the UCSF college of medicine among others to improve Wikipedia’s health
> care content.
> What Hasty did show was 1) the peer reviewed literature does not agree with
> itself (ie different peer reviewed sources come to different conclusions
> which is no surprise to anyone that has read much of it) 2) the peer review
> process is sometimes flawed as he was able to publish a "peer reviewed"
> article whose data does not support its conclusions. As someone who has
> read a lot of the peer reviewed literature this is also not surprising.
> --
> James Heilman
> MD, CCFP-EM, Wikipedian
> The Wikipedia Open Textbook of Medicine
> www.opentextbookofmedicine.com
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
> Wikimedia-l@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
Wikimedia-l mailing list, guidelines at: 
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 

Reply via email to