Dear Ron, Quite so, and who cannot laugh at the Yes Minister perfect hospital ward operating theatre sketch ( Thankyou James W).
Anyway:- Let's not get too hung up on one detail of your point 3. Your various points, including point 3, added several missing elements in this CCp4bb thread. Overall what I am saying is that to me it is good that my University at least is gearing up to provide a local Data Archive service which, since I wish to link my raw data sets in future to my publications via doi registrations, this will give a longevity to them that I cannot guarantee with a 'my raw data are in my desk drawer' approach. These could be useful in future reuse ie:- I see various improvements to understanding diffuse scattering and, secondly, to squeezing more diffraction resolution out of the Bragg data as computer hardware and software both improve. Once in my career I nearly made a mistake on a space group choice ( I wrote it up as an educational story in 1996 in Acta D); if I had made that mistake the literature would finally have caught up i suppose and said :- where are the raw data, let's check that space group choice. This latter type of challenge of course is catchable via depositing processed Bragg data as triclinic; it probably doesn't need raw images. Finally I have a project that I have worked for some years on now to solve the structure; there are two, possibly several , crystal lattices and diffuse streaks. If I have to finally give up I will make them available via doi on my a university raw data archive; meanwhile of course we make new protein and recrystallise etc, the other approach! Greetings, John Prof John R Helliwell DSc FInstP CPhys FRSC CChem F Soc Biol. Chair School of Chemistry, University of Manchester, Athena Swan Team. http://www.chemistry.manchester.ac.uk/aboutus/athena/index.html On 6 Apr 2012, at 17:23, Ronald E Stenkamp <stenk...@u.washington.edu> wrote: > Dear John, > > Your points are well taken and they're consistent with policies and practices > in the US as well. > > I wonder about the nature of the employer's responsibility though. I sit on > some university committees, and the impression I get is that much of the > time, the employers are interested in reducing their legal liabilities, not > protecting the integrity of science. The end result is the same though in > that the employers get involved and oversee the handling of scientific > misconduct. > > What is unclear to me is whether the system for dealing with misconduct is > broken. It seems to work pretty well from my viewpoint. No system is > perfect for identifying fraud, errors, etc, and I understand the idea that > improvements might be possible. However, too many "improvements" might break > the system as well. > > Ron > > On Fri, 6 Apr 2012, John R Helliwell wrote: > >> Dear Ron, >> Re (3):- >> Yes of course the investigator has that responsibility. >> The additional point I would make is that the employer has a share in >> that responsibility. Indeed in such cases the employer university >> convenes a research fraud investigating committee to form the final >> judgement on continued employment. >> A research fraud policy, at least ours, also includes the need for >> avoding inadvertent loss of raw data, which is also deemed to be >> research malpractice. >> Thus the local data repository, with doi registration for data sets >> that underpin publication, seems to me and many others, ie in other >> research fields, a practical way forward for these data sets. >> It also allows the employer to properly serve the research >> investigations of its employees and be duely diligent to the research >> sponsors whose grants it accepts. That said there is a variation of >> funding that at least our UK agencies will commit to 'Data management >> plans'. >> Greetings, >> John >> >> >> >> 2012/4/5 Ronald E Stenkamp <stenk...@u.washington.edu>: >>> This discussion has been interesting, and it's provided an interesting >>> forum for those interested in dealing with fraud in science. I've not >>> contributed anything to this thread, but the message from Alexander Aleshin >>> prodded me to say some things that I haven't heard expressed before. >>> >>> 1. The sky is not falling! The errors in the birch pollen antigen pointed >>> out by Bernhard are interesting, and the reasons behind them might be >>> troubling. However, the self-correcting functions of scientific research >>> found the errors, and current publication methods permitted an airing of >>> the problem. It took some effort, but the scientific method prevailed. >>> >>> 2. Depositing raw data frames will make little difference in identifying >>> and correcting structural problems like this one. Nor will new >>> requirements for deposition of this or that detail. What's needed for >>> finding the problems is time and interest on the part of someone who's able >>> to look at a structure critically. Deposition of additional information >>> could be important for that critical look, but deposition alone (at least >>> with today's software) will not be sufficient to find incorrect structures. >>> >>> 3. The responsibility for a fraudulent or wrong or poorly-determined >>> structure lies with the investigator, not the society of crystallographers. >>> My political leanings are left-of-central, but I still believe in >>> individual responsibility for behavior and actions. If someone messes up a >>> structure, they're accountable for the results. >>> >>> 4. Adding to the deposition requirements will not make our science more >>> efficient. Perhaps it's different in other countries, but the >>> administrative burden for doing research in the United States is growing. >>> It would be interesting to know the balance between the waste that comes >>> from a wrong structure and the waste that comes from having each of us deal >>> with additional deposition requirements. >>> >>> 5. The real danger that arises from cases of wrong or fraudulent science >>> is that it erodes the trust we have in each others results. No one has >>> time or resources to check everything, so science is based on trust. There >>> are efforts underway outside crystallographic circles to address this >>> larger threat to all science, and we should be participating in those >>> discussions as much as possible. >>> >>> Ron >>> >>> On Thu, 5 Apr 2012, aaleshin wrote: >>> >>>> Dear John,Thank you for a very informative letter about the IUCr >>>> activities towards archiving the experimental >>>> data. I feel that I did not explain myself properly. I do not object >>>> archiving the raw data, I just believe >>>> that current methodology of validating data at PDB is insufficiently >>>> robust and requires a modification. >>>> Implementation of the raw image storage and validation will take a >>>> considerable time, while the recent >>>> incidents of a presumable data frauds demonstrate that the issue is >>>> urgent. Moreover, presenting the >>>> calculated structural factors in place of the experimental data is not the >>>> only abuse that the current >>>> validation procedure encourages to do. There might be more numerous >>>> occurances of data "massaging" like >>>> overestimation of the resolution or data quality, the system does not >>>> allow to verify them. IUCr and PDB >>>> follows the American taxation policy, where the responsibility for a fraud >>>> is placed on people, and the agency >>>> does not take sufficient actions to prevent it. I believe it is >>>> inefficient and inhumane. Making a routine >>>> check of submitted data at a bit lower level would reduce a temptation to >>>> overestimate the unclearly defined >>>> quality statistics and make the model fabrication more difficult to >>>> accomplish. Many people do it unknowingly, >>>> and catching them afterwards makes no good. >>>> >>>> I suggested to turn the current incidence, which might be too complex for >>>> burning heretics, into something >>>> productive that is done as soon as possible, something that will prevent >>>> fraud from occurring. >>>> >>>> Since my persistent "trolling" at ccp4bb did not take any effect (until >>>> now), I wrote a "bad-English" letter >>>> to the PDB administration, encouraging them to take urgent actions. Those >>>> who are willing to count grammar >>>> mistakes in it can reading the message below. >>>> >>>> With best regards, >>>> Alexander Aleshin, staff scientist >>>> Sanford-Burnham Medical Research Institute >>>> 10901 North Torrey Pines Road >>>> La Jolla, California 92037 >>>> >>>> Dear PDB administrators; >>>> >>>> I am wringing to you regarding the recently publicized story about >>>> submission of calculated structural factors >>>> to the PDB entry 3k79 >>>> (http://journals.iucr.org/f/issues/2012/04/00/issconts.html). This >>>> presumable fraud (or >>>> a mistake) occurred just several years after another, more massive >>>> fabrication of PDB structures (Acta Cryst. >>>> (2010). D66, 115) that affected many scientists including myself. The >>>> repetitiveness of these events indicates >>>> that the current mechanism of structure validation by PDB is not >>>> sufficiently robust. Moreover, it is >>>> completely incapable of detecting smaller mischief such as overestimation >>>> of the data resolution and quality. >>>> >>>> There are two approaches to handling fraud problems: (1) >>>> raising policing and punishment, or (2) >>>> making a fraud too difficult to implement. Obviously, the second approach >>>> is more humane and efficient. >>>> >>>> This issue has been discussed on several occasions by the >>>> ccp4bb community, and some members began >>>> promoting the idea of submitting raw crystallographic images as a fraud >>>> repellent. However, this validation >>>> approach is not easy and cheap, moreover, it requires a considerable >>>> manpower to conduct it on a day-to-day >>>> basis. Indeed, indexing data sets is sometimes a nontrivial problem and >>>> cannot be accomplished automatically. >>>> For this reason, submitting the indexed and partially integrated data >>>> (such as .x files from HKL2000 or the >>>> output.mtz file from Mosfilm) appears as a cheaper substitute to the image >>>> storing/validating. >>>> >>>> Analysis of the partially integrated data provides almost same >>>> means to the fraud prevention as >>>> the images. Indeed, the observed cases of data fraud suggest that they >>>> would likely be attempted by a >>>> biochemist-crystallographer, who is insufficiently educated to fabricate >>>> the partially processed data. A >>>> method developer, on contrary, does not have a reasonable incentive to >>>> forge a particular structure, unless he >>>> teams up with a similarly minded biologist. But the latter scenario is >>>> very improbable and has not been >>>> detected yet. >>>> >>>> The most valuable benefit in using the partially processed >>>> data as a validation tool would be the >>>> standardization of definition for the data resolution and detection of >>>> inappropriate massaging of experimental >>>> data. >>>> >>>> Implementation of this approach requires minuscule adaptation >>>> of the current system, which most of >>>> practicing crystallographers would accept (in my humble opinion). The >>>> requirement to the data storage would be >>>> only ~1000 fold higher than the current one, and transferring the new data >>>> to PDB could be still done over the >>>> Internet. Moreover, storing the raw data is not required after the >>>> validation is done. >>>> >>>> A program such as Scala of CCP4 could be easily adopted to >>>> process the validation data and compare >>>> them with a conventional set of structural factors. Precise consistency >>>> of the two sets is not necessary. >>>> They only need to agree within statistically meaningful boundaries, and if >>>> they don’t, the author could be >>>> asked to provide a detailed algorithm of his/her data processing. Finally, >>>> the standardized method could be >>>> used to determine the resolution of submitted data, which could be >>>> reported together with values provided by >>>> the author. >>>> >>>> To implement this validation approach, PDB would need to raise >>>> some funds, but small enough to be >>>> sacrificed out of our common feeder. Anyway, it is easier and cheaper than >>>> the raw image approach and can >>>> serve as a basis for a transfer to it in a future (if required). Since it >>>> appears to be a joined project to >>>> CCP4 and PDB, I ask all crystallographers, who feel an urgent need for >>>> upgrading the structure validation >>>> protocol, to encourage them to consider this issue as quickly as possible. >>>> People who commit crimes are not >>>> always bad people; lets show our governments a good way to handle this >>>> problem. >>>> >>>> >>>> >>>> Sincerely, >>>> >>>> Alexander Aleshin, Staff Scientist >>>> >>>> Sanford-Burnham Institute for Medical Research, >>>> >>>> La Jolla, CA, USA. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >> >> >> >> -- >> Professor John R Helliwell DSc >>
