Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-14 Thread Herbert J. Bernstein
Dear James, Perhaps it is time for us to admit that this is too large, expensive and complex a problem for us to resolve without help from one or more of the commercial data managers, such as Google or Amazon. I know that dealing with ads is a nuisance, introducing a loss of time for research,

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-14 Thread James Holton
Why not just upload it to proteindiffraction.org ?  Or the SBGrid data bank (https://data.sbgrid.org/) ?  Or both for "redundancy" ? Yes, I did once do some calculations on what it would take to preserve data for tens of thousands of years, and the only proven storage medium for that

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread John R Helliwell
Dear Frank, CERN does save all its data. It also hosts the Zenodo science data archive. The Square Kilometre Array has decided to save processed data rather than its raw data as they have concluded that isn't possible. Best wishes, John Emeritus Professor John R Helliwell DSc > On 13 Jul 2018,

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread CHAVAS Leonard
Hi Frank on the same lines: do we keep our crystals after frying them? Or do molecular biologists keep their agarose gels? Hummm... evolution and technologies do progress. Yet, I would support keeping images, just as you may (but really will most likely never) want to re-process those with

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread Edward Snell
Of graeme.win...@diamond.ac.uk Sent: Friday, July 13, 2018 10:53 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Should we still keep copies of all raw data? Sergei, all, I for one would always recommend saving data - not least as this is the basis for your science and you may need to revisit

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread graeme.win...@diamond.ac.uk
Sergei, all, I for one would always recommend saving data - not least as this is the basis for your science and you may need to revisit it later, perhaps before or after publication. Re: (a) I know we’re all working on improving the methods but they are still have scope for improvement (b)

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread Frank von Delft
Are the LHC researchers that analyse the detector readout on the fly without ever storing the data also guilty of malpractice? Hardcore. Just a few more years, a few more Eiger detectors, a few more serial beamlines, a few more clusters and clouds, and a few more DIALS-style programmers,

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread Sergei Strelkov
Department of Pharmaceutical Sciences, KU Leuven <http://pharm.kuleuven.be/anafar> From: John R Helliwell Sent: Friday, July 13, 2018 12:07 To: Sergei Strelkov Cc: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Should we still keep copies of all raw data?

Re: [ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread John R Helliwell
Dear Sergei, Re “all”. As a researcher my perspective is that one’s funding agency requirement for a data management plan will be the core of what you would need to follow. Your employer may have additional policies and requirements placed on you as an employee. Eg the UK funding agency EPSRC

[ccp4bb] Should we still keep copies of all raw data?

2018-07-13 Thread Sergei Strelkov
Dear All, I believe this question may be of some interest. In the past, we always stored all raw data ever collected by the lab. With the recent advances, such as (a) automated/on-the-fly processing offered by some (European) synchrotrons, and (b) an ongoing discussion on centralized raw