Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-12-02 Thread Markus Heckmann
Hi Raquel, Are u using a compressed filesystem? I recently moved everything including /home directory to ZFS - which gave ~ 1.4X compression for old adsc images. Remember vaguely, years before, James suggested to use aufs/unionfs. You could even enable data-deduplication to save redundant

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-30 Thread James Holton
The answer depends a lot on what you mean by "long-term storage". Do you want the data to be available all the time on a mountable volume?  Or is putting it away on a shelf OK?  Do you want the storage to be as bulletproof and worry-free as possible?  Or are you OK with the fate of your data

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-30 Thread Herbert J. Bernstein
Dear Colleagues, May I suggest that those who are at Universities take a look at the G-suite for Education https://edu.google.com/products/gsuite-for-education/editions/?modal_active=none which provides unlimited cloud storage for free to educational institutions. Regards, Herbert On

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Kay Diederichs
Wladek Minor runs proteindiffraction.org . This also has E.g. JCSG data sets. BR, Kay On Thu, 29 Nov 2018 23:35:55 +, Diana Tomchick wrote: >I hope you are compressing your images, typically that makes them 1/4 the >original size. > >SBGrid and Wladek Minor also have image archival

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Tim Grüne
lease >reply to this message and follow with its deletion, so that we can >ensure such a mistake does not occur in the future. > >-Original Message- >From: CCP4 bulletin board On Behalf Of >graeme.win...@diamond.ac.uk >Sent: Thursday, November 29, 2018 4:22 PM

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Tim Grüne
Hi Graeme, I meant to say that the zenodo server sits in Europe, and the sbgrif server in the us. It was not meant to indicate any restrictions. Best, Tim On November 29, 2018 10:21:36 PM GMT+01:00, "graeme.win...@diamond.ac.uk" wrote: >Dear Tim, > >I do not think Zenodo is limited to

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Diana Tomchick
I hope you are compressing your images, typically that makes them 1/4 the original size. SBGrid and Wladek Minor also have image archival services. As I am replying from my cell phone while on vacation, the links to those services are not handy to me. But they have been mentioned many times on

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Keller, Jacob
] Long term storage for raw images/ crystallographic data sets Dear Tim, I do not think Zenodo is limited to Europeans - at least I could not find this on their policy page: http://about.zenodo.org/policies/ I know of plenty of uploads from Japan for example Best wishes Graeme On 29 Nov 2018

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread graeme.win...@diamond.ac.uk
Dear Tim, I do not think Zenodo is limited to Europeans - at least I could not find this on their policy page: http://about.zenodo.org/policies/ I know of plenty of uploads from Japan for example Best wishes Graeme On 29 Nov 2018, at 21:16, Tim Gruene mailto:tim.gru...@psi.ch>> wrote: Dear

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Tim Gruene
Dear Raquel, when they are associated with a publication, you can publish them on data.sbgrid.org in the US or at zenodo.org in Europe. Best regards, Tim On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L wrote: > Dear All, > > How do your labs handle long-term raw data

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread graeme.win...@diamond.ac.uk
Dear Raquel, For published structures you can publish the raw data, which means that somebody else is looking after it - for this I would say that the current front runner is Zenodo - https://zenodo.org/ - which is paid for by CERN / EU etc. so someone else is (currently) picking up the tab.

[ccp4bb] Long term storage for raw images/ crystallographic data sets

2018-11-29 Thread Lieberman, Raquel L
Dear All, How do your labs handle long-term raw data backups? My lab is maxing out our 6TB RAID backup (with two off-site mirrors) so I am investigating our next long term solution. The vast majority of the data sets are published structures (i.e. processed data deposited in PDB) or