Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Markus Heckmann]

Hi Raquel,
Are u using a compressed filesystem? I recently moved everything including
/home directory to ZFS - which gave ~   1.4X compression for old adsc
images. Remember vaguely, years before, James suggested to use
aufs/unionfs. You could even enable data-deduplication to save redundant
images.

In additon to James' suggestion of amazon glacier i would recommend
'backblaze'. I mainly use it for personal backup $50/year - but storage is
unlimited. You can also get a drive FedEx -ed for retrieval. But be warned
thst the GUI sucks.

One could go for the business plan - clean commandline API based
upload/download ~ about 350 per year.

If anyone is interested 'backblaze' produce fantastic harddrive statistics.

https://www.backblaze.com/blog/2018-hard-drive-failure-rates/

Markus


On Friday, November 30, 2018, James Holton <
270165b9f4cf-dmarc-requ...@jiscmail.ac.uk> wrote:

> The answer depends a lot on what you mean by "long-term storage".  Do you
> want the data to be available all the time on a mountable volume?  Or is
> putting it away on a shelf OK?  Do you want the storage to be as
> bulletproof and worry-free as possible?  Or are you OK with the fate of
> your data being somewhat nebulous, like in "the cloud"?  The price points
> for all these things are very different.
>
> You can now buy a single 8 TB drive for $230 USD.  LTO6 tapes are
> currently at ~7 USD/TB.  Both of these are the current lowest price/TB for
> disk and tape.  Using the media, of course, generally requires attaching it
> to a server that costs ~$5k-$10k USD.  Amazon Glacier is free for uploads
> and essentially free for downloading it back as long as you don't want more
> than 1 GB per month.  The other extreme is a NetApp, where you just want a
> turnkey system that keeps your data as safe as possible, but is also really
> fast.
>
> What do I do?  I am currently deploying a RAID6 array of 8 TB drives for
> high-performance storage.  For archiving I used to use DVD-R, but that
> can't keep up with a Pilatus, so now I'm on LTO6 tapes for off-line
> backups.  I know tapes are famous as "write-only media", but so far over
> the last 10 years I haven't had any real trouble reading back an old LTO
> tape.  
>
> -James Holton
> MAD Scientist
>
>
> On 11/29/2018 12:54 PM, Lieberman, Raquel L wrote:
>
> Dear All,
>
> How do your labs handle long-term raw data backups? My lab is maxing out
> our 6TB RAID backup (with two off-site mirrors) so I am investigating our
> next long term solution. The vast majority of the data sets are published
> structures (i.e. processed data deposited in PDB) or redundant/unusable so
> immediate access is not anticipated, but the size of data sets is
> increasing quickly with time, so I am looking for a scalable-yet-affordable
> solution.
>
> Would be grateful for input into various options, e.g. bigger HD/RAIDs,
> cloud backup, tape, anything else.
>
> I will compile.
>
> Thank you,
>
> Raquel
> --
> Raquel L. Lieberman, Ph.D.
> Professor
> School of Chemistry and Biochemistry
> Georgia Institute of Technology
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[James Holton]

The answer depends a lot on what you mean by "long-term storage". Do you 
want the data to be available all the time on a mountable volume?  Or is 
putting it away on a shelf OK?  Do you want the storage to be as 
bulletproof and worry-free as possible?  Or are you OK with the fate of 
your data being somewhat nebulous, like in "the cloud"?  The price 
points for all these things are very different.


You can now buy a single 8 TB drive for $230 USD.  LTO6 tapes are 
currently at ~7 USD/TB.  Both of these are the current lowest price/TB 
for disk and tape.  Using the media, of course, generally requires 
attaching it to a server that costs ~$5k-$10k USD.  Amazon Glacier is 
free for uploads and essentially free for downloading it back as long as 
you don't want more than 1 GB per month.  The other extreme is a NetApp, 
where you just want a turnkey system that keeps your data as safe as 
possible, but is also really fast.


What do I do?  I am currently deploying a RAID6 array of 8 TB drives for 
high-performance storage.  For archiving I used to use DVD-R, but that 
can't keep up with a Pilatus, so now I'm on LTO6 tapes for off-line 
backups.  I know tapes are famous as "write-only media", but so far over 
the last 10 years I haven't had any real trouble reading back an old LTO 
tape.  


-James Holton
MAD Scientist


On 11/29/2018 12:54 PM, Lieberman, Raquel L wrote:

Dear All,

How do your labs handle long-term raw data backups? My lab is maxing 
out our 6TB RAID backup (with two off-site mirrors) so I am 
investigating our next long term solution. The vast majority of the 
data sets are published structures (i.e. processed data deposited in 
PDB) or redundant/unusable so immediate access is not anticipated, but 
the size of data sets is increasing quickly with time, so I am looking 
for a scalable-yet-affordable solution.


Would be grateful for input into various options, e.g. bigger 
HD/RAIDs, cloud backup, tape, anything else.


I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1






To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Herbert J. Bernstein]

Dear Colleagues,

  May I suggest that those who are at Universities take a look at the
G-suite for Education

https://edu.google.com/products/gsuite-for-education/editions/?modal_active=none

which provides unlimited cloud storage for free to educational institutions.

  Regards,
Herbert

On Thu, Nov 29, 2018 at 3:54 PM Lieberman, Raquel L <
raquel.lieber...@chemistry.gatech.edu> wrote:

> Dear All,
>
> How do your labs handle long-term raw data backups? My lab is maxing out
> our 6TB RAID backup (with two off-site mirrors) so I am investigating our
> next long term solution. The vast majority of the data sets are published
> structures (i.e. processed data deposited in PDB) or redundant/unusable so
> immediate access is not anticipated, but the size of data sets is
> increasing quickly with time, so I am looking for a scalable-yet-affordable
> solution.
>
> Would be grateful for input into various options, e.g. bigger HD/RAIDs,
> cloud backup, tape, anything else.
>
> I will compile.
>
> Thank you,
>
> Raquel
> --
> Raquel L. Lieberman, Ph.D.
> Professor
> School of Chemistry and Biochemistry
> Georgia Institute of Technology
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Kay Diederichs]

Wladek Minor runs proteindiffraction.org . This also has E.g. JCSG data sets.
BR,
Kay

On Thu, 29 Nov 2018 23:35:55 +, Diana Tomchick 
 wrote:

>I hope you are compressing your images, typically that makes them 1/4 the 
>original size.
>
>SBGrid and Wladek Minor also have image archival services. As I am replying 
>from my cell phone while on vacation, the links to those services are not 
>handy to me. But they have been mentioned many times on this bulletin board, 
>and should be easily findable by an Internet search.
>
>Diana
>
>**
>Diana R. Tomchick
>Department of Biophysics, Rm. ND10.214A
>University of Texas Southwestern Medical Center
>5323 Harry Hines Blvd.
>Dallas, TX 75061 USA
>214-645-6383 (office)
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Tim Grüne]

Hi Jacob,
The idea of a scientific service being restricted by nationality is so strange 
to me that this possibility for misunderstanding did not occur to me.
Best,
Tim

On November 29, 2018 10:24:49 PM GMT+01:00, "Keller, Jacob" 
 wrote:
>I saw explicitly that it is not limited to EU.
>
>JPK
>
>+
>Jacob Pearson Keller
>Research Scientist / Looger Lab
>HHMI Janelia Research Campus
>19700 Helix Dr, Ashburn, VA 20147
>Desk: (571)209-4000 x3159
>Cell: (301)592-7004
>+
>
>The content of this email is confidential and intended for the
>recipient specified in message only. It is strictly forbidden to share
>any part of this message with any third party, without a written
>consent of the sender. If you received this message by mistake, please
>reply to this message and follow with its deletion, so that we can
>ensure such a mistake does not occur in the future.
>
>-Original Message-
>From: CCP4 bulletin board  On Behalf Of
>graeme.win...@diamond.ac.uk
>Sent: Thursday, November 29, 2018 4:22 PM
>To: CCP4BB@JISCMAIL.AC.UK
>Subject: Re: [ccp4bb] Long term storage for raw images/
>crystallographic data sets
>
>Dear Tim,
>
>I do not think Zenodo is limited to Europeans - at least I could not
>find this on their policy page:
>
>http://about.zenodo.org/policies/
>
>I know of plenty of uploads from Japan for example
>
>Best wishes Graeme
>
>On 29 Nov 2018, at 21:16, Tim Gruene
>mailto:tim.gru...@psi.ch>> wrote:
>
>Dear Raquel,
>
>when they are associated with a publication, you can publish them on
>data.sbgrid.org<http://data.sbgrid.org> in the US or at
>zenodo.org<http://zenodo.org> in Europe.
>
>Best regards,
>Tim
>
>On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L
>wrote:
>Dear All,
>
>How do your labs handle long-term raw data backups? My lab is maxing
>out our 6TB RAID backup (with two off-site mirrors) so I am
>investigating our next long term solution. The vast majority of the
>data sets are published structures (i.e. processed data deposited in
>PDB) or redundant/unusable so immediate access is not anticipated, but
>the size of data sets is increasing quickly with time, so I am looking
>for a scalable-yet-affordable solution.
>
>Would be grateful for input into various options, e.g. bigger HD/RAIDs,
>cloud backup, tape, anything else.
>
>I will compile.
>
>Thank you,
>
>Raquel
>--
>Raquel L. Lieberman, Ph.D.
>Professor
>School of Chemistry and Biochemistry
>Georgia Institute of Technology
>
>
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>--
>--
>Paul Scherrer Institut
>Tim Gruene
>- persoenlich -
>OSUA/204
>Forschungsstrasse 111
>CH-5232 Villigen PSI
>phone: +41 (0)56 310 5297
>
>GPG Key ID = A46BEE1A
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>--
>This e-mail and any attachments may contain confidential, copyright and
>or privileged material, and are for the use of the intended addressee
>only. If you are not the intended addressee or an authorised recipient
>of the addressee please notify us of receipt by returning the e-mail
>and do not use, copy, retain, distribute or disclose the information in
>or attached to the e-mail.
>Any opinions expressed within this e-mail are those of the individual
>and not necessarily of Diamond Light Source Ltd. 
>Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>attachments are free from viruses and we cannot accept liability for
>any damage which you may sustain as a result of software viruses which
>may be transmitted in or with the message.
>Diamond Light Source Limited (company no. 4375679). Registered in
>England and Wales with its registered office at Diamond House, Harwell
>Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United
>Kingdom
>
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Tim Grüne]

Hi Graeme,
I meant to say that the zenodo server sits in Europe, and the sbgrif server in 
the us. It was not meant to indicate any restrictions. 
Best,
Tim


On November 29, 2018 10:21:36 PM GMT+01:00, "graeme.win...@diamond.ac.uk" 
 wrote:
>Dear Tim,
>
>I do not think Zenodo is limited to Europeans - at least I could not
>find this on their policy page:
>
>http://about.zenodo.org/policies/
>
>I know of plenty of uploads from Japan for example
>
>Best wishes Graeme
>
>On 29 Nov 2018, at 21:16, Tim Gruene
>mailto:tim.gru...@psi.ch>> wrote:
>
>Dear Raquel,
>
>when they are associated with a publication, you can publish them on
>data.sbgrid.org in the US or at
>zenodo.org in Europe.
>
>Best regards,
>Tim
>
>On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L
>wrote:
>Dear All,
>
>How do your labs handle long-term raw data backups? My lab is maxing
>out our
>6TB RAID backup (with two off-site mirrors) so I am investigating our
>next
>long term solution. The vast majority of the data sets are published
>structures (i.e. processed data deposited in PDB) or redundant/unusable
>so
>immediate access is not anticipated, but the size of data sets is
>increasing quickly with time, so I am looking for a
>scalable-yet-affordable
>solution.
>
>Would be grateful for input into various options, e.g. bigger HD/RAIDs,
>cloud backup, tape, anything else.
>
>I will compile.
>
>Thank you,
>
>Raquel
>--
>Raquel L. Lieberman, Ph.D.
>Professor
>School of Chemistry and Biochemistry
>Georgia Institute of Technology
>
>
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>--
>--
>Paul Scherrer Institut
>Tim Gruene
>- persoenlich -
>OSUA/204
>Forschungsstrasse 111
>CH-5232 Villigen PSI
>phone: +41 (0)56 310 5297
>
>GPG Key ID = A46BEE1A
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>-- 
>This e-mail and any attachments may contain confidential, copyright and
>or privileged material, and are for the use of the intended addressee
>only. If you are not the intended addressee or an authorised recipient
>of the addressee please notify us of receipt by returning the e-mail
>and do not use, copy, retain, distribute or disclose the information in
>or attached to the e-mail.
>Any opinions expressed within this e-mail are those of the individual
>and not necessarily of Diamond Light Source Ltd. 
>Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>attachments are free from viruses and we cannot accept liability for
>any damage which you may sustain as a result of software viruses which
>may be transmitted in or with the message.
>Diamond Light Source Limited (company no. 4375679). Registered in
>England and Wales with its registered office at Diamond House, Harwell
>Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United
>Kingdom

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Diana Tomchick]

I hope you are compressing your images, typically that makes them 1/4 the 
original size.

SBGrid and Wladek Minor also have image archival services. As I am replying 
from my cell phone while on vacation, the links to those services are not handy 
to me. But they have been mentioned many times on this bulletin board, and 
should be easily findable by an Internet search.

Diana

**
Diana R. Tomchick
Department of Biophysics, Rm. ND10.214A
University of Texas Southwestern Medical Center
5323 Harry Hines Blvd.
Dallas, TX 75061 USA
214-645-6383 (office)

On Nov 29, 2018, at 3:54 PM, Lieberman, Raquel L 
mailto:raquel.lieber...@chemistry.gatech.edu>>
 wrote:

Dear All,

How do your labs handle long-term raw data backups? My lab is maxing out our 
6TB RAID backup (with two off-site mirrors) so I am investigating our next long 
term solution. The vast majority of the data sets are published structures 
(i.e. processed data deposited in PDB) or redundant/unusable so immediate 
access is not anticipated, but the size of data sets is increasing quickly with 
time, so I am looking for a scalable-yet-affordable solution.

Would be grateful for input into various options, e.g. bigger HD/RAIDs, cloud 
backup, tape, anything else.

I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



UT Southwestern


Medical Center



The future of medicine, today.




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Keller, Jacob]

I saw explicitly that it is not limited to EU.

JPK

+
Jacob Pearson Keller
Research Scientist / Looger Lab
HHMI Janelia Research Campus
19700 Helix Dr, Ashburn, VA 20147
Desk: (571)209-4000 x3159
Cell: (301)592-7004
+

The content of this email is confidential and intended for the recipient 
specified in message only. It is strictly forbidden to share any part of this 
message with any third party, without a written consent of the sender. If you 
received this message by mistake, please reply to this message and follow with 
its deletion, so that we can ensure such a mistake does not occur in the future.

-Original Message-
From: CCP4 bulletin board  On Behalf Of 
graeme.win...@diamond.ac.uk
Sent: Thursday, November 29, 2018 4:22 PM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] Long term storage for raw images/ crystallographic data 
sets

Dear Tim,

I do not think Zenodo is limited to Europeans - at least I could not find this 
on their policy page:

http://about.zenodo.org/policies/

I know of plenty of uploads from Japan for example

Best wishes Graeme

On 29 Nov 2018, at 21:16, Tim Gruene 
mailto:tim.gru...@psi.ch>> wrote:

Dear Raquel,

when they are associated with a publication, you can publish them on 
data.sbgrid.org<http://data.sbgrid.org> in the US or at 
zenodo.org<http://zenodo.org> in Europe.

Best regards,
Tim

On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L wrote:
Dear All,

How do your labs handle long-term raw data backups? My lab is maxing out our 
6TB RAID backup (with two off-site mirrors) so I am investigating our next long 
term solution. The vast majority of the data sets are published structures 
(i.e. processed data deposited in PDB) or redundant/unusable so immediate 
access is not anticipated, but the size of data sets is increasing quickly with 
time, so I am looking for a scalable-yet-affordable solution.

Would be grateful for input into various options, e.g. bigger HD/RAIDs, cloud 
backup, tape, anything else.

I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

--
--
Paul Scherrer Institut
Tim Gruene
- persoenlich -
OSUA/204
Forschungsstrasse 111
CH-5232 Villigen PSI
phone: +41 (0)56 310 5297

GPG Key ID = A46BEE1A



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


--
This e-mail and any attachments may contain confidential, copyright and or 
privileged material, and are for the use of the intended addressee only. If you 
are not the intended addressee or an authorised recipient of the addressee 
please notify us of receipt by returning the e-mail and do not use, copy, 
retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not 
necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments 
are free from viruses and we cannot accept liability for any damage which you 
may sustain as a result of software viruses which may be transmitted in or with 
the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and 
Wales with its registered office at Diamond House, Harwell Science and 
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[graeme.win...@diamond.ac.uk]

Dear Tim,

I do not think Zenodo is limited to Europeans - at least I could not find this 
on their policy page:

http://about.zenodo.org/policies/

I know of plenty of uploads from Japan for example

Best wishes Graeme

On 29 Nov 2018, at 21:16, Tim Gruene 
mailto:tim.gru...@psi.ch>> wrote:

Dear Raquel,

when they are associated with a publication, you can publish them on
data.sbgrid.org in the US or at 
zenodo.org in Europe.

Best regards,
Tim

On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L wrote:
Dear All,

How do your labs handle long-term raw data backups? My lab is maxing out our
6TB RAID backup (with two off-site mirrors) so I am investigating our next
long term solution. The vast majority of the data sets are published
structures (i.e. processed data deposited in PDB) or redundant/unusable so
immediate access is not anticipated, but the size of data sets is
increasing quickly with time, so I am looking for a scalable-yet-affordable
solution.

Would be grateful for input into various options, e.g. bigger HD/RAIDs,
cloud backup, tape, anything else.

I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

--
--
Paul Scherrer Institut
Tim Gruene
- persoenlich -
OSUA/204
Forschungsstrasse 111
CH-5232 Villigen PSI
phone: +41 (0)56 310 5297

GPG Key ID = A46BEE1A



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


-- 
This e-mail and any attachments may contain confidential, copyright and or 
privileged material, and are for the use of the intended addressee only. If you 
are not the intended addressee or an authorised recipient of the addressee 
please notify us of receipt by returning the e-mail and do not use, copy, 
retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not 
necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments 
are free from viruses and we cannot accept liability for any damage which you 
may sustain as a result of software viruses which may be transmitted in or with 
the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and 
Wales with its registered office at Diamond House, Harwell Science and 
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[Tim Gruene]

Dear Raquel,

when they are associated with a publication, you can publish them on 
data.sbgrid.org in the US or at zenodo.org in Europe.

Best regards,
Tim

On Thursday, November 29, 2018 9:54:02 PM CET Lieberman, Raquel L wrote:
> Dear All,
> 
> How do your labs handle long-term raw data backups? My lab is maxing out our
> 6TB RAID backup (with two off-site mirrors) so I am investigating our next
> long term solution. The vast majority of the data sets are published
> structures (i.e. processed data deposited in PDB) or redundant/unusable so
> immediate access is not anticipated, but the size of data sets is
> increasing quickly with time, so I am looking for a scalable-yet-affordable
> solution.
> 
> Would be grateful for input into various options, e.g. bigger HD/RAIDs,
> cloud backup, tape, anything else.
> 
> I will compile.
> 
> Thank you,
> 
> Raquel
> --
> Raquel L. Lieberman, Ph.D.
> Professor
> School of Chemistry and Biochemistry
> Georgia Institute of Technology
> 
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 
--
Paul Scherrer Institut
Tim Gruene
- persoenlich -
OSUA/204
Forschungsstrasse 111
CH-5232 Villigen PSI
phone: +41 (0)56 310 5297

GPG Key ID = A46BEE1A



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


signature.asc
Description: This is a digitally signed message part.

Re: [ccp4bb] Long term storage for raw images/ crystallographic data sets

[graeme.win...@diamond.ac.uk]

Dear Raquel,

For published structures you can publish the raw data, which means that 
somebody else is looking after it - for this I would say that the current front 
runner is Zenodo - https://zenodo.org/ - which is paid for by CERN / EU etc. so 
someone else is (currently) picking up the tab.

This has the happy side effect that it is useful to others :-)

Best wishes Graeme

On 29 Nov 2018, at 20:54, Lieberman, Raquel L 
mailto:raquel.lieber...@chemistry.gatech.edu>>
 wrote:

Dear All,

How do your labs handle long-term raw data backups? My lab is maxing out our 
6TB RAID backup (with two off-site mirrors) so I am investigating our next long 
term solution. The vast majority of the data sets are published structures 
(i.e. processed data deposited in PDB) or redundant/unusable so immediate 
access is not anticipated, but the size of data sets is increasing quickly with 
time, so I am looking for a scalable-yet-affordable solution.

Would be grateful for input into various options, e.g. bigger HD/RAIDs, cloud 
backup, tape, anything else.

I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


-- 
This e-mail and any attachments may contain confidential, copyright and or 
privileged material, and are for the use of the intended addressee only. If you 
are not the intended addressee or an authorised recipient of the addressee 
please notify us of receipt by returning the e-mail and do not use, copy, 
retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not 
necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments 
are free from viruses and we cannot accept liability for any damage which you 
may sustain as a result of software viruses which may be transmitted in or with 
the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and 
Wales with its registered office at Diamond House, Harwell Science and 
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

[ccp4bb] Long term storage for raw images/ crystallographic data sets

[Lieberman, Raquel L]

Dear All,

How do your labs handle long-term raw data backups? My lab is maxing out our 
6TB RAID backup (with two off-site mirrors) so I am investigating our next long 
term solution. The vast majority of the data sets are published structures 
(i.e. processed data deposited in PDB) or redundant/unusable so immediate 
access is not anticipated, but the size of data sets is increasing quickly with 
time, so I am looking for a scalable-yet-affordable solution.

Would be grateful for input into various options, e.g. bigger HD/RAIDs, cloud 
backup, tape, anything else.

I will compile.

Thank you,

Raquel
--
Raquel L. Lieberman, Ph.D.
Professor
School of Chemistry and Biochemistry
Georgia Institute of Technology





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1