As far as I can tell, S3 is powerful online storage, not literal online backup. You could use it as a data mirror, but it's not like saving the contents of the server to multiple tape sets created over the course of time and storing them in multiple secure places.
With S3 (and other online storage), any automated backup process could irreversibly overwrite the "backup" copy with corrupted/truncated/deleted versions of the original; you wouldn't have last week's tape set (or last night's differential) to fall back on. That is, there's no redundancy, so you can reach unrecoverable situations too easily. There are online services that provide the equivalent to full tape backup with offsite storage, like http://www.usdatatrust.com/service/how_it_works.asp (the "historical archive" part is the key difference between them and S3). These online backup services are not at all cheap (to back up just ten of Ari's AVIs would cost $1000+/year!), but are probably not more expensive than doing the same thing yourself, in many cases (you wouldn't have to buy or store tapes ...). --Matt On 11/6/07 8:23 AM, "Ari Davidow" <aridavidow at gmail.com> wrote: > There seems to be a burgeoning market in servers for consumer purposes--ways > to share video clips, photos, whatever. I think that places like > rememble.com or mediamax seem to represent that paradigm. We have looked, > and continue to look, at setting up an alliance with other institutions in > geographically different areas from us for mutual file sharing. The only > viable commercial service for needs like ours (hundreds of gigabytes or even > terabytes of files) seems to be Amazon's S3, which isn't cheap, but does do > a lot to ensure the integrity and security of data. Unfortunately, S3 has a > maximum file size of 5GB. With an hour of AVI clocking in around 13GB, > that's a problem. > > As big a problem of backing up this data in a reasonably secure fashion is, > MOVING it is also difficult. It takes half an hour to move that 13GB file > around on our network. On our small business pipe to the internet? > > So, hoping to hear that others are exploring this area, or that they have > other solutions, > ari > > On 11/5/07, Richard Urban <rjurban at uiuc.edu> wrote: >> >> Hi Ari, >> >> Sorry you won't make it to Chicago. I've been wondering about this >> myself after a recent round of discussion on the RCAAM list - they >> were debating the need for hard copy of collections records. Many of >> the pro-paper set pointed to the easy loss of computer data during >> disasters. I didn't ask, but I wondered where eveyone's back ups >> were. >> I think one of the things that came out of Katrina was that off-site >> backups weren't enough and people needed to think regionally or >> nationally. >> >> I do wonder about how these companies may be using the data stored on >> their servers. I think there would be a number of concerns about >> storing collections records and donor information with a third party, >> even if they had clear data usage policies. >> >> The digital preservation project I worked on still relied on FedEx to >> ship hard-drives back and forth as the fastest way to move large >> files - even though we are at a major research university with many >> fat tubes to the Interweb. >> >> Richard >> rjurban at uiuc.edu >> >> >> >> On Nov 5, 2007, at 2:21 PM, Ari Davidow wrote: >> >>> So, we have all of these digital assets. They need to be backed up >>> and they >>> are huge. It looks as though online backup, using Amazon's S3 >>> service, or a >>> company such as MediaMax, is the way to go. It looks like S3 is really >>> designed for companies that are doing more with their data than >>> just parking >>> it. That doesn't bother me, but it also looks like S3 is limited to >>> files >>> <5GB. We have AVI files that are closer to 15GB. MediaMax may have >>> similar >>> limits. >>> >>> In a worst case, I'd simply make plans to back up the working MPEGs >>> (2-3GB >>> each), and do my best to ensure that nothing bad happens to the tape >>> originals of this video. But it occurs to me that people on this >>> list may >>> have more extensive experience. >>> >>> Who is using online storage? What works for you? What provider >>> helps you >>> sleep soundly (less unsoundly?) at night? What does it cost? Does >>> anyone >>> handle 15GB files? (We won't even talk about how long it would take to >>> upload each one of those....) >>> >>> Thanks, >>> (Think of this as my way of provoking conversation in a year when I >>> can't >>> get to MCN ;-).) >>> ari >>> _______________________________________________ >>> You are currently subscribed to mcn-l, the listserv of the Museum >>> Computer Network (http://www.mcn.edu) >>> >>> To post to this list, send messages to: mcn-l at mcn.edu >>> >>> To unsubscribe or change mcn-l delivery options visit: >>> http://toronto.mediatrope.com/mailman/listinfo/mcn-l >> >> _______________________________________________ >> You are currently subscribed to mcn-l, the listserv of the Museum Computer >> Network (http://www.mcn.edu) >> >> To post to this list, send messages to: mcn-l at mcn.edu >> >> To unsubscribe or change mcn-l delivery options visit: >> http://toronto.mediatrope.com/mailman/listinfo/mcn-l >> > _______________________________________________ > You are currently subscribed to mcn-l, the listserv of the Museum Computer > Network (http://www.mcn.edu) > > To post to this list, send messages to: mcn-l at mcn.edu > > To unsubscribe or change mcn-l delivery options visit: > http://toronto.mediatrope.com/mailman/listinfo/mcn-l
