Just to chime in quickly We finally gave up on DVD storage and are exclusively using disk based solutions for data (or images).
One of our Xserve RAIDs has two 2.8TB disk banks. One bank is used for incoming data, the other mirrors the first for backup. Another Xserve RAID's bank is used as a secondary backup to the first RAID. After searching high and low for a good Backup software package, it dawned on me that Roxio Toast has a backup utility called Deja Vu, which I ignored for a long time. I gave it a spin and I have not looked back. A little tip: We bought our second Xserve RAID (5.6TB, less when formated) factory reconditioned for $6k less then a new unit. Since it was factory reconditioned, the standard one year warranty still applied, and we bought the Premium Service and Support plan for another $700, extending the warranty another two years. On Aug 31, 2006, at 9:50 AM, David Marsh wrote: > Hi Chuck... > > I'm reviewing similar issues on a much smaller scale here at the Space > Centre and Museum here in Vancouver BC. > Here's some random thoughts on my efforts. > > FYI: we're a non profit operating with minimal budgets and resources > here. > Enterprise class Solution Providers need not apply! :-) > (...open source hackers get free coffee and cookies) > > Our main shared resource is a single shared directory tree which > contains everything from planetarium visuals to accounting's Excel > spreadsheets. > > I use disk based backup with a 10 cartridge rotation. The entire > tree is > backed up daily. Using commodity IDE (or SATA) hard disks is very cost > effective. Blows tape systems out of the water regarding cost, speed, > random access, flexibility. But a single tier system like this will > inevitably run out of space eventually, so I'm looking to develop a > more > sophisticated model. > > My current line of thinking is to retain the single tree for > simplicity. > Users (all of them ...not just the technophiles) need to understand > something before they can use it. I'm intending to add a separate > archive area. This will be on a separate disk volume. The main > directory > tree will be scanned nightly, and any file not even looked at for, > say, > 6 months will be moved to the archive in an identical directory > path. It > will probably be made read-only. I may provide users direct access to > it, and that would stop them modifying the contents. I want that data > static. > > Right now I'm thinking of maintaining 3 copies of the archive. > That's a > big deal, as with the 10 cartridge rotation on the main directory > tree, > we need 10 GB of media for every 1 GB of working space. That really > holds us back from exploiting cheap disk space to the fullest. With > this > archive system, we'll only need 3, so all things being equal we'll > have > 3 (ok, 3.33...) times the archive space on the same hardware > budget. The > three, rotating copies will be 1 online, 1 physically secure on-sight > and one in a safety deposit box (size 2) at the bank we do our cash > run > with. I've also considered another step: as content is moved to the > archive, > A copy of the new stuff is buffered in a separate area. > When exactly one DVD's worth of stuff has arrived, it's burnt to > DVD as > extra insurance. Not sure this step is worth the trouble. Only 3 > copies > leaves me instinctively nervous when I'm accustomed to 10, but that is > purely psychology. I'm telling myself the chances of 3 drives failing > simultaneously must be remote (remember 2 offline, 1 offsite). > Still, my > instincts aren't quite satisfied. Intellectually, I feel burning the > DVDs is less cost effective and less flexible than simply getting more > hard drives. Hard disks are hard to beat for $/GB and optical storage > never seems to catch up, though with each new generation of CD > technology it closes the gap for a while. Even 9.something GB on a > double layer disk isn't looking very big anymore (my cheap IDE disk > cartridges are 300GB). The labor and logistics of doing the DVD > burn are > not welcome either. And of course optical disks are not famous for > reliable, long term stability that you'd bet your institution on. > > I'm also considering another class of data: Extremely bulky data > Examples would be planetarium production files (can be HUGE) and > collections digitization and cataloging (I have a conservator who's > very > busy with a shiny new digital camera right now). I'd really like to > find > a storage solution that doesn't need 10 rotation copies as that > would be > prohibitive given the size I want to achieve. But it has to be safely > backed up. > I'm considering maybe two mirrored copies online (different ends of > the > building, UPSs etc), a third offline locally, and a fourth off- > site. The > last two are essential to protect from a) a system-wide event and b) > destruction of the building(!). > The problem I've not answered yet is volume size. I want to use > JABOD or > software raid to build big, easily scalable disk volumes with > multiple, > cheap commodity disks. No problem for the online copies, but how that > could work for the offline and off-site copies is not obvious. Working > on it. > BTW: data size probably rules out online backup to an offsite service > provider due to bandwidth costs. > > Hope that's interesting to some of you ...I'd love to share any ideas > the rest of you may have. Cheap, flexible and secure storage is an > issue > many of us must be thinking about. > > Regards, > > David Marsh > > > ========================================== > David Marsh > System Administrator > H.R. MacMillan Space Centre > Vancouver Museum > 1100 Chestnut Street, Vancouver, BC V6J 3J9 > E sysadmin at hrmacmillanspacecentre.com > sysadmin at vanmuseum.bc.ca > T (604) 736 4431 ext. 5507 > C (604) 813 9667 > =========================================== > > -----Original Message----- > From: mcn-l-bounces at mcn.edu [mailto:mcn-l-bounces at mcn.edu] On > Behalf Of > Chuck Patch > Sent: Monday, August 21, 2006 2:18 PM > To: mcn-l at mcn.edu > Subject: [MCN-L] File Storage Best Practices Redux > > Last January someone posted a query asking about best practices for > file > storage across the spectrum of applications run on their system. The > only reply related to the archival storage of images, but I don't > think that was the question and I find myself asking the same one now. > Have any of you defined policies for data classification at your > institutions? What types of priorities do you give to different types > of data? Do you have retention schedules for stored digital files (of > any type -- images, office productivity, etc.) How do you > partition/allocate your online storage? Do you give people set amounts > of "scratch space" to use at their own disgression? > > We now have a big old EMC NAS with about 3 TB of space as well as a > variety of NAS and attached RAID units that will all be used within a > year if we don't start putting > limits on what gets parked there, so I'm very interested in real-world > experiences in developing policies and managing space. > > Chuck Patch > The Historic New Orleans Collection > _______________________________________________ > You are currently subscribed to mcn-l, the listserv of the Museum > Computer Network (http://www.mcn.edu) > > To post to this list, send messages to: mcn-l at mcn.edu > > To unsubscribe or change mcn-l delivery options visit: > http://toronto.mediatrope.com/mailman/listinfo/mcn-l > > _______________________________________________ > You are currently subscribed to mcn-l, the listserv of the Museum > Computer Network (http://www.mcn.edu) > > To post to this list, send messages to: mcn-l at mcn.edu > > To unsubscribe or change mcn-l delivery options visit: > http://toronto.mediatrope.com/mailman/listinfo/mcn-l >
