On Thu, 27 Aug 2009, Edward Iglesias wrote:
As I was trying to figure out what to do with half a terabyte of
archival TIFFS it occurred to me that perhaps someone else had this
problem. We are starting to produce massive amounts of digital
objects (videos, archival TIFFS, audio interviews). Up until now we
have been dealing with ways to display them to the public. Now we are
starting to look at "dark archives" like OCLC's digital archive
product. I would welcome any suggestions from those of you who have
dealt with this on an archival level. It's one thing to stick the
stuff up on a server, but then what? Our CIO suggested storage
appliances like this one
I'd recommend looking at two classes of products:
Near-line storage
MAID (Massive Array of Idle Disks)
Near line can be things like DVD juke boxes, where you don't have to have
someone manually load the items, but it can't all be accessed at once.
You put lower-res JPEGs up what we call 'browse images', and then when
someone wants to full res high quality image, it goes to the jukebox.
You might have to wait between 15 sec and 2 minutes for the file.
You can 'tune' them by adjusting the number of drives relative to the
amount of disks in them, to reduce the latency.
MAID systems are like RAID, but they spin down the disks when they're not
in use, so they have a much lower power draw when used for storage. We've
used them as both primary systems, and as storage for our backups of more
highly-available data.
...
As for your comment of Drobo, we don't use that specific brand, but we
do have a number of 4 or 6 disk RAID enclosures that we use for both
transporting files (if someone needs to copy 2TB of files, we mail it to
'em, rather than send it over the network), and for our off-site storage
of critical data.
-Joe
