Thanks to all of you who answered. Crowdsourcing does work if you pick the right crowd. We have been looking at the S3 possibility but I agree this would have to be a second copy. The policy and institutional support comments from my tokayo
see http://en.wiktionary.org/wiki/tocayo seem especially appropriate. I am going to include a link on our staff blog to this thread as a resource. Thanks again, Edward Iglesias On Thu, Aug 27, 2009 at 8:59 PM, Edward M. Corrado<ecorr...@ecorrado.us> wrote: > Joe Atzberger wrote: >> >> On Thu, Aug 27, 2009 at 4:25 PM, Edward M. Corrado >> <ecorr...@ecorrado.us>wrote: >> >> >>> >>> Nate Vack wrote: >>> >>> >>>> >>>> On Thu, Aug 27, 2009 at 1:57 PM, Ryan Ordway<rord...@oregonstate.edu> >>>> wrote: >>>> >>>> >>>> >>>>> >>>>> $213,360 over 3 years >>>>> >>>>> >>>>> >>>> >>>> If you're ONLY looking at storage costs, SATA drives in enterprise RAID >>>> >>>>> >>>>> systems range from about $1.00/GB to about $1.25/GB for online storage. >>>>> >>>>> >>>> >>>> Yeah -- but if you're looking only at storage costs, you'll have an >>>> inaccurate estimate of your costs. You've got power, cooling, sysadmin >>>> time, and replacements for failed disks. If you want an >>>> apples-to-apples comparison, you'll want an offsite mirror, as well. >>>> >>>> I'm not saying S3 is always cost-effective -- but in our experience, >>>> the costs of the disks themselves is dwarfed by the costs of the >>>> related infrastructure. >>>> >>>> I agree that the cost of storage is only one factor. I have to wonder >>>> >>> >>> though, how much more staff time do you need for local storage than cloud >>> storage? I don't know the answer but I'm not sure it is much more than >>> setting up S3 storage, especially if you have a good partnership with >>> your >>> storage vendor. >>> >> >> >> Support relationships, especially regarding storage are very costly. When >> I >> worked at a midsize datacenter, we implemented a backup solution with >> STORServer and tivoli. Both hardware and software were considerably >> costly. Initial and ongoing support, while indispensable was basically as >> much as the cost of the hardware every few years. >> > > They can be depending on what you are doing and what choices on software you > make, but for long term preservation purposes they don't have to be nearly > as expensive as what Ryan calculated S3 to cost. If you shop around you can > get a quality 36GB array with 3 yr warranty for say $30,000 that is almost > $180,000 less than S3 (probably much less, I'm be less than generous with my > Sun discounts and only briefly looked at there prices). Even if we use the > double your cost for support, it is still over $50,000 a year less for 3 > years. Yes, we might need some expertise, but running a 36TB preservation > storage array is not a $50,000 a year job and besides, what is wrong with > growing local expertise? > > ... >>> >>> Yes, maybe you save on staff time patching software on your storage >>> array, >>> but that is not a significant amount of time - esp. since you are still >>> going to have some local storage, and there isn't much difference in >>> staff >>> time in doing 2 TB vs. 20 TB. >>> >> >> >> There's a real difference. I can get 2 TB in a single HDD, for example >> this >> one for $200 at NewEgg: >> http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413 >> <http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413> >> >> Any high school kid can install that. 20 TB requires some kind of >> additional structure and additional expertise. >> > > Well building a 20 TB storage device and getting it to work can actually be > very cheap and doesn't require a PhD (just a local GNU/Linux geek who likes > to play with hardware) if you are OK with a home grown solution. I wouldn't > be satisfied with that, but I don't see how a commercial offering that adds > up to $150,000 worth of expertise and infrastructure. > >> You may some time on the initial configuration, but you still need to >> >>> >>> configure cloud storage. Is cloud storage that much easier/less time >>> consuming to configure than an iSCSI device? Replacement for disks would >>> be >>> covered under your warranty or support contract (at least I would hope >>> you >>> would have one). >>> >> >> >> Warranties expire and force you into ill-timed, hardly-afforded and >> dangerous-to-your-data upgrades. Sorta like some ILS systems with which >> we >> are all familiar. > > Yes some application upgrades can cause issues, but how is that different if > your application and/or storage is in a cloud? > >> The cloud doesn't necessarily stay the same, but the part >> you care about (data in, data out) does. >> > > How do you know they won't change their cloud models? And you don't even > have a warranty with the cloud. They won't even guarantee they won't delete > your data. > > As long as you use a common standards based method of storage, you won't > have any more issues getting it to work than you will getting future > application servers to work with the cloud. While I'm not a huge fan of NFS > I've been using it for many years with no problems due to changes in NFS or > operating systems or hardware. NFS has been available to the public for > about 20 years. Occasionally you may need to migrate it from one platform or > one machine to another but you very well need to do that with clouds as > well. Maybe you are using S3 but for whatever reason Sun gives you a better > deal with better terms and guarantees for using their cloud. Maybe Amazon > drops S3. Maybe because S3 moves servers to a country that you are not > legally allow to have your data in. Yes, you have to plan for migration to > new platforms but I fail to see how you don't need to do that with the > cloud. Really any major technological decision should have an exit plan. > Preservation storage is not different in that and the cloud doesn't change > that. > > Edward > > >>> The power and cooling can be a savings, but in many cases the library or >>> individual departments don't pay for electricity, so while *someone* pays >>> the cost, it might not be the individual department. Cooling and >>> electricity >>> costs are an actually a great argument for tape for large-scale storage. >>> Tape might seem old fashioned, but in many applications it by far offers >>> the >>> best value of long term storage per GB. >>> >> >> >> It's true, tape is still an worthwhile option. Alternatives like optical >> or >> magneto-optical media just have not kept up. >> >> Again, I'm not totally against the cloud and there are some things I think >> >>> >>> it could be very useful for, but the cloud doesn't make up for the lack >>> of >>> (or just bad) planning. >>> >> >> >> Yeah, there's no system good enough to compensate for bad planning and >> management. >> --Joe >> >
