now we have some real numbers. reputable provider 10TB/$1800
Amazon S3 10TB/$1250 Amazon EBS 10TB/$1000 its not that simple a comparison. but looking at those numbers and knowing that with AWS you can run everything else as a function of need (EC2,LB,SQS,mail services... etc etc) AWS starts looking cheap. It might be a bit more to park data ( not really ) but that is only a small part of the final equation. On Fri, Jan 28, 2011 at 10:30 AM, <[email protected]> wrote: > @Mark Yes, so big of a difference. HDDs dont even fail often : ) What can > you say about our current plan: > > Assuming we need 10TB of storage, > We get a dedicated server somewhere on a reputable provider like SoftLayer > the max HDDs they can put in a single box is 24 so assuming we get 2TB HDDs > that's 24TB in RAID 1+0. I just spoke to their sales team and with the > config Xeon 5xxx something, 12GB DDR3 of memory among other things such as > the raid card and whatnot, it would cost around $1800/month. > > Is this a viable solution? I would like to just invest in a local server but > unfortunately our internet sucks here in our country so we'll just keep > local backups and rely on the RAID in the said machine for the required > "redundancy" if you can call it that. > > On 1/29/2011 12:14 AM, Mark Korver wrote: >> >> I think current S3 pricing for 50TB at 0.125/GB comes to about >> 6250/month. That is for "Designed for 99.999999999% Durability". I >> can't even count that many 9s. I know you can buy HDs for about >> $50/TB = $2500 for that 50TB. Assuming you replace them all once a >> year its still only $5000. >> >> But we all know that to even get to 99.99 there are lot more parts to >> the above equation. Anybody have a ballpark on what it really costs >> to park that 50TB on your server room rack? >> >> On Fri, Jan 28, 2011 at 9:29 AM, Bob Basques >> <[email protected]> wrote: >>> >>> All, >>> >>> I'm working on a similar project currently. Setting up 50tb of storage, >>> we went the route of multiple CPUs, with large disks. Redundant raid >>> config, so half of physical disk available for storage. We're in the 30+tb >>> of real storage across a 4U setup right now. Cost (with >>> hardware/setup/initial config) is below those numbers below (so far), >>> because we're building from scratch and learning along the way. >>> >>> I would tend to agree on not using the off site stuff, just considering >>> the moving of the data and the idea of co-lo to some other remote location >>> starts to fall apart. The transfer costs, in bandwidth and/or time, really >>> start to eat into things cost wise. Some of this depends on the end uses as >>> well. We're building a data site for distribution of really large files and >>> datasets. >>> >>> bobb >>> >>>>>> Paul Spencer<[email protected]> wrote: >>> >>> Hi, >>> >>> I would personally recommend against AWS S3/EBS for anything of this >>> scale as the I/O is pretty pathetic unless you invest in their very high end >>> instances. We've set up a 4TB 'SAN' using glusterfs on AWS EC2 using 1TB >>> EBS volumes and separate instances for each - the performance has been so >>> poor that we have had to redesign our workflow to get copies of data onto >>> EBS attached to each mapserver instance - for scaling that sucks and even >>> then the I/O performance of EBS is not that great on the normal instances. >>> >>> I'm not a hardware guy but I think the purpose of a dedicated SAN box is >>> to provide high bandwidth access to large amounts storage so that the data >>> can effectively be distributed to/from multiple machines over a network - >>> ideal for scaling mapserver onto multiple servers but rendering from the >>> same data. I read an article about a year ago from a company that provides >>> petabyte storage for online storage, it details how they built their storage >>> devices - they say $7867 for 67 terabytes >>> >>> >>> http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ >>> >>> Seems pretty geeky, but perhaps you are the hardware type or know someone >>> who is :) >>> >>> >>> On 2011-01-28, at 3:41 AM, [email protected] wrote: >>> >>>> Hello guys, we're a startup and new to mapserver. We're expecting large >>>> amounts of data to come by (at least on our scale) around 40-60TB of raster >>>> images for mapserver to render. My question is for the infrastructure, what >>>> is the best way to store this (cost-efficiently)? >>>> >>>> - Do we just get a dedicated server with a lot of HDDs? I'm looking at a >>>> 48TB setup in RAID 1+0 so i get 24TB right what happens now if we need >>>> more? >>>> Also, how can we scale from the mapserver side? Is access to different >>>> storage servers possible? >>>> - I've considered SANs but then it's not practical right because only >>>> one machine will access the storage? >>>> - What about Amazon's S3? or EBS? Anything we can use on that? >>>> >>>> I wish to get awesome advice on this storage issue, basically what the >>>> considered best practice is for the mapserver people :P Thanks >>>> _______________________________________________ >>>> mapserver-users mailing list >>>> [email protected] >>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users >>> >>> __________________________________________ >>> >>> Paul Spencer >>> Chief Technology Officer >>> DM Solutions Group Inc >>> http://research.dmsolutions.ca/ >>> >>> _______________________________________________ >>> mapserver-users mailing list >>> [email protected] >>> http://lists.osgeo.org/mailman/listinfo/mapserver-users >>> >>> _______________________________________________ >>> mapserver-users mailing list >>> [email protected] >>> http://lists.osgeo.org/mailman/listinfo/mapserver-users >>> > > _______________________________________________ mapserver-users mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/mapserver-users
