@Mark Yes, so big of a difference. HDDs dont even fail often : ) What
can you say about our current plan:
Assuming we need 10TB of storage,
We get a dedicated server somewhere on a reputable provider like
SoftLayer the max HDDs they can put in a single box is 24 so assuming we
get 2TB HDDs that's 24TB in RAID 1+0. I just spoke to their sales team
and with the config Xeon 5xxx something, 12GB DDR3 of memory among other
things such as the raid card and whatnot, it would cost around $1800/month.
Is this a viable solution? I would like to just invest in a local server
but unfortunately our internet sucks here in our country so we'll just
keep local backups and rely on the RAID in the said machine for the
required "redundancy" if you can call it that.
On 1/29/2011 12:14 AM, Mark Korver wrote:
I think current S3 pricing for 50TB at 0.125/GB comes to about
6250/month. That is for "Designed for 99.999999999% Durability". I
can't even count that many 9s. I know you can buy HDs for about
$50/TB = $2500 for that 50TB. Assuming you replace them all once a
year its still only $5000.
But we all know that to even get to 99.99 there are lot more parts to
the above equation. Anybody have a ballpark on what it really costs
to park that 50TB on your server room rack?
On Fri, Jan 28, 2011 at 9:29 AM, Bob Basques
<[email protected]> wrote:
All,
I'm working on a similar project currently. Setting up 50tb of storage, we
went the route of multiple CPUs, with large disks. Redundant raid config, so
half of physical disk available for storage. We're in the 30+tb of real
storage across a 4U setup right now. Cost (with hardware/setup/initial config)
is below those numbers below (so far), because we're building from scratch and
learning along the way.
I would tend to agree on not using the off site stuff, just considering the
moving of the data and the idea of co-lo to some other remote location starts
to fall apart. The transfer costs, in bandwidth and/or time, really start to
eat into things cost wise. Some of this depends on the end uses as well.
We're building a data site for distribution of really large files and datasets.
bobb
Paul Spencer<[email protected]> wrote:
Hi,
I would personally recommend against AWS S3/EBS for anything of this scale as
the I/O is pretty pathetic unless you invest in their very high end instances.
We've set up a 4TB 'SAN' using glusterfs on AWS EC2 using 1TB EBS volumes and
separate instances for each - the performance has been so poor that we have had
to redesign our workflow to get copies of data onto EBS attached to each
mapserver instance - for scaling that sucks and even then the I/O performance
of EBS is not that great on the normal instances.
I'm not a hardware guy but I think the purpose of a dedicated SAN box is to
provide high bandwidth access to large amounts storage so that the data can
effectively be distributed to/from multiple machines over a network - ideal for
scaling mapserver onto multiple servers but rendering from the same data. I
read an article about a year ago from a company that provides petabyte storage
for online storage, it details how they built their storage devices - they say
$7867 for 67 terabytes
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/
Seems pretty geeky, but perhaps you are the hardware type or know someone who
is :)
On 2011-01-28, at 3:41 AM, [email protected] wrote:
Hello guys, we're a startup and new to mapserver. We're expecting large amounts
of data to come by (at least on our scale) around 40-60TB of raster images for
mapserver to render. My question is for the infrastructure, what is the best
way to store this (cost-efficiently)?
- Do we just get a dedicated server with a lot of HDDs? I'm looking at a 48TB
setup in RAID 1+0 so i get 24TB right what happens now if we need more? Also,
how can we scale from the mapserver side? Is access to different storage
servers possible?
- I've considered SANs but then it's not practical right because only one
machine will access the storage?
- What about Amazon's S3? or EBS? Anything we can use on that?
I wish to get awesome advice on this storage issue, basically what the
considered best practice is for the mapserver people :P Thanks
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
__________________________________________
Paul Spencer
Chief Technology Officer
DM Solutions Group Inc
http://research.dmsolutions.ca/
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/mapserver-users
