Nope, depending on how much you want to spend, you can cram up to 42TB into a single AWS instance. Reasonably priced instances can hold 1690GB of data on a single instance. You can see an easy to sort map over here: http://www.ec2instances.info/
I would take Amazon's own recommendations with a grain of salt; whichever engineer wrote that document is going against AWS advice for other database engines. E.g RAID 1 is great when you need capacity but your RAID volume will effectively function at the latency of the slowest drive. If you use EBS optimized instances or even PIOPS, your storage can end up costing several times what the instance itself does. E.g. an m2.4xl with 4TB of PIOPS (4000 IOPS) costs : Dedicated instance (1 year lease) $199.11 / month EBS optimization: $36.60 EBS IOPS: $400.00 EBS Storage: $50.00 For the price of that storage, you can get 2 more EC2 instances (more if you use crappier instances). It's easier and cheaper to spin up more Riak nodes than it is to tune AWS infrastructure. --- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Mon, Aug 12, 2013 at 11:38 AM, Dave Martorana <[email protected]> wrote: > Jared - thanks for the links. I'm in the same boat with Brady with > weighing deployment options in AWS. > > Jeremiah - isn't EBS the only option once your data starts reaching into > the hundreds-of-gigs? > > Dave > > > On Sun, Aug 11, 2013 at 8:57 PM, Jared Morrow <[email protected]> wrote: > >> +1 to what Jeremiah said, putting a 4 or 5 node cluster in each US West >> and US East using MDC between them would be the optimum solution. I'm also >> not buying consistent latencies between AZ's, but I've also not tested it >> personally in a production environment. We have many riak-users members on >> AWS, so hopefully more experienced people will chime in. >> >> If you haven't seen them already, here's what I have in my "Riak on AWS" >> bookmark folder: >> >> http://media.amazonwebservices.com/AWS_NoSQL_Riak.pdf >> http://docs.basho.com/riak/latest/ops/tuning/aws/ >> http://basho.com/riak-on-aws-deployment-options/ >> >> -Jared >> >> >> >> >> On Sun, Aug 11, 2013 at 6:11 PM, Jeremiah Peschka < >> [email protected]> wrote: >> >>> I'd be wary of using EBS backed nodes for Riak - with only a single >>> ethernet connection, it wil be very easy to saturate the max of 1000mbps >>> available in a single AWS NIC (unless you're using cluster compute >>> instances). I'd be more worried about temporarily losing contact with a >>> node through network saturation than through AZ failure, truthfully. >>> >>> The beauty of Riak is that a node can drop and you can replace it with >>> minimal fuss. Use that to your advantage and make every node in the cluster >>> disposable. >>> >>> As far as doubling up in one AZ goes - if you're worried about AZ >>> failure, you should treat each AZ as a separate data center and design your >>> failure scenarios accordingly. Yes, Amazon say you should put one Riak node >>> in each AZ; I'm not buying that. With no guarantee around latency, and no >>> control around between DCs, you need to be very careful how much of that >>> latency you're willing to introduce into your application. >>> >>> Were I in your position, I'd stand up a 5 node cluster in US-WEST-2 and >>> be done with it. I'd consider Riak EE for my HA/DR solution once the >>> business decides that off-site HA/DR is something it wants/needs. >>> >>> >>> --- >>> Jeremiah Peschka - Founder, Brent Ozar Unlimited >>> MCITP: SQL Server 2008, MVP >>> Cloudera Certified Developer for Apache Hadoop >>> >>> >>> On Sun, Aug 11, 2013 at 1:52 PM, Brady Wetherington < >>> [email protected]> wrote: >>> >>>> Hi all - >>>> >>>> I have some questions about how I want my Riak stuff to work - I've >>>> already asked these questions of some Basho people and gotten some answers, >>>> but thought I would toss it out into the wider world to see what you all >>>> have to say, too: >>>> >>>> First off - I know 5 instances is the "magic number" of instances to >>>> have. If I understand the thinking here, it's that at the default >>>> redundancy level ('n'?) of 3, it is most likely to start getting me some >>>> scaling (e.g., performance > just that of a single node), and yet also have >>>> redundancy; whereby I can lose one box and not start to take a performance >>>> hit. >>>> >>>> My question is - I think I can only do 4 in a way that makes sense. I >>>> only have 4 AZ's that I can use right now; AWS won't let me boot instances >>>> in 1a. My concern is if I try to do 5, I will be "doubling up" in one AZ - >>>> and in AWS you're almost as likely to lose an entire AZ as you are a single >>>> instance. And so, if I have instances doubled-up in one AZ (let's say >>>> us-east-1e), and then I lose 1e, I've now lost two instances. What are the >>>> chances that all three of my replicas of some chunk of my data are on those >>>> two instances? I know that it's not guaranteed that all replicas are on >>>> separate nodes. >>>> >>>> So is it better for me to ignore the recommendation of 5 nodes, and >>>> just do 4? Or to ignore the fact that I might be doubling-up in one AZ? >>>> Also, another note. These are designed to be 'durable' nodes, so if one >>>> should go down I would expect to bring it back up *with* its data - or, if >>>> I couldn't, I would do a force-replace or replace and rebuild it from the >>>> other replicas. I'm definitely not doing instance-store. So I don't know if >>>> that mitigates my need for a full 5 nodes. I would also consider losing one >>>> node to be "degraded" and would probably seek to fix that problem as soon >>>> as possible, so I wouldn't expect to be in that situation for long. I would >>>> probably tolerate a drop in performance during that time, too. (Not a >>>> super-severe one, but 20-30 percent? Sure.) >>>> >>>> What do you folks think? >>>> >>>> -B. >>>> >>>> >>>> _______________________________________________ >>>> riak-users mailing list >>>> [email protected] >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>>> >>> >>> _______________________________________________ >>> riak-users mailing list >>> [email protected] >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
