On Mon, Aug 12, 2013 at 2:38 PM, Dave Martorana <[email protected]> wrote: > Jared - thanks for the links. I'm in the same boat with Brady with weighing > deployment options in AWS. > > Jeremiah - isn't EBS the only option once your data starts reaching into the > hundreds-of-gigs?
Several instances give you > 1TB of instance store if you combine the volumes. [0] It may also be worth noting that if you flip the EBS-optimized bit on instances that support it, you can get 500Mbps-1000Mbps between your instance and EBS. [1] > Dave [0] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#StorageOnInstanceTypes [1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#EBSOptimized > On Sun, Aug 11, 2013 at 8:57 PM, Jared Morrow <[email protected]> wrote: >> >> +1 to what Jeremiah said, putting a 4 or 5 node cluster in each US West >> and US East using MDC between them would be the optimum solution. I'm also >> not buying consistent latencies between AZ's, but I've also not tested it >> personally in a production environment. We have many riak-users members on >> AWS, so hopefully more experienced people will chime in. >> >> If you haven't seen them already, here's what I have in my "Riak on AWS" >> bookmark folder: >> >> http://media.amazonwebservices.com/AWS_NoSQL_Riak.pdf >> http://docs.basho.com/riak/latest/ops/tuning/aws/ >> http://basho.com/riak-on-aws-deployment-options/ >> >> -Jared >> >> >> >> >> On Sun, Aug 11, 2013 at 6:11 PM, Jeremiah Peschka >> <[email protected]> wrote: >>> >>> I'd be wary of using EBS backed nodes for Riak - with only a single >>> ethernet connection, it wil be very easy to saturate the max of 1000mbps >>> available in a single AWS NIC (unless you're using cluster compute >>> instances). I'd be more worried about temporarily losing contact with a node >>> through network saturation than through AZ failure, truthfully. >>> >>> The beauty of Riak is that a node can drop and you can replace it with >>> minimal fuss. Use that to your advantage and make every node in the cluster >>> disposable. >>> >>> As far as doubling up in one AZ goes - if you're worried about AZ >>> failure, you should treat each AZ as a separate data center and design your >>> failure scenarios accordingly. Yes, Amazon say you should put one Riak node >>> in each AZ; I'm not buying that. With no guarantee around latency, and no >>> control around between DCs, you need to be very careful how much of that >>> latency you're willing to introduce into your application. >>> >>> Were I in your position, I'd stand up a 5 node cluster in US-WEST-2 and >>> be done with it. I'd consider Riak EE for my HA/DR solution once the >>> business decides that off-site HA/DR is something it wants/needs. >>> >>> >>> --- >>> Jeremiah Peschka - Founder, Brent Ozar Unlimited >>> MCITP: SQL Server 2008, MVP >>> Cloudera Certified Developer for Apache Hadoop >>> >>> >>> On Sun, Aug 11, 2013 at 1:52 PM, Brady Wetherington >>> <[email protected]> wrote: >>>> >>>> Hi all - >>>> >>>> I have some questions about how I want my Riak stuff to work - I've >>>> already asked these questions of some Basho people and gotten some answers, >>>> but thought I would toss it out into the wider world to see what you all >>>> have to say, too: >>>> >>>> First off - I know 5 instances is the "magic number" of instances to >>>> have. If I understand the thinking here, it's that at the default >>>> redundancy >>>> level ('n'?) of 3, it is most likely to start getting me some scaling >>>> (e.g., >>>> performance > just that of a single node), and yet also have redundancy; >>>> whereby I can lose one box and not start to take a performance hit. >>>> >>>> My question is - I think I can only do 4 in a way that makes sense. I >>>> only have 4 AZ's that I can use right now; AWS won't let me boot instances >>>> in 1a. My concern is if I try to do 5, I will be "doubling up" in one AZ - >>>> and in AWS you're almost as likely to lose an entire AZ as you are a single >>>> instance. And so, if I have instances doubled-up in one AZ (let's say >>>> us-east-1e), and then I lose 1e, I've now lost two instances. What are the >>>> chances that all three of my replicas of some chunk of my data are on those >>>> two instances? I know that it's not guaranteed that all replicas are on >>>> separate nodes. >>>> >>>> So is it better for me to ignore the recommendation of 5 nodes, and just >>>> do 4? Or to ignore the fact that I might be doubling-up in one AZ? Also, >>>> another note. These are designed to be 'durable' nodes, so if one should go >>>> down I would expect to bring it back up *with* its data - or, if I >>>> couldn't, >>>> I would do a force-replace or replace and rebuild it from the other >>>> replicas. I'm definitely not doing instance-store. So I don't know if that >>>> mitigates my need for a full 5 nodes. I would also consider losing one node >>>> to be "degraded" and would probably seek to fix that problem as soon as >>>> possible, so I wouldn't expect to be in that situation for long. I would >>>> probably tolerate a drop in performance during that time, too. (Not a >>>> super-severe one, but 20-30 percent? Sure.) >>>> >>>> What do you folks think? >>>> >>>> -B. >>>> >>>> >>>> _______________________________________________ >>>> riak-users mailing list >>>> [email protected] >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>> >>> >>> _______________________________________________ >>> riak-users mailing list >>> [email protected] >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
