We have pretty much the same setup as this, running on m1.large instances. On Mon, Nov 19, 2012 at 11:44 AM, Matthew Rathbone <matt...@foursquare.com>wrote:
> We RAID-0 4 EBS disks, we find that to be most performant, although it does > leave you more vulnerable to EBS network errors and outages. Ideally, if > you're spread across AZ's you could do some clever routing to > geographically local brokers, whilst keeping the others as a backup in case > of failure. > > In the same vein, we have N+1 brokers, where N is how many we reasonably > think we need, this way we can hopefully survive outages. > > > On Mon, Nov 19, 2012 at 1:30 PM, David Arthur <mum...@gmail.com> wrote: > > > I'd only consider m1.xlarge and higher for Kafka. The m1.xlarge have > > "high" I/O performance according to Amazon. This is disk I/O and network > > I/O performance. Of course you need to use EBS volumes if you want your > > Kafka brokers to survive reboots - you can expect reboots on AWS. Some > > people have reported I/O improvements by RAIDing EBS volumes ( > > http://alestic.com/2009/06/ec2-ebs-raid). Deploying in the same region > as > > your application will also improve performance. > > > > > > On Nov 16, 2012, at 8:04 PM, Senthilvel Rangaswamy wrote: > > > > > Have folks implemented large installations on Kafka on Amazon EC2. I am > > > looking for best practices. Like the kind of nodes, EBS vs Instance > store > > > etc., > > > > > > -- > > > ..Senthil > > > > > > "If there's anything more important than my ego around, I want it > > > caught and shot now." > > > - Douglas Adams. > > > > > > > -- > Matthew Rathbone > Foursquare | Software Engineer | Server Engineering Team > matt...@foursquare.com | @rathboma <http://twitter.com/rathboma> | > 4sq<http://foursquare.com/rathboma> >
