Re: Kafka Best Practices in AWS

2015-10-26 Thread Jennifer Fountain
On Mon, Oct 26, 2015 at 1:18 PM, Steve Brandon wrote: > Depending on the consumer, I tend to work with c4 types and have found > better success. Although we run about 200+ consumers per server on 16 core > servers and see moderate load. YMMV > We are currently using storm for our consumer.W

Re: Kafka Best Practices in AWS

2015-10-26 Thread Steve Brandon
Depending on the consumer, I tend to work with c4 types and have found better success. Although we run about 200+ consumers per server on 16 core servers and see moderate load. YMMV On Oct 26, 2015 8:32 AM, "Jennifer Fountain" wrote: > We installed four nodes (r3.xlarge) with approx 100 topics.

Kafka Best Practices in AWS

2015-10-26 Thread Jennifer Fountain
We installed four nodes (r3.xlarge) with approx 100 topics. Some topics have a replication factor of 4, some have 3 and some have 2. Each node is running about 90% cpu usage and we are seeing performance issues with messages consumption. Would anyone have any words of wisdom or recommendations fo