Hey Tom, I tried to expand the FAQ on this, let me know if you feel like it answers your question:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowdoIchoosethenumberofpartitionsforatopic%3F Folks--let me know if I missed anything there. I tried to give both a simple answer and a more complete answer. -Jay On Wed, Aug 21, 2013 at 9:52 AM, Tom Brown <[email protected]> wrote: > Philip, > > How many topics per broker (just one?) And what is the read/write profile > of your setup? > > --Tom > > > On Wed, Aug 21, 2013 at 12:24 PM, Philip O'Toole <[email protected]> > wrote: > > > On Wed, Aug 21, 2013 at 10:06:50AM -0400, Tom Brown wrote: > > > I have not been able to find reliable advice regarding how many > > partitions > > > should exist on a single broker. How many partitions have you used, and > > > what kind of throughput have you seen? > > > > We settled on 8 per broker, and usually run 3 brokers in production > > environments, giving us a total of 24 partitions. Throughput has been > > superb. > > > > For integration testing however, we usually use just 1 or 2 partitions. > > > > Philip > > > > > > > > Thanks in advance! > > > > > > --Tom > > > > -- > > Philip O'Toole > > > > Senior Developer > > Loggly, Inc. > > San Francisco, CA. > > www.loggly.com > > > > Come join us! > > http://loggly.com/company/careers/ > > >
