Batching with new Producer API

2015-02-26 Thread Akshat Aranya
Hi, I am using the new Producer API in Kafka 0.8.2. I am writing messages to Kafka that are ~310 bytes long with the same partition key to one single . I'm mostly using the default Producer config, which sets the max batch size to 16,384. However, looking at the JMX stats on the broker side, I

Re: Batching with new Producer API

2015-02-26 Thread Akshat Aranya
, at 09:49 AM, Akshat Aranya wrote: Hi, I am using the new Producer API in Kafka 0.8.2. I am writing messages to Kafka that are ~310 bytes long with the same partition key to one single . I'm mostly using the default Producer config, which sets the max batch size to 16,384. However

Re: Kafka - deployment size and topologies

2015-04-08 Thread Akshat Aranya
Thanks for the info, Todd. This is very useful. Please see my question inline: On Mon, Apr 6, 2015 at 10:24 AM, Todd Palino tpal...@gmail.com wrote: - Partition count (leader and follower combined) on each broker should stay under 4000 As far as topic volume goes, it varies widely. We

Re: Deploying Kafka Streams to production (Ubuntu 16.04 / systemd)

2016-07-18 Thread Akshat Aranya
Hi Phillip, In general, it is trivial to take an existing java -jar command and systemd-ify it. All you need to do is to run it as an ExecStart in the foreground in the systemd unit file, possibly with a User setting. Something like this should work: [Unit] Description=My stream app [Service]

Re: Deploying Kafka Streams to production (Ubuntu 16.04 / systemd)

2016-07-18 Thread Akshat Aranya
ks again for your help. > > > > Phillip > > > > *From: *Akshat Aranya <aara...@gmail.com> > *Date: *Monday, July 18, 2016 at 10:12 AM > *To: *"users@kafka.apache.org" <users@kafka.apache.org> > *Cc: *Phillip Mann <pm...@trulia.com>