batch.size, linger.ms, max.in.flight.requests.per.connection, etc all effect the trade off between latency and throughput. Your throughput is very low so you should be able to tune for much better latency. Very easy to get 1-2 ms of latency out of Kafka.
Here is a video on Tuning Kafka for low latency guaranteed messaging -- Jiangjie (Becket) Qin (LinkedIn) <https://www.youtube.com/watch?v=oQe7PpDDdzA> which might give you much better context and understanding of what these parameters mean and how they work. -hans /** * Hans Jespersen, Principal Systems Engineer, Confluent Inc. * h...@confluent.io (650)924-2670 */ On Fri, Jan 13, 2017 at 12:36 AM, Patricia Callejo < patricia.call...@imdea.org> wrote: > Hi, > we are developing a kafka environment in order to be able to process up > hundreds of messages per second. The point is that if we send in different > partitions a lot of messages, the time of the process is very good in ms, > however if we send just a single message, the minimum time required is very > high, at least 1 or 2 seconds, even more. > Is there any configuration we are missing to avoid this minimum delay? or > is the minimum time required by kafka for the whole process? > > Best Regards, > Patricia
