Have you increased the producer linger.ms property [1] to ~1s? (Default is 0 so records are sent immediately; it generally shouldn't be more than a few ms - depending on how long your serialization takes).
If not, perhaps your serialization (or compression) might be the problem; best to profile the app to see where the time is being spent. [1]: https://kafka.apache.org/documentation/#linger.ms ________________________________ From: John Coleman <john...@central.tech> Sent: Friday, November 13, 2020 4:09 AM To: users@kafka.apache.org <users@kafka.apache.org> Subject: High latency for Kafka transactions Hi, We have very high latency ~1 second when we send a small number of small messages to Kafka. The whole group of messages is less than 1k. We are thinking better to just send the messages together as 1 message containing a collection of all the messages. Is there something that we can tweak to reduce this latency? We have 10 producers and they publish to a topic with 5 consumers. Be nice if we don’t have to change our message structure. TIA John