Hi,
We send small transactions (2 to 3 messages) a few times per second from each
of 5 to 10 producers, depending on the consumer partition size.
Regards,
John
On 14/11/20 03:55, "Guozhang Wang" <wangg...@gmail.com> wrote:
Hello John,
It's a bit hard to reason what's your total producing traffic to the Kafka
brokers, e.g. how frequently do you send a `whole group of messages`? If it
is every 10ms e.g. then your traffic would be 100K per sec, which is pretty
large enough with transactions.
Guozhang
On Fri, Nov 13, 2020 at 1:09 AM John Coleman <john...@central.tech> wrote:
> Hi,
>
> We have very high latency ~1 second when we send a small number of small
> messages to Kafka. The whole group of messages is less than 1k. We are
> thinking better to just send the messages together as 1 message containing
> a collection of all the messages.
>
> Is there something that we can tweak to reduce this latency? We have 10
> producers and they publish to a topic with 5 consumers. Be nice if we
don’t
> have to change our message structure.
>
> TIA
> John
>
--
-- Guozhang
