Re: High latency for Kafka transactions

Have you increased the producer linger.ms property [1] to ~1s? (Default is 0 so records are sent immediately; it generally shouldn't be more than a few ms - depending on how long your serialization takes). If not, perhaps your serialization (or compression) might be the problem; best to

Re: High latency for Kafka transactions

Hello John, It's a bit hard to reason what's your total producing traffic to the Kafka brokers, e.g. how frequently do you send a `whole group of messages`? If it is every 10ms e.g. then your traffic would be 100K per sec, which is pretty large enough with transactions. Guozhang On Fri, Nov

Re: StreamBuilder#stream multiple topic and consumed api

Hi Uirco, The method that doesn’t take Consumed will fall back to the configured “default serdes”. If you don’t have that confit set, it will just keep them as byte arrays, which will probably give you an exception at runtime. You’ll probably want to use the Consumed argument to set your

Re: MirrorMaker 2 Reload Configuration

Hi, I tried as well to stop all instances of MM2, but it didn't help for me. I had to stop all MM2 instances, delete the mm2-config and mm2-status topics on the destination cluster and start up all MM2 instances again. Peter

Connect: Setting topic creation config via file (vs. API)

Hi all, I’m trying to make use of the somewhat new functionality of Connect to create new topics on the fly as described in KIP-158. My test

Re: MirrorMaker 2 Reload Configuration

Hi All, After inspecting a few internal topics and running console consumer to see the payloads in the mm2-configs topic identified that properties are indeed not getting refreshed, I've assumed that mm2 internally is joining the existing cluster, so to refresh config I've tried to completely

Re: MirrorMaker 2 Reload Configuration

Hi Ryanne, I will open an issue in Jira. I see mm2-config and mm2-status topics on both the source and destination clusters. Should I purge all of them? Or is it enough to purge just the destination topics? Thanks, Peter On Wed, 11 Nov 2020 at 19:33, Ryanne Dolan wrote: > Hey guys, this is

High latency for Kafka transactions

Hi, We have very high latency ~1 second when we send a small number of small messages to Kafka. The whole group of messages is less than 1k. We are thinking better to just send the messages together as 1 message containing a collection of all the messages. Is there something that we can tweak