Re: Increased latency when consuming from the closest ISR

2022-05-11 Thread benitocm
l/11965 > > I know one improvement will be included in the v3.2.0 release, which will > be released soon. > Maybe you can give it a try to see if it improves the throughput. > > Thank you. > Luke > > On Wed, May 11, 2022 at 2:56 PM benitocm wrote: > > > Hi, >

Increased latency when consuming from the closest ISR

2022-05-11 Thread benitocm
Hi, We are using the functionality provided by KIP-392 (a consumer can fetch the data from a ISR replica instead of the partition leader) in a Kafka cluster stretched between two very close DCs (average round-trip latency about 2 milliseconds). What we have seen is that, on average, when the

How to reduce the latency to interact with a topic?

2021-05-24 Thread benitocm
Hi all, I am using a Kafka topic to hanle invalidation events inside a system A that consists of different nodes. When a node of System A detects that a situation for invalidation happens, it produces an event to the invalidation topic. The rest of the nodes in System A consume that invalidation

Kafka Connect metrics as Yammer metrics

2021-05-21 Thread benitocm
Hi, Currently I am using the library https://github.com/RTBHOUSE/kafka-graphite-reporter in the brokers and clients to extract metrics and publish them into Graphite. This can be done for the metrics that are available as Yammer metrics. The thing is that I wanted to do the same with Connect

Re: MM2 for DR

2020-02-12 Thread benitocm
e DCs > and apps keep running like nothing happened. > > On Wed, Feb 12, 2020 at 12:05 AM benitocm wrote: > > > Hi Ryanne, > > > > Please could you elaborate a bit more about the active-active > > recommendation? > > > > Thanks in advance > &g

Re: [External] Re: MM2 for DR

2020-02-12 Thread benitocm
se, but > goes over some problems they experienced and addressed w.r.t. inter-cluster > replication and federation: > > https://www.confluent.io/kafka-summit-san-francisco-2019/kafka-cluster-federation-at-uber > > On Tue, Feb 11, 2020 at 10:05 PM benitocm wrote: > > > Hi Ry

Re: MM2 for DR

2020-02-11 Thread benitocm
Hi Ryanne, Please could you elaborate a bit more about the active-active recommendation? Thanks in advance On Mon, Feb 10, 2020 at 10:21 PM benitocm wrote: > Thanks very much for the response. > > Please could you elaborate a bit more about "I'd > arc in that direction. Ins

Re: MM2 for DR

2020-02-10 Thread benitocm
ch of times would leave a trail of > replication hops. > > Also, you might look into implementing a custom ReplicationPolicy. For > example, you could squash "secondary.primary.topic1" into something shorter > if you like. > > Ryanne > > On Mon, Feb 10, 2020 at 1:2

MM2 for DR

2020-02-10 Thread benitocm
Hi, After having a look to the talk https://www.confluent.io/kafka-summit-lon19/disaster-recovery-with-mirrormaker-2-0 and the https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382 I am trying to understand how I would use it in the setup that I have. For now, we