Re: Passing states stores around

Hi John, I'm using PAPI to create my topology which has 5 process functions. Out which 3 are large functions (more than 1000 lines of code) and they have about 2 KV stores each. Since the code is fairly large per function, I have them split into classes by functionalities. and some method in the

[VOTE] 2.5.0 RC1

Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 2.5.0. The first release candidate included an erroneous NOTICE file, so another RC was needed to fix that. This is a major release of Kafka which includes many new features,

Re: Passing states stores around

Hi Navneeth, This sounds like an unusual use case. Can you provide more information on why this is required? Thanks, John On Mon, Mar 9, 2020, at 12:48, Navneeth Krishnan wrote: > Hi All, > > Any suggestions? > > Thanks > > On Sat, Mar 7, 2020 at 10:13 AM Navneeth Krishnan > wrote: > > >

Re: Subject: [VOTE] 2.4.1 RC0

Is this a blocker given that it's been like this for months and no-one noticed? 2.4.1 seemingly has all the votes needed for the release. Why not go ahead with it. When KAFKA-9675 is merged, it can be included in the next release. Ismael On Mon, Mar 9, 2020 at 8:43 PM Bill Bejeck wrote: >

Re: Subject: [VOTE] 2.4.1 RC0

Thanks to everyone for voting. A new blocker has surfaced https://issues.apache.org/jira/browse/KAFKA-9675, so I'll do another RC soon. Thanks again. Bill On Mon, Mar 9, 2020 at 1:35 PM Levani Kokhreidze wrote: > +1 non-binding. > > - Built from source > - Ran unit tests. All passed. > -

Re: Passing states stores around

Hi All, Any suggestions? Thanks On Sat, Mar 7, 2020 at 10:13 AM Navneeth Krishnan wrote: > Hi All, > > Is there a recommended way of passing state stores around across different > classes? The problem is state store can be fetched only if you have access > to the context and in most of the

Re: MirrorMaker2 to mirror new topics?

Hey Peter, often the problem is the replication factor or min ISRs being misconfigured, which can prevent MM2 (or anything else) from sending. Hope that helps! Ryanne On Mon, Mar 9, 2020, 10:35 AM Péter Sinóros-Szabó wrote: > Hi, > > it is mirroring but the problem was that on of the task

Re: Question about log flusher real frequency

Hi Alexandre, Thank you for your quick answer. I want to monitor it cause I'm trying to find out the reason why our existing Kafka cluster is configured to flush data every10 milliseconds! (people who configured it are not available anymore to answer). As that value seems really low to me, I

Question about log flusher real frequency

Hello, By default, both log.flush.interval.ms and log.flush.interval.messages are set to Long.MAX_VALUE. As I understand, it makes Kafka flush log to disk (fsync) only depends on file system. Is there any simple way to monitor that frequency ? Is there a rule of thumb to estimate that value

Re: MirrorMaker2 to mirror new topics?

Hi, it is mirroring but the problem was that on of the task failed with not being able to produce messages to the destination cluster and the task stopped there. Peter On Mon, 9 Mar 2020 at 11:20, Péter Sinóros-Szabó < peter.sinoros-sz...@transferwise.com> wrote: > Hi, > > Should MM2

MirrorMaker2 to mirror new topics?

Hi, Should MM2 automatically start to mirror new topics? I see in the logs that MM2 created the remote topic, but it is not mirroring the new messages. In the source cluster, topic was created 3 days ago and there is about 200 million messages in it. Just to test if the messages are there, I