Re: KStreams in-memory state-store

2020-02-03 Thread Matthias J. Sax
I created https://issues.apache.org/jira/browse/KAFKA-9501 for tracking. On 1/30/20 10:17 AM, Matthias J. Sax wrote: > What you say is correct. This is a severe bug rendering standby tasks > useless for in-memory state stores. Can you pleas open a ticket so we > can fix it asap? > > > -Matthias

Re: High CPU in 2.2.0 kafka cluster

2020-02-03 Thread Ismael Juma
Yes, there were some improvements in 2.4.0. However, the puzzling aspect of your description is that the CPU usage basically disappeared when you removed the consumers even though your profiler claimed that the replica fetchers were using a large chunk of the CPU. Ismael On Mon, Feb 3, 2020 at 9:

Re: High CPU in 2.2.0 kafka cluster

2020-02-03 Thread Brandon Barron
Haven't tried updating to 2.4.0 yet. Were there any related fixes or improvements in that version? I skimmed the changelog but I didn't see anything. I found this issue https://issues.apache.org/jira/browse/KAFKA-9039 which I thought could be related to our problems but it seems like it's projec

Re: [EXTERNAL] Re: Kafka consumers freeze (or topics are freezing)

2020-02-03 Thread Sullivan, Tim
Thanks M. At this point I’m hoping that there is something utility wise in kafka that I can use, or perhaps a configuration setting that I can “correct”. We may be decommissioning the application that uses the data from kafka. If this happened then I can probably decommission Kafka

Re: High CPU in 2.2.0 kafka cluster

2020-02-03 Thread Ismael Juma
Hi Brandon, Are you still seeing this behavior with Apache Kafka 2.4.0? Ismael On Fri, Jan 31, 2020 at 10:51 AM Brandon Barron wrote: > We were running client version 2.3.0 for a while, then bumped to 2.3.1 for > a particular kafka streams bug fix. We saw this issue while both versions > were

Re: [EXTERNAL] Re: Kafka consumers freeze

2020-02-03 Thread Sullivan, Tim
I’m not seeing any errors, or I don’t know where to look. I inherited my Kafka cluster from another dev team. What I see are e-mails from my downstream users saying that their jobs are stuck as Kafka isn’t taking in their messages. So maybe it is a Topic issue for me vs a Consumer i

Re: Kafka audit logs in Apache Kafka?

2020-02-03 Thread Ivan Yurchenko
Hi, The KIP for this is being discussed now: https://cwiki.apache.org/confluence/display/KAFKA/KIP-567%3A+Kafka+Cluster+Audit Ivan On Fri, 31 Jan 2020 at 17:52, Otis Gospodnetić wrote: > Hi, > > Does Apache Kafka have audit logs equivalent of > https://docs.confluent.io/current/security/audit