Re: Ensuring that the message is persisted after acknowledgement

2021-08-24 Thread Peter Bukowinski
Kunal, I recommend looking at the broker and topic parameters that include the term “flush” , such as https://kafka.apache.org/documentation/#topicconfigs_flush.messages Kafka lets you configure how often log messages are

Re: Ensuring that the message is persisted after acknowledgement

2021-08-24 Thread Kunal Goyal
Hi Sunil The article that you shared talks about acks. But even if the message is received by all in-sync replicas and kafka sends response back to the producer, it is possible that none of the replicas did not flush the messages to disk. So, if all the replicas crash for some reason, the

How to use CRL (Certificate Revocation List) with Kafka

2021-08-24 Thread Darshan
Hi We have a private CA and our Kafka Brokers are signed by a private CA. Bunch of external clients connect to our broker and before connecting they download the private CA's cert and add it to truststore. Everything works fine. On the Kafka broker side, we want to use CRL before we authenticate

Conditional Produce in Kafka (KAFKA-2260)

2021-08-24 Thread Niket Goel
Hi all, I am new to the Kafka project and while having a discussion with a colleague, I was pointed to the JIRA KAFKA-2260 [1] which talks about adding the ability to perform "conditional produce" requests to Kafka. This idea sounded very

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-24 Thread Yoda Jedi Master
Thank you for your help, I will check it and try it :-) On Mon, Aug 16, 2021 at 11:45 AM Bruno Cadonna wrote: > Hi Yoda, > > for certain cases, Kafka Streams allows you to specify handlers that > skip the problematic record. Those handlers are: > > 1. deserialization exception handler

Re: Kafka attempt rediscovery never happen

2021-08-24 Thread Luke Chen
Hi Antonio, What you can do is to check the broker side log, to see if there's any info. If no, you can also enable DEBUG log to have more info for troubleshooting. Thank you. Luke On Mon, Aug 23, 2021 at 11:27 PM Antonio Pires wrote: > Hello fellow Kafka users, > > I've been trying to