Kafka streams - runs out of memory

2018-08-14 Thread AshokKumar J
Hi, we have a stream application that uses the low level API. We persist the data into the key value state store. For each record that we retrieve from the topic we perform a lookup against the store to see if it exists, if it does then we update the existing, else we simply add the new record.

Re: unable to reconfigure ssl truststore dynamically on broker

2018-08-14 Thread Manikumar
Hi, There is no specific doc. "--command-config" option takes admin client configs (. AdminClient configs are listed here: http://kafka.apache.org/documentation/#adminclientconfigs On Tue, Aug 14, 2018 at 10:35 PM John Calcote wrote: > Manikumar, > > Thank you. You are right - the security.pro

Re: Question how to drop a unused group id?

2018-08-14 Thread Vahid S Hashemian
These monitors are not consuming / committing offsets as part of the group, are they? Normally that should not be the case (e.g. the consumer group command tool retrieves the current position and the log end offset but it doesn't affect how long the group lives). Only consumers that commit offs

Re: unable to reconfigure ssl truststore dynamically on broker

2018-08-14 Thread John Calcote
Manikumar, Thank you. You are right - the security.protocol is NOT being passed to adminclient.properties file. Where exactly is the doc for that file? I searched for hours and finally had to guess at which options should be in there. J On Tue, Aug 14, 2018, 8:54 AM Manikumar wrote: > looks li

Re: unable to reconfigure ssl truststore dynamically on broker

2018-08-14 Thread Daniel Roy
unsuscribe On Tue, Aug 14, 2018 at 4:54 PM, Manikumar wrote: > looks like port is wrong or security.protocol config is not passed to > adminclient.properties file > > On Tue, Aug 14, 2018 at 7:23 PM John Calcote > wrote: > > > Hi, > > > > I'm using the latest kafka broker - 2.0.0 with scala 2.1

Re: unable to reconfigure ssl truststore dynamically on broker

2018-08-14 Thread Manikumar
looks like port is wrong or security.protocol config is not passed to adminclient.properties file On Tue, Aug 14, 2018 at 7:23 PM John Calcote wrote: > Hi, > > I'm using the latest kafka broker - 2.0.0 with scala 2.12. I have a > complete SSL configuration working, but I add clients occasionally

unable to reconfigure ssl truststore dynamically on broker

2018-08-14 Thread John Calcote
Hi, I'm using the latest kafka broker - 2.0.0 with scala 2.12. I have a complete SSL configuration working, but I add clients occasionally and want to be able to tell the broker to reload it's ssl truststore (as new certs have been added to it). I've followed the doc here: https://cwiki.apache.or

help regarding log.flush.offset.checkpoint.interval.ms & log.flush.start.offset.checkpoint.interval.ms

2018-08-14 Thread Kalpa 1977
Hi all, As I read, I believe log.flush.offset.checkpoint.interval.ms is log recovery point from replicas across different brokers. However, I am not getting start.offset.checkpoint. basically, what is this parameter, what does start mean opposed to currently flushed? how the latter help? An