Re: Kafka Streams not auto-creating the state store changelog topic

Hello Eno, So, if I change the input topic partitions, it affects the ability of kafka streams to find partitions for the state store changelog? I think I'm missing something here. In my case, the application was new, so it's for sure that there were no changes. Also, if I have regex for the input

Re: Kafka streams regex match

Damian, I am getting a syntax error. I have responded on gist. Appreciate any inputs. - Shekar On Sat, Jul 29, 2017 at 1:57 AM, Damian Guy wrote: > Hi, > > I left a comment on your gist. > > Thanks, > Damian > > On Fri, 28 Jul 2017 at 21:50 Shekar Tippur wrote: > > > Damien, > > > > Here is a

Kafka (0.9) ConsumerRebalanceListener onPartitionsRevoked not called.

Hello, We’re using custom storage for kafka offsets. Partitions are assigned automatically, and we’re using ConsumerRebalanceListener to save offsets, clear threads and so on. In ‘onPartitionsAssinged’ method, we look up committed offsets for assigned partitions from the custom store. If no comm

KafkaProducer (0.9): Single producer multiple topics.

Hello, We’re currently using one producer per topic. We’re considering switching to one producer for multiple topics for obvious reasons. Now, is there a case where one topic might be slower for some reason, and not the others? And if there is such a case, how would it play out with a single pr

Console Producer/Consumer - Leader Not Available when implemented with TLS/SSL

Hello, I wanted to add TLS/SSL to my kafka setup. To start with, I went through the kafka SSL documenation on main website. I have done the following: 1) Imported the signed certificates to keystore 2) Imported the root CA 3) Verified that the keystore and trust store password are correct by usin

Re: Kafka Streams not auto-creating the state store changelog topic

Hi, Could you check if this helps: https://stackoverflow.com/questions/42329387/failed-to-rebalance-error-in-kafka-streams-with-more-than-one-topic-partition Thanks Eno >

Re: Adding partitons | Unaffected producers

See the producer param called metadata.max.age.ms which is "The period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions." -hans > On Aug 4, 2017, at 5:17 AM, Sameer

Re: Kafka Streams: why aren't offsets being committed?

Thank you Matthias and Bill, Just want to confirm that was my offsets *were *being committed but I was being affected by `offsets.retention.minutes` which I did not know about. I set offsets.retention.minutes=2147483647 offsets.retention.check.interval.ms=9223372036854775807 Will keep an eye on

Re: [DISCUSS] Streams DSL/StateStore Refactoring

Hi Guozhang, thank you very much for the reply. It explained a lot more of your reasoning to me once again! I have to disagree with you on the first point. As you mentioned the Join Case. A Join is usually a "logically" materialized table and its KTableValueGetterSupplier is to be used when

Adding partitons | Unaffected producers

According to Kafka docs, producer decides on which partition the data shall reside. I am aware that neither broker nor producer needs to be restarted to detect added partitions. Would like to understand if there is some frequency through which producer detects new partitions. Though consumers wer

Re: Kafka Streams not auto-creating the state store changelog topic

Hello Eno, Thanks for considering the question. How I am creating the state stores: StateStoreSupplier stateStoreSupplier = StateStorStores.create("testing-2-store").withKeys(keySerde).withValues(valueSerde).persistent().build(); TopologyBuilder builder = ... builder.addStateStore(stateStoreSuppl

Re: Kafka Streams not auto-creating the state store changelog topic

Hi Anish, Could you give more info on how you create the state stores in your code? Also could you copy-paste the exact error message from the log? Thanks Eno > On Aug 4, 2017, at 9:05 AM, Anish Mashankar wrote: > > I have a new application, call it streamsApp with state stores S1 and S2. > So

Kafka Streams not auto-creating the state store changelog topic

I have a new application, call it streamsApp with state stores S1 and S2. So, according to the documentation, upon the first time startup, the application should've created the changelog topics streamsApp-S1-changelog and streamsApp-S2-changelog. But I see that these topics are not created. Also, t

Re: Kafka 0.11.0 problem with transactions.

Hi, Many thanks for the explanation. I just wanted to do some tests and look around, I think the perf tool will be OK for me. Indeed, it’s a bit tricky with the console producer. To do a commit after every new line (enter) sounds not right. Something like begin; message, message… commit;

RE: Need clarification on Kafka Usage within our product..

Thank you so much Shane for responding to the query and confirming the usage of logo. Regards, Rinu -Original Message- From: Shane Curcuru [mailto:a...@shanecurcuru.org] Sent: Thursday, August 3, 2017 11:33 PM To: Sunil, Rinu Cc: tradema...@apache.org; users@kafka.apache.org Subject: Re