Re: Kafka Streams Processor API state stores not restored via changelog topics

Great to hear! Always a pleasure. Guozhang On Tue, Mar 30, 2021 at 8:04 PM Upesh Desai wrote: > Hi Guozhang, > > > > We can confirm the behavior with the 2.7.1 release. Appreciate all the > help! > > > > Cheers, > > Upesh > > > Upesh Desai​ | Senior Software Developer |

Re: Kafka Streams Processor API state stores not restored via changelog topics

Hi Guozhang, We can confirm the behavior with the 2.7.1 release. Appreciate all the help! Cheers, Upesh Upesh Desai | Senior Software Developer | ude...@itrsgroup.com www.itrsgroup.com From: Guozhang Wang Date: Tuesday, March 30, 2021 at 2:10 PM To: Users Cc: Bart Lilje Subject: Re: Kafka

Re: [VOTE] 2.8.0 RC0

Hello again, all, I just wanted to mention that I am aware of Justin's concerns in the 2.6.2 thread: https://lists.apache.org/thread.html/r2df54c11c10d3d38443054998bc7dd92d34362641733c2fb7c579b50%40%3Cdev.kafka.apache.org%3E I plan to make sure we address these concerns before the actual 2.8.0

[VOTE] 2.8.0 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 2.8.0. This is a major release that includes many new features, including: * Early-access release of replacing Zookeeper with a self- managed quorum * Add Describe Cluster API * Support

Re: MirrorMaker 2 and Negative Lag

Hey Ning, I believe "if the CG offsets do not contain a pair of , simply sync the offsets from source" could be the problematic behavior here? I'm not very familiar with Mirrormaker's internals, so can't say for sure. As I described previously, this "negative lag" problem happens when a target

Re: Kafka Streams Processor API state stores not restored via changelog topics

Great, I think https://issues.apache.org/jira/browse/KAFKA-12323 is indeed the root cause then. Note that this is only an issue with punctuation triggered events, where `context.timestamp()` would return 0 (and it is fixed in the yet-to-release 2.7.1/2.8.0). You can consider applying the patch if

Re: Kafka Streams Processor API state stores not restored via changelog topics

Hi Guozhang, Great to hear we might have found the issue! To answer your question, the changelog record is generated by us calling ‘store.put(key,value)’ from the punctuate callback, which makes sense then because the timestamp would be 0 like you saw in your test as well. Best, Upesh Upesh

Re: Kafka Streams Processor API state stores not restored via changelog topics

Hello Upesh, These are super helpful logs, and I think I'm very close to the root cause of it. You see, the written changelog record's timestamp is set to 0 (i.e. January 1st 1970 at midnight GMT), and hence given a reasonable Kafka server start time (presumingly in 21st century), the retention

Re: Learning Kafka

Thank you a lot, I'm really gonna learn something cool with all these examples! ‐‐‐ Original Message ‐‐‐ On Tuesday, March 30, 2021 9:17 AM, David Klein wrote: > Here are some use case blog posts: > https://www.confluent.io/blog/category/use-cases >

Re: Learning Kafka

Here are some use case blog posts: https://www.confluent.io/blog/category/use-cases And here are some use case podcast episodes: https://developer.confluent.io/podcast/topic/use-cases

Re: Learning Kafka

Hey, check this, https://www.confluent.io/resources Among others you should read IN DETAILS Kafka: The Definitive Guide v2 and v1, and there are many technical online talks available in the link, each talk describe a partiuclar subject you might be interested in (delivery semantics,

Learning Kafka

Hi to all!... I learned a while last year about apache Kafka and, even though I have some 'experience', I now see that there are a lot of things that I don't understand. I'm asking if there are templates, or advanced use cases of Kafka I can learn about, so I 'open' my mind with the things