Jdbc Source Connector Holding more active sessions

2021-04-19 Thread vishnu murali
Hi everyone we are using Kafka JDBC connector to connect to Redshift database we created three connectors to pull the data from three different table to three different topic we observed that each connector holding multiple active sessions even if we delete the connectors, active sessions are

Re: Kafka Stream: State replication seems unpredictable.

2021-04-19 Thread Guozhang Wang
Hello Mangat, What you've encountered is a "zombie writer" issue, that is, Thread-2 did not know there's already a new rebalance and hence its partitions have been migrated out, until it tries to commit and then got notified of the illegal-generation error and realize itself is the "zombie"

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

2021-04-19 Thread Konstantine Karantasis
Congratulations Randall! Konstantine On Mon, Apr 19, 2021 at 1:14 AM Bruno Cadonna wrote: > Congrats Randall! Well deserved! > > Bruno > > On 17.04.21 01:43, Matthias J. Sax wrote: > > Hi, > > > > It's my pleasure to announce that Randall Hauch in now a member of the > > Kafka PMC. > > > >

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread Ran Lupovich
Hello, Maybe I missed it in the documentations but where can I read about what is the future plan for the zookeeper managed ACLs ? בתאריך יום ב׳, 19 באפר׳ 2021, 22:48, מאת Israel Ekpo ‏: > This is fantastic news! > > Thanks everyone for contributing and thanks John for managing the release. > >

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread Adam Bellemare
Wow, 2.8! I remember 0.8.2.0 :) Thanks for all the work everyone, and thank you John for putting together the release! On Mon, Apr 19, 2021 at 3:48 PM Israel Ekpo wrote: > This is fantastic news! > > Thanks everyone for contributing and thanks John for managing the release. > > On Mon, Apr 19,

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread Israel Ekpo
This is fantastic news! Thanks everyone for contributing and thanks John for managing the release. On Mon, Apr 19, 2021 at 1:10 PM Guozhang Wang wrote: > This is great! Thanks to everyone who has contributed to the release. > > On Mon, Apr 19, 2021 at 9:36 AM John Roesler wrote: > >> The

value.subject.name.strategy for multiple event types not applied to internal topics in kafka

2021-04-19 Thread Sidney
hello, in kafka, we have a requirement for multiple event types in the same topic (we use avro w/ schema registry). we were able to achieve this using the following configuration:

Re: [ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread Guozhang Wang
This is great! Thanks to everyone who has contributed to the release. On Mon, Apr 19, 2021 at 9:36 AM John Roesler wrote: > The Apache Kafka community is pleased to announce the > release for Apache Kafka 2.8.0 > > Kafka 2.8.0 includes a number of significant new features. > Here is a summary

[ANNOUNCE] Apache Kafka 2.8.0

2021-04-19 Thread John Roesler
The Apache Kafka community is pleased to announce the release for Apache Kafka 2.8.0 Kafka 2.8.0 includes a number of significant new features. Here is a summary of some notable changes: * Early access of replace ZooKeeper with a self-managed quorum * Add Describe Cluster API * Support mutual

Re: MirrorMaker 2 with SSL

2021-04-19 Thread Men Lim
I got this working late last week. I did the shotgun approach with all the values in both the distributor and connector.json and once it worked, I started to remove things to see when it stop working. On Sun, Apr 18, 2021 at 10:00 PM Ning Zhang wrote: > if the source kafka cluster is

Re: State Store Data Retention

2021-04-19 Thread Bruno Cadonna
Hi Upesh, The answers to your questions are: 1. The configs cleanup.policy and retention.ms are topic configs. Hence, they only affect the changelog of a state store, not the local state store in a Kafka Streams client. Locally, window and session stores remove data they do not need

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

2021-04-19 Thread Bruno Cadonna
Congrats Randall! Well deserved! Bruno On 17.04.21 01:43, Matthias J. Sax wrote: Hi, It's my pleasure to announce that Randall Hauch in now a member of the Kafka PMC. Randall has been a Kafka committer since Feb 2019. He has remained active in the community since becoming a committer.

Re: Kafka Stream: State replication seems unpredictable.

2021-04-19 Thread mangat rai
Thanks, Guozhang, I was knocking myself with Kafka's various consumer rebalancing algorithms in the last 2 days. Could I generalize this problem as *Any in-memory state store backed by a changelog topic will always risk having interleaved writes from two different writers during rebalancing?*

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

2021-04-19 Thread Robin Moffatt
Great news, congratulations Randall! -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Sat, 17 Apr 2021 at 00:45, Matthias J. Sax wrote: > Hi, > > It's my pleasure to announce that Randall Hauch in now a member of the > Kafka PMC. > > Randall has been a Kafka