Re: [DISCUSS] Apache Kafka 3.8.0 release
Hi Kafka developers! Today is the KIP freeze deadline. All KIPs should be accepted by EOD today. Tomorrow morning (CEST timezone) I'll start summarizing all KIPs that have been approved. Please any KIP approved after tomorrow should be adopted in a future release version, not 3.8. Other relevant upcoming deadlines: - Feature freeze is on May 29th - Code freeze is June 12th Best, On Fri, May 3, 2024 at 3:59 PM Josep Prat wrote: > Hi Kafka developers! > I just wanted to remind you all of the upcoming relevant dates for Kafka > 3.8.0: > - KIP freeze is on May 15th (this is in a little less than 2 weeks) > - Feature freeze is on May 29th (this is in a little more than 25 days). > > If there is a KIP you really want to have in the 3.8 series, now is the > time to make the last push. Once the deadline for KIP freeze is over I'll > update the release plan with the final list of KIPs accepted and that may > make it to the release. > > Best! > > On Wed, Mar 6, 2024 at 10:40 AM Josep Prat wrote: > >> Hi all, >> >> Thanks for your support. I updated the skeleton release plan created by >> Colin. You can find it here: >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.8.0 >> >> Our last release stumbled upon some problems while releasing and was >> delayed by several weeks, so I won't try to shave some weeks from our plan >> for 3.8.0 (we might end up having delays again). Please raise your concerns >> if you don't agree with the proposed dates. >> >> The current proposal on dates are: >> >>- KIP Freeze: *15nd May *(Wednesday) >> - A KIP must be accepted by this date in order to be considered >> for this release. Note, any KIP that may not be implemented in a week, >> or >> that might destabilize the release, should be deferred. >>- Feature Freeze: *29th May *(Wednesday) >> - *major features merged & working on stabilisation, minor >> features have PR, release branch cut; anything not in this state will >> be >> automatically moved to the next release in JIRA* >>- Code Freeze: *12th June *(Wednesday) >>- At least two weeks of stabilization will follow Code Freeze, >>meaning we expect to release *no earlier* than *June 26th*. We will >>move as fast as we can, and aim for completion the earliest we can in >> June. >> >> I went through the KIP list, and found that these are the ones that might >> make it into the release: >> KIP-853: KRaft Controller Membership Changes (still under discussion) >> KIP-942: Add Power(ppc64le) support >> KIP-966: Eligible Leader Replicas >> KIP-974: Docker Image for GraalVM based Native Kafka Broker >> KIP-977: Partition-Level Throughput Metrics >> KIP-993: Allow restricting files accessed by File and Directory >> ConfigProviders >> KIP-994: Minor Enhancements to ListTransactions and DescribeTransactions >> APIs >> KIP-996: Pre-Vote >> KIP-1004: Enforce tasks.max property in Kafka Connect >> KIP-1005: Expose EarliestLocalOffset and TieredOffset >> KIP-1007: Introduce Remote Storage Not Ready Exception >> KIP-1019: Expose method to determine Metric Measurability >> >> Please review the plan and provide any additional information or updates >> regarding KIPs that target this release version (3.8). >> If you have authored any KIPs that have an inaccurate status in the list, >> or are not in the list and should be, or are in the list and should not be >> - please share it in this thread so that I can keep the document accurate >> and up to date. >> >> Looking forward to your feedback. >> >> Best, >> >> On Wed, Feb 28, 2024 at 10:07 AM Satish Duggana >> wrote: >> >>> Thanks Josep, +1. >>> >>> On Tue, 27 Feb 2024 at 17:29, Divij Vaidya >>> wrote: >>> > >>> > Thank you for volunteering Josep. +1 from me. >>> > >>> > -- >>> > Divij Vaidya >>> > >>> > >>> > >>> > On Tue, Feb 27, 2024 at 9:35 AM Bruno Cadonna >>> wrote: >>> > >>> > > Thanks Josep! >>> > > >>> > > +1 >>> > > >>> > > Best, >>> > > Bruno >>> > > >>> > > On 2/26/24 9:53 PM, Chris Egerton wrote: >>> > > > Thanks Josep, I'm +1 as well. >>> > > > >>> > > > On Mon, Feb 26, 2024 at 12:32 PM Justine Olshan >>> > > > wrote: >>> > > > >>> > > >>
Re: [DISCUSS] Apache Kafka 3.8.0 release
Hi Kafka developers! I just wanted to remind you all of the upcoming relevant dates for Kafka 3.8.0: - KIP freeze is on May 15th (this is in a little less than 2 weeks) - Feature freeze is on May 29th (this is in a little more than 25 days). If there is a KIP you really want to have in the 3.8 series, now is the time to make the last push. Once the deadline for KIP freeze is over I'll update the release plan with the final list of KIPs accepted and that may make it to the release. Best! On Wed, Mar 6, 2024 at 10:40 AM Josep Prat wrote: > Hi all, > > Thanks for your support. I updated the skeleton release plan created by > Colin. You can find it here: > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.8.0 > > Our last release stumbled upon some problems while releasing and was > delayed by several weeks, so I won't try to shave some weeks from our plan > for 3.8.0 (we might end up having delays again). Please raise your concerns > if you don't agree with the proposed dates. > > The current proposal on dates are: > >- KIP Freeze: *15nd May *(Wednesday) > - A KIP must be accepted by this date in order to be considered for > this release. Note, any KIP that may not be implemented in a week, or > that > might destabilize the release, should be deferred. >- Feature Freeze: *29th May *(Wednesday) > - *major features merged & working on stabilisation, minor features > have PR, release branch cut; anything not in this state will be > automatically moved to the next release in JIRA* >- Code Freeze: *12th June *(Wednesday) >- At least two weeks of stabilization will follow Code Freeze, meaning >we expect to release *no earlier* than *June 26th*. We will move as >fast as we can, and aim for completion the earliest we can in June. > > I went through the KIP list, and found that these are the ones that might > make it into the release: > KIP-853: KRaft Controller Membership Changes (still under discussion) > KIP-942: Add Power(ppc64le) support > KIP-966: Eligible Leader Replicas > KIP-974: Docker Image for GraalVM based Native Kafka Broker > KIP-977: Partition-Level Throughput Metrics > KIP-993: Allow restricting files accessed by File and Directory > ConfigProviders > KIP-994: Minor Enhancements to ListTransactions and DescribeTransactions > APIs > KIP-996: Pre-Vote > KIP-1004: Enforce tasks.max property in Kafka Connect > KIP-1005: Expose EarliestLocalOffset and TieredOffset > KIP-1007: Introduce Remote Storage Not Ready Exception > KIP-1019: Expose method to determine Metric Measurability > > Please review the plan and provide any additional information or updates > regarding KIPs that target this release version (3.8). > If you have authored any KIPs that have an inaccurate status in the list, > or are not in the list and should be, or are in the list and should not be > - please share it in this thread so that I can keep the document accurate > and up to date. > > Looking forward to your feedback. > > Best, > > On Wed, Feb 28, 2024 at 10:07 AM Satish Duggana > wrote: > >> Thanks Josep, +1. >> >> On Tue, 27 Feb 2024 at 17:29, Divij Vaidya >> wrote: >> > >> > Thank you for volunteering Josep. +1 from me. >> > >> > -- >> > Divij Vaidya >> > >> > >> > >> > On Tue, Feb 27, 2024 at 9:35 AM Bruno Cadonna >> wrote: >> > >> > > Thanks Josep! >> > > >> > > +1 >> > > >> > > Best, >> > > Bruno >> > > >> > > On 2/26/24 9:53 PM, Chris Egerton wrote: >> > > > Thanks Josep, I'm +1 as well. >> > > > >> > > > On Mon, Feb 26, 2024 at 12:32 PM Justine Olshan >> > > > wrote: >> > > > >> > > >> Thanks Joesp. +1 from me. >> > > >> >> > > >> On Mon, Feb 26, 2024 at 3:37 AM Josep Prat >> > > > > >> > > >> wrote: >> > > >> >> > > >>> Hi all, >> > > >>> >> > > >>> I'd like to volunteer as release manager for the Apache Kafka >> 3.8.0 >> > > >>> release. >> > > >>> If there are no objections, I'll start building a release plan (or >> > > >> adapting >> > > >>> the one Colin made some weeks ago) in the wiki in the next days. >> > > >>> >> > > >>> Thank you. >> > > >>> >> > > >>> -- >> > > >>> [image: Aiven] <https://www.aiven.io> &
Re: Permission to contribute to Apache Kafka Project
Hi Robin, You are now set up. Thanks for your interest in Apache Kafka. Best, On Tue, Apr 16, 2024 at 3:31 PM Robin Han wrote: > Hi there, > > My Jira ID is 'robinhan' and I'd like to ask permission to contribute to > the Apache Kafka Project. > > > I have encountered an error when upgrading from version 3.4.0 to 3.7.0 in > Kraft mode. I would like to fix this issue by submitting a Jira Ticket and > a Github PR. Unfortunately, I don't have the permission to create a Jira > Ticket at the moment. > [image: image.png] > > --- > Robin Han > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris
Congrats Greg!!! Best, Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Sun, Apr 14, 2024, 12:30 Divij Vaidya wrote: > Congratulations Greg! > > -- > Divij Vaidya > > > > On Sun, Apr 14, 2024 at 6:39 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Congratulations, Greg! > > > > On Sun, Apr 14, 2024 at 8:57 AM Yash Mayya wrote: > > > > > Congrats Greg! > > > > > > On Sun, 14 Apr, 2024, 05:56 Randall Hauch, wrote: > > > > > > > Congratulations, Greg! > > > > > > > > On Sat, Apr 13, 2024 at 6:36 PM Luke Chen wrote: > > > > > > > > > Congrats, Greg! > > > > > > > > > > On Sun, Apr 14, 2024 at 7:05 AM Viktor Somogyi-Vass > > > > > wrote: > > > > > > > > > > > Congrats Greg! :) > > > > > > > > > > > > On Sun, Apr 14, 2024, 00:35 Bill Bejeck > wrote: > > > > > > > > > > > > > Congrats Greg! > > > > > > > > > > > > > > -Bill > > > > > > > > > > > > > > On Sat, Apr 13, 2024 at 4:25 PM Boudjelda Mohamed Said < > > > > > > bmsc...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Congratulations Greg > > > > > > > > > > > > > > > > On Sat 13 Apr 2024 at 20:42, Chris Egerton < > > ceger...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > Greg Harris has been a Kafka committer since July 2023. He > > has > > > > > > remained > > > > > > > > > very active and instructive in the community since > becoming a > > > > > > > committer. > > > > > > > > > It's my pleasure to announce that Greg is now a member of > > Kafka > > > > > PMC. > > > > > > > > > > > > > > > > > > Congratulations, Greg! > > > > > > > > > > > > > > > > > > Chris, on behalf of the Apache Kafka PMC > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] Personal branches under apache/kafka
Hi Michael, I think it's a good idea. Only "official" branches should exist in the upstream repo. I guess the only exception would be if a massive feature would be done by different individuals collaborating and they would need a "neutral" place for the branch to be. But This didn't happen yet and I doubt it will in the near future. Best, --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Wed, Mar 13, 2024, 19:27 José Armando García Sancio wrote: > On Wed, Mar 13, 2024 at 11:02 AM Mickael Maison > wrote: > > What do you think? > > I agree. I wouldn't be surprised if these branches (not trunk or > release branches) were created by mistake by the committer. > > Thanks, > -- > -José >
Re: [DISCUSS] Apache Kafka 3.6.2 release
+1 thanks for volunteering! Best --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Wed, Mar 13, 2024, 17:17 Divij Vaidya wrote: > +1 > > Thank you for volunteering. > > -- > Divij Vaidya > > > > On Wed, Mar 13, 2024 at 4:58 PM Justine Olshan > > wrote: > > > Thanks Manikumar! > > +1 from me > > > > Justine > > > > On Wed, Mar 13, 2024 at 8:52 AM Manikumar > > wrote: > > > > > Hi, > > > > > > I'd like to volunteer to be the release manager for a bug fix release > of > > > the 3.6 line. > > > If there are no objections, I'll send out the release plan soon. > > > > > > Thanks, > > > Manikumar > > > > > >
Re: [DISCUSS] Apache Kafka 3.8.0 release
Hi all, Thanks for your support. I updated the skeleton release plan created by Colin. You can find it here: https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.8.0 Our last release stumbled upon some problems while releasing and was delayed by several weeks, so I won't try to shave some weeks from our plan for 3.8.0 (we might end up having delays again). Please raise your concerns if you don't agree with the proposed dates. The current proposal on dates are: - KIP Freeze: *15nd May *(Wednesday) - A KIP must be accepted by this date in order to be considered for this release. Note, any KIP that may not be implemented in a week, or that might destabilize the release, should be deferred. - Feature Freeze: *29th May *(Wednesday) - *major features merged & working on stabilisation, minor features have PR, release branch cut; anything not in this state will be automatically moved to the next release in JIRA* - Code Freeze: *12th June *(Wednesday) - At least two weeks of stabilization will follow Code Freeze, meaning we expect to release *no earlier* than *June 26th*. We will move as fast as we can, and aim for completion the earliest we can in June. I went through the KIP list, and found that these are the ones that might make it into the release: KIP-853: KRaft Controller Membership Changes (still under discussion) KIP-942: Add Power(ppc64le) support KIP-966: Eligible Leader Replicas KIP-974: Docker Image for GraalVM based Native Kafka Broker KIP-977: Partition-Level Throughput Metrics KIP-993: Allow restricting files accessed by File and Directory ConfigProviders KIP-994: Minor Enhancements to ListTransactions and DescribeTransactions APIs KIP-996: Pre-Vote KIP-1004: Enforce tasks.max property in Kafka Connect KIP-1005: Expose EarliestLocalOffset and TieredOffset KIP-1007: Introduce Remote Storage Not Ready Exception KIP-1019: Expose method to determine Metric Measurability Please review the plan and provide any additional information or updates regarding KIPs that target this release version (3.8). If you have authored any KIPs that have an inaccurate status in the list, or are not in the list and should be, or are in the list and should not be - please share it in this thread so that I can keep the document accurate and up to date. Looking forward to your feedback. Best, On Wed, Feb 28, 2024 at 10:07 AM Satish Duggana wrote: > Thanks Josep, +1. > > On Tue, 27 Feb 2024 at 17:29, Divij Vaidya > wrote: > > > > Thank you for volunteering Josep. +1 from me. > > > > -- > > Divij Vaidya > > > > > > > > On Tue, Feb 27, 2024 at 9:35 AM Bruno Cadonna > wrote: > > > > > Thanks Josep! > > > > > > +1 > > > > > > Best, > > > Bruno > > > > > > On 2/26/24 9:53 PM, Chris Egerton wrote: > > > > Thanks Josep, I'm +1 as well. > > > > > > > > On Mon, Feb 26, 2024 at 12:32 PM Justine Olshan > > > > wrote: > > > > > > > >> Thanks Joesp. +1 from me. > > > >> > > > >> On Mon, Feb 26, 2024 at 3:37 AM Josep Prat > > > > > > > >> wrote: > > > >> > > > >>> Hi all, > > > >>> > > > >>> I'd like to volunteer as release manager for the Apache Kafka 3.8.0 > > > >>> release. > > > >>> If there are no objections, I'll start building a release plan (or > > > >> adapting > > > >>> the one Colin made some weeks ago) in the wiki in the next days. > > > >>> > > > >>> Thank you. > > > >>> > > > >>> -- > > > >>> [image: Aiven] <https://www.aiven.io> > > > >>> > > > >>> *Josep Prat* > > > >>> Open Source Engineering Director, *Aiven* > > > >>> josep.p...@aiven.io | +491715557497 > > > >>> aiven.io <https://www.aiven.io> | < > > > >> https://www.facebook.com/aivencloud > > > >>>> > > > >>><https://www.linkedin.com/company/aiven/> < > > > >>> https://twitter.com/aiven_io> > > > >>> *Aiven Deutschland GmbH* > > > >>> Alexanderufer 3-7, 10117 Berlin > > > >>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > >>> Amtsgericht Charlottenburg, HRB 209739 B > > > >>> > > > >> > > > > > > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] Apache Kafka 3.7.0
Thanks Stanislav for running the release! And thanks to all members of the community who were part of the release! Best, On Fri, Mar 1, 2024 at 8:27 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Thanks Stanislav for running the release! > > On Thu, Feb 29, 2024, 21:03 Guozhang Wang > wrote: > > > Thanks Stan for running the release! > > > > On Thu, Feb 29, 2024 at 5:39 AM Boudjelda Mohamed Said > > wrote: > > > > > > Thanks Stanislav for running the release! > > > > > > On Wed, Feb 28, 2024 at 10:36 PM Kirk True wrote: > > > > > > > Thanks Stanislav > > > > > > > > > On Feb 27, 2024, at 10:01 AM, Stanislav Kozlovski < > > > > stanislavkozlov...@apache.org> wrote: > > > > > > > > > > The Apache Kafka community is pleased to announce the release of > > > > > Apache Kafka 3.7.0 > > > > > > > > > > This is a minor release that includes new features, fixes, and > > > > > improvements from 296 JIRAs > > > > > > > > > > An overview of the release and its notable changes can be found in > > the > > > > > release blog post: > > > > > > https://kafka.apache.org/blog#apache_kafka_370_release_announcement > > > > > > > > > > All of the changes in this release can be found in the release > notes: > > > > > https://www.apache.org/dist/kafka/3.7.0/RELEASE_NOTES.html > > > > > > > > > > You can download the source and binary release (Scala 2.12, 2.13) > > from: > > > > > https://kafka.apache.org/downloads#3.7.0 > > > > > > > > > > > > > > > > > --- > > > > > > > > > > > > > > > Apache Kafka is a distributed streaming platform with four core > APIs: > > > > > > > > > > > > > > > ** The Producer API allows an application to publish a stream of > > records > > > > to > > > > > one or more Kafka topics. > > > > > > > > > > ** The Consumer API allows an application to subscribe to one or > more > > > > > topics and process the stream of records produced to them. > > > > > > > > > > ** The Streams API allows an application to act as a stream > > processor, > > > > > consuming an input stream from one or more topics and producing an > > > > > output stream to one or more output topics, effectively > transforming > > the > > > > > input streams to output streams. > > > > > > > > > > ** The Connector API allows building and running reusable producers > > or > > > > > consumers that connect Kafka topics to existing applications or > data > > > > > systems. For example, a connector to a relational database might > > > > > capture every change to a table. > > > > > > > > > > > > > > > With these APIs, Kafka can be used for two broad classes of > > application: > > > > > > > > > > ** Building real-time streaming data pipelines that reliably get > data > > > > > between systems or applications. > > > > > > > > > > ** Building real-time streaming applications that transform or > react > > > > > to the streams of data. > > > > > > > > > > > > > > > Apache Kafka is in use at large and small companies worldwide, > > including > > > > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, > > Rabobank, > > > > > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > > > > > > > > > A big thank you to the following 146 contributors to this release! > > > > > (Please report an unintended omission) > > > > > > > > > > Abhijeet Kumar, Akhilesh Chaganti, Alieh, Alieh Saeedi, Almog > Gavra, > > > > > Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew > > > > > Schofield, Anna Sophie Blee-Goldman, Anton Agestam, Apoorv Mittal, > > > > > Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj, > > > > > ashwinpankaj, atu-sharm, bachmanity1, Bob Barrett, Bruno Cadonna, > > > > > Calvin Liu, Cerchie, chern, Chri
[DISCUSS] Apache Kafka 3.8.0 release
Hi all, I'd like to volunteer as release manager for the Apache Kafka 3.8.0 release. If there are no objections, I'll start building a release plan (or adapting the one Colin made some weeks ago) in the wiki in the next days. Thank you. -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.7.0 RC4
are considering for release > of > > >>> Apache > > >>> > > >> Kafka > > >>> > > >>> 3.7.0. > > >>> > > >>> > > >>> > > >>> Major changes include: > > >>> > > >>> - Early Access to KIP-848 - the next generation of the > consumer > > >>> > rebalance > > >>> > > >>> protocol > > >>> > > >>> - Early Access to KIP-858: Adding JBOD support to KRaft > > >>> > > >>> - KIP-714: Observability into Client metrics via a > standardized > > >>> > interface > > >>> > > >>> > > >>> > > >>> Release notes for the 3.7.0 release: > > >>> > > >>> > > >>> > > >>> > > >>> > > >> > > >>> > > > >>> > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html > > >>> > > >>> > > >>> > > >>> *** Please download, test and vote by Thursday, February > 15th, > > >>> 9AM > > >>> > PST > > >>> > > >> *** > > >>> > > >>> > > >>> > > >>> Kafka's KEYS file containing PGP keys we use to sign the > > release: > > >>> > > >>> https://kafka.apache.org/KEYS > > >>> > > >>> > > >>> > > >>> * Release artifacts to be voted upon (source and binary): > > >>> > > >>> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/ > > >>> > > >>> > > >>> > > >>> * Docker release artifact to be voted upon: > > >>> > > >>> apache/kafka:3.7.0-rc4 > > >>> > > >>> > > >>> > > >>> * Maven artifacts to be voted upon: > > >>> > > >>> > > >>> > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > >>> > > >>> > > >>> > > >>> * Javadoc: > > >>> > > >>> > > >>> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/ > > >>> > > >>> > > >>> > > >>> * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag: > > >>> > > >>> https://github.com/apache/kafka/releases/tag/3.7.0-rc4 > > >>> > > >>> > > >>> > > >>> * Documentation: > > >>> > > >>> https://kafka.apache.org/37/documentation.html > > >>> > > >>> > > >>> > > >>> * Protocol: > > >>> > > >>> https://kafka.apache.org/37/protocol.html > > >>> > > >>> > > >>> > > >>> * Successful Jenkins builds for the 3.7 branch: > > >>> > > >>> > > >>> > > >>> Unit/integration tests: I am in the process of running and > > >>> analyzing > > >>> > > >> these. > > >>> > > >>> System tests: I am in the process of running these. > > >>> > > >>> > > >>> > > >>> Expect a follow-up over the weekend > > >>> > > >>> > > >>> > > >>> * Successful Docker Image Github Actions Pipeline for 3.7 > > branch: > > >>> > > >>> Docker Build Test Pipeline: > > >>> > > >>> https://github.com/apache/kafka/actions/runs/7845614846 > > >>> > > >>> > > >>> > > >>> /** > > >>> > > >>> > > >>> > > >>> Best, > > >>> > > >>> Stanislav > > >>> > > >>> > > >>> > > >> > > >>> > > > > >>> > > > >>> > > >> > > >> > > >> -- > > >> Best, > > >> Stanislav > > >> > > > > > > > > > -- > > > Best, > > > Stanislav > > > > > > > > > -- > > Best, > > Stanislav > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.7.0 RC4
Hi Stanislav, Thanks for running the release. It gets a +1 (non-binding) from me. I run the following steps to validate: - Compiled with Java 17 and Scala 2.13.12 - Run unit and integration tests - "LogDirFailureTest > testIOExceptionDuringLogRoll(String).quorum=kraft" failed[1] several times before I could get a clear build. - Checked JavaDoc and clicked links pointing to JDK - Run getting started with ZK and KRaft - Verified artifact's signatures and hashes Best, [1]: Gradle Test Run :core:test > Gradle Test Executor 159 > LogDirFailureTest > testIOExceptionDuringLogRoll(String) > testIOExceptionDuringLogRoll(String).quorum=kraft FAILED org.opentest4j.AssertionFailedError: expected: but was: at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63) at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36) at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:31) at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:179) at app//kafka.utils.TestUtils$.causeLogDirFailure(TestUtils.scala:1671) at app//kafka.server.LogDirFailureTest.testProduceAfterLogDirFailureOnLeader(LogDirFailureTest.scala:186) at app//kafka.server.LogDirFailureTest.testIOExceptionDuringLogRoll(LogDirFailureTest.scala:70) On Fri, Feb 9, 2024 at 4:20 PM Stanislav Kozlovski wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate we are considering for release of Apache Kafka > 3.7.0. > > Major changes include: > - Early Access to KIP-848 - the next generation of the consumer rebalance > protocol > - Early Access to KIP-858: Adding JBOD support to KRaft > - KIP-714: Observability into Client metrics via a standardized interface > > Release notes for the 3.7.0 release: > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html > > *** Please download, test and vote by Thursday, February 15th, 9AM PST *** > > Kafka's KEYS file containing PGP keys we use to sign the release: > https://kafka.apache.org/KEYS > > * Release artifacts to be voted upon (source and binary): > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/ > > * Docker release artifact to be voted upon: > apache/kafka:3.7.0-rc4 > > * Maven artifacts to be voted upon: > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > * Javadoc: > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/ > > * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag: > https://github.com/apache/kafka/releases/tag/3.7.0-rc4 > > * Documentation: > https://kafka.apache.org/37/documentation.html > > * Protocol: > https://kafka.apache.org/37/protocol.html > > * Successful Jenkins builds for the 3.7 branch: > > Unit/integration tests: I am in the process of running and analyzing these. > System tests: I am in the process of running these. > > Expect a follow-up over the weekend > > * Successful Docker Image Github Actions Pipeline for 3.7 branch: > Docker Build Test Pipeline: > https://github.com/apache/kafka/actions/runs/7845614846 > > /** > > Best, > Stanislav > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] Kafka-Streams-Scala for Scala 3
Hi Matthias, If I understand it right (I'm not an expert in CI infra), Kafka uses the ASF shared Jenkins, meaning this would potentially increase the time-to-built for the rest of the projects. Best, On Mon, Feb 12, 2024 at 11:30 AM Matthias Berndt < matthias.ber...@ttmzero.com> wrote: > Hey Josep, > > wouldn't it be possible to run this in a separate CI job that runs in > parallel? Then it shouldn't take any longer than today – assuming enough CI > resources are available. > > Best, > Matthias > > Am Mo., 12. Feb. 2024 um 09:14 Uhr schrieb Josep Prat > : > > > Hi Matthias, > > > > One of the problems of adding support for Scala 3 for the scala-streams > > submodule is that we would need to have another build run on CI (which is > > already extremely long). I guess we could restrict this only when having > > changes on the Stream module(s). > > If I remember it correctly, when I tried to port Kafka to Scala 3 (as a > > whole) the sentiment was that only 2 versions of Scala should be > supported > > at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait for > > others to chime in. Maybe Ismael has some thoughts about it. > > > > Best, > > > > On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt < > > matthias.ber...@ttmzero.com> > > wrote: > > > > > Hey there, > > > > > > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala > library. > > > As you might have seen already, I have recently created a ticket > > > https://issues.apache.org/jira/browse/KAFKA-16237 > > > and a PR > > > https://github.com/apache/kafka/pull/15338 > > > to move this forward. The changes required to make Kafka-Streams-Scala > > > compile with Scala 3 are trivial; the trickier part is the build system > > and > > > the release process > > > I have made some changes to the build system (feel free to comment on > the > > > above PR about that) that make it possible to test Kafka-Streams-Scala > > and > > > build the jar. What remains to be done is the CI and release process. > > There > > > is a `release.py` file in the Kafka repository's root directory, which > > > assumes that all artifacts are available for all supported Scala > > versions. > > > This is no longer the case with my changes because while porting > > > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is > > less > > > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I > > > would appreciate some guidance as to what the release process should > look > > > like in the future. > > > > > > Oh and I've made a PR to remove a syntax error from release.py. > > > https://github.com/apache/kafka/pull/15350 > > > > > > All the best, > > > Matthias > > > > > > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud > > > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] Kafka-Streams-Scala for Scala 3
Hi Matthias, One of the problems of adding support for Scala 3 for the scala-streams submodule is that we would need to have another build run on CI (which is already extremely long). I guess we could restrict this only when having changes on the Stream module(s). If I remember it correctly, when I tried to port Kafka to Scala 3 (as a whole) the sentiment was that only 2 versions of Scala should be supported at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait for others to chime in. Maybe Ismael has some thoughts about it. Best, On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt wrote: > Hey there, > > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library. > As you might have seen already, I have recently created a ticket > https://issues.apache.org/jira/browse/KAFKA-16237 > and a PR > https://github.com/apache/kafka/pull/15338 > to move this forward. The changes required to make Kafka-Streams-Scala > compile with Scala 3 are trivial; the trickier part is the build system and > the release process > I have made some changes to the build system (feel free to comment on the > above PR about that) that make it possible to test Kafka-Streams-Scala and > build the jar. What remains to be done is the CI and release process. There > is a `release.py` file in the Kafka repository's root directory, which > assumes that all artifacts are available for all supported Scala versions. > This is no longer the case with my changes because while porting > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is less > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I > would appreciate some guidance as to what the release process should look > like in the future. > > Oh and I've made a PR to remove a syntax error from release.py. > https://github.com/apache/kafka/pull/15350 > > All the best, > Matthias > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Kafka-Streams-Scala for Scala 3
Hi Matthias, Yes, it's just a matter of adding the [DISCUSS] prefix in the subject. By the way, I didn't say this won't need a KIP, just that I won't be pushing for it, but other maintainers might think it's needed. For the discuss thread, you should write down what changes in the build and what steps would be needed to create the artifacts. Best, Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Fri, Feb 9, 2024, 19:55 Matthias Berndt wrote: > Hey Josep, > > I'm glad you agree that a KIP is not needed here, and I agree with you that > how to publish these artifacts should be discussed with the Kafka team. In > fact, this is what I created this thread for This is my first time > contributing to Kafka, so I'm going to have to ask what a DISCUSS thread > is. Is it just a mailing list thread with a subject that starts with > [DISCUSS], or is there more behind it? > > Best regards, > Matthias > > Am Fr., 9. Feb. 2024 um 18:31 Uhr schrieb Josep Prat > : > > > Hi Matthias, > > It's not adding a new functionality but it's changing the way to generate > > artifacts. In the end we are talking about generating a new binary. > > > > I could live with not having a KIP, but a DISCUSS thread I think it's > > necessary. This signals the community members and maintainers that their > > input is needed. > > > > I could help you with writing the KIP if you want. > > > > Best, > > > > Josep Prat > > Open Source Engineering Director, aivenjosep.p...@aiven.io | > > +491715557497 | aiven.io > > Aiven Deutschland GmbH > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > > On Fri, Feb 9, 2024, 18:02 Matthias Berndt > > wrote: > > > > > Hi Matthias, Hi Josep, > > > > > > I'm afraid I can't do the KIP thing as the signup process for Apache > > > Confluence requires sending me a password reset link via E-Mail and > said > > > E-Mail doesn't seem to reach me for some reason. I've contacted the > > Apache > > > infrastructure team but haven't yet heard back from them. > > > That said, I'd like to push back on the notion that a KIP is really > > > necessary for this change. It's certainly not a “major new feature” as > it > > > adds zero extra functionality, and it doesn't affect binary > compatibility > > > either as all the currently supported Scala versions are still > supported. > > > This looks like a routine upgrade to me. Please, let's try to keep the > > > administrative overhead to the required minimum, shall we? > > > Thanks btw for merging Github PR #15239 (removal of > > scala-collection-compat > > > dependency from the 2.13 artifact). That will already improve life for > > > Scala 3 users. > > > > > > All the best, > > > Matthias > > > > > > > > > Am Do., 8. Feb. 2024 um 18:02 Uhr schrieb Matthias J. Sax < > > > mj...@apache.org > > > >: > > > > > > > Josep, > > > > > > > > thanks for helping with this. I was also thinking if we might need a > > KIP > > > > for this change. Since you had the same though, I would say, yes, > let's > > > > do a KIP. > > > > > > > > @Matthias: can you prepare a KIP? You can read up on the details on > the > > > > wiki page: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > > > > If you have any questions about the process, please let us know. > > > > > > > > Thanks for pushing this forward! > > > > > > > > > > > > -Matthias > > > > > > > > On 2/8/24 8:08 AM, Matthias Berndt wrote: > > > > > Hey Josep et al, > > > > > > > > > > I've created a ticket regarding this. > > > > > https://issues.apache.org/jira/browse/KAFKA-16237 > > > > > > > > > > All the best, > > > > > Matthias > > > > > > > > > > Am Do., 8. Feb. 2024 um 11:42 Uhr schrieb Josep Prat > > > > > : > > > > >> > > > > >> Go ahead and ask for a JIRA and Wiki account (Confluence). Let us > > know
Re: Kafka-Streams-Scala for Scala 3
Hi Matthias, It's not adding a new functionality but it's changing the way to generate artifacts. In the end we are talking about generating a new binary. I could live with not having a KIP, but a DISCUSS thread I think it's necessary. This signals the community members and maintainers that their input is needed. I could help you with writing the KIP if you want. Best, Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Fri, Feb 9, 2024, 18:02 Matthias Berndt wrote: > Hi Matthias, Hi Josep, > > I'm afraid I can't do the KIP thing as the signup process for Apache > Confluence requires sending me a password reset link via E-Mail and said > E-Mail doesn't seem to reach me for some reason. I've contacted the Apache > infrastructure team but haven't yet heard back from them. > That said, I'd like to push back on the notion that a KIP is really > necessary for this change. It's certainly not a “major new feature” as it > adds zero extra functionality, and it doesn't affect binary compatibility > either as all the currently supported Scala versions are still supported. > This looks like a routine upgrade to me. Please, let's try to keep the > administrative overhead to the required minimum, shall we? > Thanks btw for merging Github PR #15239 (removal of scala-collection-compat > dependency from the 2.13 artifact). That will already improve life for > Scala 3 users. > > All the best, > Matthias > > > Am Do., 8. Feb. 2024 um 18:02 Uhr schrieb Matthias J. Sax < > mj...@apache.org > >: > > > Josep, > > > > thanks for helping with this. I was also thinking if we might need a KIP > > for this change. Since you had the same though, I would say, yes, let's > > do a KIP. > > > > @Matthias: can you prepare a KIP? You can read up on the details on the > > wiki page: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > If you have any questions about the process, please let us know. > > > > Thanks for pushing this forward! > > > > > > -Matthias > > > > On 2/8/24 8:08 AM, Matthias Berndt wrote: > > > Hey Josep et al, > > > > > > I've created a ticket regarding this. > > > https://issues.apache.org/jira/browse/KAFKA-16237 > > > > > > All the best, > > > Matthias > > > > > > Am Do., 8. Feb. 2024 um 11:42 Uhr schrieb Josep Prat > > > : > > >> > > >> Go ahead and ask for a JIRA and Wiki account (Confluence). Let us know > > when > > >> your accounts are created and we'll properly set them up so you can > > create > > >> and assign tickets to you. > > >> > > >> Best, > > >> > > >> On Thu, Feb 8, 2024 at 11:32 AM Matthias Berndt < > > matthias.ber...@ttmzero.com> > > >> wrote: > > >> > > >>> Thanks Josep, I've applied for a JIRA account and addressed your > > >>> review comments. > > >>> > > >>> Am Do., 8. Feb. 2024 um 09:19 Uhr schrieb Josep Prat > > >>> : > > >>>> > > >>>> Hi Matthias, > > >>>> > > >>>> I think for this particular case it would be worth creating a JIRA > > ticket > > >>>> for this as it's a new "feature". > > >>>> Regarding the change itself, I think we need to clarify how the > > release > > >>>> process would work. Right now, the script `gradlewAll` is used > (which > > >>>> basically runs the build with Scala version 2.12 and 2.13). If I > > >>> understand > > >>>> your changes correctly, we would need to run the build 3 times: > > >>>> - 1 with property scalaVersion 2.12 > > >>>> - 1 with scalaVersion 2.13 and streamsScalaVersion 2.13 > > >>>> - 1 with scalaVersion 2.13 and streamsScalaVersion 3.1 > > >>>> > > >>>> I think we should document this and discuss when to have this > feature. > > >>> If I > > >>>> remember correctly from when I tried to update Kafka to Scala 3, the > > idea > > >>>> was to push this to a Kafka 4.0 version because we didn't want to > > >>> maintain > > >>>> more than 2 Scala versions at the same time. I would encourage if > not > > >>>> ha
Re: Github build queue
Regarding "Split our CI "test" job into unit and integration so we can
start collecting data on those suites", can we run these 2 tasks in the
same machine? So they won't need to compile classes twice for the same
exact code?
On Fri, Feb 9, 2024 at 6:05 PM Ismael Juma wrote:
> Why can't we add @Tag("integration") for all of those tests? Seems like
> that would not be too hard.
>
> Ismael
>
> On Fri, Feb 9, 2024 at 9:03 AM Greg Harris
> wrote:
>
> > Hi David,
> >
> > +1 on that strategy.
> >
> > I see several flaky tests that aren't marked with @Tag("integration")
> > or @IntegrationTest, and I think those would make using the unitTest
> > target ineffective here. We could also start a new tag @Tag("flaky")
> > and exclude that.
> >
> > Thanks,
> > Greg
> >
> > On Fri, Feb 9, 2024 at 8:57 AM David Arthur wrote:
> > >
> > > I do think we can add a PR to the merge queue while bypassing branch
> > > potections (like we do for the Merge button today), but I'm not 100%
> > sure.
> > > I like the idea of running unit tests, though I don't think we have
> data
> > on
> > > how long just the unit tests run on Jenkins (since we run the "test"
> > target
> > > which includes all tests). I'm also not sure how flaky the unit test
> > suite
> > > is alone.
> > >
> > > Since we already bypass the PR checks when merging, it seems that
> adding
> > a
> > > required compile/check step before landing on trunk is strictly an
> > > improvement.
> > >
> > > What about this as a short term plan:
> > >
> > > 1) Add the merge queue, only run compile/check
> > > 2) Split our CI "test" job into unit and integration so we can start
> > > collecting data on those suites
> > > 3) Add "unitTest" to merge queue job once we're satisfied it won't
> cause
> > > disruption
> > >
> > >
> > >
> > >
> > > On Fri, Feb 9, 2024 at 11:43 AM Josep Prat >
> > > wrote:
> > >
> > > > Hi David,
> > > > I like the idea, it will solve the problem we've seen a couple of
> > times in
> > > > the last 2 weeks where compilation for some Scala version failed, it
> > was
> > > > probably overlooked during the PR build because of the flakiness of
> > tests
> > > > and the compilation failure was buried among the amount of failed
> > tests.
> > > >
> > > > Regarding the type of check, I'm not sure what's best, have a real
> > quick
> > > > check or a longer one including unit tests. A full test suite will
> run
> > per
> > > > each commit in each PR (these we have definitely more than 8 per day)
> > and
> > > > this should be used to ensure changes are safe and sound. I'm not
> sure
> > if
> > > > having unit tests run as well before the merge itself would cause too
> > much
> > > > of an extra load on the CI machines.
> > > > We can go with `gradlew unitTest` and see if this takes too long or
> > causes
> > > > too many delays with the normal pipeline.
> > > >
> > > > Best,
> > > >
> > > > On Fri, Feb 9, 2024 at 4:16 PM Ismael Juma
> wrote:
> > > >
> > > > > Hi David,
> > > > >
> > > > > I think this is a helpful thing (and something I hoped we would use
> > when
> > > > I
> > > > > learned about it), but it does require the validation checks to be
> > > > reliable
> > > > > (or else the PR won't be merged). Sounds like you are suggesting to
> > skip
> > > > > the tests for the merge queue validation. Could we perhaps include
> > the
> > > > unit
> > > > > tests as well? That would incentivize us to ensure the unit tests
> are
> > > > fast
> > > > > and reliable. Getting the integration tests to the same state will
> > be a
> > > > > longer journey.
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Fri, Feb 9, 2024 at 7:04 AM David Arthur
> > wrote:
> > > > >
> > > > > > Hey folks,
> > > > > >
> > > > > > I recently learned about Github's Merge Queue feature, and I
> think
> > it
> > > > > could
> > > >
Re: Github build queue
Hi David, I like the idea, it will solve the problem we've seen a couple of times in the last 2 weeks where compilation for some Scala version failed, it was probably overlooked during the PR build because of the flakiness of tests and the compilation failure was buried among the amount of failed tests. Regarding the type of check, I'm not sure what's best, have a real quick check or a longer one including unit tests. A full test suite will run per each commit in each PR (these we have definitely more than 8 per day) and this should be used to ensure changes are safe and sound. I'm not sure if having unit tests run as well before the merge itself would cause too much of an extra load on the CI machines. We can go with `gradlew unitTest` and see if this takes too long or causes too many delays with the normal pipeline. Best, On Fri, Feb 9, 2024 at 4:16 PM Ismael Juma wrote: > Hi David, > > I think this is a helpful thing (and something I hoped we would use when I > learned about it), but it does require the validation checks to be reliable > (or else the PR won't be merged). Sounds like you are suggesting to skip > the tests for the merge queue validation. Could we perhaps include the unit > tests as well? That would incentivize us to ensure the unit tests are fast > and reliable. Getting the integration tests to the same state will be a > longer journey. > > Ismael > > On Fri, Feb 9, 2024 at 7:04 AM David Arthur wrote: > > > Hey folks, > > > > I recently learned about Github's Merge Queue feature, and I think it > could > > help us out. > > > > Essentially, when you hit the Merge button on a PR, it will add the PR > to a > > queue and let you run a CI job before merging. Just something simple like > > compile + static analysis would probably save us from a lot of headaches > on > > trunk. > > > > I can think of two situations this would help us avoid: > > * Two valid PRs are merged near one another, but they create a code > > breakage (rare) > > * A quick little "fixup" commit on a PR actually breaks something (less > > rare) > > > > Looking at our Github stats, we are averaging under 40 commits per week. > > Assuming those primarily come in on weekdays, that's 8 commits per day. > If > > we just run "gradlew check -x tests" for the merge queue job, I don't > think > > we'd get backlogged. > > > > Thoughts? > > David > > > > > > > > > > -- > > David Arthur > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Kafka-Streams-Scala for Scala 3
Go ahead and ask for a JIRA and Wiki account (Confluence). Let us know when your accounts are created and we'll properly set them up so you can create and assign tickets to you. Best, On Thu, Feb 8, 2024 at 11:32 AM Matthias Berndt wrote: > Thanks Josep, I've applied for a JIRA account and addressed your > review comments. > > Am Do., 8. Feb. 2024 um 09:19 Uhr schrieb Josep Prat > : > > > > Hi Matthias, > > > > I think for this particular case it would be worth creating a JIRA ticket > > for this as it's a new "feature". > > Regarding the change itself, I think we need to clarify how the release > > process would work. Right now, the script `gradlewAll` is used (which > > basically runs the build with Scala version 2.12 and 2.13). If I > understand > > your changes correctly, we would need to run the build 3 times: > > - 1 with property scalaVersion 2.12 > > - 1 with scalaVersion 2.13 and streamsScalaVersion 2.13 > > - 1 with scalaVersion 2.13 and streamsScalaVersion 3.1 > > > > I think we should document this and discuss when to have this feature. > If I > > remember correctly from when I tried to update Kafka to Scala 3, the idea > > was to push this to a Kafka 4.0 version because we didn't want to > maintain > > more than 2 Scala versions at the same time. I would encourage if not > > having a KIP, at least open up a [DISCUSS] thread to clarify some of > these > > points. > > > > I'll add some feedback on the PR itself regarding the changes. > > > > Best, > > > > On Thu, Feb 8, 2024 at 1:57 AM Matthias Berndt < > matthias.ber...@ttmzero.com> > > wrote: > > > > > Hi Matthias J., Hi Lucas, Hi Josep, > > > > > > Thank you for your encouraging responses regarding a Scala 3 port of > > > Kafka-Streams-Scala, and apologies for the late response from my side. > > > I have now created a PR to port Kafka-Streams-Scala to Scala 3 (while > > > retaining support for 2.13 and 2.12). Almost no changes to the code > > > were required and the tests also pass. Please take a look and let me > > > know what you think :-) > > > https://github.com/apache/kafka/pull/15338 > > > > > > All the best > > > Matthias > > > > > > Am Do., 1. Feb. 2024 um 16:35 Uhr schrieb Josep Prat > > > : > > > > > > > > Hi, > > > > > > > > For reference, prior work on this: > > > > https://github.com/apache/kafka/pull/11350 > > > > https://github.com/apache/kafka/pull/11432 > > > > > > > > Best, > > > > > > > > On Thu, Feb 1, 2024, 15:55 Lucas Brutschy > > .invalid> > > > > wrote: > > > > > > > > > Hi Matthiases, > > > > > > > > > > I know Scala 2 fairly well, so I'd be happy to review changes that > add > > > > > Scala 3 support. However, as Matthias S. said, it has to be driven > by > > > > > people who use Scala day-to-day, since I believe most Kafka Streams > > > > > committers are working with Java. > > > > > > > > > > Rewriting the tests to not use EmbeddedKafkaCluster seems like a > large > > > > > undertaking, so option 1 is the first thing we should explore. > > > > > > > > > > I don't have any experience with Scala 3 migration topics, but on > the > > > > > Scala website it says > > > > > > The first piece of good news is that the Scala 3 compiler is > able to > > > > > read the Scala 2.13 Pickle format and thus it can type check code > that > > > > > depends on modules or libraries compiled with Scala 2.13. > > > > > > One notable example is the Scala 2.13 library. We have indeed > decided > > > > > that the Scala 2.13 library is the official standard library for > Scala > > > 3. > > > > > So wouldn't that mean that we are safe in terms of standard library > > > > > upgrades if we use core_2.13 in the tests? > > > > > > > > > > Cheers, > > > > > Lucas > > > > > > > > > > > > > > > On Wed, Jan 31, 2024 at 9:20 PM Matthias J. Sax > > > wrote: > > > > > > > > > > > > Thanks for raising this. The `kafka-streams-scala` module seems > to > > > be an > > > > > > important feature for Kafka Streams and I am generally in favor > of > > > your > > > &
Re: Kafka-Streams-Scala for Scala 3
Hi Matthias, I think for this particular case it would be worth creating a JIRA ticket for this as it's a new "feature". Regarding the change itself, I think we need to clarify how the release process would work. Right now, the script `gradlewAll` is used (which basically runs the build with Scala version 2.12 and 2.13). If I understand your changes correctly, we would need to run the build 3 times: - 1 with property scalaVersion 2.12 - 1 with scalaVersion 2.13 and streamsScalaVersion 2.13 - 1 with scalaVersion 2.13 and streamsScalaVersion 3.1 I think we should document this and discuss when to have this feature. If I remember correctly from when I tried to update Kafka to Scala 3, the idea was to push this to a Kafka 4.0 version because we didn't want to maintain more than 2 Scala versions at the same time. I would encourage if not having a KIP, at least open up a [DISCUSS] thread to clarify some of these points. I'll add some feedback on the PR itself regarding the changes. Best, On Thu, Feb 8, 2024 at 1:57 AM Matthias Berndt wrote: > Hi Matthias J., Hi Lucas, Hi Josep, > > Thank you for your encouraging responses regarding a Scala 3 port of > Kafka-Streams-Scala, and apologies for the late response from my side. > I have now created a PR to port Kafka-Streams-Scala to Scala 3 (while > retaining support for 2.13 and 2.12). Almost no changes to the code > were required and the tests also pass. Please take a look and let me > know what you think :-) > https://github.com/apache/kafka/pull/15338 > > All the best > Matthias > > Am Do., 1. Feb. 2024 um 16:35 Uhr schrieb Josep Prat > : > > > > Hi, > > > > For reference, prior work on this: > > https://github.com/apache/kafka/pull/11350 > > https://github.com/apache/kafka/pull/11432 > > > > Best, > > > > On Thu, Feb 1, 2024, 15:55 Lucas Brutschy .invalid> > > wrote: > > > > > Hi Matthiases, > > > > > > I know Scala 2 fairly well, so I'd be happy to review changes that add > > > Scala 3 support. However, as Matthias S. said, it has to be driven by > > > people who use Scala day-to-day, since I believe most Kafka Streams > > > committers are working with Java. > > > > > > Rewriting the tests to not use EmbeddedKafkaCluster seems like a large > > > undertaking, so option 1 is the first thing we should explore. > > > > > > I don't have any experience with Scala 3 migration topics, but on the > > > Scala website it says > > > > The first piece of good news is that the Scala 3 compiler is able to > > > read the Scala 2.13 Pickle format and thus it can type check code that > > > depends on modules or libraries compiled with Scala 2.13. > > > > One notable example is the Scala 2.13 library. We have indeed decided > > > that the Scala 2.13 library is the official standard library for Scala > 3. > > > So wouldn't that mean that we are safe in terms of standard library > > > upgrades if we use core_2.13 in the tests? > > > > > > Cheers, > > > Lucas > > > > > > > > > On Wed, Jan 31, 2024 at 9:20 PM Matthias J. Sax > wrote: > > > > > > > > Thanks for raising this. The `kafka-streams-scala` module seems to > be an > > > > important feature for Kafka Streams and I am generally in favor of > your > > > > proposal to add Scala 3 support. However, I am personally no Scala > > > > person and it sounds like quite some overhead. > > > > > > > > If you are willing to drive and own this initiative happy to support > you > > > > to the extend I can. > > > > > > > > About the concrete proposal: my understanding is that :core will move > > > > off Scala long-term (not 100% sure what the timeline is, but new > modules > > > > are written in Java only). Thus, down the road the compatibility > issue > > > > would go away naturally, but it's unclear when. > > > > > > > > Thus, if we can test kafak-stream-scala_3 with core_2.13 it seems we > > > > could add support for Scala 3 now, taking a risk that it might break > in > > > > the future assume that the migration off Scala from core is not fast > > > enough. > > > > > > > > For proposal (2), I don't think that it would be easily possible for > > > > unit/integration tests. We could fall back to system tests though, > but > > > > they would be much more heavy weight of course. > > > > > > > > Might be good to hear from others. We might actually also want to do > a >
Re: Kafka-Streams-Scala for Scala 3
Hi, For reference, prior work on this: https://github.com/apache/kafka/pull/11350 https://github.com/apache/kafka/pull/11432 Best, On Thu, Feb 1, 2024, 15:55 Lucas Brutschy wrote: > Hi Matthiases, > > I know Scala 2 fairly well, so I'd be happy to review changes that add > Scala 3 support. However, as Matthias S. said, it has to be driven by > people who use Scala day-to-day, since I believe most Kafka Streams > committers are working with Java. > > Rewriting the tests to not use EmbeddedKafkaCluster seems like a large > undertaking, so option 1 is the first thing we should explore. > > I don't have any experience with Scala 3 migration topics, but on the > Scala website it says > > The first piece of good news is that the Scala 3 compiler is able to > read the Scala 2.13 Pickle format and thus it can type check code that > depends on modules or libraries compiled with Scala 2.13. > > One notable example is the Scala 2.13 library. We have indeed decided > that the Scala 2.13 library is the official standard library for Scala 3. > So wouldn't that mean that we are safe in terms of standard library > upgrades if we use core_2.13 in the tests? > > Cheers, > Lucas > > > On Wed, Jan 31, 2024 at 9:20 PM Matthias J. Sax wrote: > > > > Thanks for raising this. The `kafka-streams-scala` module seems to be an > > important feature for Kafka Streams and I am generally in favor of your > > proposal to add Scala 3 support. However, I am personally no Scala > > person and it sounds like quite some overhead. > > > > If you are willing to drive and own this initiative happy to support you > > to the extend I can. > > > > About the concrete proposal: my understanding is that :core will move > > off Scala long-term (not 100% sure what the timeline is, but new modules > > are written in Java only). Thus, down the road the compatibility issue > > would go away naturally, but it's unclear when. > > > > Thus, if we can test kafak-stream-scala_3 with core_2.13 it seems we > > could add support for Scala 3 now, taking a risk that it might break in > > the future assume that the migration off Scala from core is not fast > enough. > > > > For proposal (2), I don't think that it would be easily possible for > > unit/integration tests. We could fall back to system tests though, but > > they would be much more heavy weight of course. > > > > Might be good to hear from others. We might actually also want to do a > > KIP for this? > > > > > > -Matthias > > > > On 1/20/24 10:34 AM, Matthias Berndt wrote: > > > Hey there, > > > > > > I'd like to discuss a Scala 3 port of the kafka-streams-scala library. > > > Currently, the build system is set up such that kafka-streams-scala > > > and core (i. e. kafka itself) are compiled with the same Scala > > > compiler versions. This is not an optimal situation because it means > > > that a Scala 3 release of kafka-streams-scala cannot happen > > > independently of kafka itself. I think this should be changed > > > > > > The production codebase of scala-streams-kafka actually compiles just > > > fine on Scala 3.3.1 with two lines of trivial syntax changes. The > > > problem is with the tests. These use the `EmbeddedKafkaCluster` class, > > > which means that kafka is pulled into the classpath, potentially > > > leading to binary compatibility issues. > > > I can see several approaches to fixing this: > > > > > > 1. Run the kafka-streams-scala tests using the compatible version of > > > :core if one is available. Currently, this means that everything can > > > be tested (test kafka-streams-scala_2.12 using core_2.12, > > > kafka-streams-scala_2.13 using core_2.13 and kafka-streams-scala_3 > > > using core_2.13, as these should be compatible), but when a new > > > scala-library version is released that is no longer compatible with > > > 2.13, we won't be able to test that. > > > 2. Rewrite the tests to run without EmbeddedKafkaCluster, instead > > > running the test cluster in a separate JVM or perhaps even a > > > container. > > > > > > I'd be willing to get my hands dirty working on this, but before I > > > start I'd like to get some feedback from the Kafka team regarding the > > > approaches outlined above. > > > > > > All the best > > > Matthias Berndt > KJosep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Broken build
Hi all, PR has been merged, I recommend rebasing to everyone who created a PR in the last hours. Best! On Tue, Jan 30, 2024 at 12:52 PM Luke Chen wrote: > Thanks for the quick fix, Josep! > > On Tue, Jan 30, 2024 at 7:21 PM Josep Prat > wrote: > > > Hi Kafka Devs! > > I noticed the current trunk build is broken under Scala 2.12 only. > > I found the faulty error and sent a PR to fix it: > > https://github.com/apache/kafka/pull/15290 > > > > If someone has the time and could review it, it would be great. Thanks! > > > > Best!, > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud > > > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Broken build
Hi Kafka Devs! I noticed the current trunk build is broken under Scala 2.12 only. I found the faulty error and sent a PR to fix it: https://github.com/apache/kafka/pull/15290 If someone has the time and could review it, it would be great. Thanks! Best!, -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Permission to contribute
Hi Rowland, Thanks for your interest in Apache Kafka! Your accounts are all set. Let me know if you have any questions. Best, On Sun, Jan 14, 2024 at 2:17 AM Rowland Smith wrote: > I would like permission to contribute to Kafka. I have created Wiki and > Jira ID's 'rowls'. > > I will be working with a KIP for XA support. > > -- > *Rowland E. Smith* > P: (862) 260-4163 > M: (201) 396-3842 > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Kafka 3.0 support Java 8
Hi, We attempt to support the last 3 non-patch versions. This would mean we would try to Backport security vulnerabilities to a 3.x (probably 3.8) for 6 to 9 months after the last release. Best, --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Wed, Jan 10, 2024, 21:43 Devinder Saggu wrote: > Thanks. > > And how long Kafka 3.x will be supported. > > Thanks > > On Wed, Jan 10, 2024 at 3:40 PM Divij Vaidya > wrote: > > > All versions in the 3.x series of Kafka will support Java 8. > > > > Starting Kafka 4.0, we will drop support for Java 8. Clients will support > > >= JDK 11 and other packages will support >= JDK 17. More details about > > Java in Kafka 4.0 can be found here: > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=284789510 > > > > Does this answer your question? > > > > -- > > Divij Vaidya > > > > > > > > On Wed, Jan 10, 2024 at 9:37 PM Devinder Saggu < > saggusinghsu...@gmail.com> > > wrote: > > > > > Hi, > > > > > > I wonder how long Kafka 3.0 can support Java 8. > > > > > > Thanks & Regards, > > > > > > *Devinder Singh* > > > P *Please consider the environment before printing this email* > > > > > > > > -- > Thanks & Regards, > > *Devinder Singh* > P *Please consider the environment before printing this email* >
Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release
Hi all, the vote passes with: - 4 binding votes: Colin, Chris, Greg and myself - 1 non-binding vote: Anton Thanks everyone for the discussion and feedback! Best, On Mon, Jan 8, 2024 at 11:08 PM Colin McCabe wrote: > Hi Chris, > > Based on some of the offline discussions we had, people generally feel > that major/minor releases have a large enough overhead that they don't want > them more frequently than every 4 months. (Obviously dot releases are a > different story) So Josep and I didn't want to raise the issue here. > > I also feel that 4 months should be enough (for anyone? :) ) But I'm > always an optimist. > > best, > Colin > > > On Mon, Jan 8, 2024, at 13:03, Chris Egerton wrote: > > Hi Colin, > > > > The idea isn't to hold up 4.0.0 any longer; in fact, it's the opposite. > If > > things slip a bit with 3.8.0 and we take, for example, an extra month to > > deliver it (or even to cut the branch), I don't think this should > > necessarily translate to an extra month of latency between now and 4.0.0, > > given exactly what you mention about the major changes we plan to include > > in 4.0.0 (which consist more of dropping support for existing things than > > adding support for new things). > > > > If we want to avoid confusion, we could say something like "no later > than 3 > > to 4 months after the 3.8 branch is created". Frankly though, I think > it's > > unnecessary to specify an exact timeline for 4.0 in this KIP, since > nothing > > in the proposal actually diverges from the usual time-based release plan > we > > follow. The only necessary part seems to be to state that 4.0 will > directly > > follow 3.8 (as opposed to 3.9, 3.10, etc.). But perhaps I'm missing > > something? > > > > Cheers, > > > > Chris > > > > On Mon, Jan 8, 2024 at 2:38 PM Colin McCabe wrote: > > > >> On Mon, Jan 8, 2024, at 09:05, Chris Egerton wrote: > >> > Hi Josep, > >> > > >> > Thanks for the KIP! +1 (binding). > >> > > >> > One small nit: I don't think we necessarily have to hold ourselves to > >> > releasing 4.0.0 "3 to 4 months after 3.8 branch is created" (quoting > the > >> > timeline section of the KIP). IMO it's fine to leave some wiggle room > for > >> > the 4.0.0 release without codifying a timeline in this KIP. Maybe > >> something > >> > like "some time after 3.8 branch is created" would be sufficient? > >> Anyways, > >> > not a huge thing, I'm sure we'll all give 4.0.0 the flexibility it > needs > >> > with the understanding that this KIP is more focused on 3.8.0 than > 4.0.0. > >> > > >> > >> Hmm... I don't see any obstacles in the path of releasing 4.0 after the > >> traditional 4 months of development. Keep in mind, we're removing things > >> from the code (the ability to support JDK8, ZooKeeper mode, etc.), not > >> adding things. We already support JDK11 so saying that it's the minimum > is > >> a very quick change. Similarly, we already support KRaft so saying that > >> it's the only mode should be a pretty quick change. > >> > >> Also, we added a note that "the timeline is very rough" to KIP-833 and > it > >> caued all kinds of confusion. So overall I'd prefer to leave the > language > >> about 4.0 unchanged. > >> > >> best, > >> Colin > >> > >> > > >> > Cheers, > >> > > >> > Chris > >> > > >> > On Mon, Jan 8, 2024 at 11:41 AM Greg Harris > >> > > >> > wrote: > >> > > >> >> Thanks Josep for leading the KIP and building consensus on 3.8! > >> >> > >> >> +1 (binding) > >> >> > >> >> Greg > >> >> > >> >> On Sun, Jan 7, 2024 at 11:45 PM Josep Prat > >> > > >> >> wrote: > >> >> > > >> >> > Hi all, > >> >> > > >> >> > Thanks for your comments, > >> >> > I reworded some parts of the KIP to express that: > >> >> > - The KIP is to agree that we need at least one more minor version > in > >> the > >> >> > 3.x series > >> >> > - Explicitly saying that the list of KIPs is not exhaustive and > that > >> if > >> >> > some are not done, we would need another minor version > >> >> > - W
Re: Requesting permissions to contribute to Apache Kafka
Hi Abhinav, This one I could find :) You are now all set! Happy contributing! Best, On Mon, Jan 8, 2024 at 2:39 PM Abhinav Dixit wrote: > Hi Josep, > Thanks for the quick response. My WIKI ID is adixit, sorry for the > confusion. > Thanks, > Abhinav > > On Mon, Jan 8, 2024 at 6:59 PM Josep Prat > wrote: > > > Hi Abhinav, > > Thanks for your interest in Apache Kafka. > > I could set up your Jira account accordingly, but I can't find your Wiki > ID > > in the system. Can you confirm you created it and named it > > "adixitconfluent"? > > > > Best, > > > > On Mon, Jan 8, 2024 at 2:10 PM Abhinav Dixit > > > wrote: > > > > > Hi Team, > > > I am writing this email to request permissions to contribute to Apache > > > Kafka. Here are my details - > > > JIRA ID - adixitconfluent > > > WIKI ID - adixitconfluent > > > > > > Please let me know if anything is missing. > > > Thanks, > > > Abhinav > > > > > > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud > > > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Requesting permissions to contribute to Apache Kafka
Hi Abhinav, Thanks for your interest in Apache Kafka. I could set up your Jira account accordingly, but I can't find your Wiki ID in the system. Can you confirm you created it and named it "adixitconfluent"? Best, On Mon, Jan 8, 2024 at 2:10 PM Abhinav Dixit wrote: > Hi Team, > I am writing this email to request permissions to contribute to Apache > Kafka. Here are my details - > JIRA ID - adixitconfluent > WIKI ID - adixitconfluent > > Please let me know if anything is missing. > Thanks, > Abhinav > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release
Hi all, Copying here the same message as in the VOTE thread just in case someone is not following it (yet). "*Thanks for your comments,* *I reworded some parts of the KIP to express that:* *- The KIP is to agree that we need at least one more minor version in the 3.x series* *- Explicitly saying that the list of KIPs is not exhaustive and that if some are not done, we would need another minor version* *- Which are the KIPs/Features the community identified that should be present in a 3.x version so they can safely migrate to a potential 4.0 version* *- The timeline of 4.0 (starting after branching, not after release)* *- Wording is now focusing more on the need to have at least another minor release in 3.x to enable and ease migration to a potential 4.0 version* *I always mention potential in terms of 4.0 as we don't know what will be in there yet, and this KIP's scope is not meant to define this.*" Best, On Fri, Jan 5, 2024 at 11:50 PM Greg Harris wrote: > Hi Colin, > > Thanks for the quick reply! > > > If we cannot get KIP-853 done in time for 3.8, then we'd move to have > another 3.x release. > > This is the crux of my concern, and this is satisfactory. This means > that the "must-haves" are advisory only, and don't constitute a > binding feature list or a feature-based release. Thank you! > > >The intention behind saying the timeline was "rough" was to make the > obvious comment that if we shipped 3.7 in February rather than in January, > it would still be in the spirit of following the KIP. > > I kind of regret putting that comment in there now, since apparently > there was a lot of misinterpretation! It didn't mean that the KIP itself > was just a suggestion and not binding, or that we hadn't come to a > consensus about 3.7 being the last release. > > I certainly did not get that sense from reading KIP-833 or the > discussion thread after-the-fact. I was reading it using the > contemporary interpretation you posted here: [1]. If I were voting on > that KIP, I would not think that I was voting for 3.7 to be the last > 3.x release. > > > To make this totally clear: > > NO vote for this KIP ===> 3.7 is the last 3.x release, as per KIP-833. > > YES vote for this KIP ===> 3.8 is the last 3.x release. > > This was not the precedent set by previous major version bump > discussions. We need a mutual consensus to advance the major version, > and lack of consensus means by default the next release will be a > minor release. > If KIP-833 actually changed this precedent, then I'll vote for this > KIP just to restore the norm, as long as it is clear to everyone that > after 3.8 the version will bump to 3.9 (and etc) unless another > discussion reaches consensus about the project being ready for 4.0. > > [1] https://lists.apache.org/thread/k3bbz7k0hsb4y0b2xn1lh7bjrmjowmcq > > Thanks! > Greg > > On Fri, Jan 5, 2024 at 2:20 PM Colin McCabe wrote: > > > > Hi Justine, > > > > If there are more things you think are needed in 3.8 for migration, > let's discuss in the VOTE thread. > > > > best, > > Colin > > > > > > On Fri, Jan 5, 2024, at 09:23, Justine Olshan wrote: > > > While I agree we should have this release and should vote on it soon, > is it > > > worth determining the exact items we need before we vote? Just so we > are > > > all in agreement? > > > There is still some discussion on the road to 4.0 thread that may be > worth > > > having here. > > > > > > On Fri, Jan 5, 2024 at 1:25 AM Josep Prat > > > > wrote: > > > > > >> Hi Colin, > > >> Sorry for being quiet these last days (PTO). > > >> > > >> I will start the vote thread right away. > > >> > > >> Best, > > >> > > >> > > >> --- > > >> Josep Prat > > >> Open Source Engineering Director, aivenjosep.p...@aiven.io | > > >> +491715557497 | aiven.io > > >> Aiven Deutschland GmbH > > >> Alexanderufer 3-7, 10117 Berlin > > >> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > >> Amtsgericht Charlottenburg, HRB 209739 B > > >> > > >> On Fri, Jan 5, 2024, 00:24 Colin McCabe wrote: > > >> > > >> > Hi all, > > >> > > > >> > Since this has been open for a few weeks, are there any objections > to > > >> > starting the vote? What do you think, Josep? > > >> > > > >> > Since 3.8 is going to be the next release (according to the KIP) we > > >> should > > >> > really vote th
Re: [VOTE] KIP-1012: The need for a Kafka 3.8.x release
Hi all, Thanks for your comments, I reworded some parts of the KIP to express that: - The KIP is to agree that we need at least one more minor version in the 3.x series - Explicitly saying that the list of KIPs is not exhaustive and that if some are not done, we would need another minor version - Which are the KIPs/Features the community identified that should be present in a 3.x version so they can safely migrate to a potential 4.0 version - The timeline of 4.0 (starting after branching, not after release) - Wording is now focusing more on the need to have at least another minor release in 3.x to enable and ease migration to a potential 4.0 version I always mention potential in terms of 4.0 as we don't know what will be in there yet, and this KIP's scope is not meant to define this. Best, On Fri, Jan 5, 2024 at 10:46 PM Ismael Juma wrote: > I agree with Colin. Also, 4.0 would be started after 3.8 is branched (not > after 3.8.0 is released). The rest looks good. > > Thanks! > > Ismael > > On Fri, Jan 5, 2024 at 11:27 PM Colin McCabe wrote: > > > Hi, > > > > Thanks for calling the vote, Josep. > > > > I re-checked this, and saw something that we missed updating. One thing > we > > talked about earlier is that KIP-966 is actually not required for 3.8. > What > > is required is some way of enabling unclean leader election by default in > > KRaft. (Which could be KIP-966, or something else). Please revise this. > > > > best, > > Colin > > > > On Fri, Jan 5, 2024, at 02:50, Anton Agestam wrote: > > > +1 from me. > > > > > > Den fre 5 jan. 2024 kl 10:33 skrev Josep Prat > > : > > > > > >> Hi all, > > >> > > >> I'd like to start a vote on KIP-1012: > > >> > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8.x+release > > >> > > >> Discussion thread is here: > > >> https://lists.apache.org/thread/kvdp2gmq5gd9txkvxh5vk3z2n55b04s5 > > >> > > >> Thanks! > > >> > > >> --- > > >> Josep Prat > > >> Open Source Engineering Director, aivenjosep.p...@aiven.io | > > >> +491715557497 | aiven.io > > >> Aiven Deutschland GmbH > > >> Alexanderufer 3-7, 10117 Berlin > > >> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > >> Amtsgericht Charlottenburg, HRB 209739 B > > >> > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
[VOTE] KIP-1012: The need for a Kafka 3.8.x release
Hi all, I'd like to start a vote on KIP-1012: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8.x+release Discussion thread is here: https://lists.apache.org/thread/kvdp2gmq5gd9txkvxh5vk3z2n55b04s5 Thanks! --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release
Hi Colin, Sorry for being quiet these last days (PTO). I will start the vote thread right away. Best, --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Fri, Jan 5, 2024, 00:24 Colin McCabe wrote: > Hi all, > > Since this has been open for a few weeks, are there any objections to > starting the vote? What do you think, Josep? > > Since 3.8 is going to be the next release (according to the KIP) we should > really vote this in as soon as possible. > > Also, I created a wiki page about the 3.8 release with a tentative > schedule. > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.8.0 > > Please let me know if these dates make sense -- they're just proposals > right now. > > best, > Colin > > > On Thu, Dec 28, 2023, at 20:14, Colin McCabe wrote: > > On Thu, Dec 28, 2023, at 18:17, Justine Olshan wrote: > >> Hey Colin, > >> > >> Some folks were concerned about the lack of automatic unclean leader > >> election. I mentioned that KIP-966 would actually be better with its > >> aggressive recovery option. > >> I think folks were hoping for some availability over durability solution > >> for KRaft, so if we don't do KIP-966 we should provide an alternative > or be > >> able to convince ourselves it is not needed. > > > > Hi Justine, > > > > That's a fair point. We should specify in KIP-1012 that we need to have > > some way to configure the system to automatically do unclean leader > > election. If we run out of time implementing KIP-966, this could be > > something quite simple, like honoring the static > > unclean.leader.election = true configuration. > > > >> > >> I think while many folks decided KIP-853 was a blocker, there were a > lot of > >> other features that many folks were expecting so I don't think we can > say > >> definitively the only must-have is KIP-853 (and hence the discussion > thread > >> here :) ) > >> > >> Also as an aside, I filed a ticket to remove ZK from the top of the > >> quickstart guide. https://issues.apache.org/jira/browse/KAFKA-15975 > >> > > > > Yeah. There is a bunch of docs and quickstart cleanup that we should > > do. I don't think any of it is a blocker for 3.8 or 4.0, but the new > > year is a good time to clean things up. > > > > best, > > Colin > > > > > >> Justine > >> > >> On Thu, Dec 28, 2023 at 5:12 PM Colin McCabe > wrote: > >> > >>> Hi Josep, > >>> > >>> Thanks for the KIP. Based on the discussions we had previously, I agree > >>> that we need a 3.8. > >>> > >>> It would be good to link to KIP-833 in the motivation section, since > this > >>> KIP builds on that one. > >>> > >>> Also, I think we should mention in KIP-1012 that 3.8 will be a > >>> general-purpose release that may add some new features. This was > something > >>> that we were on the fence about previously, so it would be good to > clarify > >>> it here. > >>> > >>> On another note. I don't think KIP-966 is a "must-have" for Kafka 3.8, > as > >>> the KIP currently states. I certainly hope that it makes it for 3.8, > but if > >>> it doesn't, it can go into 4.0. It's not needed for migration, so it > could > >>> just as easily go into 4.0 as 3.8. The only thing that KIP-966 really > needs > >>> is "KIP-853: KRaft Controller Membership Changes." > >>> > >>> Along these lines, I think we should drop the language about "strategic > >>> feature parity with Zookeeper." Kafka isn't competing with ZooKeeper, > and > >>> doesn't need feature parity with it. For example, ZK implemented > >>> Netty-TcNative OpenSSL Support, but we don't have that (and probably > won't > >>> in 3.8). We probably won't add this -- or if we do, it won't be so > that we > >>> can have "parity with ZK." Really the only must-have in 3.8 is > KIP-853, and > >>> we should be clear about that. > >>> > >>> I think we should start issuing a deprecation log message at ERROR > level > >>> when brokers start up in ZK mode. This message could point out that > some > >>> safety mechanisms and new features will not
Re: [DISCUSS] KIP-1012: The need for a Kafka 3.8.x release
On this note, I'd like to add that I would volunteer to be the release manager of such release 3.8.0. Best, On Fri, Dec 22, 2023 at 1:31 PM Josep Prat wrote: > Hi all! > As agreed on the "Road to Kafka 4.0" email thread, I created KIP-1012 to > discuss and I'd like to open it up for discussion: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8.x+release > > Let's use this KIP to: > a) Leave a papertrail agreement for the need of a 3.8 version > b) Define which KIPs are the must-haves in regards to KRaft that should be > included there. > > Please let me know your feedback and suggestions. > > Best, > > -- > [image: Aiven] <https://www.aiven.io> > > *Josep Prat* > Open Source Engineering Director, *Aiven* > josep.p...@aiven.io | +491715557497 > aiven.io <https://www.aiven.io> | > <https://www.facebook.com/aivencloud> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> > *Aiven Deutschland GmbH* > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > Amtsgericht Charlottenburg, HRB 209739 B > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] Road to Kafka 4.0
Hi all, Discuss thread for KIP-1012 (The need for a Kafka 3.8 release) can be found under https://lists.apache.org/thread/kvdp2gmq5gd9txkvxh5vk3z2n55b04s5 Best, On Fri, Dec 22, 2023 at 4:00 AM Luke Chen wrote: > For release 3.8, I think we should also include the unclean leader election > support in KRaft. > But we can discuss more details in the KIP. > > Thank you, Josep! > And thank you all for the comments! > > Luke > > On Fri, Dec 22, 2023 at 1:14 AM Ismael Juma wrote: > > > Thank you Josep! > > > > Ismael > > > > On Thu, Dec 21, 2023, 9:09 AM Josep Prat > > wrote: > > > > > Hi Ismael, > > > > > > I can volunteer to write the KIP. Unless somebody else has any > > objections, > > > I'll get to write it by the end of this week. > > > > > > Best, > > > > > > Josep Prat > > > Open Source Engineering Director, aivenjosep.p...@aiven.io | > > > +491715557497 | aiven.io > > > Aiven Deutschland GmbH > > > Alexanderufer 3-7, 10117 Berlin > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > On Thu, Dec 21, 2023, 17:58 Ismael Juma wrote: > > > > > > > Hi all, > > > > > > > > After understanding the use case Josep and Anton described in more > > > detail, > > > > I think it's fair to say that quorum reconfiguration is necessary for > > > > migration of Apache Kafka users who follow this pattern. Given that, > I > > > > think we should have a 3.8 release before the 4.0 release. > > > > > > > > The next question is whether we should do something special when it > > comes > > > > to timeline, parallel releases, etc. After careful consideration, I > > think > > > > we should simply follow our usual approach: regular 3.8 release > around > > > > early May 2024 and regular 4.0 release around early September 2024. > The > > > > community will be able to start working on items specific to 4.0 > after > > > 3.8 > > > > is branched in late March/early April - I don't think we need to deal > > > with > > > > the overhead of maintaining multiple long-lived branches for > > > > feature development. > > > > > > > > If the proposal above sounds reasonable, I suggest we write a KIP and > > > vote > > > > on it. Any volunteers? > > > > > > > > Ismael > > > > > > > > On Tue, Nov 21, 2023 at 8:18 PM Ismael Juma > wrote: > > > > > > > > > Hi Luke, > > > > > > > > > > I think we're conflating different things here. There are 3 > separate > > > > > points in your email, but only 1 of them requires 3.8: > > > > > > > > > > 1. JBOD may have some bugs in 3.7.0. Whatever bugs exist can be > fixed > > > in > > > > > 3.7.x. We have already said that we will backport critical fixes to > > > 3.7.x > > > > > for some time. > > > > > 2. Quorum reconfiguration is important to include in 4.0, the > release > > > > > where ZK won't be supported. This doesn't need a 3.8 release > either. > > > > > 3. Quorum reconfiguration is necessary for migration use cases and > > > hence > > > > > needs to be in a 3.x release. This one would require a 3.8 release > if > > > > true. > > > > > But we should have a debate on whether it is indeed true. It's not > > > clear > > > > to > > > > > me yet. > > > > > > > > > > Ismael > > > > > > > > > > On Tue, Nov 21, 2023 at 7:30 PM Luke Chen > wrote: > > > > > > > > > >> Hi Colin and Jose, > > > > >> > > > > >> I revisited the discussion of KIP-833 here > > > > >> <https://lists.apache.org/thread/90zkqvmmw3y8j6tkgbg3md78m7hs4yn6 > >, > > > and > > > > >> you > > > > >> can see I'm the first one to reply to the discussion thread to > > express > > > > my > > > > >> excitement at that time. Till now, I personally still think having > > > KRaft > > > > >> in > > > > >> Kafka is a good direction we have to move forward. But to move to > > this > > > > >> destination, we need to make our user
[DISCUSS] KIP-1012: The need for a Kafka 3.8.x release
Hi all! As agreed on the "Road to Kafka 4.0" email thread, I created KIP-1012 to discuss and I'd like to open it up for discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1012%3A+The+need+for+a+Kafka+3.8.x+release Let's use this KIP to: a) Leave a papertrail agreement for the need of a 3.8 version b) Define which KIPs are the must-haves in regards to KRaft that should be included there. Please let me know your feedback and suggestions. Best, -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Request permissions to contribute to Apache Kafka
Hi Jiao, Your accounts are now all set. The Jira one was already properly set up. Let me know if you have any problems with the accounts. Thanks for showing your interest in Apache Kafka, Best, On Fri, Dec 22, 2023 at 5:50 AM Jiao Zhang wrote: > Hi team, > > May I request permissions? > My wiki ID is "zhangjiao.thu" > My jira ID is "Jiao-zhang" > > Thank you! > > -- > Jiao Zhang > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] Road to Kafka 4.0
Hi Ismael, I can volunteer to write the KIP. Unless somebody else has any objections, I'll get to write it by the end of this week. Best, Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Thu, Dec 21, 2023, 17:58 Ismael Juma wrote: > Hi all, > > After understanding the use case Josep and Anton described in more detail, > I think it's fair to say that quorum reconfiguration is necessary for > migration of Apache Kafka users who follow this pattern. Given that, I > think we should have a 3.8 release before the 4.0 release. > > The next question is whether we should do something special when it comes > to timeline, parallel releases, etc. After careful consideration, I think > we should simply follow our usual approach: regular 3.8 release around > early May 2024 and regular 4.0 release around early September 2024. The > community will be able to start working on items specific to 4.0 after 3.8 > is branched in late March/early April - I don't think we need to deal with > the overhead of maintaining multiple long-lived branches for > feature development. > > If the proposal above sounds reasonable, I suggest we write a KIP and vote > on it. Any volunteers? > > Ismael > > On Tue, Nov 21, 2023 at 8:18 PM Ismael Juma wrote: > > > Hi Luke, > > > > I think we're conflating different things here. There are 3 separate > > points in your email, but only 1 of them requires 3.8: > > > > 1. JBOD may have some bugs in 3.7.0. Whatever bugs exist can be fixed in > > 3.7.x. We have already said that we will backport critical fixes to 3.7.x > > for some time. > > 2. Quorum reconfiguration is important to include in 4.0, the release > > where ZK won't be supported. This doesn't need a 3.8 release either. > > 3. Quorum reconfiguration is necessary for migration use cases and hence > > needs to be in a 3.x release. This one would require a 3.8 release if > true. > > But we should have a debate on whether it is indeed true. It's not clear > to > > me yet. > > > > Ismael > > > > On Tue, Nov 21, 2023 at 7:30 PM Luke Chen wrote: > > > >> Hi Colin and Jose, > >> > >> I revisited the discussion of KIP-833 here > >> <https://lists.apache.org/thread/90zkqvmmw3y8j6tkgbg3md78m7hs4yn6>, and > >> you > >> can see I'm the first one to reply to the discussion thread to express > my > >> excitement at that time. Till now, I personally still think having KRaft > >> in > >> Kafka is a good direction we have to move forward. But to move to this > >> destination, we need to make our users comfortable with this decision. > The > >> worst scenario is, we said 4.0 is ready, and ZK is removed. Then, some > >> users move to 4.0 and say, wait a minute, why does it not support xxx > >> feature? And then start to search for other alternatives to replace > Apache > >> Kafka. We all don't want to see this, right? So, that's why some > community > >> users start to express their concern to move to 4.0 too quickly, > including > >> me. > >> > >> > >> Quoting Colin: > >> > While dynamic quorum reconfiguration is a nice feature, it doesn't > block > >> anything: not migration, not deployment. > >> > >> Clearly Confluent team might deploy ZooKeeper in a particular way and > >> didn’t depend on its ability to support reconfiguration. So KRaft is > ready > >> from your point of view. But users of Apache Kafka might have come to > >> depend on some ZooKeeper functionality, such as the ability to > reconfigure > >> ZooKeeper quorums, that is not available in KRaft, yet. I don’t think > the > >> Apache Kafka documentation has ever said “do not depend on this ability > of > >> Apache Kafka or Zookeeper”, so it doesn’t seem unreasonable for users to > >> have deployed ZooKeeper in this way. In KIP-833 > >> < > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-MissingFeatures > >> >, > >> we said: “Modifying certain dynamic configurations on the standalone > KRaft > >> controller” was an important missing feature. Unfortunately it wasn’t as > >> explicit as it could have been. While no one expects KRaft to support > all > >> the features of ZooKeeper, it looks to me that users might depend on > this > >
Re: [DISCUSS] KIP-853: KRaft Voters Change
Hi Jose, Thanks for the update. Best, Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B On Wed, Dec 20, 2023, 20:48 José Armando García Sancio wrote: > Yes. I have been working on the KIP. I don't expect to change too much > from what is currently documented. I am mainly focusing on improving > the UX and readability of the KIP. > > I am hoping to be able to restart the discussion first thing next year. > > Thanks > > On Wed, Dec 20, 2023 at 6:11 AM Josep Prat > wrote: > > > > Hi Jose, > > Did you have any time to update the KIP? Let me know if we can help in > any > > way. > > > > Thanks in advance, > > > > On Mon, Nov 27, 2023 at 7:04 PM José Armando García Sancio > > wrote: > > > > > On Mon, Nov 27, 2023 at 2:32 AM Josep Prat wrote: > > > > I wanted to revive this thread and see if there is anything > preventing > > > it to be voted on. Happy to help unblocking anything that might be > holding > > > this back. > > > > > > Hi Josep, > > > > > > Thanks for reviving the thread. I need to make some changes to the > > > KIP. My thinking has changed a bit since I wrote this KIP. The core of > > > the design still holds. I just want to improve the wording and > > > usability. > > > > > > I should have an updated KIP this week. I'll restart the discussion > > > thread at that point. > > > > > > Thanks, > > > -- > > > -José > > > > > > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud> > > <https://www.linkedin.com/company/aiven/> < > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > > -- > -José >
Re: [DISCUSS] Road to Kafka 4.0
> > > how > > > > > >> to handle controller disk failures earlier in this thread. > > > > > >> > > > > > >> I should note here that the broker in ZooKeeper mode also > requires > > > > > manual > > > > > >> handling of disk failures. Restarting a broker with the same ID, > > but > > > > an > > > > > >> empty disk, breaks the invariants of replication when in ZK > mode. > > > > > Consider: > > > > > >> > > > > > >> 1. Broker 1 goes down. A ZK state change notification for > /brokers > > > > fires > > > > > >> and goes on the controller queue. > > > > > >> > > > > > >> 2. Broker 1 comes back up with an empty disk. > > > > > >> > > > > > >> 3. The controller processes the zk state change notification for > > > > > /brokers. > > > > > >> Since broker 1 is up no action is taken. > > > > > >> > > > > > >> 4. Now broker 1 is in the ISR for any partitions it was > > previously, > > > > but > > > > > >> has no data. If it is or becomes leader for any partitions, > > > > irreversable > > > > > >> data loss will occur. > > > > > >> > > > > > >> This problem is more than theoretical. We at Confluent have > > observed > > > > it > > > > > in > > > > > >> production and put in place special workarounds for the ZK > > clusters > > > we > > > > > >> still have. > > > > > >> > > > > > >> KRaft has never had this problem because brokers are removed > from > > > ISRs > > > > > >> when a new incarnation of the broker registers. > > > > > >> > > > > > >> So perhaps ZK mode is not ready for production for Aiven? Since > > disk > > > > > >> failures do in fact require special handling there. (And/or > > bringing > > > > up > > > > > new > > > > > >> nodes with empty disks, which seems to be their main concern.) > > > > > >> > > > > > >> > > > > > > >> > It’s also worth thinking about how Apache Kafka users who > depend > > > on > > > > > JBOD > > > > > >> > might look at the risks of not having a 3.8 release. JBOD > > support > > > on > > > > > >> KRaft > > > > > >> > is planned to be added in 3.7, and is still in progress so > far. > > So > > > > > it’s > > > > > >> > hard to say it’s a blocker or not. But in practice, even if > the > > > > > feature > > > > > >> is > > > > > >> > made into 3.7 in time, a lot of new code for this feature is > > > > unlikely > > > > > to > > > > > >> be > > > > > >> > entirely bug free. We need to maintain the confidence of those > > > > users, > > > > > and > > > > > >> > forcing them to migrate through 3.7 where this new code is > > hardly > > > > > >> > battle-tested doesn’t appear to do that. > > > > > >> > > > > > > >> > > > > > >> As Ismael said, if there are JBOD bugs in 3.7, we will do > > follow-on > > > > > point > > > > > >> releases to address them. > > > > > >> > > > > > >> > Our goal for 4.0 should be that all the “main” features in > KRaft > > > are > > > > > in > > > > > >> > production ready state. To reach the goal, I think having one > > more > > > > > >> release > > > > > >> > makes sense. We can have different opinions about what the > “main > > > > > >> features” > > > > > >> > in KRaft are, but we should all agree, JBOD is one of them. > > > > > >> > > > > > >> The current plan is for JBOD to be production-ready in the 3.7 > > > branch. > > > > > >> > > > > > >> The other features of KRaft have been in production-ready state > > > since &g
Re: [DISCUSS] KIP-853: KRaft Voters Change
Hi Jose, Did you have any time to update the KIP? Let me know if we can help in any way. Thanks in advance, On Mon, Nov 27, 2023 at 7:04 PM José Armando García Sancio wrote: > On Mon, Nov 27, 2023 at 2:32 AM Josep Prat wrote: > > I wanted to revive this thread and see if there is anything preventing > it to be voted on. Happy to help unblocking anything that might be holding > this back. > > Hi Josep, > > Thanks for reviving the thread. I need to make some changes to the > KIP. My thinking has changed a bit since I wrote this KIP. The core of > the design still holds. I just want to improve the wording and > usability. > > I should have an updated KIP this week. I'll restart the discussion > thread at that point. > > Thanks, > -- > -José > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: New Release Branch 3.7
Hi Stanislav, I cherry-picked this change to 3.7: https://github.com/apache/kafka/pull/15002 Best, On Thu, Dec 14, 2023 at 5:30 PM Satish Duggana wrote: > Hi Stanislav, > +1 to what Luke mentioned in the earlier email. I have also replied to your > comment on https://issues.apache.org/jira/browse/KAFKA-15147 > > These are minor changes and we plan to target the remaining changes to be > merged in the next few days(before 20th Dec). > > Thanks, > Satish. > > On Thu, 14 Dec 2023 at 17:54, Luke Chen wrote: > > > Hi Stanislav, > > > > For KIP-963, there are only 1 remaining task and 2 PRs which should be > > close to get merged. (ref: > > https://issues.apache.org/jira/browse/KAFKA-15147) > > Since this is an improvement for tiered storage metrics, not a huge > > change, I'd like to backport them into 3.7 branch if no objection from > > you. > > > > Thanks. > > Luke > > > > On Thu, Dec 14, 2023 at 5:15 PM Stanislav Kozlovski > > wrote: > > > > > > Hey all, > > > > > > (thanks to Josep for reviewing the 3.8 bump PR) > > > > > > I have two more PRs to get reviewed regarding the release: > > > - targeting trunk: MINOR: Update documentation.html with the 3.7 > release > > > #15010 <https://github.com/apache/kafka/pull/15010> > > > - targeting 3.7: MINOR: Update documentation.html with the 3.7 release > > > #15011 <https://github.com/apache/kafka/pull/15011> > > > > > > Additionally, I have one ask: > > > - if you are reading this message, can you double-check the list of > KIPs > > > being released in the Release Page > > > <https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.7.0> > > and > > > if you recognize any KIP you are involved it - can you ensure that the > > data > > > in the page & the associated JIRA/KIP (target release, merge status, > vote > > > status) is up to date? > > > > > > I have myself went over the list of KIPs and a bit of the commit > > history. I > > > think KIPs: > > > - KIP-998: Give ProducerConfig(props, doLog) constructor protected > access > > > - KIP-938: Add more metrics for measuring KRaft performance > > > are slipping this release. > > > > > > By far our biggest feature this release, KIP-848 The Next Generation of > > the > > > Consumer Rebalance Protocol, is a bit on the border. The main PR that > KIP > > > is dependent on is this KAFKA-15456: Commit/Fetch error handling > > > improvements and V9 support #14557 > > > <https://github.com/apache/kafka/pull/14557>. > > > While it's a gray area, I am weighing this change more on the > > stabilization > > > side as well as allowing it to come in the middle of feature freeze for > > two > > > chief reasons: a) it fixes issues that were recently discovered, hence > > can > > > be marked as stabilization work and b) it only touches the code path > used > > > for Early Access, and not the existing production code path (hence > > doesn't > > > substantially risk the release) > > > > > > Additionally, KIP-858 has one minor but important change pending > > > <https://github.com/apache/kafka/pull/14984>. > > > > > > With that, I remind you that there are only 6 days to code freeze! It's > > not > > > long until we will have our very first Apache Kafka 3.7 RC! Let's get > > this > > > shipped. > > > > > > Best, > > > Stanislav > > > > > > On Tue, Dec 12, 2023 at 2:56 PM Stanislav Kozlovski < > > stanis...@confluent.io> > > > wrote: > > > > > > > Hello Kafka developers and friends, > > > > > > > > As promised, we now have a release branch for 3.7 release. > > > > Trunk is being bumped to 3.8.0-SNAPSHOT (please help review the PR < > > https://github.com/apache/kafka/pull/14993>). > > > > > > > > I'll be going over the JIRAs to move every non-blocker from this > > release to the next release. > > > > > > > > From this point, most changes should go to trunk. > > > > *- Blockers (existing and new that we discover while testing the > > release) will be double-committed.* > > > > *- Please discuss with your reviewer whether your PR should go to > > trunk or to trunk+release so they can merge accordingly.* > > > > *- Please help us test the release!* > > > > > > > > Thanks! > > > > > > > > -- > > > > Best, > > > > Stanislav > > > > > > > > > > > > > -- > > > Best, > > > Stanislav > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Request contributor permissions
Hi Ayoub, Your accounts are all set. Let me know if you have any questions. Thanks for your interest in Apache Kafka! Best, On Wed, Dec 13, 2023 at 3:15 PM Ayoub wrote: > Hello, > > I am interested in contributing to Apache Kafka. > Could you please grant me contributor access ? > > JIRA username: ayoubomari > Wiki username: ayoubomari > Github username: AyoubOm > > Thank you, > Ayoub > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] Apache Kafka 3.5.2
Thanks Luke for running the release! Best! On Mon, Dec 11, 2023 at 12:34 PM Luke Chen wrote: > The Apache Kafka community is pleased to announce the release for > Apache Kafka 3.5.2 > > This is a bugfix release. It contains many bug fixes including > upgrades the Snappy and Rocksdb dependencies. > > All of the changes in this release can be found in the release notes: > https://www.apache.org/dist/kafka/3.5.2/RELEASE_NOTES.html > > > You can download the source and binary release from: > https://kafka.apache.org/downloads#3.5.2 > > > --- > > > Apache Kafka is a distributed streaming platform with four core APIs: > > > ** The Producer API allows an application to publish a stream of records to > one or more Kafka topics. > > ** The Consumer API allows an application to subscribe to one or more > topics and process the stream of records produced to them. > > ** The Streams API allows an application to act as a stream processor, > consuming an input stream from one or more topics and producing an > output stream to one or more output topics, effectively transforming the > input streams to output streams. > > ** The Connector API allows building and running reusable producers or > consumers that connect Kafka topics to existing applications or data > systems. For example, a connector to a relational database might > capture every change to a table. > > > With these APIs, Kafka can be used for two broad classes of application: > > ** Building real-time streaming data pipelines that reliably get data > between systems or applications. > > ** Building real-time streaming applications that transform or react > to the streams of data. > > > Apache Kafka is in use at large and small companies worldwide, including > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > A big thank you for the following contributors to this release! > > A. Sophie Blee-Goldman, atu-sharm, bachmanity1, Calvin Liu, Chase > Thomas, Chris Egerton, Colin Patrick McCabe, David Arthur, Divij > Vaidya, Federico Valeri, flashmouse, Florin Akermann, Greg Harris, > hudeqi, José Armando García Sancio, Levani Kokhreidze, Lucas Brutschy, > Luke Chen, Manikumar Reddy, Matthias J. Sax, Mickael Maison, Nick > Telford, Okada Haruki, Omnia G.H Ibrahim, Robert Wagner, Rohan, Said > Boudjelda, sciclon2, Vincent Jiang, Xiaobing Fang, Yash Mayya > > We welcome your help and feedback. For more information on how to > report problems, and to get involved, visit the project website at > https://kafka.apache.org/ > > Thank you! > > Regards, > Luke > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Requesting contributor permissions
Hi Kondrát, You should be good to go now! Thanks for showing interest in Apache Kafka! Best, On Mon, Dec 11, 2023 at 11:27 AM Kondrát Bertalan wrote: > Hi All, > Please add me as a contributor. > > JIRA username: k0b3rit > GitHub username: k0b3rit > Wiki username: k0b3rit > > Thank you > Berci > -- > *Bertalan Kondrat* | Founder > t. +36(70) 413-4801 > servy.hu <https://www.servy.hu/> > > > [image: Servy] <https://www.cloudera.com/> > -- > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] Apache Kafka 3.6.1
Thanks for running this release! Best, On Mon, Dec 11, 2023 at 8:35 AM Bruno Cadonna wrote: > Thanks for managing the release, Mickael! > > Best, > Bruno > > On 12/8/23 12:44 PM, Luke Chen wrote: > > Hi Mickael, > > > > Thanks for running this release! > > > > Luke > > > > On Thu, Dec 7, 2023 at 7:13 PM Mickael Maison > wrote: > > > >> The Apache Kafka community is pleased to announce the release for > >> Apache Kafka 3.6.1 > >> > >> This is a bug fix release and it includes fixes and improvements from 30 > >> JIRAs. > >> > >> All of the changes in this release can be found in the release notes: > >> https://www.apache.org/dist/kafka/3.6.1/RELEASE_NOTES.html > >> > >> You can download the source and binary release (Scala 2.12 and Scala > 2.13) > >> from: > >> https://kafka.apache.org/downloads#3.6.1 > >> > >> > >> > --- > >> > >> Apache Kafka is a distributed streaming platform with four core APIs: > >> > >> ** The Producer API allows an application to publish a stream of > records to > >> one or more Kafka topics. > >> > >> ** The Consumer API allows an application to subscribe to one or more > >> topics and process the stream of records produced to them. > >> > >> ** The Streams API allows an application to act as a stream processor, > >> consuming an input stream from one or more topics and producing an > >> output stream to one or more output topics, effectively transforming the > >> input streams to output streams. > >> > >> ** The Connector API allows building and running reusable producers or > >> consumers that connect Kafka topics to existing applications or data > >> systems. For example, a connector to a relational database might > >> capture every change to a table. > >> > >> > >> With these APIs, Kafka can be used for two broad classes of application: > >> > >> ** Building real-time streaming data pipelines that reliably get data > >> between systems or applications. > >> > >> ** Building real-time streaming applications that transform or react > >> to the streams of data. > >> > >> > >> Apache Kafka is in use at large and small companies worldwide, including > >> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, > >> Target, The New York Times, Uber, Yelp, and Zalando, among others. > >> > >> A big thank you for the following 39 contributors to this release! > >> (Please report an unintended omission) > >> > >> Anna Sophie Blee-Goldman, Arpit Goyal, atu-sharm, Bill Bejeck, Chris > >> Egerton, Colin P. McCabe, David Arthur, David Jacot, Divij Vaidya, > >> Federico Valeri, Greg Harris, Guozhang Wang, Hao Li, hudeqi, > >> iit2009060, Ismael Juma, Jorge Esteban Quilcate Otoya, Josep Prat, > >> Jotaniya Jeel, Justine Olshan, Kamal Chandraprakash, kumarpritam863, > >> Levani Kokhreidze, Lucas Brutschy, Luke Chen, Manikumar Reddy, > >> Matthias J. Sax, Mayank Shekhar Narula, Mickael Maison, Nick Telford, > >> Philip Nee, Qichao Chu, Rajini Sivaram, Robert Wagner, Sagar Rao, > >> Satish Duggana, Walker Carlson, Xiaobing Fang, Yash Mayya > >> > >> We welcome your help and feedback. For more information on how to > >> report problems, and to get involved, visit the project website at > >> https://kafka.apache.org/ > >> > >> Thank you! > >> > >> Regards, > >> Mickael > >> > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.6.1 RC0
Hi Mickael, This PR[1] made me realize NOTICE-binary is missing the notice for commons-io. I don't know if it's a blocker or not. I can cherry pick the commit to the 3.6 branch if you want. Best, [1]: https://github.com/apache/kafka/pull/14865 On Tue, Nov 28, 2023 at 10:25 AM Josep Prat wrote: > Hi Mickael, > Thanks for running the release. It's a +1 for me (non-binding). > I did the following: > - Verified artifact's signatures and hashes > - Checked JavaDoc (with navigation to Oracle JavaDoc) > - Compiled source code > - Run unit tests and integration tests > - Run getting started with ZK and KRaft > > Best, > > On Tue, Nov 28, 2023 at 8:51 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > >> +1 (non-binding) >> >> 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and 2.13 >> 2. Ran all the unit and integration tests. >> 3. Ran quickstart and verified the produce-consume on a 3 node cluster. >> 4. Verified the tiered storage functionality with local-tiered storage. >> >> On Tue, Nov 28, 2023 at 12:55 AM Federico Valeri >> wrote: >> >> > Hi Mickael, >> > >> > - Build from source (Java 17, Scala 2.13) >> > - Run unit and integration tests >> > - Run custom client apps using staging artifacts >> > >> > +1 (non binding) >> > >> > Thanks >> > Fede >> > >> > >> > >> > On Sun, Nov 26, 2023 at 11:34 AM Jakub Scholz wrote: >> > > >> > > +1 non-binding. I used the staged Scala 2.13 artifacts and the staged >> > Maven >> > > repo for my tests. All seems to work fine. >> > > >> > > Thanks >> > > Jakub >> > > >> > > On Fri, Nov 24, 2023 at 4:37 PM Mickael Maison >> > wrote: >> > > >> > > > Hello Kafka users, developers and client-developers, >> > > > >> > > > This is the first candidate for release of Apache Kafka 3.6.1. >> > > > >> > > > This is a bugfix release with several fixes, including dependency >> > > > version bumps for CVEs. >> > > > >> > > > Release notes for the 3.6.1 release: >> > > > >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/RELEASE_NOTES.html >> > > > >> > > > *** Please download, test and vote by Friday, December 1 >> > > > >> > > > Kafka's KEYS file containing PGP keys we use to sign the release: >> > > > https://kafka.apache.org/KEYS >> > > > >> > > > * Release artifacts to be voted upon (source and binary): >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/ >> > > > >> > > > * Maven artifacts to be voted upon: >> > > > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ >> > > > >> > > > * Javadoc: >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/javadoc/ >> > > > >> > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.1 tag: >> > > > https://github.com/apache/kafka/releases/tag/3.6.1-rc0 >> > > > >> > > > PR for updating docs: >> > > > https://github.com/apache/kafka-site/pull/568 >> > > > >> > > > * Documentation: >> > > > https://kafka.apache.org/36/documentation.html >> > > > >> > > > * Protocol: >> > > > https://kafka.apache.org/36/protocol.html >> > > > >> > > > * Successful Jenkins builds for the 3.6 branch: >> > > > Unit/integration tests: We still have a lot of flaky tests in the >> 3.6 >> > > > branch. Looking at the last few 3.6 builds in >> > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/ it seems >> all >> > > > tests passed at least once apart from >> > > > ClusterConnectionStatesTest.testSingleIP(). There's >> > > > https://issues.apache.org/jira/browse/KAFKA-15762 to fix that test. >> > > > System tests: Still running I'll post an update once they complete. >> > > > >> > > > Thanks, >> > > > Mickael >> > > > >> > >> > > > -- > [image: Aiven] <https://www.aiven.io> > > *Josep Prat* > Open Source Engineering Director, *Aiven* > josep.p...@aiven.io | +491715557497 > aiven.io <https://www.aiven.io> | > <https://www.facebook.com/aivencloud> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> > *Aiven Deutschland GmbH* > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > Amtsgericht Charlottenburg, HRB 209739 B > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.6.1 RC0
Hi Mickael, Thanks for running the release. It's a +1 for me (non-binding). I did the following: - Verified artifact's signatures and hashes - Checked JavaDoc (with navigation to Oracle JavaDoc) - Compiled source code - Run unit tests and integration tests - Run getting started with ZK and KRaft Best, On Tue, Nov 28, 2023 at 8:51 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding) > > 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and 2.13 > 2. Ran all the unit and integration tests. > 3. Ran quickstart and verified the produce-consume on a 3 node cluster. > 4. Verified the tiered storage functionality with local-tiered storage. > > On Tue, Nov 28, 2023 at 12:55 AM Federico Valeri > wrote: > > > Hi Mickael, > > > > - Build from source (Java 17, Scala 2.13) > > - Run unit and integration tests > > - Run custom client apps using staging artifacts > > > > +1 (non binding) > > > > Thanks > > Fede > > > > > > > > On Sun, Nov 26, 2023 at 11:34 AM Jakub Scholz wrote: > > > > > > +1 non-binding. I used the staged Scala 2.13 artifacts and the staged > > Maven > > > repo for my tests. All seems to work fine. > > > > > > Thanks > > > Jakub > > > > > > On Fri, Nov 24, 2023 at 4:37 PM Mickael Maison > > wrote: > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > This is the first candidate for release of Apache Kafka 3.6.1. > > > > > > > > This is a bugfix release with several fixes, including dependency > > > > version bumps for CVEs. > > > > > > > > Release notes for the 3.6.1 release: > > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/RELEASE_NOTES.html > > > > > > > > *** Please download, test and vote by Friday, December 1 > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > https://kafka.apache.org/KEYS > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/ > > > > > > > > * Maven artifacts to be voted upon: > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > * Javadoc: > > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/javadoc/ > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.1 tag: > > > > https://github.com/apache/kafka/releases/tag/3.6.1-rc0 > > > > > > > > PR for updating docs: > > > > https://github.com/apache/kafka-site/pull/568 > > > > > > > > * Documentation: > > > > https://kafka.apache.org/36/documentation.html > > > > > > > > * Protocol: > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > Unit/integration tests: We still have a lot of flaky tests in the 3.6 > > > > branch. Looking at the last few 3.6 builds in > > > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/ it seems > all > > > > tests passed at least once apart from > > > > ClusterConnectionStatesTest.testSingleIP(). There's > > > > https://issues.apache.org/jira/browse/KAFKA-15762 to fix that test. > > > > System tests: Still running I'll post an update once they complete. > > > > > > > > Thanks, > > > > Mickael > > > > > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] KIP-853: KRaft Voters Change
Hi community, I wanted to revive this thread and see if there is anything preventing it to be voted on. Happy to help unblocking anything that might be holding this back. Best, Josep Prat On 2022/07/27 15:08:24 José Armando García Sancio wrote: > Hi all, > > Community members Jason Gustafson, Colin P. McCabe and I have been > having some offline conversations. > > At a high-level KIP-853 solves the problems: > 1) How can KRaft detect and recover from disk failures on the minority > of the voters? > 2) How can KRaft support a changing set of voter nodes? > > I think that problem 2) is a superset of problem 1). The mechanism for > solving problem 2) can be used to solve problem 1). This is the reason > that I decided to design them together and proposed this KIP. Problem > 2) adds the additional requirement of how observers (Brokers and new > Controllers) discover the leader? KIP-853 solves this problem by > returning the endpoint of the leader in all of the KRaft RPCs. There > are some concerns with this approach. > > To solve problem 1) we don't need to return the leader's endpoint > since it is expressed in the controller.quorum.voters property. To > make faster progress on 1) I have decided to create "KIP-856: KRaft > Disk Failure Recovery" that just addresses this problem. I will be > starting a discussion thread for KIP-856 soon. > > We can continue the discussion of KIP-853 here. If KIP-856 gets > approved I will either: > 3) Modify KIP-853 to just describe the improvement needed on top of KIP-856. > 4) Create a new KIP and abandon KIP-853. This new KIP will take into > account all of the discussion from this thread. > > Thanks! > -- > -José >
Re: [VOTE] 3.5.2 RC1
Hi Luke, Thanks for running the release. I did the following: - Verified artifact's signatures and hashes - Checked JavaDoc (with navigation to Oracle JavaDoc) - Compiled source code - Run unit tests and integration tests - Run getting started with ZK and KRaft It gets a +1 from my side (non-binding) Best, On Tue, Nov 21, 2023 at 11:09 AM Luke Chen wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka 3.5.2. > > This is a bugfix release with several fixes since the release of 3.5.1, > including dependency version bumps for CVEs. > > Release notes for the 3.5.2 release: > https://home.apache.org/~showuon/kafka-3.5.2-rc1/RELEASE_NOTES.html > > *** Please download, test and vote by Nov. 28. > > Kafka's KEYS file containing PGP keys we use to sign the release: > https://kafka.apache.org/KEYS > > * Release artifacts to be voted upon (source and binary): > https://home.apache.org/~showuon/kafka-3.5.2-rc1/ > > * Maven artifacts to be voted upon: > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > * Javadoc: > https://home.apache.org/~showuon/kafka-3.5.2-rc1/javadoc/ > > * Tag to be voted upon (off 3.5 branch) is the 3.5.2 tag: > https://github.com/apache/kafka/releases/tag/3.5.2-rc1 > > * Documentation: > https://kafka.apache.org/35/documentation.html > > * Protocol: > https://kafka.apache.org/35/protocol.html > > * Successful Jenkins builds for the 3.5 branch: > Unit/integration tests: > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.5/98/ > There are some falky tests, including the testSingleIP test failure. It > failed because of some infra change and we fixed it > <https://github.com/apache/kafka/pull/14741> recently. > > System tests: running, will update the results later. > > > > Thank you. > Luke > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
[jira] [Created] (KAFKA-15875) Snapshot class is package protected but returned in public methods
Josep Prat created KAFKA-15875: -- Summary: Snapshot class is package protected but returned in public methods Key: KAFKA-15875 URL: https://issues.apache.org/jira/browse/KAFKA-15875 Project: Kafka Issue Type: Task Affects Versions: 3.6.0 Reporter: Josep Prat Assignee: Josep Prat org.apache.kafka.timeline.Snapshot class is package protected but it is part of the public API of org.apache.kafka.timeline.SnapshotRegistry. This might cause compilation errors if we ever try to assign the returned object of these methods to a variable. org.apache.kafka.controller.OffsetControlManager is calling SnapshotRegistry's methods that return a Snapshot and OffsetControlManager is in another package. The SnapshotRegistry class seems to not be public API so I don't think this needs a KIP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Requesting permissions to contribute to Apache Kafka
Hi Ria, You are now set. Thanks for your interest in Apache Kafka! Best, On Mon, Nov 20, 2023 at 5:48 PM Ria Pradeep (BLOOMBERG/ 919 3RD A) < rprade...@bloomberg.net> wrote: > I would like to request permission to contribute to Apache Kafka. > > wiki ID: rpradeep > JIRA ID: rpradeep > > Thanks, > Ria -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [DISCUSS] Road to Kafka 4.0
Hi Colin, I think it's great that Confluent runs KRaft clusters in production, and it means that it is production ready for Confluent and it's users. But luckily for Kafka, the community is bigger than this (self managed in the cloud or in-prem, or customers of other SaaS companies). We've heard at least from 1 SaaS company, Aiven (disclaimer, it is my employer) where the current feature set makes it not trivial to migrate. This same issue might happen not only at Aiven but with any user of Kafka who uses immutable infrastructure. Another case is for users that have hundreds (or more) of clusters and more than 100k nodes experience node failures multiple times during a single day. In this situation, not having KIP 853 makes these power users unable to join the game as introducing a new error-prone manual (or needed to automate) operation is usually a huge no-go. But I hear the concerns of delaying 4.0 for another 3 to 4 months. Would it help if we would aim at shortening the timeline for 3.8.0 and start with the 4.0.0 a bit earlier help? Maybe we could work on 3.8.0 almost in parallel with 4.0.0: - Start with 3.8.0 release process - After a small time (let's say a week) create the release branch - Start with 4.0.0 release process as usual - Cherry pick KRaft related issues to 3.8.0 - Release 3.8.0 I suspect 4.0.0 will need a bit more time than usual to ensure the code is cleaned up of deprecated classes and methods on top of the usual work we have. For this reason I think there would be enough time between releasing 3.8.0 and 4.0.0. What do you all think? Best, Josep Prat On 2023/11/20 20:03:18 Colin McCabe wrote: > Hi Josep, > > I think there is some confusion here. Quorum reconfiguration is not needed > for KRaft to become production ready. Confluent runs thousands of KRaft > clusters without quorum reconfiguration, and has for years. While dynamic > quorum reconfiguration is a nice feature, it doesn't block anything: not > migration, not deployment. As best as I understand it, the use-case Aiven has > isn't even reconfiguration per se, just wiping a disk. There are ways to > handle this -- I discussed some earlier in the thread. I think it would be > productive to continue that discussion -- especially the part around > documentation and testing of these cases. > > A lot of people have done a lot of work to get Kafka 4.0 ready. I would not > want to delay that because we want an additional feature. And we will always > want additional features. So I am concerned we will end up in an infinite > loop of people asking for "just one more feature" before they migrate. > > best, > Colin > > > On Mon, Nov 20, 2023, at 04:15, Josep Prat wrote: > > Hi all, > > > > I wanted to share my opinion regarding this topic. I know some > > discussions happened some time ago (over a year) but I believe it's > > wise to reflect and re-evaluate if those decisions are still valid. > > KRaft, as of Kafka 3.6.x and 3.7.x, has not yet feature parity with > > Zookeeper. By dropping Zookeeper altogether before achieving such > > parity, we are opening the door to leaving a chunk of Apache Kafka > > users without an easy way to upgrade to 4.0. > > In pro of making upgrades as smooth as possible, I propose to have a > > Kafka version where KIP-853 is merged and Zookeeper still is supported. > > This will enable community members who can't migrate yet to KRaft to do > > so in a safe way (rolling back is something goes wrong). Additionally, > > this will give us more confidence on having KRaft replacing > > successfully Zookeeper without any big problems by discovering and > > fixing bugs or by confirming that KRaft works as expected. > > For this I strongly believe we should have a 3.8.x version before 4.0.x. > > > > What do other think in this regard? > > > > Best, > > > > On 2023/11/14 20:47:10 Colin McCabe wrote: > >> On Tue, Nov 14, 2023, at 04:37, Anton Agestam wrote: > >> > Hi Colin, > >> > > >> > Thank you for your thoughtful and comprehensive response. > >> > > >> >> KIP-853 is not a blocker for either 3.7 or 4.0. We discussed this in > >> >> several KIPs that happened this year and last year. The most notable was > >> >> probably KIP-866, which was approved in May 2022. > >> > > >> > I understand this is the case, I'm raising my concern because I was > >> > foreseeing some major pain points as a consequence of this decision. Just > >> > to make it clear though: I am not asking for anyone to do work for me, > >> > and > >> > I understand the limitations of resources available to implement >
Re: [DISCUSS] Road to Kafka 4.0
Hi all, I wanted to share my opinion regarding this topic. I know some discussions happened some time ago (over a year) but I believe it's wise to reflect and re-evaluate if those decisions are still valid. KRaft, as of Kafka 3.6.x and 3.7.x, has not yet feature parity with Zookeeper. By dropping Zookeeper altogether before achieving such parity, we are opening the door to leaving a chunk of Apache Kafka users without an easy way to upgrade to 4.0. In pro of making upgrades as smooth as possible, I propose to have a Kafka version where KIP-853 is merged and Zookeeper still is supported. This will enable community members who can't migrate yet to KRaft to do so in a safe way (rolling back is something goes wrong). Additionally, this will give us more confidence on having KRaft replacing successfully Zookeeper without any big problems by discovering and fixing bugs or by confirming that KRaft works as expected. For this I strongly believe we should have a 3.8.x version before 4.0.x. What do other think in this regard? Best, On 2023/11/14 20:47:10 Colin McCabe wrote: > On Tue, Nov 14, 2023, at 04:37, Anton Agestam wrote: > > Hi Colin, > > > > Thank you for your thoughtful and comprehensive response. > > > >> KIP-853 is not a blocker for either 3.7 or 4.0. We discussed this in > >> several KIPs that happened this year and last year. The most notable was > >> probably KIP-866, which was approved in May 2022. > > > > I understand this is the case, I'm raising my concern because I was > > foreseeing some major pain points as a consequence of this decision. Just > > to make it clear though: I am not asking for anyone to do work for me, and > > I understand the limitations of resources available to implement features. > > What I was asking is rather to consider the implications of _removing_ > > features before there exists a replacement for them. > > > > I understand that the timeframe for 3.7 isn't feasible, and because of that > > I think what I was asking is rather: can we make sure that there are more > > 3.x releases until controller quorum online resizing is implemented? > > > > From your response, I gather that your stance is that it's important to > > drop ZK support sooner rather than later and that the necessary pieces for > > doing so are already in place. > > Hi Anton, > > Yes. I'm basically just repeating what we agreed upon in 2022 as part of > KIP-833. > > > > > --- > > > > I want to make sure I've understood your suggested sequence for controller > > node replacement. I hope the mentions of Kubernetes are rather for examples > > of how to carry things out, rather than saying "this is only supported on > > Kubernetes"? > > Apache Kafka is supported in lots of environments, including non-k8s ones. I > was just pointing out that using k8s means that you control your own DNS > resolution, which simplifies matters. If you don't control DNS there are some > extra steps for changing the quorum voters. > > > > > Given we have three existing nodes as such: > > > > - a.local -> 192.168.0.100 > > - b.local -> 192.168.0.101 > > - c.local -> 192.168.0.102 > > > > As well as a candidate node 192.168.0.103 that we want to replace for the > > role of c.local. > > > > 1. Shut down controller process on node .102 (to make sure we don't "go > > back in time"). > > 2. rsync state from leader to .103. > > 3. Start controller process on .103. > > 4. Point the c.local entry at .103. > > > > I have a few questions about this sequence: > > > > 1. Would this sequence be safe against leadership changes? > > > > If the leader changes, the new leader should have all of the committed > entries that the old leader had. > > > 2. Does it work > > Probably the biggest issue is dealing with "torn writes" that happen because > you're copying the current log segment while it's being written to. The > system should be robust against this. However, we don't regularly do this, so > there hasn't been a lot of testing. > > I think Jose had a PR for improving the handling of this which we might want > to dig up. We'd want the system to auto-truncate the partial record at the > end of the log, if there is one. > > > 3. By "state", do we mean `metadata.log.dir`? Something else? > > Yes, the state of the metadata.log.dir. Keep in mind you will need to change > the node ID in meta.properties after copying, of course. > > > 4. What are the effects on cluster availability? (I think this is the same > > as asking what happens if a or b crashes during the process, or if network > > partitions occur). > > Cluster metadata state tends to be pretty small. typically a hundred > megabytes or so. Therefore, I do not think it will take more than a second or > two to copy from one node to another. However, if you do experience a crash > when one node out of three is down, then you will be unavailable until you > can bring up a second node to regain a majority. > > > > > --- > > > > If this is considered the
Re: Request permissions to contribute to Apache Kafka
Hi there! Thanks for your interest in Apache Kafka. Your accounts are all set, let me know if you have any questions. Best, On Thu, Nov 16, 2023 at 10:46 PM shang xinli wrote: > I would like to contribute to Apache Kafka. Could you please review and > approve the request for permission to contribute to Apache Kafka with the > following Wiki & Jira ID? > > Wiki ID: shangxinli > Jira ID: shangxinli > > > Xinli Shang > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Request required permissions to contribute to Apache Kafka
Hi there! Thanks for your interest in Apache Kafka, your accounts are all set! Best, On Tue, Nov 7, 2023 at 8:22 AM 104664078 <104664...@163.com> wrote: > Hi Teams, > > > I request required permissions to contribute to Apache Kafka. > > > > Jira ID: Zihao Lin (linzihao1999) > Wiki ID: Zihao Lin (linzihao1999) > > > Thanks, > Zihao Lin > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Permission to contribute to Apache Kafka
Hi there! Thanks for your interest in Apache Kafka, your accounts are all set! Best, On Tue, Nov 7, 2023 at 1:20 AM Zhiyuan Lei wrote: > I am requesting permission to contribute to Apache Kafka. > > Wiki ID: Zhiyuan Lei (leizhiyuan) > Jira ID: Zhiyuan Lei (leizhiyuan) > > Thanks! > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Permission to contribute to Apache Kafka
Hi Herman, You are good to go. Thanks for your interest in Apache Kafka! Best, On Sun, Nov 5, 2023 at 1:55 PM Herman K. Jakobsen wrote: > I am requesting permission to contribute to Apache Kafka. > > Wiki ID: Herman Kolstad Jakobsen (hermankj) > Jira ID: Herman Kolstad Jakobsen (hermankj) > > Thanks! > > --- > Herman Kolstad Jakobsen > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: ACCESS to Apache Pony Mail
Hi Arpit, Pony Mail can be seen as the archive of the mailing list. We usually share these links because they are always accessible. That being said, if you want to reply to an email that you can only find on Pony Mail (maybe because the mail was sent before you subscribed or because you deleted the email), there is a button with a pen icon that lets you reply with your preferred mail client. Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Wed, Nov 1, 2023, 16:42 Arpit Goyal wrote: > Thanks Joseph for providing detailed information. > I recently started contributing in the Kafka project and i observe > developers shares pony mail link for discussion around the design.How could > i be part of the thread and able to share the opinion around the design. > > On Wed, Nov 1, 2023, 17:22 Josep Prat wrote: > > > Hi Arpit, > > > > By committer it is meant a person with write access to an ASF project > (for > > example Apache Kafka). Towards the end of this page you can see what > needs > > to be done to become a committer: > > https://kafka.apache.org/contributing.html > > . > > Committership happens on invite basis and it's done by the merits > described > > in the link above. > > > > Best, > > > > ——— > > Josep Prat > > > > Aiven Deutschland GmbH > > > > Alexanderufer 3-7, 10117 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > m: +491715557497 > > > > w: aiven.io > > > > e: josep.p...@aiven.io > > > > On Wed, Nov 1, 2023, 04:02 Arpit Goyal wrote: > > > > > I am already a committer of Apache Kafka. > > > On Wed, Nov 1, 2023, 05:18 Matthias J. Sax wrote: > > > > > > > Only committers can login using their ASF account. > > > > > > > > -Matthias > > > > > > > > On 10/30/23 10:19 PM, Arpit Goyal wrote: > > > > > Hi > > > > > Can anyone help me provide access to Apache Pony Mail. I tried > login > > > > using > > > > > the jira credential but it didn't work. > > > > > Thanks and Regards > > > > > Arpit Goyal > > > > > 8861094754 > > > > > > > > > > > > > > >
Re: ACCESS to Apache Pony Mail
Hi Arpit, By committer it is meant a person with write access to an ASF project (for example Apache Kafka). Towards the end of this page you can see what needs to be done to become a committer: https://kafka.apache.org/contributing.html . Committership happens on invite basis and it's done by the merits described in the link above. Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Wed, Nov 1, 2023, 04:02 Arpit Goyal wrote: > I am already a committer of Apache Kafka. > On Wed, Nov 1, 2023, 05:18 Matthias J. Sax wrote: > > > Only committers can login using their ASF account. > > > > -Matthias > > > > On 10/30/23 10:19 PM, Arpit Goyal wrote: > > > Hi > > > Can anyone help me provide access to Apache Pony Mail. I tried login > > using > > > the jira credential but it didn't work. > > > Thanks and Regards > > > Arpit Goyal > > > 8861094754 > > > > > >
Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana
Congrats Satish! Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Mon, Oct 30, 2023, 11:37 Bruno Cadonna wrote: > Congrats, Satish! > > Bruno > > On 10/29/23 2:42 PM, John Roesler wrote: > > Congratulations, Satish! > > -John > > > > On Sun, Oct 29, 2023, at 08:09, Randall Hauch wrote: > >> Congratulations, Satish! > >> > >> On Sun, Oct 29, 2023 at 1:47 AM Tom Bentley > wrote: > >> > >>> Congratulations! > >>> > >>> On Sun, 29 Oct 2023 at 5:41 PM, Guozhang Wang < > guozhang.wang...@gmail.com> > >>> wrote: > >>> > >>>> Congratulations Satish! > >>>> > >>>> On Sat, Oct 28, 2023 at 12:59 AM Luke Chen wrote: > >>>>> > >>>>> Congrats Satish! > >>>>> > >>>>> Luke > >>>>> > >>>>> On Sat, Oct 28, 2023 at 11:16 AM ziming deng < > dengziming1...@gmail.com > >>>> > >>>>> wrote: > >>>>> > >>>>>> Congratulations Satish! > >>>>>> > >>>>>>> On Oct 27, 2023, at 23:03, Jun Rao > >>> wrote: > >>>>>>> > >>>>>>> Hi, Everyone, > >>>>>>> > >>>>>>> Satish Duggana has been a Kafka committer since 2022. He has been > >>>> very > >>>>>>> instrumental to the community since becoming a committer. It's my > >>>>>> pleasure > >>>>>>> to announce that Satish is now a member of Kafka PMC. > >>>>>>> > >>>>>>> Congratulations Satish! > >>>>>>> > >>>>>>> Jun > >>>>>>> on behalf of Apache Kafka PMC > >>>>>> > >>>>>> > >>>> > >>>> > >>> >
Re: [DISCUSS] Apache Kafka 3.5.2 release
Hi Luke, Thanks for taking this one! Best, On Tue, Oct 17, 2023 at 8:12 AM Luke Chen wrote: > Hi all, > > I'd like to volunteer as release manager for the Apache Kafka 3.5.2, to > have an important bug/vulnerability fix release for 3.5.1. > > If there are no objections, I'll start building a release plan in thewiki > in the next couple of weeks. > > Thanks, > Luke > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Apache Kafka 3.7.0 Release
Thanks Stanislav! ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Mon, Oct 9, 2023, 20:05 Chris Egerton wrote: > +1, thanks Stanislav! > > On Mon, Oct 9, 2023, 14:02 Bill Bejeck wrote: > > > +1 > > > > Thanks, Stanislav! > > > > -Bill > > > > On Mon, Oct 9, 2023 at 1:59 PM Ismael Juma wrote: > > > > > Thanks for volunteering Stanislav! > > > > > > Ismael > > > > > > On Mon, Oct 9, 2023 at 10:51 AM Stanislav Kozlovski > > > wrote: > > > > > > > Hey all! > > > > > > > > I would like to volunteer to be the release manager driving the next > > > > release - Apache Kafka *3.7.0*. > > > > > > > > If there are no objections, I will start and share a release plan > soon > > > > enough! > > > > > > > > Cheers, > > > > Stanislav > > > > > > > > > >
Re: Upgrading from 2.8.0 to 3.5.1
Hi Chester, If I'm not mistaken, what it means is to update the Kafka version on the broker. Broker with version 2.8.0 -> Shutdown -> Replace jars and binaries so it contains Kafka 3.5.1 -> Start it Best, On Wed, Oct 4, 2023 at 9:07 AM Walchester Gaw wrote: > Hello. > > I would like to seek clarification on what it means to "update the code" > in your tutorial <https://kafka.apache.org/documentation/#upgrade_3_5_0>on > how to upgrade. How do I update the code? Which code should I update? > > [image: image.png] > > Thanks, > Chester > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Need generate a KIP
Done. Thanks for showing interest in Apache Kafka! Best, On Mon, Oct 2, 2023 at 6:13 PM Hanyu (Peter) Zheng wrote: > Thank you, Josep. Yes I also need Jira rights. > > On Mon, Oct 2, 2023 at 9:05 AM Josep Prat > wrote: > > > Hi Peter, > > > > You are set :) Please share if you also need Jira rights. > > > > Best, > > > > On Mon, Oct 2, 2023 at 5:59 PM Hanyu (Peter) Zheng > > wrote: > > > > > wiki ID: pzheng > > > Jira ID: hanyuzheng > > > > > > -- > > > > > > [image: Confluent] <https://www.confluent.io> > > > Hanyu (Peter) Zheng he/him/his > > > Software Engineer Intern > > > +1 (213) 431-7193 <+1+(213)+431-7193> > > > Follow us: [image: Blog] > > > < > > > > > > https://www.confluent.io/blog?utm_source=footer_medium=email_campaign=ch.email-signature_type.community_content.blog > > > >[image: > > > Twitter] <https://twitter.com/ConfluentInc>[image: LinkedIn] > > > <https://www.linkedin.com/in/hanyu-peter-zheng/>[image: Slack] > > > <https://slackpass.io/confluentcommunity>[image: YouTube] > > > <https://youtube.com/confluent> > > > > > > [image: Try Confluent Cloud for Free] > > > < > > > > > > https://www.confluent.io/get-started?utm_campaign=tm.fm-apac_cd.inbound_source=gmail_medium=organic > > > > > > > > > > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud > > > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > -- > > [image: Confluent] <https://www.confluent.io> > Hanyu (Peter) Zheng he/him/his > Software Engineer Intern > +1 (213) 431-7193 <+1+(213)+431-7193> > Follow us: [image: Blog] > < > https://www.confluent.io/blog?utm_source=footer_medium=email_campaign=ch.email-signature_type.community_content.blog > >[image: > Twitter] <https://twitter.com/ConfluentInc>[image: LinkedIn] > <https://www.linkedin.com/in/hanyu-peter-zheng/>[image: Slack] > <https://slackpass.io/confluentcommunity>[image: YouTube] > <https://youtube.com/confluent> > > [image: Try Confluent Cloud for Free] > < > https://www.confluent.io/get-started?utm_campaign=tm.fm-apac_cd.inbound_source=gmail_medium=organic > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Need generate a KIP
Hi Peter, You are set :) Please share if you also need Jira rights. Best, On Mon, Oct 2, 2023 at 5:59 PM Hanyu (Peter) Zheng wrote: > wiki ID: pzheng > Jira ID: hanyuzheng > > -- > > [image: Confluent] <https://www.confluent.io> > Hanyu (Peter) Zheng he/him/his > Software Engineer Intern > +1 (213) 431-7193 <+1+(213)+431-7193> > Follow us: [image: Blog] > < > https://www.confluent.io/blog?utm_source=footer_medium=email_campaign=ch.email-signature_type.community_content.blog > >[image: > Twitter] <https://twitter.com/ConfluentInc>[image: LinkedIn] > <https://www.linkedin.com/in/hanyu-peter-zheng/>[image: Slack] > <https://slackpass.io/confluentcommunity>[image: YouTube] > <https://youtube.com/confluent> > > [image: Try Confluent Cloud for Free] > < > https://www.confluent.io/get-started?utm_campaign=tm.fm-apac_cd.inbound_source=gmail_medium=organic > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
[jira] [Reopened] (KAFKA-14956) Flaky test org.apache.kafka.connect.integration.OffsetsApiIntegrationTest#testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted
[ https://issues.apache.org/jira/browse/KAFKA-14956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat reopened KAFKA-14956: It happened again here: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14458/3/testReport/junit/org.apache.kafka.connect.integration/OffsetsApiIntegrationTest/Build___JDK_11_and_Scala_2_13___testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted/ > Flaky test > org.apache.kafka.connect.integration.OffsetsApiIntegrationTest#testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted > -- > > Key: KAFKA-14956 > URL: https://issues.apache.org/jira/browse/KAFKA-14956 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Sagar Rao >Assignee: Yash Mayya >Priority: Major > Labels: flaky-test > Fix For: 3.5.0 > > > ``` > h4. Error > org.opentest4j.AssertionFailedError: Condition not met within timeout 15000. > Sink connector consumer group offsets should catch up to the topic end > offsets ==> expected: but was: > h4. Stacktrace > org.opentest4j.AssertionFailedError: Condition not met within timeout 15000. > Sink connector consumer group offsets should catch up to the topic end > offsets ==> expected: but was: > at > app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) > at > app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) > at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63) > at app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36) > at app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:211) > at > app//org.apache.kafka.test.TestUtils.lambda$waitForCondition$4(TestUtils.java:337) > at > app//org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:385) > at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:334) > at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:318) > at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:291) > at > app//org.apache.kafka.connect.integration.OffsetsApiIntegrationTest.getAndVerifySinkConnectorOffsets(OffsetsApiIntegrationTest.java:150) > at > app//org.apache.kafka.connect.integration.OffsetsApiIntegrationTest.testGetSinkConnectorOffsetsDifferentKafkaClusterTargeted(OffsetsApiIntegrationTest.java:131) > at > java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) > at > app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > app//org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > app//org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > app//org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at app//org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at > app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExe
[jira] [Created] (KAFKA-15524) Flaky test org.apache.kafka.connect.integration.OffsetsApiIntegrationTest.testResetSinkConnectorOffsetsZombieSinkTasks
Josep Prat created KAFKA-15524:
--
Summary: Flaky test
org.apache.kafka.connect.integration.OffsetsApiIntegrationTest.testResetSinkConnectorOffsetsZombieSinkTasks
Key: KAFKA-15524
URL: https://issues.apache.org/jira/browse/KAFKA-15524
Project: Kafka
Issue Type: Bug
Components: connect
Affects Versions: 3.5.1, 3.6.0
Reporter: Josep Prat
Last seen:
[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14458/3/testReport/junit/org.apache.kafka.connect.integration/OffsetsApiIntegrationTest/Build___JDK_17_and_Scala_2_13___testResetSinkConnectorOffsetsZombieSinkTasks/]
h3. Error Message
{code:java}
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: The request timed out.{code}
h3. Stacktrace
{code:java}
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: The request timed out. at
org.apache.kafka.connect.util.clusters.EmbeddedKafkaCluster.createTopic(EmbeddedKafkaCluster.java:427)
at
org.apache.kafka.connect.util.clusters.EmbeddedKafkaCluster.createTopic(EmbeddedKafkaCluster.java:401)
at
org.apache.kafka.connect.util.clusters.EmbeddedKafkaCluster.createTopic(EmbeddedKafkaCluster.java:392)
at
org.apache.kafka.connect.integration.OffsetsApiIntegrationTest.testResetSinkConnectorOffsetsZombieSinkTasks(OffsetsApiIntegrationTest.java:763)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at
org.junit.runners.ParentRunner.run(ParentRunner.java:413) at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
at
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:60)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at jdk.proxy1/jdk.proxy1.$Proxy2.processTestClass(Unknown Source) at
org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
at
org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
at
org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100
[jira] [Created] (KAFKA-15522) Flaky test org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest.testOneWayReplicationWithFrequentOffsetSyncs
Josep Prat created KAFKA-15522:
--
Summary: Flaky test
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest.testOneWayReplicationWithFrequentOffsetSyncs
Key: KAFKA-15522
URL: https://issues.apache.org/jira/browse/KAFKA-15522
Project: Kafka
Issue Type: Bug
Affects Versions: 3.5.1, 3.6.0
Reporter: Josep Prat
h3. Last seen:
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14458/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationExactlyOnceTest/Build___JDK_17_and_Scala_2_13___testOneWayReplicationWithFrequentOffsetSyncs__/
h3. Error Message
{code:java}
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: The request timed out.{code}
h3. Stacktrace
{code:java}
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: The request timed out. at
org.apache.kafka.connect.util.clusters.EmbeddedKafkaCluster.createTopic(EmbeddedKafkaCluster.java:427)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.createTopics(MirrorConnectorsIntegrationBaseTest.java:1276)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.startClusters(MirrorConnectorsIntegrationBaseTest.java:235)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.startClusters(MirrorConnectorsIntegrationBaseTest.java:149)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest.startClusters(MirrorConnectorsIntegrationExactlyOnceTest.java:51)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568) at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:128)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeEachMethod(TimeoutExtension.java:78)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:521)
at
org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$23(ClassBasedTestDescriptor.java:506)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$3(TestMethodTestDescriptor.java:175)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$6(TestMethodTestDescriptor.java:203)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:203)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:172)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda
[jira] [Created] (KAFKA-15523) Flaky test org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testSyncTopicConfigs
Josep Prat created KAFKA-15523:
--
Summary: Flaky test
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testSyncTopicConfigs
Key: KAFKA-15523
URL: https://issues.apache.org/jira/browse/KAFKA-15523
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.5.1, 3.6.0
Reporter: Josep Prat
Last seen:
[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14458/3/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationSSLTest/Build___JDK_17_and_Scala_2_13___testSyncTopicConfigs__/]
h3. Error Message
{code:java}
org.opentest4j.AssertionFailedError: Condition not met within timeout 3.
Topic: mm2-status.backup.internal didn't get created in the cluster ==>
expected: but was: {code}
h3. Stacktrace
{code:java}
org.opentest4j.AssertionFailedError: Condition not met within timeout 3.
Topic: mm2-status.backup.internal didn't get created in the cluster ==>
expected: but was: at
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at app//org.junit.jupiter.api.AssertTrue.failNotTrue(AssertTrue.java:63) at
app//org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:36) at
app//org.junit.jupiter.api.Assertions.assertTrue(Assertions.java:210) at
app//org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:331)
at
app//org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:379)
at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:328)
at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:312) at
app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:302) at
app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.waitForTopicCreated(MirrorConnectorsIntegrationBaseTest.java:1041)
at
app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.startClusters(MirrorConnectorsIntegrationBaseTest.java:224)
at
app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.startClusters(MirrorConnectorsIntegrationBaseTest.java:149)
at
app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.startClusters(MirrorConnectorsIntegrationSSLTest.java:63)
at
java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) at
app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at
app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:128)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeEachMethod(TimeoutExtension.java:78)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
app//org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:521)
at
app//org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$23(ClassBasedTestDescriptor.java:506)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.
[jira] [Reopened] (KAFKA-13531) Flaky test NamedTopologyIntegrationTest
[
https://issues.apache.org/jira/browse/KAFKA-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josep Prat reopened KAFKA-13531:
> Flaky test NamedTopologyIntegrationTest
> ---
>
> Key: KAFKA-13531
> URL: https://issues.apache.org/jira/browse/KAFKA-13531
> Project: Kafka
> Issue Type: Test
> Components: streams, unit tests
>Reporter: Matthias J. Sax
>Assignee: Matthew de Detrich
>Priority: Critical
> Labels: flaky-test
> Attachments:
> org.apache.kafka.streams.integration.NamedTopologyIntegrationTest.shouldRemoveOneNamedTopologyWhileAnotherContinuesProcessing().test.stdout
>
>
> org.apache.kafka.streams.integration.NamedTopologyIntegrationTest.shouldRemoveNamedTopologyToRunningApplicationWithMultipleNodesAndResetsOffsets
> {quote}java.lang.AssertionError: Did not receive all 3 records from topic
> output-stream-2 within 6 ms, currently accumulated data is [] Expected:
> is a value equal to or greater than <3> but: <0> was less than <3> at
> org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.lambda$waitUntilMinKeyValueRecordsReceived$1(IntegrationTestUtils.java:648)
> at
> org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:368)
> at
> org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:336)
> at
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:644)
> at
> org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilMinKeyValueRecordsReceived(IntegrationTestUtils.java:617)
> at
> org.apache.kafka.streams.integration.NamedTopologyIntegrationTest.shouldRemoveNamedTopologyToRunningApplicationWithMultipleNodesAndResetsOffsets(NamedTopologyIntegrationTest.java:439){quote}
> STDERR
> {quote}java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.GroupSubscribedToTopicException: Deleting
> offsets of a topic is forbidden while the consumer group is actively
> subscribed to it. at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) at
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
> at
> org.apache.kafka.streams.processor.internals.namedtopology.KafkaStreamsNamedTopologyWrapper.lambda$removeNamedTopology$3(KafkaStreamsNamedTopologyWrapper.java:213)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.lambda$whenComplete$2(KafkaFutureImpl.java:107)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at
> org.apache.kafka.common.internals.KafkaCompletableFuture.kafkaComplete(KafkaCompletableFuture.java:39)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.complete(KafkaFutureImpl.java:122)
> at
> org.apache.kafka.streams.processor.internals.TopologyMetadata.maybeNotifyTopologyVersionWaiters(TopologyMetadata.java:154)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.checkForTopologyUpdates(StreamThread.java:916)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:598)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:575)
> Caused by: org.apache.kafka.common.errors.GroupSubscribedToTopicException:
> Deleting offsets of a topic is forbidden while the consumer group is actively
> subscribed to it. java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.GroupSubscribedToTopicException: Deleting
> offsets of a topic is forbidden while the consumer group is actively
> subscribed to it. at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) at
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
> at
> org.apache.kafka.streams.processor.internals.namedtopology.KafkaStreamsNamedTopologyWrapper.lambda$removeNamedTopology$3(KafkaStreamsNamedTopologyWrapper.java:213)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.lambda$whenComplete$2(KafkaFutureImpl.java:107)
>
[jira] [Reopened] (KAFKA-13966) Flaky test `QuorumControllerTest.testUnregisterBroker`
[
https://issues.apache.org/jira/browse/KAFKA-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josep Prat reopened KAFKA-13966:
> Flaky test `QuorumControllerTest.testUnregisterBroker`
> --
>
> Key: KAFKA-13966
> URL: https://issues.apache.org/jira/browse/KAFKA-13966
> Project: Kafka
> Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: David Arthur
>Priority: Major
>
> We have seen the following assertion failure in
> `QuorumControllerTest.testUnregisterBroker`:
> {code:java}
> org.opentest4j.AssertionFailedError: expected: <2> but was: <0>
> at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
> at
> org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)
> at
> org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:161)
> at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:628)
> at
> org.apache.kafka.controller.QuorumControllerTest.testUnregisterBroker(QuorumControllerTest.java:494)
> {code}
> I reproduced it by running the test in a loop. It looks like what happens is
> that the BrokerRegistration request is able to get interleaved between the
> leader change event and the write of the bootstrap metadata. Something like
> this:
> # handleLeaderChange() start
> # appendWriteEvent(registerBroker)
> # appendWriteEvent(bootstrapMetadata)
> # handleLeaderChange() finish
> # registerBroker() -> writes broker registration to log
> # bootstrapMetadata() -> writes bootstrap metadata to log
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [ANNOUNCE] New committer: Yash Mayya
Congrats Yash! ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Thu, Sep 21, 2023, 18:55 Raymond Ng wrote: > Congrats Yash! Well-deserved! > > /Ray > > On Thu, Sep 21, 2023 at 9:40 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Congratulations Yash! > > > > On Thu, Sep 21, 2023, 22:03 Bill Bejeck wrote: > > > > > Congrats Yash! > > > > > > On Thu, Sep 21, 2023 at 12:26 PM Divij Vaidya > > > > wrote: > > > > > > > Congratulations Yash! > > > > > > > > Divij Vaidya > > > > > > > > > > > > On Thu, Sep 21, 2023 at 6:18 PM Sagar > > wrote: > > > > > > > > > > Congrats Yash ! > > > > > On Thu, 21 Sep 2023 at 9:38 PM, Ashwin > > > > > > > wrote: > > > > > > > > > > > Awesome ! Congratulations Yash !! > > > > > > > > > > > > On Thu, Sep 21, 2023 at 9:25 PM Edoardo Comar < > > edoardli...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Congratulations Yash > > > > > > > > > > > > > > On Thu, 21 Sept 2023 at 16:28, Bruno Cadonna < > cado...@apache.org > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > The PMC of Apache Kafka is pleased to announce a new Kafka > > > > committer > > > > > > > > Yash Mayya. > > > > > > > > > > > > > > > > Yash's major contributions are around Connect. > > > > > > > > > > > > > > > > Yash authored the following KIPs: > > > > > > > > > > > > > > > > KIP-793: Allow sink connectors to be used with topic-mutating > > > SMTs > > > > > > > > KIP-882: Kafka Connect REST API configuration validation > > timeout > > > > > > > > improvements > > > > > > > > KIP-970: Deprecate and remove Connect's redundant task > > > > configurations > > > > > > > > endpoint > > > > > > > > KIP-980: Allow creating connectors in a stopped state > > > > > > > > > > > > > > > > Overall, Yash is known for insightful and friendly input to > > > > discussions > > > > > > > > and his high quality contributions. > > > > > > > > > > > > > > > > Congratulations, Yash! > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Bruno (on behalf of the Apache Kafka PMC) > > > > > > > > > > > > > > > > > > > > > > >
Re: [kafka-clients] [VOTE] 3.6.0 RC1
org/36/documentation.html > > > > > > > > * Protocol: > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > There are a few runs of unit/integration tests. You can see the > latest > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will > > continue running a few more iterations. > > > > System tests: > > > > We will send an update once we have the results. > > > > > > > > Thanks, > > > > Satish. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups "kafka-clients" group. > > > > To unsubscribe from this group and stop receiving emails from it, > send > > an email to kafka-clients+unsubscr...@googlegroups.com. > > > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/kafka-clients/CAM-aUZ%3DuJ-SKeVFtBZwBjhLHKw4CbxF_ws%2BvQqaymGHFsC%2Bmdg%40mail.gmail.com > > . > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.6.0 RC0
Hi Satish, Thanks for running the release. I ran the following validation steps: - Built from source with Java 11 and Scala 2.13 - Verified Signatures and hashes of the artifacts generated - Navigated through Javadoc including links to JDK classes - Run the unit tests - Run integration tests - Run the quickstart in KRaft and Zookeeper mode - Checked License-binary against libs and matched them I +1 this release (non-binding) Best, On Mon, Sep 18, 2023 at 6:02 PM David Arthur wrote: > Hey Satish, thanks for getting the RC underway! > > I noticed that the PR for the 3.6 blog post is merged. This makes the blog > post live on the Kafka website https://kafka.apache.org/blog.html. The > blog > post (along with other public announcements) is usually the last thing we > do as part of the release. I think we should probably take this down until > we're done with the release, otherwise users stumbling on this post could > get confused. It also contains some broken links. > > Thanks! > David > > On Sun, Sep 17, 2023 at 1:31 PM Satish Duggana > wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the first candidate for the release of Apache Kafka 3.6.0. Some > of > > the major features include: > > > > * KIP-405 : Kafka Tiered Storage > > * KIP-868 : KRaft Metadata Transactions > > * KIP-875: First-class offsets support in Kafka Connect > > * KIP-898: Modernize Connect plugin discovery > > * KIP-938: Add more metrics for measuring KRaft performance > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > * KIP-917: Additional custom metadata for remote log segment > > > > Release notes for the 3.6.0 release: > > https://home.apache.org/~satishd/kafka-3.6.0-rc0/RELEASE_NOTES.html > > > > *** Please download, test and vote by Wednesday, September 21, 12pm PT > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > https://kafka.apache.org/KEYS > > > > * Release artifacts to be voted upon (source and binary): > > https://home.apache.org/~satishd/kafka-3.6.0-rc0/ > > > > * Maven artifacts to be voted upon: > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > * Javadoc: > > https://home.apache.org/~satishd/kafka-3.6.0-rc0/javadoc/ > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag: > > https://github.com/apache/kafka/releases/tag/3.6.0-rc0 > > > > * Documentation: > > https://kafka.apache.org/36/documentation.html > > > > * Protocol: > > https://kafka.apache.org/36/protocol.html > > > > * Successful Jenkins builds for the 3.6 branch: > > There are a few runs of unit/integration tests. You can see the latest at > > https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will > > continue > > running a few more iterations. > > System tests: > > We will send an update once we have the results. > > > > Thanks, > > Satish. > > > > > -- > David Arthur > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Requesting permission to contribute
Hi Nikhil, My bad, I might have forgotten to save the right permissions. Now you are good to go. Best, On Mon, Sep 11, 2023 at 4:16 PM Nikhil Ramakrishnan < ramakrishnan.nik...@gmail.com> wrote: > Hello Josep, > > Thanks for the quick response! However, I am still unable to edit/create > documents in the Kafka space on the Confluence wiki (my username is > nikrmk). Do I need to do anything else to enable this? > > Thanks, > Nikhil > > > On Thu, Sep 7, 2023 at 4:23 PM Josep Prat > wrote: > > > Hi Nikhil, > > > > Thanks for your interest in contributing to Apache Kafka. Your accounts > are > > all set now. > > > > Let me know if you have any questions. > > > > Best, > > > > On Thu, Sep 7, 2023 at 4:59 PM Nikhil Ramakrishnan < > > ramakrishnan.nik...@gmail.com> wrote: > > > > > Hi, > > > I'm requesting permission to contribute to the Apache Kafka project, as > > > described on the WIki: > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > > My Wiki ID: nikrmk > > > My JIRA ID: nikramakrishnan > > > > > > Thanks in advance, > > > Nikhil > > > > > > > > > -- > > [image: Aiven] <https://www.aiven.io> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io> | < > https://www.facebook.com/aivencloud > > > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Requesting permission to contribute
Hi Nikhil, Thanks for your interest in contributing to Apache Kafka. Your accounts are all set now. Let me know if you have any questions. Best, On Thu, Sep 7, 2023 at 4:59 PM Nikhil Ramakrishnan < ramakrishnan.nik...@gmail.com> wrote: > Hi, > I'm requesting permission to contribute to the Apache Kafka project, as > described on the WIki: > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > My Wiki ID: nikrmk > My JIRA ID: nikramakrishnan > > Thanks in advance, > Nikhil > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Request: Permissions to contribute to Apache Kafka
Hi Krishna, Thank you for your interest in contributing to Apache Kafka. Your accounts are all set. Let me know if you have any questions. Best, On Tue, Aug 29, 2023 at 4:06 PM Krishna Agarwal < krishna0608agar...@gmail.com> wrote: > Hi, > I would like to propose a KIP and contribute to Apache Kafka. > Can I please be provided with the necessary permissions. > > Wiki ID: krishna06 > Jira ID: krishna06 > > Regards, > Krishna > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Need Access to create KIP & Jira Tickets
Hi Raghu, Thanks for your interest in Apache Kafka. As Justine rightly points out, for the change of the Jira ID you should file a Jira issue under the INFRA project [1]. If you end up creating another account, I would also recommend creating a Jira issue under INFRA to merge the 2 accounts. Best, [1]: https://issues.apache.org/jira/projects/INFRA/issues On Mon, Aug 28, 2023 at 10:29 PM Justine Olshan wrote: > Hey Raghu, > > I've added your ID to give you permissions to the wiki. > > I'm not sure if committers can change your jira ID. You may want to try to > create a new account or file a ticket with apache for that. > > Let me know if there are any issues. > > Justine > > On Mon, Aug 28, 2023 at 11:54 AM Raghu Baddam wrote: > > > Hi Team, > > > > Please find wiki ID and Jira ID and help me by providing me access to > > create KIP's and Jira Tickets on Apache Kafka space. > > > > wiki ID: rbaddam > > Jira ID: raghu98...@gmail.com > > > > Also If possible I also need help with changing my Jira ID same as wiki > ID > > i.e. *rbaddam* > > > > Thanks, > > Raghu > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: new KIP doubt
Hi Sergio, Thanks for contributing to Apache Kafka! You don't need any credentials to start a discussion thread. The discussion thread is started by sending an email to this very same mailing list with a subject like [DISUSS] KIP-964: Have visibility when produce requests become "async". And then in the body state that you'd like to start a discuss thread for this KIP and link to the KIP page. That should be it! Here you can find one of the latest DISCUSS threads created: https://lists.apache.org/thread/0f20kvfqkmhdqrwcb8vqgqn80szcrcdd Let me know if you have any questions! Best, On Mon, Aug 7, 2023 at 2:20 PM Sergio Daniel Troiano wrote: > hey everyone, > > Sorry for bothering you, I created a KIP long time ago and now I am > creating a new one here > < > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263426647 > > > The problem is I lost my apache credentials for starting a discussion > thread, could you please tell me how I can recover them? > > Thanks in advance. > Sergio Troiano > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Subscribe to mailing list
Hi Mital, Thanks for your interest in helping out with Apache Kafka. You can find the instructions on how to subscribe to the mailing list here: https://kafka.apache.org/contact.html |> *Developer mailing list: A list for discussion on Kafka® development. To subscribe, send an email to dev-subscr...@kafka.apache.org . Once subscribed, you can have discussion on Kafka® development by mailing to dev@kafka.apache.org . Archives are available here <https://lists.apache.org/list.html?dev@kafka.apache.org>. To unsubscribe, send an email to dev-unsubscr...@kafka.apache.org .* Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Sun, Jul 16, 2023, 15:54 Mital Awachat wrote: > Hi, > > I would like to subscribe to the mailing list. > > -- > Regards > Mital Awachat >
Re: Requesting permissions to contribute to Apache Kafka
Hi Jack, Thanks for your interest in contributing to Apache Kafka! Your accounts are all set now. Let me know if you have any questions. Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Sat, Jul 15, 2023, 11:41 Jack Tomy wrote: > Hi Team, > > Please find my wiki ID and Jira ID below. I would like to get access > to contribute > to Apache Kafka > > wiki ID : hijker > JIRA ID : hijker > > TIA > > -- > Best Regards > *Jack* >
Re: [ANNOUNCE] New committer: Greg Harris
Congrats Greg! ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Mon, Jul 10, 2023, 17:47 Matthias J. Sax wrote: > Congrats! > > On 7/10/23 8:45 AM, Chris Egerton wrote: > > Hi all, > > > > The PMC for Apache Kafka has invited Greg Harris to become a committer, > and > > we are happy to announce that he has accepted! > > > > Greg has been contributing to Kafka since 2019. He has made over 50 > commits > > mostly around Kafka Connect and Mirror Maker 2. His most notable > > contributions include KIP-898: "Modernize Connect plugin discovery" and a > > deep overhaul of the offset syncing logic in MM2 that addressed several > > technically-difficult, long-standing, high-impact issues. > > > > He has also been an active participant in discussions and reviews on the > > mailing lists and on GitHub. > > > > Thanks for all of your contributions, Greg. Congratulations! > > >
Re: Apply to contribute to the Kafka project
Hi there! Thanks for your interest in contributing to Apache Kafka, you should be all set now. Let me know if there is any problem. Best, On Wed, Jun 28, 2023 at 9:47 AM isding_l wrote: > > Hi， > > > I'd like to request permissions to contribute to Kafka to propose a KIP. > > > Wiki ID: isDing_L > Kira ID: isDing_L > > > Thank you > > > -- > 发自我的网易邮箱手机智能版 -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: How to Integrate MySQL database with Kafka? Need Demo
Hi Avani, Thanks for taking interest in Apache Kafka. Regarding the example code involving a MySQL database as a sink you can take a look at this blog post: https://aiven.io/developer/db-technology-migration-with-apache-kafka-and-kafka-connect The example in there contains more moving pieces, but I believe it contains your use case. Disclaimer, I'm employed by Aiven. Regarding the "Book a demo with Kafka" you won't find this on the kafka.apache.org page, because this is the project's website. For commercial offerings you should head over to the websites of the copmanies that offer Apache Kafka as a service. aiven.io, confluent.io and instaclustr.com should have an option to request a demo. Best, On Mon, Jun 26, 2023 at 8:12 AM Avani Panchal wrote: > Hi, > In my application I want to sync my client's data to my SQL server. at > client place the database is MYSQL. > > How can I achieve this using Kafka? I read a lot of documents but I don't > understand which setup I need and how I can achieve it. > > I was also wondering about "Book a demo with Kafka" but didn't find it. > > Please help me. > > Thank you, > Avani > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] KIP-917: Additional custom metadata for remote log segment
Hi Ivan, Thanks and congrats for your first accepted KIP. Regarding the process, yes you are right. You should create an issue (this can be done pre-KIP as well), it would be good if you could link this vote thread in the JIRA. Then you need to update your KIP's wiki page with the status (approved now), in your case adding the Jira ticket link as well in the right section. Then you should go to the general KIP overview page and move your KIP from "under discussion" to "approved" and 3.6.0 as the version. Best, On Mon, Jun 19, 2023 at 12:20 PM Ivan Yurchenko wrote: > Hi all, > > Thank you for your votes. > > The voting has been open for 7 days and we have: > - 4 binding +1: Divij Vaidya, Luke Chen, Josep Prat, and Satish Duggana > (based on today's state of https://kafka.apache.org/committers). > - 2 non-binding +1: Kamal Chandraprakash and Matthew Benedict de Detrich. > - No -1. > > AFAIU, we can declare the KIP as accepted. I will create a ticket and also > volunteer to implement it. > > Please correct me if I'm wrong, that's the first time I'm bringing a KIP to > this stage :) > > Thanks! > > Best, > Ivan > > > On Wed, 14 Jun 2023 at 18:28, Satish Duggana > wrote: > > > Thanks Ivan for addressing the comments in the KIP, LGTM. > > > > +1 > > > > On Tue, 13 Jun 2023 at 18:31, Luke Chen wrote: > > > > > > Looks good. Thanks for the update. > > > > > > On Tue, Jun 13, 2023 at 8:08 PM Ivan Yurchenko < > ivan0yurche...@gmail.com > > > > > > wrote: > > > > > > > Hi all! > > > > > > > > Thank you for your votes. > > > > > > > > Based on the proposal of Satish in the discussion thread, I modified > > the > > > > KIP a little bit by lowering the default value of > > > > `remote.log.metadata.custom.metadata.max.size` from 10 KiB to 128 > > bytes. I > > > > hope this doesn't change your vote, but feel free to raise your > > concerns. > > > > Thank you! > > > > > > > > Best, > > > > Ivan > > > > > > > > > > > > On Tue, 13 Jun 2023 at 11:09, Josep Prat > > > > > wrote: > > > > > > > > > Hi Ivan, > > > > > > > > > > Thank you very much for this KIP. +1 (binding) from me. > > > > > > > > > > Best, > > > > > > > > > > > > > > > On Tue, Jun 13, 2023 at 10:03 AM Luke Chen > > wrote: > > > > > > > > > > > +1 (binding) from me. > > > > > > > > > > > > Thanks. > > > > > > Luke > > > > > > > > > > > > On Tue, Jun 13, 2023 at 3:44 PM Matthew Benedict de Detrich > > > > > > wrote: > > > > > > > > > > > > > > +1 (non binding). Thanks for KIP > > > > > > > > > > > > > > On Tue, Jun 13, 2023 at 3:38 AM Kamal Chandraprakash < > > > > > > > kamal.chandraprak...@gmail.com> wrote: > > > > > > > > > > > > > > > +1 (non-binding). Thanks for the KIP! > > > > > > > > > > > > > > > > On Mon, Jun 12, 2023, 21:23 Divij Vaidya < > > divijvaidy...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > I have reviewed the proposal and feel that it would be > > beneficial > > > > > to > > > > > > > > > implement. > > > > > > > > > > > > > > > > > > Vote +1 (non-binding) > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 12, 2023 at 4:39 PM Ivan Yurchenko < > > > > > > ivan0yurche...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > Some interest in KIP-917 was expressed in the discussion > > thread > > > > > > [1]. > > > > >
[jira] [Created] (KAFKA-15105) Flaky test FetchFromFollowerIntegrationTest.testFetchFromLeaderWhilePreferredReadReplicaIsUnavailable
Josep Prat created KAFKA-15105:
--
Summary: Flaky test
FetchFromFollowerIntegrationTest.testFetchFromLeaderWhilePreferredReadReplicaIsUnavailable
Key: KAFKA-15105
URL: https://issues.apache.org/jira/browse/KAFKA-15105
Project: Kafka
Issue Type: Bug
Reporter: Josep Prat
Test
integration.kafka.server.FetchFromFollowerIntegrationTest.testFetchFromLeaderWhilePreferredReadReplicaIsUnavailable()
became flaky. An example can be found here:
[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13865/2/testReport/junit/integration.kafka.server/FetchFromFollowerIntegrationTest/Build___JDK_11_and_Scala_2_13___testFetchFromLeaderWhilePreferredReadReplicaIsUnavailable__/]
The error might be caused because of a previous kafka cluster used for another
test wasn't cleaned up properly before this one run.
h3. Error Message
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists.{code}
h3. Stacktrace
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-15104) Flaky test MetadataQuorumCommandTest for method testDescribeQuorumReplicationSuccessful
Josep Prat created KAFKA-15104:
--
Summary: Flaky test MetadataQuorumCommandTest for method
testDescribeQuorumReplicationSuccessful
Key: KAFKA-15104
URL: https://issues.apache.org/jira/browse/KAFKA-15104
Project: Kafka
Issue Type: Bug
Components: tools
Affects Versions: 3.5.0
Reporter: Josep Prat
The MetadataQuorumCommandTest has become flaky on CI, I saw this failing:
org.apache.kafka.tools.MetadataQuorumCommandTest.[1] Type=Raft-Combined,
Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.6-IV0,
Security=PLAINTEXT
Link to the CI:
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13865/2/testReport/junit/org.apache.kafka.tools/MetadataQuorumCommandTest/Build___JDK_8_and_Scala_2_121__Type_Raft_Combined__Name_testDescribeQuorumReplicationSuccessful__MetadataVersion_3_6_IV0__Security_PLAINTEXT/
h3. Error Message
{code:java}
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Received a
fatal error while waiting for the controller to acknowledge that we are caught
up{code}
h3. Stacktrace
{code:java}
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Received a
fatal error while waiting for the controller to acknowledge that we are caught
up at java.util.concurrent.FutureTask.report(FutureTask.java:122) at
java.util.concurrent.FutureTask.get(FutureTask.java:192) at
kafka.testkit.KafkaClusterTestKit.startup(KafkaClusterTestKit.java:419) at
kafka.test.junit.RaftClusterInvocationContext.lambda$getAdditionalExtensions$5(RaftClusterInvocationContext.java:115)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeTestExecutionCallbacks$5(TestMethodTestDescriptor.java:191)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$6(TestMethodTestDescriptor.java:202)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:202)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeTestExecutionCallbacks(TestMethodTestDescriptor.java:190)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:136){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-15103) Flaky test KRaftClusterTest.testCreateClusterAndPerformReassignment
Josep Prat created KAFKA-15103:
--
Summary: Flaky test
KRaftClusterTest.testCreateClusterAndPerformReassignment
Key: KAFKA-15103
URL: https://issues.apache.org/jira/browse/KAFKA-15103
Project: Kafka
Issue Type: Bug
Components: core
Reporter: Josep Prat
Fix For: 3.5.0
{{The test
kafka.server.KRaftClusterTest.testCreateClusterAndPerformReassignment() is
failing:
[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13865/2/testReport/junit/kafka.server/KRaftClusterTest/Build___JDK_8_and_Scala_2_12___testCreateClusterAndPerformReassignment__/]}}
h3. Error Message
{code:java}
org.opentest4j.AssertionFailedError: Timed out waiting for replica assignments
for topic foo. Wanted: List(List(2, 1, 0), List(0, 1, 2), List(2, 3), List(3,
2, 0, 1)). Got: ArrayBuffer(ArrayBuffer(2, 1, 0), ArrayBuffer(0, 1, 2, 3),
ArrayBuffer(2, 3), ArrayBuffer(3, 2, 0, 1)){code}
h3. Stacktrace
{code:java}
org.opentest4j.AssertionFailedError: Timed out waiting for replica assignments
for topic foo. Wanted: List(List(2, 1, 0), List(0, 1, 2), List(2, 3), List(3,
2, 0, 1)). Got: ArrayBuffer(ArrayBuffer(2, 1, 0), ArrayBuffer(0, 1, 2, 3),
ArrayBuffer(2, 3), ArrayBuffer(3, 2, 0, 1)) at
org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at
org.junit.jupiter.api.Assertions.fail(Assertions.java:135) at
kafka.server.KRaftClusterTest.testCreateClusterAndPerformReassignment(KRaftClusterTest.scala:479)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Permissions to contribute to Apache Kafka.
Hi Vaibhav, I just checked and you have all the right permissions, someone was faster than me setting it up for you. Best, On Fri, Jun 16, 2023 at 8:10 AM Vaibhav Nazare wrote: > HI Team, > I work with power(ppc64le) team and need permissions to contribute to > Apache Kafka to enable CI for ppc64le arch. > > Requesting you to grant access to contribute to Apache Kafka. > > Following are the details for > Wiki id > Email: vaibhav.naz...@ibm.com<mailto:vaibhav.naz...@ibm.com> > Username: vnazare > > Jira id > Email: vaibhav.naz...@ibm.com<mailto:vaibhav.naz...@ibm.com> > Username: vnazare > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] Apache Kafka 3.5.0
Thank you Mickael for running the release! Best ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Thu, Jun 15, 2023, 12:34 Manyanda Chitimbo wrote: > Thank you for running the release! > > On Thu 15 Jun 2023 at 10:28, Mickael Maison wrote: > > > The Apache Kafka community is pleased to announce the release for Apache > > Kafka 3.5.0. > > > > This is a minor release and it includes fixes and improvements from 201 > > JIRAs. > > > > All of the changes in this release can be found in the release notes: > > https://downloads.apache.org/kafka/3.5.0/RELEASE_NOTES.html > > > > An overview of the release can be found in our announcement blog post: > > https://kafka.apache.org/blog > > > > You can download the source and binary release (Scala 2.12 and Scala > 2.13) > > from: > > https://kafka.apache.org/downloads#3.5.0 > > > > > > > --- > > > > Apache Kafka is a distributed streaming platform with four core APIs: > > ** The Producer API allows an application to publish a stream records to > > one or more Kafka topics. > > ** The Consumer API allows an application to subscribe to one or more > > topics and process the stream of records produced to them. > > ** The Streams API allows an application to act as a stream processor, > > consuming an input stream from one or more topics and producing an output > > stream to one or more output topics, effectively transforming the input > > streams to output streams. > > ** The Connector API allows building and running reusable producers or > > consumers that connect Kafka topics to existing applications or data > > systems. For example, a connector to a relational database might capture > > every change to a table. > > > > With these APIs, Kafka can be used for two broad classes of application: > > ** Building real-time streaming data pipelines that reliably get data > > between systems or applications. > > ** Building real-time streaming applications that transform or react to > the > > streams of data. > > > > Apache Kafka is in use at large and small companies worldwide, including > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, > > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > > > A big thank you for the following 103 contributors to this release! > > A. Sophie Blee-Goldman, Akhilesh Chaganti, Alex Sorokoumov, Alexandre > > Dupriez, Alyssa Huang, Anastasia Vela, Andreas Maechler, andymg3, Artem > > Livshits, atu-sharm, bachmanity1, Bill Bejeck, Brendan Ribera, Calvin > Liu, > > Chaitanya Mukka, Cheryl Simmons, Chia-Ping Tsai, Chris Egerton, Christo > > Lolov, Colin P. McCabe, csolidum, Daniel Scanteianu, David Arthur, David > > Jacot, David Karlsson, David Mao, Dejan Stojadinović, Divij Vaidya, > dorwi, > > drgnchan, Dániel Urbán, Edoardo Comar, egyedt, emilnkrastev, Eric Haag, > > Farooq Qaiser, Federico Valeri, Gantigmaa Selenge, Greg Harris, Guozhang > > Wang, Hao Li, Hector Geraldino, Himani Arora, Hoki Min, hudeqi, iamazy, > > Iblis Lin, Ismael Juma, Ivan Yurchenko, Jakub Scholz, Jason Gustafson, > Jeff > > Kim, Jim Galasyn, Jorge Esteban Quilcate Otoya, Josep Prat, José Armando > > García Sancio, Juan José Ramos, Junyang Liu, Justine Olshan, Kamal > > Chandraprakash, Kirk True, Kowshik Prakasam, littlehorse-eng, liuzc9, > Lucas > > Brutschy, Lucia Cerchie, Luke Chen, Manikumar Reddy, Manyanda Chitimbo, > > Matthew Wong, Matthias J. Sax, Matthias Seiler, Michael Marshall, Mickael > > Maison, nicolasguyomar, Nikolay, Paolo Patierno, Philip Nee, Pierangelo > Di > > Pilato, Proven Provenzano, Purshotam Chauhan, Qing, Rajini Sivaram, > > RivenSun, Robert Young, Rohan, Roman Schmitz, Ron Dagostino, Ruslan > > Krivoshein, Satish Duggana, Shay Elkin, Shekhar Rajak, Simon Woodman, > > Spacrocket, stejani-cflt, Terry, Tom Bentley, vamossagar12, Victoria Xia, > > Viktor Somogyi-Vass, Vladimir Korenev, Yash Mayya, Zheng-Xian Li > > > > We welcome your help and feedback. For more information on how to report > > problems, and to get involved, visit the project website at > > https://kafka.apache.org/ > > > > Thank you! > > > > Regards, > > Mickael Maison > > > -- > Manyanda Chitimbo. >
Re: [ANNOUNCE] New committer: Divij Vaidya
Congrats Divij! On Wed, Jun 14, 2023 at 11:45 AM Lucas Brutschy wrote: > Congrats!! > > On Wed, Jun 14, 2023 at 11:02 AM Federico Valeri > wrote: > > > > Congrats Divij! > > > > On Wed, Jun 14, 2023 at 10:02 AM Divij Vaidya > wrote: > > > > > > Thank you everyone! I promise to shoulder this new responsibility to > the > > > best of my abilities. > > > > > > -- > > > Divij Vaidya > > > > > > > > > > > > On Wed, Jun 14, 2023 at 4:13 AM ziming deng > > > wrote: > > > > > > > Congratulations, Divij! > > > > Well deserved! > > > > > > > > -- > > > > Ziming > > > > > > > > > On Jun 14, 2023, at 09:41, Luke Chen wrote: > > > > > > > > > > Congratulations, Divij! > > > > > Well deserved! > > > > > > > > > > Luke > > > > > > > > > > On Wed, Jun 14, 2023 at 7:01 AM Viktor Somogyi-Vass > > > > > wrote: > > > > > > > > > >> Congrats Divij! > > > > >> > > > > >> On Tue, Jun 13, 2023, 20:27 Philip Nee > wrote: > > > > >> > > > > >>> Congrats! > > > > >>> > > > > >>> On Tue, Jun 13, 2023 at 8:17 PM Randall Hauch > > > > wrote: > > > > >>> > > > > >>>> Congratulations! > > > > >>>> > > > > >>>> On Tue, Jun 13, 2023 at 12:48 PM Matthias J. Sax < > mj...@apache.org> > > > > >>> wrote: > > > > >>>> > > > > >>>>> Congrats! > > > > >>>>> > > > > >>>>> On 6/13/23 10:24 AM, Satish Duggana wrote: > > > > >>>>>> Congratulations Divij!! > > > > >>>>>> > > > > >>>>>> On Tue, 13 Jun 2023 at 22:41, Manyanda Chitimbo > > > > >>>>>> wrote: > > > > >>>>>>> > > > > >>>>>>> Congratulations Divij. > > > > >>>>>>> > > > > >>>>>>> On Tue 13 Jun 2023 at 17:50, Bruno Cadonna < > cado...@apache.org> > > > > >>>> wrote: > > > > >>>>>>> > > > > >>>>>>>> Hi all, > > > > >>>>>>>> > > > > >>>>>>>> The PMC of Apache Kafka is pleased to announce a new Kafka > > > > >>> committer > > > > >>>>>>>> Divij Vaidya. > > > > >>>>>>>> > > > > >>>>>>>> Divij's major contributions are: > > > > >>>>>>>> > > > > >>>>>>>> GDPR compliance enforcement of kafka-site - > > > > >>>>>>>> https://issues.apache.org/jira/browse/KAFKA-13868 > > > > >>>>>>>> > > > > >>>>>>>> Performance improvements: > > > > >>>>>>>> > > > > >>>>>>>> Improve performance of VarInt encoding and decoding - > > > > >>>>>>>> https://github.com/apache/kafka/pull/13312 > > > > >>>>>>>> > > > > >>>>>>>> Reduce data copy & buffer allocation during decompression - > > > > >>>>>>>> https://github.com/apache/kafka/pull/13135 > > > > >>>>>>>> > > > > >>>>>>>> He also was heavily involved in the migration to Mockito. > > > > >>>>>>>> > > > > >>>>>>>> Furthermore, Divij is very active on the mailing lists as > well as > > > > >>> in > > > > >>>>>>>> maintaining and reviewing pull requests. > > > > >>>>>>>> > > > > >>>>>>>> Congratulations, Divij! > > > > >>>>>>>> > > > > >>>>>>>> Thanks, > > > > >>>>>>>> > > > > >>>>>>>> Bruno (on behalf of the Apache Kafka PMC) > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> -- > > > > >>>>>>> Manyanda Chitimbo. > > > > >>>>> > > > > >>>> > > > > >>> > > > > >> > > > > > > > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] KIP-917: Additional custom metadata for remote log segment
Hi Ivan, Thank you very much for this KIP. +1 (binding) from me. Best, On Tue, Jun 13, 2023 at 10:03 AM Luke Chen wrote: > +1 (binding) from me. > > Thanks. > Luke > > On Tue, Jun 13, 2023 at 3:44 PM Matthew Benedict de Detrich > wrote: > > > > +1 (non binding). Thanks for KIP > > > > On Tue, Jun 13, 2023 at 3:38 AM Kamal Chandraprakash < > > kamal.chandraprak...@gmail.com> wrote: > > > > > +1 (non-binding). Thanks for the KIP! > > > > > > On Mon, Jun 12, 2023, 21:23 Divij Vaidya > wrote: > > > > > > > I have reviewed the proposal and feel that it would be beneficial to > > > > implement. > > > > > > > > Vote +1 (non-binding) > > > > > > > > > > > > -- > > > > Divij Vaidya > > > > > > > > > > > > > > > > On Mon, Jun 12, 2023 at 4:39 PM Ivan Yurchenko < > ivan0yurche...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hello, > > > > > > > > > > Some interest in KIP-917 was expressed in the discussion thread > [1]. > > > > After > > > > > addressing all the comments there, I'm putting it up to a vote. > > > > > Thanks! > > > > > > > > > > Best, > > > > > Ivan > > > > > > > > > > [1] > https://lists.apache.org/thread/qpccqd3jy5rzvbt5ngtzo3dg9pzp722y > > > > > > > > > > > > > > > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: Permissions to contribute to Apache Kafka.
Hi Vaibhav, You are all set. Thanks for your interest in Apache Kafka! Best, On Mon, Jun 12, 2023 at 3:21 PM Vaibhav Nazare wrote: > HI Team, > > Following are the details for > > Wiki id > Email: vaibhav.naz...@ibm.com<mailto:vaibhav.naz...@ibm.com> > Username: vnazare > > Jira id > Email: vaibhav.naz...@ibm.com<mailto:vaibhav.naz...@ibm.com> > Username: vnazare > > > Thanks, > Vaibhav Nazare > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [VOTE] 3.5.0 RC1
Hi MIckael, Just for the record, I +1'd this release. However, my vote falls onto the non-binding ones as well. Best, On Sun, Jun 11, 2023 at 11:38 AM Mickael Maison wrote: > Hi, > > We've passed the deadline but we are still missing votes. > So far we have have 2 binding votes (Luke and John) and 2 non-binding > votes (Federico and Jakub). > > Please take time to test this release candidate and vote. > > Thanks, > Mickael > > On Wed, Jun 7, 2023 at 6:26 PM John Roesler wrote: > > > > Thanks for running this release, Mickael! > > > > I've verified: > > * the signature > > * that I can compile the project > > * that I can run the tests. I saw one flaky test failure, but I don't > think it should block us. Reported as > https://issues.apache.org/jira/browse/KAFKA-13531?focusedCommentId=17730190 > > * the Kafka, Consumer, and Streams quickstarts with ZK and KRaft > > > > I'm +1 (binding) > > > > Thanks, > > -John > > > > On Wed, Jun 7, 2023, at 06:16, Josep Prat wrote: > > > Hi MIckael, > > > > > > Apparently you did it in this PR already :) : > > > https://github.com/apache/kafka/pull/13749 (this PR among other things > > > removes classgraph. > > > > > > Without being a lawyer, I think I agree with you as stating we depend > on > > > something we don't would be less problematic than the other way around. > > > > > > Best, > > > > > > On Wed, Jun 7, 2023 at 12:14 PM Mickael Maison < > mickael.mai...@gmail.com> > > > wrote: > > > > > >> Hi Josep, > > >> > > >> Thanks for spotting this. If not already done, can you open a > > >> ticket/PR to fix this on trunk? It looks like the last couple of > > >> releases already had that issue. Since we're including a license for a > > >> dependency we don't ship, I think we can consider this non blocking. > > >> The other way around (shipping a dependency without its license) would > > >> be blocking. > > >> > > >> Thanks, > > >> Mickael > > >> > > >> On Tue, Jun 6, 2023 at 10:10 PM Jakub Scholz wrote: > > >> > > > >> > +1 (non-binding) ... I used the staged binaries with Scala 2.13 and > > >> staged > > >> > artifacts to run my tests. All seems to work fine. > > >> > > > >> > Thanks for running the release Mickael! > > >> > > > >> > Jakub > > >> > > > >> > On Mon, Jun 5, 2023 at 3:39 PM Mickael Maison > > >> wrote: > > >> > > > >> > > Hello Kafka users, developers and client-developers, > > >> > > > > >> > > This is the second candidate for release of Apache Kafka 3.5.0. > Some > > >> > > of the major features include: > > >> > > - KIP-710: Full support for distributed mode in dedicated > MirrorMaker > > >> > > 2.0 clusters > > >> > > - KIP-881: Rack-aware Partition Assignment for Kafka Consumers > > >> > > - KIP-887: Add ConfigProvider to make use of environment variables > > >> > > - KIP-889: Versioned State Stores > > >> > > - KIP-894: Use incrementalAlterConfig for syncing topic > configurations > > >> > > - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM > for > > >> > > Kafka Brokers > > >> > > > > >> > > Release notes for the 3.5.0 release: > > >> > > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/RELEASE_NOTES.html > > >> > > > > >> > > *** Please download, test and vote by Friday June 9, 5pm PT > > >> > > > > >> > > Kafka's KEYS file containing PGP keys we use to sign the release: > > >> > > https://kafka.apache.org/KEYS > > >> > > > > >> > > * Release artifacts to be voted upon (source and binary): > > >> > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/ > > >> > > > > >> > > * Maven artifacts to be voted upon: > > >> > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > >> > > > > >> > > * Javadoc: > > >> > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/javadoc/ > > >> > > > > >> > > * Tag to be voted upon (off 3.5 branch) is the 3.5.0 t
Re: [DISCUSS] Conventional commits
Hi Ismael, Thanks for raising this topic Ismael, I like the idea, I think this would bring some more meaningful first-liners on the commits. I guess we could use the scope (what comes within parentheses) to indicate the module the commit refers to (if it's single module) However, I doubt if this can be used for the release notes as we rely on JIra for this purpose. Some questions that come to mind: - should `feat` only be used for KIPs? - should we use conventional commits to help cut the release (for example, no `feat` commit in a patch version? - should we use the scope to indicate the module the commit touches? - should this be a convention for comitters to rewrite the message before squash-merging? Or will we enforce PR authors to follow this convention? Best, On Thu, Jun 8, 2023 at 8:13 AM Ismael Juma wrote: > Hi, > > A number of open source projects have adopted the conventional commits > specification. What do people think about using this for Apache Kafka? > > https://www.conventionalcommits.org/en/v1.0.0/ > > Thanks, > Ismael > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
[jira] [Created] (KAFKA-15072) Flaky test MirrorConnectorsIntegrationExactlyOnceTest.testReplicationWithEmptyPartition
Josep Prat created KAFKA-15072:
--
Summary: Flaky test
MirrorConnectorsIntegrationExactlyOnceTest.testReplicationWithEmptyPartition
Key: KAFKA-15072
URL: https://issues.apache.org/jira/browse/KAFKA-15072
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.5.0
Reporter: Josep Prat
Test
MirrorConnectorsIntegrationExactlyOnceTest.testReplicationWithEmptyPartition
became flaky again, but it's a different error this time.
Occurrence:
[https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13824/1/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationExactlyOnceTest/Build___JDK_17_and_Scala_2_13___testReplicationWithEmptyPartition__/]
h3. Error Message
{code:java}
java.lang.AssertionError: Connector MirrorHeartbeatConnector tasks did not
start in time on cluster: backup-connect-cluster{code}
h3. Stacktrace
{code:java}
java.lang.AssertionError: Connector MirrorHeartbeatConnector tasks did not
start in time on cluster: backup-connect-cluster at
org.apache.kafka.connect.util.clusters.EmbeddedConnectClusterAssertions.assertConnectorAndAtLeastNumTasksAreRunning(EmbeddedConnectClusterAssertions.java:301)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.waitUntilMirrorMakerIsRunning(MirrorConnectorsIntegrationBaseTest.java:912)
at
org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testReplicationWithEmptyPartition(MirrorConnectorsIntegrationBaseTest.java:415)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568) at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute
[jira] [Created] (KAFKA-15071) Flaky test kafka.admin.LeaderElectionCommandTest.testPreferredReplicaElection for Type=ZK, MetadataVersion=3.5-IV2, Security=PLAINTEXT
Josep Prat created KAFKA-15071:
--
Summary: Flaky test
kafka.admin.LeaderElectionCommandTest.testPreferredReplicaElection for Type=ZK,
MetadataVersion=3.5-IV2, Security=PLAINTEXT
Key: KAFKA-15071
URL: https://issues.apache.org/jira/browse/KAFKA-15071
Project: Kafka
Issue Type: Bug
Reporter: Josep Prat
Test became kafka.admin.LeaderElectionCommandTest.testPreferredReplicaElection
flaky again but failing because of different reason. In this case it might be a
missing cleanup
The values of the parameters are Type=ZK, MetadataVersion=3.5-IV2,
Security=PLAINTEXT
Related to https://issues.apache.org/jira/browse/KAFKA-13737
Ocurred:
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13824/1/testReport/junit/kafka.admin/LeaderElectionCommandTest/Build___JDK_8_and_Scala_2_123__Type_ZK__Name_testPreferredReplicaElection__MetadataVersion_3_5_IV2__Security_PLAINTEXT/
h3. Error Message
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists.{code}
h3. Stacktrace
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists.{code}
{{ }}
h3. Standard Output
{code:java}
Successfully completed leader election (UNCLEAN) for partitions unclean-topic-0
[2023-06-07 14:42:33,845] ERROR [QuorumController id=3000] writeNoOpRecord:
unable to start processing because of RejectedExecutionException. Reason: The
event queue is shutting down (org.apache.kafka.controller.QuorumController:467)
[2023-06-07 14:42:42,699] WARN [AdminClient clientId=adminclient-65] Connection
to node -2 (localhost/127.0.0.1:35103) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) Successfully
completed leader election (UNCLEAN) for partitions unclean-topic-0 [2023-06-07
14:42:44,416] ERROR [QuorumController id=0] writeNoOpRecord: unable to start
processing because of RejectedExecutionException. Reason: The event queue is
shutting down (org.apache.kafka.controller.QuorumController:467) [2023-06-07
14:42:44,716] WARN maxCnxns is not configured, using default value 0.
(org.apache.zookeeper.server.ServerCnxnFactory:309) [2023-06-07 14:42:44,765]
WARN No meta.properties file under dir
/tmp/kafka-2117748934951771120/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:44,986] WARN No
meta.properties file under dir /tmp/kafka-5133306871105583937/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:45,214] WARN No
meta.properties file under dir /tmp/kafka-8449809620400833553/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:45,634] WARN
[ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-0. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
14:42:45,634] WARN [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0]
Received UNKNOWN_TOPIC_ID from the leader for partition __consumer_offsets-4.
This error may be returned transiently when the partition is being created or
deleted, but it is not expected to persist.
(kafka.server.ReplicaFetcherThread:70) [2023-06-07 14:42:45,872] WARN
[ReplicaFetcher replicaId=2, leaderId=0, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-1. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
14:42:46,010] WARN [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error
in response for fetch request (type=FetchRequest, replicaId=0, maxWait=500,
minBytes=1, maxBytes=10485760,
fetchData={__consumer_offsets-3=PartitionData(topicId=vAlEsYVbTFClcpnVRp3AOw,
fetchOffset=0, logStartOffset=0, maxBytes=1048576,
currentLeaderEpoch=Optional[0], lastFetchedEpoch=Optional.empty)},
isolationLevel=READ_UNCOMMITTED, removed=, replaced=,
metadata=(sessionId=INVALID, epoch=INITIAL), rackId=)
(kafka.server.ReplicaFetcherThread:72) java.io.IOException: Connection to 2 was
disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.server.BrokerBlockingSender.sendRequest(BrokerBlockingSender.scala:113)
at kafka.server.RemoteLeaderEndPoint.fetch(RemoteLeaderEndPoint.scala:79) at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:316)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:130)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:129)
at scala.Option.foreach(Option.scala:407
[jira] [Created] (KAFKA-15070) Flaky test kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic for codec zstd
Josep Prat created KAFKA-15070: -- Summary: Flaky test kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic for codec zstd Key: KAFKA-15070 URL: https://issues.apache.org/jira/browse/KAFKA-15070 Project: Kafka Issue Type: Bug Components: core Affects Versions: 3.5.0 Reporter: Josep Prat Flaky tests with the following traces and output: h3. Error Message org.opentest4j.AssertionFailedError: Timed out waiting for deletion of old segments h3. Stacktrace org.opentest4j.AssertionFailedError: Timed out waiting for deletion of old segments at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at org.junit.jupiter.api.Assertions.fail(Assertions.java:135) at kafka.log.LogCleanerParameterizedIntegrationTest.testCleansCombinedCompactAndDeleteTopic(LogCleanerParameterizedIntegrationTest.scala:123) ... h3. Standard Output [2023-06-07 16:03:59,974] WARN [LocalLog partition=log-0, dir=/tmp/kafka-6339499869249617477] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:04:01,691] WARN [LocalLog partition=log-0, dir=/tmp/kafka-6391328203703920459] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:04:02,661] WARN [LocalLog partition=log-0, dir=/tmp/kafka-7107559685120209313] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:04:04,449] WARN [LocalLog partition=log-0, dir=/tmp/kafka-2334095685379242376] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:04:12,059] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-4306370019245327987] Could not find offset index file corresponding to log file /tmp/kafka-4306370019245327987/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:04:21,424] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-8549848301585177643] Could not find offset index file corresponding to log file /tmp/kafka-8549848301585177643/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:04:42,679] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-8308685679443421785] Could not find offset index file corresponding to log file /tmp/kafka-8308685679443421785/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:04:50,435] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-2686097435338562303] Could not find offset index file corresponding to log file /tmp/kafka-2686097435338562303/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:07:16,263] WARN [LocalLog partition=log-0, dir=/tmp/kafka-5435804108212698551] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:07:35,193] WARN [LocalLog partition=log-0, dir=/tmp/kafka-4310277229895025994] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:07:55,323] WARN [LocalLog partition=log-0, dir=/tmp/kafka-3364951894697258113] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:08:16,286] WARN [LocalLog partition=log-0, dir=/tmp/kafka-3161518940405121110] Record format version has been downgraded from V2 to V0. (kafka.log.LocalLog:70) [2023-06-07 16:35:03,765] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-2385863108707929062] Could not find offset index file corresponding to log file /tmp/kafka-2385863108707929062/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:35:06,406] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-5380450050465409057] Could not find offset index file corresponding to log file /tmp/kafka-5380450050465409057/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:35:09,061] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-7510941634638265317] Could not find offset index file corresponding to log file /tmp/kafka-7510941634638265317/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:35:11,593] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-7423113520781905391] Could not find offset index file corresponding to log file /tmp/kafka-7423113520781905391/log-0/0300.log, recovering segment and rebuilding index files... (kafka.log.LogLoader:74) [2023-06-07 16:35:14,159] ERROR [LogLoader partition=log-0, dir=/tmp/kafka-2120426496175304835] Could not find offset index file corresponding to log file /tmp/kafka
Re: [DISCUSS] Regarding Old PRs
Thanks David! ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Wed, Jun 7, 2023, 20:28 David Arthur wrote: > Hey all, I started poking around at Github actions on my fork. > > https://github.com/mumrah/kafka/actions > > I'll post a PR if I get it working and we can discuss what kind of settings > we want (or if we want it all) > > -David > > On Tue, Jun 6, 2023 at 1:18 PM Chris Egerton > wrote: > > > Hi Josep, > > > > Thanks for bringing this up! Will try to keep things brief. > > > > I'm generally in favor of this initiative. A couple ideas that I really > > liked: requiring a component label (producer, consumer, connect, streams, > > etc.) before closing, and disabling auto-close (i.e., automatically > tagging > > PRs as stale, but leaving it to a human being to actually close them). > > > > We might replace the "stale" label with a "close-by-" label so that > > it becomes even easier for us to find the PRs that are ready to be closed > > (as opposed to the ones that have just been labeled as stale without > giving > > the contributor enough time to respond). > > > > I've also gone ahead and closed some of my stale PRs. Others I've > > downgraded to draft to signal that I'd like to continue to pursue them, > but > > have to iron out merge conflicts first. For the last ones, I'll ping for > > review. > > > > One question that came to mind--do we want to distinguish between regular > > and draft PRs? I'm guessing not, since they still add up to the total PR > > count against the project, but since they do also implicitly signal that > > they're not intended for review (yet) it may be friendlier to leave them > > up. > > > > Cheers, > > > > Chris > > > > On Tue, Jun 6, 2023 at 10:18 AM Mickael Maison > > > wrote: > > > > > Hi Josep, > > > > > > Thanks for looking into this. This is clearly one aspect where we need > > > to improve. > > > > > > We had a similar initiative last year > > > (https://lists.apache.org/thread/66yj9m6tcyz8zqb3lqlbnr386bqwsopt) and > > > we closed many PRs. Unfortunately we did not follow up with a process > > > or automation and we are back to the same situation. > > > > > > Manually reviewing all these PRs is a huge task, so I think we should > > > at least partially automate it. I'm not sure if we should manually > > > review the oldest PRs (pre 2020). There's surely many interesting > > > things but I wonder if we should instead focus on the more recent ones > > > as they have a higher chance of 1) still making sense, 2) getting > > > updates from their authors, 3) needing less rebasing. If something has > > > been broken since 2016 but we never bothered to fix the PR it means it > > > can't be anything critical! > > > > > > Finally as Colin mentioned, it looks like a non negligible chunk of > > > stale PRs comes from committers and regular contributors. So I'd > > > suggest we each try to clean our own backlog too. > > > > > > I wonder if we also need to do something in JIRA. Querying for > > > unresolved tickets returns over 4000 items. Considering we're not > > > quite at KAFKA-15000 yet, that's a lot. > > > > > > Thanks, > > > Mickael > > > > > > > > > On Tue, Jun 6, 2023 at 11:35 AM Josep Prat > > > > wrote: > > > > > > > > Hi Devs, > > > > I would say we can split the problem in 2. > > > > > > > > Waiting for Author feedback: > > > > We could have a bot that would ping authors for the cases where we > have > > > PRs > > > > that are stalled and have either: > > > > - Merge conflict > > > > - Unaddressed reviews > > > > > > > > Waiting for reviewers: > > > > For the PRs where we have no reviewers and there are no conflicts, I > > > think > > > > we would need some human interaction to determine modules (maybe this > > can > > > > be automated) and ping people who could review. > > > > > > > > What do you think? > > > > > > > > Best, > > > > > > > > On Tue, Jun 6, 2023 at 11:30 AM Josep Prat > > wrote: > > > > > > > > > Hi Nikolay, > >
Re: [VOTE] 3.5.0 RC1
Hi MIckael, Apparently you did it in this PR already :) : https://github.com/apache/kafka/pull/13749 (this PR among other things removes classgraph. Without being a lawyer, I think I agree with you as stating we depend on something we don't would be less problematic than the other way around. Best, On Wed, Jun 7, 2023 at 12:14 PM Mickael Maison wrote: > Hi Josep, > > Thanks for spotting this. If not already done, can you open a > ticket/PR to fix this on trunk? It looks like the last couple of > releases already had that issue. Since we're including a license for a > dependency we don't ship, I think we can consider this non blocking. > The other way around (shipping a dependency without its license) would > be blocking. > > Thanks, > Mickael > > On Tue, Jun 6, 2023 at 10:10 PM Jakub Scholz wrote: > > > > +1 (non-binding) ... I used the staged binaries with Scala 2.13 and > staged > > artifacts to run my tests. All seems to work fine. > > > > Thanks for running the release Mickael! > > > > Jakub > > > > On Mon, Jun 5, 2023 at 3:39 PM Mickael Maison > wrote: > > > > > Hello Kafka users, developers and client-developers, > > > > > > This is the second candidate for release of Apache Kafka 3.5.0. Some > > > of the major features include: > > > - KIP-710: Full support for distributed mode in dedicated MirrorMaker > > > 2.0 clusters > > > - KIP-881: Rack-aware Partition Assignment for Kafka Consumers > > > - KIP-887: Add ConfigProvider to make use of environment variables > > > - KIP-889: Versioned State Stores > > > - KIP-894: Use incrementalAlterConfig for syncing topic configurations > > > - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for > > > Kafka Brokers > > > > > > Release notes for the 3.5.0 release: > > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/RELEASE_NOTES.html > > > > > > *** Please download, test and vote by Friday June 9, 5pm PT > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > https://kafka.apache.org/KEYS > > > > > > * Release artifacts to be voted upon (source and binary): > > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/ > > > > > > * Maven artifacts to be voted upon: > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > * Javadoc: > > > https://home.apache.org/~mimaison/kafka-3.5.0-rc1/javadoc/ > > > > > > * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag: > > > https://github.com/apache/kafka/releases/tag/3.5.0-rc1 > > > > > > * Documentation: > > > https://kafka.apache.org/35/documentation.html > > > > > > * Protocol: > > > https://kafka.apache.org/35/protocol.html > > > > > > * Successful Jenkins builds for the 3.5 branch: > > > Unit/integration tests: I'm struggling to get all tests to pass in the > > > same build. I'll run a few more builds to ensure each test pass at > > > least once in the CI. All tests passed locally. > > > System tests: The build is still running, I'll send an update once I > > > have the results. > > > > > > Thanks, > > > Mickael > > > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739 B
Re: [ANNOUNCE] Apache Kafka 3.4.1
Thanks for running the release! Best, ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen m: +491715557497 w: aiven.io e: josep.p...@aiven.io On Wed, Jun 7, 2023, 05:57 Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Thanks Luke for running this release! > > On Wed, Jun 7, 2023 at 8:08 AM Chia-Ping Tsai wrote: > > > Thank Luke for this hard work!!! > > > > > Chris Egerton 於 2023年6月7日 上午10:35 寫道： > > > > > > Thanks for running this release, Luke! > > > > > > On Tue, Jun 6, 2023, 22:31 Luke Chen wrote: > > > > > >> The Apache Kafka community is pleased to announce the release for > > >> Apache Kafka 3.4.1. > > >> > > >> This is a bug fix release and it includes fixes and improvements from > > >> 58 JIRAs, including a few critical bugs: > > >> - core > > >> KAFKA-14644 Process should stop after failure in raft IO thread > > >> KAFKA-14946 KRaft controller node shutting down while renouncing > > leadership > > >> KAFKA-14887 ZK session timeout can cause broker to shutdown > > >> - client > > >> KAFKA-14639 Kafka CooperativeStickyAssignor revokes/assigns partition > > >> in one rebalance cycle > > >> - connect > > >> KAFKA-12558 MM2 may not sync partition offsets correctly > > >> KAFKA-14666 MM2 should translate consumer group offsets behind > > replication > > >> flow > > >> - stream > > >> KAFKA-14172 bug: State stores lose state when tasks are reassigned > under > > >> EOS > > >> > > >> All of the changes in this release can be found in the release notes: > > >> > > >> https://www.apache.org/dist/kafka/3.4.1/RELEASE_NOTES.html > > >> > > >> You can download the source and binary release (Scala 2.12 and Scala > > 2.13) > > >> from: > > >> > > >> https://kafka.apache.org/downloads#3.4.1 > > >> > > >> > > >> > > > --- > > >> > > >> Apache Kafka is a distributed streaming platform with four core APIs: > > >> > > >> ** The Producer API allows an application to publish a stream records > > >> to one or more Kafka topics. > > >> > > >> ** The Consumer API allows an application to subscribe to one or more > > >> topics and process the stream of records produced to them. > > >> > > >> ** The Streams API allows an application to act as a stream processor, > > >> consuming an input stream from one or more topics and producing an > > >> output stream to one or more output topics, effectively transforming > > >> the input streams to output streams. > > >> > > >> ** The Connector API allows building and running reusable producers or > > >> consumers that connect Kafka topics to existing applications or data > > >> systems. For example, a connector to a relational database might > > >> capture every change to a table. > > >> > > >> > > >> With these APIs, Kafka can be used for two broad classes of > application: > > >> > > >> ** Building real-time streaming data pipelines that reliably get data > > >> between systems or applications. > > >> > > >> ** Building real-time streaming applications that transform or react > > >> to the streams of data. > > >> > > >> Apache Kafka is in use at large and small companies worldwide, > > >> including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, > > >> Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and > > >> Zalando, among others. > > >> > > >> A big thank you for the following 32 contributors to this release! > > >> > > >> atu-sharm, Chia-Ping Tsai, Chris Egerton, Colin Patrick McCabe, > > >> csolidum, David Arthur, David Jacot, Divij Vaidya, egyedt, > > >> emilnkrastev, Eric Haag, Greg Harris, Guozhang Wang, Hector Geraldino, > > >> hudeqi, Jason Gustafson, Jeff Kim, Jorge Esteban Quilcate Otoya, José > > >> Armando García Sancio, Lucia Cerchie, Luke Chen, Manikumar Reddy, > > >> Matthias J. Sax, Mickael Maison, Philip Nee, Purshotam Chauhan, Rajini > > >> Sivaram, Ron Dagostino, Terry, Victoria Xia, Viktor Somogyi-Vass, Yash > > >> Mayya > > >> > > >> We welcome your help and feedback. For more information on how to > > >> report problems, and to get involved, visit the project website at > > >> https://kafka.apache.org/ > > >> > > >> > > >> Thank you! > > >> > > >> Regards, > > >> Luke > > >> > > > > >
