Hi Konstantine, Thanks for your work on this release! We discovered three blocker bugs which are worth bringing up here:
1. KAFKA-13112: Controller's committed offset get out of sync with raft client listener context 2. KAFKA-13095: TransactionsTest is failing in kraft mode 3. KAFKA-12851: Flaky Test RaftEventSimulationTest.canMakeProgressIfMajorityIsReachable There are two subtasks for #1 which we are working on. We suspect that #3 has been fixed by a previous fix we made... we're looking into it. best, Colin On Mon, Jul 19, 2021, at 20:23, Konstantine Karantasis wrote: > Hi all, > > Since last week, we have reached the stage of Code Freeze for the 3.0.0 > Apache Kafka release. > > From this point forward and until the official release of 3.0.0, only > critical fixes for blocker issues should be merged to the 3.0 release > branch. > > The release plan currently includes ten (10) such known blockers. > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0 > > Besides these issues, any new issue that potentially gets discovered will > need to be reported on dev@kafka.apache.org (under this thread) and be > evaluated as a release blocker. At this point, the bar for such issues is > high; they need to be regressions or critical issues without an acceptable > workaround to be considered as release blockers. > > Exceptions of changes that may be merged to the 3.0 branch without a > mention on the dev mailing list are fixes for test failures that will help > stabilize the build and small documentation changes. > > If by the end of this week we are down to zero blockers and have green > builds and passing system tests, I will attempt to generate the first > Release Candidate (RC) for 3.0.0 on Friday. > > Thank you all for the hard work so far. > Konstantine > > > On Mon, Jul 12, 2021 at 1:59 PM Konstantine Karantasis > <konstant...@confluent.io.invalid> wrote: > > > Hi all, > > > > This is a reminder that Code Freeze for Apache Kafka 3.0 is coming up this > > week and is set to take place by the end of day Wednesday, July 14th. > > > > Currently in the project we have 22 blocker issues for 3.0, out of 41 total > > tickets targeting 3.0. > > > > You may find the list of open issues in the release plan page: > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0 > > > > Thanks for all the hard work so far and for reducing the number of open > > issues in the recent days. > > Please take another look and help us resolve all the blockers for this > > upcoming major release. > > > > Best, > > Konstantine > > > > On Mon, Jul 12, 2021 at 1:57 PM Konstantine Karantasis < > > konstant...@confluent.io> wrote: > > > > > > > > Thanks for the update Levani, > > > > > > KIP-708 is now on the list of postponed KIPs. > > > > > > Konstantine > > > > > > On Thu, Jul 1, 2021 at 10:48 PM Levani Kokhreidze < > > levani.co...@gmail.com> > > > wrote: > > > > > >> Hi Konstantine, > > >> > > >> FYI, I don’t think we will be able to have KIP-708 ready on time. > > >> Feel free to remove it from the release plan. > > >> > > >> Best, > > >> Levani > > >> > > >> > On 1. Jul 2021, at 01:27, Konstantine Karantasis < > > >> konstant...@confluent.io.INVALID> wrote: > > >> > > > >> > Hi all, > > >> > > > >> > Today we have reached the Feature Freeze milestone for Apache Kafka > > 3.0. > > >> > Exciting! > > >> > > > >> > I'm going to allow for any pending changes to settle within the next > > >> couple > > >> > of days. > > >> > I trust that we all approve and merge adopted features and changes > > >> which we > > >> > consider to be in good shape for 3.0. > > >> > > > >> > Given the 4th of July holiday in the US, the 3.0 release branch will > > >> appear > > >> > sometime on Tuesday, July 6th. > > >> > Until then, please keep merging to trunk only the changes you intend > > to > > >> > include in Apache Kafka 3.0. > > >> > > > >> > Regards, > > >> > Konstantine > > >> > > > >> > > > >> > On Wed, Jun 30, 2021 at 3:25 PM Konstantine Karantasis < > > >> > konstant...@confluent.io> wrote: > > >> > > > >> >> > > >> >> Done. Thanks Luke! > > >> >> > > >> >> On Tue, Jun 29, 2021 at 6:39 PM Luke Chen <show...@gmail.com> wrote: > > >> >> > > >> >>> Hi Konstantine, > > >> >>> We've decided that the KIP-726 will be released in V3.1, not V3.0. > > >> >>> KIP-726: Make the "cooperative-sticky, range" as the default > > assignor > > >> >>> > > >> >>> Could you please remove this KIP from the 3.0 release plan wiki > > page? > > >> >>> > > >> >>> Thank you. > > >> >>> Luke > > >> >>> > > >> >>> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis > > >> >>> <konstant...@confluent.io.invalid> wrote: > > >> >>> > > >> >>>> Thanks for the update Colin. > > >> >>>> They are now both in the release plan. > > >> >>>> > > >> >>>> Best, > > >> >>>> Konstantine > > >> >>>> > > >> >>>> On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe <cmcc...@apache.org> > > >> >>> wrote: > > >> >>>> > > >> >>>>> Hi Konstantine, > > >> >>>>> > > >> >>>>> Can you please add two KIPs to the 3.0 release plan wiki page? > > >> >>>>> > > >> >>>>> I'm thinking of: > > >> >>>>> KIP-630: Kafka Raft Snapshots > > >> >>>>> KIP-746: Revise KRaft Metadata Records > > >> >>>>> > > >> >>>>> These are marked as 3.0 on the KIP page but I guess we don't have > > >> >>> them on > > >> >>>>> the page yet. > > >> >>>>> > > >> >>>>> Many thanks. > > >> >>>>> Colin > > >> >>>>> > > >> >>>>> > > >> >>>>> On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote: > > >> >>>>>> Hi there, > > >> >>>>>> > > >> >>>>>> As the feature freeze date is approaching, I just wanted to > > kindly > > >> >>> ask > > >> >>>>> for > > >> >>>>>> some reviews on the already submitted PR ( > > >> >>>>>> https://github.com/apache/kafka/pull/10840) that implements the > > >> >>>> approved > > >> >>>>>> KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR > > has > > >> >>>> been > > >> >>>>>> ready for review for 2 weeks, and I simply want to make sure > > there > > >> >>> is > > >> >>>>>> enough time to address any possible changes that might be > > >> requested. > > >> >>>>>> > > >> >>>>>> Thanks in advance and sorry for any inconvenience caused, > > >> >>>>>> -- > > >> >>>>>> Josep > > >> >>>>>> On Mon, Jun 21, 2021 at 11:54 PM Konstantine Karantasis > > >> >>>>>> <konstant...@confluent.io.invalid> wrote: > > >> >>>>>> > > >> >>>>>>> Thanks for the update Bruno. > > >> >>>>>>> I've moved KIP-698 to the list of postponed KIPs in the plan. > > >> >>>>>>> > > >> >>>>>>> Konstantine > > >> >>>>>>> > > >> >>>>>>> On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna < > > cado...@apache.org > > >> >>>> > > >> >>>>> wrote: > > >> >>>>>>> > > >> >>>>>>>> Hi Konstantine, > > >> >>>>>>>> > > >> >>>>>>>> The implementation of > > >> >>>>>>>> > > >> >>>>>>>> KIP-698: Add Explicit User Initialization of Broker-side State > > >> >>> to > > >> >>>>> Kafka > > >> >>>>>>>> Streams > > >> >>>>>>>> > > >> >>>>>>>> will not be ready for 3.0, so you can remove it from the list. > > >> >>>>>>>> > > >> >>>>>>>> Best, > > >> >>>>>>>> Bruno > > >> >>>>>>>> > > >> >>>>>>>> On 15.06.21 07:33, Konstantine Karantasis wrote: > > >> >>>>>>>>> Done. Moved it into the table of Adopted KIPs targeting 3.0.0 > > >> >>> and > > >> >>>>> to > > >> >>>>>>> the > > >> >>>>>>>>> release plan of course. > > >> >>>>>>>>> Thanks for catching this Israel. > > >> >>>>>>>>> > > >> >>>>>>>>> Best, > > >> >>>>>>>>> Konstantine > > >> >>>>>>>>> > > >> >>>>>>>>> On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo < > > >> >>>> israele...@gmail.com> > > >> >>>>>>>> wrote: > > >> >>>>>>>>> > > >> >>>>>>>>>> Konstantine, > > >> >>>>>>>>>> > > >> >>>>>>>>>> One of mine is missing from this list > > >> >>>>>>>>>> > > >> >>>>>>>>>> KIP-633: Drop 24 hour default of grace period in Streams > > >> >>>>>>>>>> Please could you include it? > > >> >>>>>>>>>> > > >> >>>>>>>>>> Voting has already concluded a long time ago > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> On Mon, Jun 14, 2021 at 6:08 PM Konstantine Karantasis > > >> >>>>>>>>>> <konstant...@confluent.io.invalid> wrote: > > >> >>>>>>>>>> > > >> >>>>>>>>>>> Hi all. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> KIP Freeze for the next major release of Apache Kafka was > > >> >>>> reached > > >> >>>>>>> last > > >> >>>>>>>>>>> week. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> As of now, 36 KIPs have concluded their voting process and > > >> >>> have > > >> >>>>> been > > >> >>>>>>>>>>> adopted. > > >> >>>>>>>>>>> These KIPs are targeting 3.0 (unless it's noted otherwise in > > >> >>>> the > > >> >>>>>>>> release > > >> >>>>>>>>>>> plan) and their inclusion as new features will be finalized > > >> >>>> right > > >> >>>>>>> after > > >> >>>>>>>>>>> Feature Freeze. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> At the high level, out of these 36 KIPs, 11 have been > > >> >>>> implemented > > >> >>>>>>>> already > > >> >>>>>>>>>>> and 25 are open or in progress. > > >> >>>>>>>>>>> Here is the full list of adopted KIPs: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> KIP-751: Drop support for Scala 2.12 in Kafka 4.0 > > >> >>> (deprecate in > > >> >>>>> 3.0) > > >> >>>>>>>>>>> KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in > > >> >>>> 3.0) > > >> >>>>>>>>>>> KIP-746: Revise KRaft Metadata Records > > >> >>>>>>>>>>> KIP-745: Connect API to restart connector and tasks > > >> >>>>>>>>>>> KIP-744: Migrate TaskMetadata and ThreadMetadata to an > > >> >>>> interface > > >> >>>>> with > > >> >>>>>>>>>>> internal implementation > > >> >>>>>>>>>>> KIP-743: Remove config value 0.10.0-2.4 of Streams built-in > > >> >>>>> metrics > > >> >>>>>>>>>> version > > >> >>>>>>>>>>> config > > >> >>>>>>>>>>> KIP-741: Change default serde to be null > > >> >>>>>>>>>>> KIP-740: Clean up public API in TaskId and fix > > >> >>>>> TaskMetadata#taskId() > > >> >>>>>>>>>>> KIP-738: Removal of Connect's internal converter properties > > >> >>>>>>>>>>> KIP-734: Improve AdminClient.listOffsets to return timestamp > > >> >>>> and > > >> >>>>>>> offset > > >> >>>>>>>>>> for > > >> >>>>>>>>>>> the record with the largest timestamp > > >> >>>>>>>>>>> KIP-733: Change Kafka Streams default replication factor > > >> >>> config > > >> >>>>>>>>>>> KIP-732: Deprecate eos-alpha and replace eos-beta with > > >> >>> eos-v2 > > >> >>>>>>>>>>> KIP-730: Producer ID generation in KRaft mode > > >> >>>>>>>>>>> KIP-726: Make the "cooperative-sticky, range" as the default > > >> >>>>> assignor > > >> >>>>>>>>>>> KIP-725: Streamlining configurations for WindowedSerializer > > >> >>> and > > >> >>>>>>>>>>> WindowedDeserializer. > > >> >>>>>>>>>>> KIP-724: Drop support for message formats v0 and v1 > > >> >>>>>>>>>>> KIP-722: Enable connector client overrides by default > > >> >>>>>>>>>>> KIP-721: Enable connector log contexts in Connect Log4j > > >> >>>>> configuration > > >> >>>>>>>>>>> KIP-720: Deprecate MirrorMaker v1 > > >> >>>>>>>>>>> KIP-716: Allow configuring the location of the offset-syncs > > >> >>>> topic > > >> >>>>>>> with > > >> >>>>>>>>>>> MirrorMaker2 > > >> >>>>>>>>>>> KIP-715: Expose Committed offset in streams > > >> >>>>>>>>>>> KIP-709: Extend OffsetFetch requests to accept multiple > > >> >>> group > > >> >>>>> ids. > > >> >>>>>>>>>>> KIP-708: Rack awareness for Kafka Streams > > >> >>>>>>>>>>> KIP-707: The future of KafkaFuture > > >> >>>>>>>>>>> KIP-699: Update FindCoordinator to resolve multiple > > >> >>>> Coordinators > > >> >>>>> at a > > >> >>>>>>>>>> time > > >> >>>>>>>>>>> KIP-698: Add Explicit User Initialization of Broker-side > > >> >>> State > > >> >>>> to > > >> >>>>>>> Kafka > > >> >>>>>>>>>>> Streams > > >> >>>>>>>>>>> KIP-695: Further Improve Kafka Streams Timestamp > > >> >>>> Synchronization > > >> >>>>>>>>>>> KIP-691: Enhance Transactional Producer Exception Handling > > >> >>>>>>>>>>> KIP-679: Producer will enable the strongest delivery > > >> >>> guarantee > > >> >>>> by > > >> >>>>>>>> default > > >> >>>>>>>>>>> KIP-666: Add Instant-based methods to ReadOnlySessionStore > > >> >>>>>>>>>>> KIP-653: Upgrade log4j to log4j2 > > >> >>>>>>>>>>> KIP-623: Add "internal-topics" option to streams application > > >> >>>>> reset > > >> >>>>>>> tool > > >> >>>>>>>>>>> KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to > > >> >>>>>>>>>>> ProcessorContext > > >> >>>>>>>>>>> KIP-466: Add support for List<T> serialization and > > >> >>>>> deserialization > > >> >>>>>>>>>>> KIP-405: Kafka Tiered Storage > > >> >>>>>>>>>>> KIP-390: Support Compression Level > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> If you notice that a KIP is missing from the list above and > > >> >>>>> should be > > >> >>>>>>>>>> part > > >> >>>>>>>>>>> of the release plan for 3.0, please reply below. > > >> >>>>>>>>>>> The single source of truth remains the official release plan > > >> >>>> for > > >> >>>>> 3.0, > > >> >>>>>>>>>> which > > >> >>>>>>>>>>> you may read at any time here: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>> > > >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0 > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Kind reminder that for all the adopted KIPs any required > > >> >>>> changes > > >> >>>>> to > > >> >>>>>>> the > > >> >>>>>>>>>>> documentation are also part of their respective feature. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> For the KIPs that are still in progress, please work closely > > >> >>>> with > > >> >>>>>>> your > > >> >>>>>>>>>>> reviewers to make sure that the features are stable and > > >> >>> land on > > >> >>>>> time > > >> >>>>>>>> for > > >> >>>>>>>>>>> Feature Freeze. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> The next milestone for the Apache Kafka 3.0 release is > > >> >>> Feature > > >> >>>>> Freeze > > >> >>>>>>>> on > > >> >>>>>>>>>>> June 30th, 2021. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Best regards, > > >> >>>>>>>>>>> Konstantine > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> On Fri, Jun 4, 2021 at 3:47 PM Konstantine Karantasis < > > >> >>>>>>>>>>> konstant...@confluent.io> wrote: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> Hi all, > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> Just a quick reminder that KIP Freeze is next Wednesday, > > >> >>> June > > >> >>>>> 9th. > > >> >>>>>>>>>>>> A vote thread needs to be open for at least 72 hours, so to > > >> >>>>> everyone > > >> >>>>>>>>>> that > > >> >>>>>>>>>>>> is working hard on proposals targeting 3.0.0, please make > > >> >>> sure > > >> >>>>> that > > >> >>>>>>>>>> your > > >> >>>>>>>>>>>> [VOTE] threads are started on time. > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> Best, > > >> >>>>>>>>>>>> Konstantine > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> On Wed, May 26, 2021 at 8:10 PM Israel Ekpo < > > >> >>>>> israele...@gmail.com> > > >> >>>>>>>>>>> wrote: > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>>> +1 on the new schedule. > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> On Wed, May 26, 2021 at 8:14 PM Sophie Blee-Goldman > > >> >>>>>>>>>>>>> <sop...@confluent.io.invalid> wrote: > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> Ah ok, thanks Konstantine. I won't bug you about every > > >> >>> new > > >> >>>> KIP > > >> >>>>>>> that > > >> >>>>>>>>>>>>> comes > > >> >>>>>>>>>>>>>> in between now and KIP Freeze :P > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> +1 on the scheduling changes as well > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> On Wed, May 26, 2021 at 4:00 PM David Arthur < > > >> >>>>> mum...@gmail.com> > > >> >>>>>>>>>>> wrote: > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> The new schedule looks good to me, +1 > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> On Wed, May 26, 2021 at 6:29 PM Ismael Juma < > > >> >>>>> ism...@juma.me.uk> > > >> >>>>>>>>>>>>> wrote: > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>> Thanks Konstantine, +1 from me. > > >> >>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>> Ismael > > >> >>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>> On Wed, May 26, 2021 at 2:48 PM Konstantine Karantasis > > >> >>>>>>>>>>>>>>>> <konstant...@confluent.io.invalid> wrote: > > >> >>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> Hi all, > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> Please find below the updated release plan for the > > >> >>> Apache > > >> >>>>>>>>>> Kafka > > >> >>>>>>>>>>>>> 3.0.0 > > >> >>>>>>>>>>>>>>>>> release. > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177046466 > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> New suggested dates for the release are as follows: > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> KIP Freeze is 09 June 2021 (same date as in the > > >> >>> initial > > >> >>>>> plan) > > >> >>>>>>>>>>>>>>>>> Feature Freeze is 30 June 2021 (new date, extended by > > >> >>> two > > >> >>>>>>>>>> weeks) > > >> >>>>>>>>>>>>>>>>> Code Freeze is 14 July 2021 (new date, extended by two > > >> >>>>> weeks) > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> At least two weeks of stabilization will follow Code > > >> >>>>> Freeze. > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> The release plan is up to date and currently includes > > >> >>> all > > >> >>>>> the > > >> >>>>>>>>>>>>>> approved > > >> >>>>>>>>>>>>>>>> KIPs > > >> >>>>>>>>>>>>>>>>> that are targeting 3.0.0. > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> Please let me know if you have any objections with the > > >> >>>>> recent > > >> >>>>>>>>>>>>>> extension > > >> >>>>>>>>>>>>>>>> of > > >> >>>>>>>>>>>>>>>>> Feature Freeze and Code Freeze or any other concerns. > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>>> Regards, > > >> >>>>>>>>>>>>>>>>> Konstantine > > >> >>>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> -- > > >> >>>>>>>>>>>>>>> David Arthur > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> -- > > >> >>>>>> > > >> >>>>>> Josep Prat > > >> >>>>>> > > >> >>>>>> *Aiven Deutschland GmbH* > > >> >>>>>> > > >> >>>>>> Immanuelkirchstraße 26, 10405 Berlin > > >> >>>>>> > > >> >>>>>> Amtsgericht Charlottenburg, HRB 209739 B > > >> >>>>>> > > >> >>>>>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > >> >>>>>> > > >> >>>>>> *m:* +491715557497 > > >> >>>>>> > > >> >>>>>> *w:* aiven.io > > >> >>>>>> > > >> >>>>>> *e:* josep.p...@aiven.io > > >> >>>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> >> > > >> > > >> > > >
