Thanks Luke for helping with the reviews and adding a few tests in a couple of PRs.
Hi Mickael, I raised 3 PRs recently for tiered storage, one is merged. The other 2 PRs are in the critical path of non-tiered storage changes also. Especially, in consumer fetch and retention cleanup paths. These need to be thoroughly reviewed and avoid any regressions in that area. We should merge them to the trunk as soon as possible to make it easier to work on follow-up PRs. IMO, we can avoid merging these PRs in 3.5 just before the release without baking for a longer duration. We can take a call on this later after the reviews are done. Many of the individual functionalities related to tiered storage like default topic based RLMM implementation, enhanced follower fetch protocol implementation for tiered storage, copying remote log segments are merged. There are 2 PRs for consumer fetch for remote record reads, remote retention cleanup and topic deletion functionality are under review. I do not think it can be considered as an early access review even with the 2 PRs in review. Luke and I synced up and agreed on the same. Most of the recent functionality is added with a few unit tests. We plan to have follow-up PRs on the immediate pending items and also raise PRs in the next few weeks on unit tests, integration test framework and several integration tests for many of these functionalities tying together. Thanks, Satish. On Fri, 14 Apr 2023 at 12:52, Matthias J. Sax <mj...@apache.org> wrote: > > Hey Mickael, > > we have one open PR for KIP-914 left. Would be great if you could merge > it before cutting the 3.5 branch. If you don't want to merge it and > prefer that I cherry-pick it to 3.5 branch later, also works for me. > > I did close the ticket already as resolved. It's just a minor change the > KIP. > > https://github.com/apache/kafka/pull/13565 > > > Thanks a lot! > -Matthias > > > On 4/13/23 4:32 AM, David Jacot wrote: > > Hi Mickael, > > > > Thanks for the heads up. As raised by Jeff earlier in this thread, we would > > like to get the two small patches [1][2] for KIP-915 in 3.5. The PRs are in > > review and I should be able to merge them in the next few days. I will > > cherry-pick them to the release branch in your create it in the meantime. > > > > [1] https://github.com/apache/kafka/pull/13511 > > [2] https://github.com/apache/kafka/pull/13526 > > > > Best, > > David > > > > On Thu, Apr 13, 2023 at 12:55 PM Mickael Maison <mickael.mai...@gmail.com> > > wrote: > > > >> Hi Luke, > >> > >> Thanks for the heads up. This would be great to get Tiered Storage in > >> Early Access. Let me know if you can't get everything done this week. > >> > >> Mickael > >> > >> On Thu, Apr 13, 2023 at 12:54 PM Mickael Maison > >> <mickael.mai...@gmail.com> wrote: > >>> > >>> Hi, > >>> > >>> We've now reached feature freeze for 3.5.0. From now on, only bug > >>> fixes and changes related to stabilizing the release should be merged. > >>> > >>> I plan to create the release branch tomorrow (Friday 14). After this > >>> point, you'll have to cherry pick changes to the release branch when > >>> merging a PR. I'll send another message once the branch has been > >>> created. > >>> > >>> I've updated the release plan and started moving KIPs that are not > >>> complete to the postponed section. For now I've kept a few KIPs that > >>> are still in progress. If they are not fully merged when I create a > >>> branch, I'll mark them as postponed too. > >>> > >>> The next milestone is code freeze on April 26. > >>> > >>> Thanks, > >>> Mickael > >>> > >>> On Wed, Apr 12, 2023 at 12:24 PM Luke Chen <show...@gmail.com> wrote: > >>>> > >>>> Hi Mickael, > >>>> > >>>> I'd like to ask for some more days for KIP-405 tiered storage PRs to > >>>> include in v3.5. > >>>> Currently, we have 1 PR under reviewing ( > >>>> https://github.com/apache/kafka/pull/13535), and 1 PR soon will be > >> opened > >>>> for review. > >>>> After these 2 PRs merged, we can have an "Early Access" for tiered > >> storage > >>>> feature, that allow users to use in non-production environments. > >>>> Does that work for you? > >>>> > >>>> Thank you. > >>>> Luke > >>>> > >>>> On Thu, Apr 6, 2023 at 2:49 AM Jeff Kim <jeff....@confluent.io.invalid > >>> > >>>> wrote: > >>>> > >>>>> Hi Mickael, > >>>>> > >>>>> Thank you. > >>>>> > >>>>> Best, > >>>>> Jeff > >>>>> > >>>>> On Wed, Apr 5, 2023 at 1:28 PM Mickael Maison < > >> mickael.mai...@gmail.com> > >>>>> wrote: > >>>>> > >>>>>> Hi Jeff, > >>>>>> > >>>>>> Ok, I've added KIP-915 to the release plan. > >>>>>> > >>>>>> Thanks, > >>>>>> Mickael > >>>>>> > >>>>>> On Wed, Apr 5, 2023 at 6:48 PM Jeff Kim > >> <jeff....@confluent.io.invalid> > >>>>>> wrote: > >>>>>>> > >>>>>>> Hi Mickael, > >>>>>>> > >>>>>>> I would like to bring up that KIP-915 proposes to patch 3.5 > >>>>>>> although it missed the KIP freeze date. If the patch is done > >> before the > >>>>>>> feature freeze date, 4/13, would this be acceptable? If so, > >> should this > >>>>>>> be added to the 3.5.0 Release Plan wiki? > >>>>>>> > >>>>>>> Best, > >>>>>>> Jeff > >>>>>>> > >>>>>>> On Mon, Mar 27, 2023 at 1:02 PM Greg Harris > >>>>> <greg.har...@aiven.io.invalid > >>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Mickael, > >>>>>>>> > >>>>>>>> Just wanted to let you know that I will not be including > >> KIP-898 in > >>>>> the > >>>>>>>> 3.5.0 release. > >>>>>>>> I think the change needed is not reviewable before the feature > >> freeze > >>>>>>>> deadline, and would take resources away from other more > >> necessary > >>>>>> changes. > >>>>>>>> > >>>>>>>> Thanks! > >>>>>>>> Greg > >>>>>>>> > >>>>>>>> On Thu, Mar 23, 2023 at 9:01 AM Chia-Ping Tsai < > >> chia7...@gmail.com> > >>>>>> wrote: > >>>>>>>> > >>>>>>>>>> If you have a KIP that is accepted, make sure it is listed > >> in > >>>>>>>>>> > >>>>>> > >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 > >>>>>>>>>> and that it's status is accurate. > >>>>>>>>> > >>>>>>>>> Thanks for the reminder. Have added KIP-641 to the list. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Chia-Ping > >>>>>>>>> > >>>>>>>>>> Mickael Maison <mickael.mai...@gmail.com> 於 2023年3月23日 > >> 下午11:51 > >>>>> 寫道： > >>>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> KIP Freeze was yesterday. The next milestone is feature > >> freeze on > >>>>>> April > >>>>>>>>> 12. > >>>>>>>>>> If you have a KIP that is accepted, make sure it is listed > >> in > >>>>>>>>>> > >>>>>> > >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 > >>>>>>>>>> and that it's status is accurate. > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> Mickael > >>>>>>>>>> > >>>>>>>>>> On Fri, Mar 17, 2023 at 6:22 PM Christo Lolov < > >>>>>> christolo...@gmail.com> > >>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hello! > >>>>>>>>>>> > >>>>>>>>>>> What would you suggest as the best way to get more eyes on > >>>>>> KIP-902 as > >>>>>>>> I > >>>>>>>>> would like it to be included it in 3.5.0? > >>>>>>>>>>> > >>>>>>>>>>> Best, > >>>>>>>>>>> Christo > >>>>>>>>>>> > >>>>>>>>>>>> On 16 Mar 2023, at 10:33, Mickael Maison < > >>>>>> mickael.mai...@gmail.com> > >>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> Hi, > >>>>>>>>>>>> > >>>>>>>>>>>> This is a reminder that KIP freeze is less than a week > >> away (22 > >>>>>> Mar). > >>>>>>>>>>>> For a KIP to be considered for this release, it must be > >> voted > >>>>> and > >>>>>>>>>>>> accepted by that date. > >>>>>>>>>>>> > >>>>>>>>>>>> Feature freeze will be 3 weeks after this, so if you > >> want KIPs > >>>>> or > >>>>>>>>>>>> other significant changes in the release, please get > >> them ready > >>>>>> soon. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Mickael > >>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Feb 14, 2023 at 10:44 PM Ismael Juma < > >>>>> ism...@juma.me.uk > >>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>> > >>>>>>>>>>>>> Ismael > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Feb 14, 2023 at 1:07 PM Mickael Maison < > >>>>>>>>> mickael.mai...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Ismael, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Good call. I shifted all dates by 2 weeks and moved > >> them to > >>>>>>>>> Wednesdays. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Mickael > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Feb 14, 2023 at 6:01 PM Ismael Juma < > >>>>> ism...@juma.me.uk > >>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks Mickael. A couple of notes: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 1. We typically choose a Wednesday for the various > >> freeze > >>>>>> dates - > >>>>>>>>> there > >>>>>>>>>>>>>> are > >>>>>>>>>>>>>>> often 1-2 day slips and it's better if that doesn't > >> require > >>>>>> people > >>>>>>>>>>>>>>> working through the weekend. > >>>>>>>>>>>>>>> 2. Looks like we're over a month later compared to the > >>>>>> equivalent > >>>>>>>>> release > >>>>>>>>>>>>>>> last year ( > >>>>>>>>>>>>>>> > >>>>>>>>> > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.2.0 > >> ). > >>>>>> I > >>>>>>>>>>>>>>> understand that some of it is due to 3.4.0 slipping, > >> but I > >>>>>> wonder > >>>>>>>>> if we > >>>>>>>>>>>>>>> could perhaps aim for the KIP freeze to be one or two > >> weeks > >>>>>>>> earlier. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Ismael > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Tue, Feb 14, 2023 at 8:00 AM Mickael Maison < > >>>>>>>>> mickael.mai...@gmail.com > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I've created a release plan for 3.5.0 in the wiki: > >>>>>>>>>>>>>>>> > >>>>>>>>> > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Current dates are: > >>>>>>>>>>>>>>>> 1) KIP Freeze: 07 Apr 2023 > >>>>>>>>>>>>>>>> 2) Feature Freeze: 27 Apr 2023 > >>>>>>>>>>>>>>>> 3) Code Freeze: 11 May 2023 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Please take a look at the plan. Let me know if there > >> are > >>>>>> other > >>>>>>>> KIPs > >>>>>>>>>>>>>>>> targeting 3.5.0. > >>>>>>>>>>>>>>>> Also if you are the author of one of the KIPs that's > >>>>> missing > >>>>>> a > >>>>>>>>> status > >>>>>>>>>>>>>>>> (or the status is incorrect) please update it and > >> let me > >>>>>> know. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Mickael > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Thu, Feb 9, 2023 at 9:23 AM Bruno Cadonna < > >>>>>> cado...@apache.org > >>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, Mickael! > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>> Bruno > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On 09.02.23 03:15, Luke Chen wrote: > >>>>>>>>>>>>>>>>>> Hi Mickael, > >>>>>>>>>>>>>>>>>> Thanks for volunteering! > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Luke > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Thu, Feb 9, 2023 at 6:23 AM Chris Egerton < > >>>>>>>>>>>>>> fearthecel...@gmail.com> > >>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks for volunteering, Mickael! > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Wed, Feb 8, 2023 at 1:12 PM José Armando García > >>>>> Sancio > >>>>>>>>>>>>>>>>>>> <jsan...@confluent.io.invalid> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks for volunteering Mickael. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>> -José > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>> > >>>>> > >> > >
