Hi All, Whoever has KIP entries in the 3.6.0 release plan. Please update it with the latest status by tomorrow(end of the day 29th Jul UTC ).
Thanks Satish. On Fri, 28 Jul 2023 at 12:01, Satish Duggana <satish.dugg...@gmail.com> wrote: > > Thanks Ismael and Divij for the suggestions. > > One way was to follow the earlier guidelines that we set for any early > access release. It looks Ismael already mentioned the example of > KRaft. > > KIP-405 mentions upgrade/downgrade and limitations sections. We can > clarify that in the release notes for users on how this feature can be > used for early access. > > Divij, We do not want users to enable this feature on production > environments in early access release. Let us work together on the > followups Ismael suggested. > > ~Satish. > > On Fri, 28 Jul 2023 at 02:24, Divij Vaidya <divijvaidy...@gmail.com> wrote: > > > > Those are great suggestions, thank you. We will continue this discussion > > forward in a separate KIP for release plan for Tiered Storage. > > > > On Thu 27. Jul 2023 at 21:46, Ismael Juma <m...@ismaeljuma.com> wrote: > > > > > Hi Divij, > > > > > > I think the points you bring up for discussion are all good. My main > > > feedback is that they should be discussed in the context of KIPs vs the > > > release template. That's why we have a backwards compatibility section for > > > every KIP, it's precisely to ensure we think carefully about some of the > > > points you're bringing up. When it comes to defining the meaning of early > > > access, we have two options: > > > > > > 1. Have a KIP specifically for tiered storage. > > > 2. Have a KIP to define general guidelines for what early access means. > > > > > > Does this make sense? > > > > > > Ismael > > > > > > On Thu, Jul 27, 2023 at 6:38 PM Divij Vaidya <divijvaidy...@gmail.com> > > > wrote: > > > > > > > Thank you for the response, Ismael. > > > > > > > > 1. Specifically in context of 3.6, I wanted this compatibility > > > > guarantee point to encourage a discussion on > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-952%3A+Regenerate+segment-aligned+producer+snapshots+when+upgrading+to+a+Kafka+version+supporting+Tiered+Storage > > > > . > > > > Due to lack of producer snapshots in <2.8 versions, a customer may not > > > > be able to upgrade to 3.6 and use TS on a topic which was created when > > > > the cluster was on <2.8 version (see motivation for details). We can > > > > discuss and agree that it does not break compatibility, which is fine. > > > > But I want to ensure that we have a discussion soon on this to reach a > > > > conclusion. > > > > > > > > 2. I will start a KIP on this for further discussion. > > > > > > > > 3. In the context of 3.6, this would mean that there should be > > > > no-regression, if a user does "not" turn-on remote storage (early > > > > access feature) at a cluster level. We have some known cases (such as > > > > https://issues.apache.org/jira/browse/KAFKA-15189) which violate this > > > > compatibility requirement. Having this guarantee mentioned in the > > > > release plan will ensure that we are all in agreement with which cases > > > > are truly blockers and which aren't. > > > > > > > > 4. Fair, instead of a general goal, let me put it specifically in the > > > > context of 3.6. Let me know if this is not the right forum for this > > > > discussion. > > > > Once a user "turns on" tiered storage (TS) at a cluster level, I am > > > > proposing that they should have the ability to turn it off as well at > > > > a cluster level. Since this is a topic level feature, folks may not > > > > spin up a separate cluster to try this feature, hence, we need to > > > > ensure that we provide them with the ability to try tiered storage for > > > > a topic which could be deleted and featured turned-off, so that rest > > > > of the production cases are not impacted. > > > > > > > > 5. Agree on not making public interface change as a requirement but we > > > > should define what "early access" means in that case. Users may not be > > > > aware that "early access" public APIs may change (unless I am missing > > > > some documentation somewhere completely, in which case I apologize for > > > > bringing this naive point). > > > > > > > > -- > > > > Divij Vaidya > > > > > > > > On Thu, Jul 27, 2023 at 2:27 PM Ismael Juma <m...@ismaeljuma.com> wrote: > > > > > > > > > > Hi Divij, > > > > > > > > > > Some of these are launch checklist items (not really goals) and some > > > are > > > > > compatibility guarantees. More below. > > > > > > > > > > On Thu, Jul 27, 2023, 12:10 PM Divij Vaidya <divijvaidy...@gmail.com> > > > > wrote: > > > > > > > > > > > Hey Satish > > > > > > > > > > > > Could we consider adding "launch goals" in the release plan. While > > > > > > some of these may be implicit, it would be nice to list them down in > > > > > > the release plan. For this release, our launch requirements would > > > > > > be: > > > > > > 1. Users should be able to upgrade from any prior Kafka version to > > > this > > > > > > version. > > > > > > > > > > > > > > > > This is part of the compatibility guarantees. The upgrade notes > > > > > mention > > > > > this already. If there is a change in a given release, it should > > > > definitely > > > > > be highlighted. > > > > > > > > > > 2. On release, this version (or it's dependencies) would not have any > > > > > > known MEDIUM/HIGH CVE. > > > > > > > > > > > > > > > > This is a new policy and the details should be discussed. In > > > particular, > > > > > the threshold (medium or high). > > > > > > > > > > 3. Presence of any "early access"/"beta" feature should not impact > > > > > > other production features when it is not enabled. > > > > > > > > > > > > > > > > This is a general guideline for early access features and not specific > > > to > > > > > this release. It would be good to have a page that talks about these > > > > things. > > > > > > > > > > 4. Once enabled, users should have an option to disable any "early > > > > > > access"/"beta" feature and resume normal production features, i.e. > > > > > > impact of beta features should be reversible. > > > > > > > > > > > > > > > > This needs discussion and I don't think it's reasonable as a general > > > > rule. > > > > > For example, Kraft early access wasn't reversible and it was not > > > feasible > > > > > for it to be. > > > > > > > > > > 5. KIP-405 will be available in "early access"/"beta" mode. Early > > > > > > access/beta means that the public facing interfaces won't change in > > > > > > future but the implementation is not recommended to be used in > > > > > > production. > > > > > > > > > > > > > > > I don't think it's ok to make this a requirement. Early access is a > > > > > way > > > > to > > > > > get early feedback and all types of changes should be on the table. > > > They > > > > > would be discussed via KIPs as usual. I believe there were some > > > > > incompatible changes for Kraft during the early access period although > > > > the > > > > > team aimed to minimize work required during upgrades. I have mentioned > > > > > Kraft a couple of times since it's a good example of a large feature > > > that > > > > > went through this process. > > > > > > > > > > Ismael > > > > > > > > > -- > > Divij Vaidya
