Hey Colin, I had some offline discussions with Proven previously and it seems like he said something different so I'm glad I brought it up here.
Let's clarify if we are ok with reordering unstable metadata versions :) Justine On Mon, Jan 8, 2024 at 1:56 PM Colin McCabe <cmcc...@apache.org> wrote: > On Mon, Jan 8, 2024, at 13:19, Justine Olshan wrote: > > Hey all, > > > > I was wondering how often we plan to update LATEST_PRODUCTION metadata > > version. Is this something we should do as soon as the feature is > complete > > or something we do when we are releasing kafka. When is the time we > abandon > > a MV so that other features can be unblocked? > > Hi Justine, > > Thanks for reviewing. > > The idea is that you should bump LATEST_PRODUCTION when you want to take a > feature to production. That could mean deploying it internally somewhere to > production, or doing an Apache release that lets everyone deploy the thing > to production. > > Not in production? No need to care about this. Make any changes you like. > > As a corollary, we should keep the LATEST_PRODUCTION version as low as it > can be. If you haven't tested the feature, don't freeze it in stone yet. > > > > > I am just considering a feature that may end up missing a release. It > seems > > like maybe that MV would block future metadata versions until we decide > the > > feature won't make the cut. From that point, all "ready" features should > be > > able to be released. > > The intention is the opposite. A feature in an unstable metadata version > doesn't block anything. You can always move a feature from one unstable > metadata version to another if the feature starts taking too long to finish. > > > I'm also wondering if the KIP should include some information about how a > > metadata should be abandoned. Maybe there is a specific message to write > in > > the file? So folks who were maybe waiting on that version know they can > > release their feature? > > > > I am also assuming that we don't shift all the waiting metadata versions > > when we abandon a version, but it would be good to clarify and include in > > the KIP. > > I'm not sure what you mean by abandoning a version. We never abandon a > version once it's stable. > > Unstable versions can change. I wouldn't describe this as "abandonment", > just the MV changing prior to release. > > In a similar way, the contents of the 3.7 branch will change up until > 3.7.0 is released. Once it gets released, it's never unreleased. We just > move on to 3.7.1. Same thing here. > > best, > Colin > > > > > Thanks, > > > > Justine > > > > On Mon, Jan 8, 2024 at 12:44 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Hi Proven, > >> > >> Thanks for the KIP. I think there is a need for this capability, for > those > >> of us who deploy from trunk (or branches dervied from trunk). > >> > >> With regard to "unstable.metadata.versions.enable": is this going to be > a > >> documented configuration, or an internal one? I am guessing we want it > to > >> be documented, so that users can use it. If we do, we should probably > also > >> very prominently warn that THIS WILL BREAK UPGRADES FOR YOUR CLUSTER. > That > >> includes logging an ERROR message on startup, etc. > >> > >> It would be good to document if a release can go out that contains > "future > >> MVs" that are unstable. Like can we make a 3.8 release that contains > >> IBP_4_0_IV0 in MetadataVersion.java, as an unstable future MV? > Personally I > >> think the answer should be "yes," but with the usual caveats. When the > >> actual 4.0 comes out, the unstable 4.0 MV that shipped in 3.8 probably > >> won't work, and you won't be able to upgrade. (It was unstable, we told > you > >> not to use it.) > >> > >> best, > >> Colin > >> > >> > >> On Fri, Jan 5, 2024, at 07:32, Proven Provenzano wrote: > >> > Hey folks, > >> > > >> > I am starting a discussion thread for managing unstable metadata > >> > versions > >> > in Apache Kafka. > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1014%3A+Managing+Unstable+Metadata+Versions+in+Apache+Kafka > >> > > >> > This KIP is actually already implemented in 3.7 with PR > >> > https://github.com/apache/kafka/pull/14860. > >> > I have created this KIP to explain the motivation and how managing > >> Metadata > >> > Versions is expected to work. > >> > Comments are greatly appreciated as this process can always be > improved. > >> > > >> > -- > >> > --Proven > >> >
