Hi Colin, Thanks for the KIP. AS1: I think that the KIP is not going to prove that useful because of the introduction of new features as the recent releases have occurred. I'll give a couple of examples.
ELR was introduced as a preview in AK 4.0 and a feature enabled by default in AK 4.1. The AK 4.0 binaries understand how to do ELR, but earlier versions would not. Doesn't that mean that it would be disallowed to downgrade the MV beyond 4.0 if ELR had been enabled on the cluster? We have a similar situation occurring with KIP-932 in the future (preview in AK 4.1 using a feature, enabled by default in AK 4.2). And again with KIP-1071 (preview in AK 4.2 using a feature, enabled by default in AK 4.3, I believe is the plan). Similarly, would we need to be able to re-enable the old group coordinator even though it doesn't understand the new records on __consumer_offsets? So, my point is that the scope of downgrades which are safe or even possible is very limited and the prerequisites of features which we already have defined mean that we might need to disable features which were previously defaulted on, and potentially clean up behind them so that they properly re-initialize if the cluster is subsequently upgraded back to a point where the features are re-enabled. I worry that we are making a combinatorial testing situation which will be expensive. Thanks, Andrew ________________________________________ From: Colin McCabe <cmcc...@apache.org> Sent: 08 April 2025 20:46 To: dev@kafka.apache.org <dev@kafka.apache.org> Subject: Re: KIP-1155: Metadata Version Downgrades [DISCUSS] On Tue, Apr 8, 2025, at 09:55, Justine Olshan wrote: > Hey PoAn, > > Just wanted to make a clarification. TV2 doesn't need to be downgraded if > the MV is lower than the bootstrap MV for the TV. > The bootstrap MV is only used when the cluster is created to get the > default feature versions. As long as the TV version is a production version > for the Kafka release, you can change MV to whatever you choose. It is > decoupled from the MV at this point. > > Thanks, > Justine > Hi Justine, Thanks for the clarification. Yes, the "default versions" of features don't necessarily imply hard dependencies. But if there are hard dependencies (where a feature needs a certain MV), we will enforce that. best, Colin > On Tue, Apr 8, 2025 at 2:01 AM PoAn Yang <yangp...@gmail.com> wrote: > >> Hi Colin, >> >> Thanks for the KIP. It’s good to give users more flexibility to downgrade >> metadata.version. >> >> PY1: Each feature has bootstrapMetadataVersion. For example, TV_2 is >> IBP_4_0_IV2. >> Assume that a cluster uses TV_2 and metadata.version 4.0-IV2. If a user >> downgrades >> metadata.version to 3.9-IV0, will the controller check TransactionVersion >> and reject >> the request? >> >> PY2: After users downgrade metadata.version, can they also restart server >> with old binary >> version? If they can, we need to add related end-to-end ducktape test. >> >> Thanks, >> PoAn >> >> > On Apr 8, 2025, at 6:04 AM, Colin McCabe <cmcc...@apache.org> wrote: >> > >> > Hi all, >> > >> > I posted a new KIP about support metadata.version downgrades. Take a >> look here: >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1155%3A+Metadata+Version+Downgrades >> > >> > thanks, >> > Colin >> >>
