Looks like this KIP has gone quiet for a bit, but here with an update
(maybe this will revitalize the conversation too :) )
For KIP-1022 we want to introduce more features and will have a config `
unstable.version.enable`
I know that this KIP proposed a config `unstable.metadata.version.enable`
Hi, Colin,
Thanks for the reply.
"If Y changed the same file as X, we can just edit the file so that Y's
change come first."
It seems the effort is more than just editing the file. One has to change
all the logic around the old IV. Also, what happens to the client? A client
may have implemented
On Tue, Jan 23, 2024, at 11:21, Jun Rao wrote:
> Hi, Proven,
>
> Thanks for the KIP.
>
> I am not sure about the reordering approach proposed in the KIP. Let's say
> in a release we have features X and Y, depending on MV IV1 and IV2,
> respectively. At the release time, feature Y is ready, but X
Hi, Proven,
Thanks for the KIP.
I am not sure about the reordering approach proposed in the KIP. Let's say
in a release we have features X and Y, depending on MV IV1 and IV2,
respectively. At the release time, feature Y is ready, but X is not. I
guess the proposal is to move IV1 to a new MV IV3?
Hi Colin,
> I think feature flags are somewhat orthogonal to the stable / unstable
discussion
I think feature flags can be used as an alternative to achieve similar
results as stable / unstable functionality. As well as long-lived feature
branches. In my experience, I've seen feature flags to
On Fri, Jan 12, 2024, at 11:32, Artem Livshits wrote:
> I think using feature flags (whether we support a framework and tooling for
> feature flags or just an ad-hoc XyzEnabled flag) can be an alternative to
> this KIP. I think the value of this KIP is that it's trying to propose a
> systemic
I think using feature flags (whether we support a framework and tooling for
feature flags or just an ad-hoc XyzEnabled flag) can be an alternative to
this KIP. I think the value of this KIP is that it's trying to propose a
systemic approach for gating functionality that may take multiple releases
Hi Ismael,
I debated including something about feature flags in my last comment, but
maybe I should have.
What you said makes sense.
Justine
On Fri, Jan 12, 2024 at 9:31 AM Ismael Juma wrote:
> Justine,
>
> For features that are not production-ready, they should have an additional
>
Justine,
For features that are not production-ready, they should have an additional
configuration (not the metadata version) that enables/disables it. The MV
specific features we ship are something we have to support and make sure we
don't break going forward.
Ismael
On Fri, Jan 12, 2024 at
Hi Ismael,
I think the concern I have about a MV for a feature that is not production
ready is that it blocks any development/features with higher MV versions
that could be production ready.
I do see your point though. Previously MV/IBP was about pure broker
compatibility and not about the
Hi,
Thanks for the KIP.
Reading the discussion, I think a lot of the confusion is due to the
"unstable" naming. People are then trying to figure out when we think
something is stable in the "this is battle-tested" sense. But this flag
should not be about that. We can have an MV for a feature
I also agree with 2 & 2 with reasoning along the same lines as Artem.
Thanks,
Andrew
> On 12 Jan 2024, at 09:15, Federico Valeri wrote:
>
> On Thu, Jan 11, 2024 at 10:43 PM Artem Livshits
> wrote:
>>
>> Hi Proven,
>>
>> I'd say that we should do 2 & 2. The idea is that for small features
On Thu, Jan 11, 2024 at 10:43 PM Artem Livshits
wrote:
>
> Hi Proven,
>
> I'd say that we should do 2 & 2. The idea is that for small features that
> can be done and stabilized within a short period of time (with one or very
> few commits) that's exactly what happens -- people interested in
Hi Proven,
I'd say that we should do 2 & 2. The idea is that for small features that
can be done and stabilized within a short period of time (with one or very
few commits) that's exactly what happens -- people interested in testing
in-progress feature could take unstable code from a patch (or
We have two approaches here for how we update unstable metadata versions.
1. The update will only increase MVs of unstable features to a value
greater than the new stable feature. The idea is that a specific unstable
MV may support some set of features and in the future that set is
Hi Federico,
Thank you for the suggestions. I've added them to the KIP.
--Proven
On Wed, Jan 10, 2024 at 4:16 AM Federico Valeri
wrote:
> Hi folks,
>
> > If you use an unstable MV, you probably won't be able to upgrade your
> software. Because whenever something changes, you'll probably get
>
On Wed, Jan 10, 2024, at 09:16, Justine Olshan wrote:
> Hmm it seems like Colin and Proven are disagreeing with whether we can swap
> unstable metadata versions.
>
>> When we reorder, we are always allocating a new MV and we are never
> reusing an existing MV even if it was also unstable.
>
>>
Hmm it seems like Colin and Proven are disagreeing with whether we can swap
unstable metadata versions.
> When we reorder, we are always allocating a new MV and we are never
reusing an existing MV even if it was also unstable.
> Given that this is true, there's no reason to have special rules
Hi folks,
> If you use an unstable MV, you probably won't be able to upgrade your
> software. Because whenever something changes, you'll probably get
> serialization exceptions being thrown inside the controller. Fatal ones.
Thanks for this clarification. I think this concrete risk should be
On Tue, Jan 9, 2024, at 18:56, Proven Provenzano wrote:
> Hi folks,
>
> Thank you for the questions.
>
> Let me clarify about reorder first. The reorder of unstable metadata
> versions should be infrequent.
Why does it need to be infrequent? We should be able to reorder unstable
metadata
Hi folks,
Thank you for the questions.
Let me clarify about reorder first. The reorder of unstable metadata
versions should be infrequent. The time you reorder is when a feature that
requires a higher metadata version to enable becomes "production ready" and
the features with unstable metadata
Hi Justine,
Yes, this is an important point to clarify. Proven can comment more, but my
understanding is that we can do anything to unstable metadata versions. Reorder
them, delete them, change them in any other way. There are no stability
guarantees. If the current text is unclear let's add
Hey Colin,
I had some offline discussions with Proven previously and it seems like he
said something different so I'm glad I brought it up here.
Let's clarify if we are ok with reordering unstable metadata versions :)
Justine
On Mon, Jan 8, 2024 at 1:56 PM Colin McCabe wrote:
> On Mon, Jan
On Mon, Jan 8, 2024, at 13:19, Justine Olshan wrote:
> Hey all,
>
> I was wondering how often we plan to update LATEST_PRODUCTION metadata
> version. Is this something we should do as soon as the feature is complete
> or something we do when we are releasing kafka. When is the time we abandon
> a
Hey all,
I was wondering how often we plan to update LATEST_PRODUCTION metadata
version. Is this something we should do as soon as the feature is complete
or something we do when we are releasing kafka. When is the time we abandon
a MV so that other features can be unblocked?
I am just
Hi Proven,
Thanks for the KIP. I think there is a need for this capability, for those of
us who deploy from trunk (or branches dervied from trunk).
With regard to "unstable.metadata.versions.enable": is this going to be a
documented configuration, or an internal one? I am guessing we want it
Hey folks,
I am starting a discussion thread for managing unstable metadata versions
in Apache Kafka.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1014%3A+Managing+Unstable+Metadata+Versions+in+Apache+Kafka
This KIP is actually already implemented in 3.7 with PR
27 matches
Mail list logo
