Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Chris Egerton]

Thanks Ziming, LGTM!

On Mon, Jan 15, 2024 at 12:00 AM ziming deng 
wrote:

> Hello Luke,
>
> thank you for finding this error, I have rectified it, and I will start a
> vote process soon.
>
> --
> Best,
> Ziming
>
>
> > On Jan 12, 2024, at 16:32, Luke Chen  wrote:
> >
> > Hi Ziming,
> >
> > Thanks for the KIP!
> > LGTM!
> > Using incremental by defaul and fallback automatically if it's not
> > supported is a good idea!
> >
> > One minor comment:
> > 1. so I'm inclined to move it to incrementalAlterConfigs  and "provide a
> > flag" to still use alterConfigs  for new client to interact with old
> > servers.
> > I don't think we will "provide any flag" after the discussion. We should
> > remove it.
> >
> > Thanks.
> > Luke
> >
> > On Fri, Jan 12, 2024 at 12:29 PM ziming deng  >
> > wrote:
> >
> >> Thank you for your clarification, Chris,
> >>
> >> I have spent some time to review KIP-894 and I think it's automatic way
> is
> >> better and bring no side effect, and I will also adopt this way here.
> >> As you mentioned, the changes in semantics is minor, the most important
> >> reason for this change is fixing bug brought by sensitive configs.
> >>
> >>
> >>> We
> >>> don't appear to support appending/subtracting from list properties via
> >> the
> >>> CLI for any other entity type right now,
> >> You are right about this, I tried and found that we can’t subtract or
> >> append configs, I will change the KIP to "making way for
> >> appending/subtracting list properties"
> >>
> >> --
> >> Best,
> >> Ziming
> >>
> >>> On Jan 6, 2024, at 01:34, Chris Egerton 
> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> Can we clarify any changes in the user-facing semantics for the CLI
> tool
> >>> that would come about as a result of this KIP? I think the debate over
> >> the
> >>> necessity of an opt-in flag, or waiting for 4.0.0, ultimately boils
> down
> >> to
> >>> this.
> >>>
> >>> My understanding is that the only changes in semantics are fairly minor
> >>> (semantic versioning pun intended):
> >>>
> >>> - Existing sensitive broker properties no longer have to be explicitly
> >>> specified on the command line if they're not being changed
> >>> - A small race condition is fixed where the broker config is updated
> by a
> >>> separate operation in between when the CLI reads the existing broker
> >> config
> >>> and writes the new broker config
> >>> - Usage of a new broker API that has been supported since version
> 2.3.0,
> >>> but which does not require any new ACLs and does not act any
> differently
> >>> apart from the two small changes noted above
> >>>
> >>> If this is correct, then I'm inclined to agree with Ismael's suggestion
> >> of
> >>> starting with incrementalAlterConfigs, and falling back on alterConfigs
> >> if
> >>> the former is not supported by the broker, and do not believe it's
> >>> necessary to wait for 4.0.0 or provide opt-in or opt-out flags to
> release
> >>> this change. This would also be similar to changes we made to
> >> MirrorMaker 2
> >>> in KIP-894 [1], where the default behavior for syncing topic configs is
> >> now
> >>> to start with incrementalAlterConfigs and fall back on alterConfigs if
> >> it's
> >>> not supported.
> >>>
> >>> If there are other, more significant changes to the user-facing
> semantics
> >>> for the CLI, then these should be called out here and in the KIP, and
> we
> >>> might consider a more cautious approach.
> >>>
> >>> [1] -
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
> >>>
> >>>
> >>> Also, regarding this part of the KIP:
> >>>
>  incrementalAlterConfigs is more convenient especially for updating
> >>> configs of list data type, such as
> >> "leader.replication.throttled.replicas"
> >>>
> >>> While this is true for the Java admin client and the corresponding
> broker
> >>> APIs, it doesn't appear to be relevant to the kafka-configs.sh CLI
> tool.
> >> We
> >>> don't appear to support appending/subtracting from list properties via
> >> the
> >>> CLI for any other entity type right now, and there's nothing in the KIP
> >>> that leads me to believe we'd be adding it for broker configs.
> >>>
> >>> Cheers,
> >>>
> >>> Chris
> >>>
> >>> On Thu, Jan 4, 2024 at 10:12 PM ziming deng  >> >
> >>> wrote:
> >>>
>  Hi Ismael,
>  I added this automatically approach to “Rejected alternatives”
> >> concerning
>  that we need to unify the semantics between alterConfigs and
>  incrementalAlterConfigs, so I choose to give this privilege to users.
> 
>  After reviewing these code and doing some tests I found that they
>  following the similar approach, I think the simplest way is to let the
>  client choose the best method heuristically.
> 
>  Thank you for pointing out this, I will change the KIP later.
> 
>  Best,
>  Ziming
> 
> > On Jan 4, 2024, at 17:28, 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Hello Luke,

thank you for finding this error, I have rectified it, and I will start a vote 
process soon.

--
Best,
Ziming


> On Jan 12, 2024, at 16:32, Luke Chen  wrote:
> 
> Hi Ziming,
> 
> Thanks for the KIP!
> LGTM!
> Using incremental by defaul and fallback automatically if it's not
> supported is a good idea!
> 
> One minor comment:
> 1. so I'm inclined to move it to incrementalAlterConfigs  and "provide a
> flag" to still use alterConfigs  for new client to interact with old
> servers.
> I don't think we will "provide any flag" after the discussion. We should
> remove it.
> 
> Thanks.
> Luke
> 
> On Fri, Jan 12, 2024 at 12:29 PM ziming deng  >
> wrote:
> 
>> Thank you for your clarification, Chris,
>> 
>> I have spent some time to review KIP-894 and I think it's automatic way is
>> better and bring no side effect, and I will also adopt this way here.
>> As you mentioned, the changes in semantics is minor, the most important
>> reason for this change is fixing bug brought by sensitive configs.
>> 
>> 
>>> We
>>> don't appear to support appending/subtracting from list properties via
>> the
>>> CLI for any other entity type right now,
>> You are right about this, I tried and found that we can’t subtract or
>> append configs, I will change the KIP to "making way for
>> appending/subtracting list properties"
>> 
>> --
>> Best,
>> Ziming
>> 
>>> On Jan 6, 2024, at 01:34, Chris Egerton  wrote:
>>> 
>>> Hi all,
>>> 
>>> Can we clarify any changes in the user-facing semantics for the CLI tool
>>> that would come about as a result of this KIP? I think the debate over
>> the
>>> necessity of an opt-in flag, or waiting for 4.0.0, ultimately boils down
>> to
>>> this.
>>> 
>>> My understanding is that the only changes in semantics are fairly minor
>>> (semantic versioning pun intended):
>>> 
>>> - Existing sensitive broker properties no longer have to be explicitly
>>> specified on the command line if they're not being changed
>>> - A small race condition is fixed where the broker config is updated by a
>>> separate operation in between when the CLI reads the existing broker
>> config
>>> and writes the new broker config
>>> - Usage of a new broker API that has been supported since version 2.3.0,
>>> but which does not require any new ACLs and does not act any differently
>>> apart from the two small changes noted above
>>> 
>>> If this is correct, then I'm inclined to agree with Ismael's suggestion
>> of
>>> starting with incrementalAlterConfigs, and falling back on alterConfigs
>> if
>>> the former is not supported by the broker, and do not believe it's
>>> necessary to wait for 4.0.0 or provide opt-in or opt-out flags to release
>>> this change. This would also be similar to changes we made to
>> MirrorMaker 2
>>> in KIP-894 [1], where the default behavior for syncing topic configs is
>> now
>>> to start with incrementalAlterConfigs and fall back on alterConfigs if
>> it's
>>> not supported.
>>> 
>>> If there are other, more significant changes to the user-facing semantics
>>> for the CLI, then these should be called out here and in the KIP, and we
>>> might consider a more cautious approach.
>>> 
>>> [1] -
>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
>>> 
>>> 
>>> Also, regarding this part of the KIP:
>>> 
 incrementalAlterConfigs is more convenient especially for updating
>>> configs of list data type, such as
>> "leader.replication.throttled.replicas"
>>> 
>>> While this is true for the Java admin client and the corresponding broker
>>> APIs, it doesn't appear to be relevant to the kafka-configs.sh CLI tool.
>> We
>>> don't appear to support appending/subtracting from list properties via
>> the
>>> CLI for any other entity type right now, and there's nothing in the KIP
>>> that leads me to believe we'd be adding it for broker configs.
>>> 
>>> Cheers,
>>> 
>>> Chris
>>> 
>>> On Thu, Jan 4, 2024 at 10:12 PM ziming deng > >
>>> wrote:
>>> 
 Hi Ismael,
 I added this automatically approach to “Rejected alternatives”
>> concerning
 that we need to unify the semantics between alterConfigs and
 incrementalAlterConfigs, so I choose to give this privilege to users.
 
 After reviewing these code and doing some tests I found that they
 following the similar approach, I think the simplest way is to let the
 client choose the best method heuristically.
 
 Thank you for pointing out this, I will change the KIP later.
 
 Best,
 Ziming
 
> On Jan 4, 2024, at 17:28, Ismael Juma  wrote:
> 
> Hi Ziming,
> 
> Why is the flag required at all? Can we use incremental and fallback
 automatically if it's not supported by the broker? At this point, the
>> vast
 majority of clusters should support it.
> 
> Ismael
> 
> On Mon, Dec 18, 2023 at 7:58 PM ziming deng >>> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Luke Chen]

Hi Ziming,

Thanks for the KIP!
LGTM!
Using incremental by defaul and fallback automatically if it's not
supported is a good idea!

One minor comment:
1. so I'm inclined to move it to incrementalAlterConfigs  and "provide a
flag" to still use alterConfigs  for new client to interact with old
servers.
I don't think we will "provide any flag" after the discussion. We should
remove it.

Thanks.
Luke

On Fri, Jan 12, 2024 at 12:29 PM ziming deng 
wrote:

> Thank you for your clarification, Chris,
>
> I have spent some time to review KIP-894 and I think it's automatic way is
> better and bring no side effect, and I will also adopt this way here.
> As you mentioned, the changes in semantics is minor, the most important
> reason for this change is fixing bug brought by sensitive configs.
>
>
> >  We
> > don't appear to support appending/subtracting from list properties via
> the
> > CLI for any other entity type right now,
> You are right about this, I tried and found that we can’t subtract or
> append configs, I will change the KIP to "making way for
> appending/subtracting list properties"
>
> --
> Best,
> Ziming
>
> > On Jan 6, 2024, at 01:34, Chris Egerton  wrote:
> >
> > Hi all,
> >
> > Can we clarify any changes in the user-facing semantics for the CLI tool
> > that would come about as a result of this KIP? I think the debate over
> the
> > necessity of an opt-in flag, or waiting for 4.0.0, ultimately boils down
> to
> > this.
> >
> > My understanding is that the only changes in semantics are fairly minor
> > (semantic versioning pun intended):
> >
> > - Existing sensitive broker properties no longer have to be explicitly
> > specified on the command line if they're not being changed
> > - A small race condition is fixed where the broker config is updated by a
> > separate operation in between when the CLI reads the existing broker
> config
> > and writes the new broker config
> > - Usage of a new broker API that has been supported since version 2.3.0,
> > but which does not require any new ACLs and does not act any differently
> > apart from the two small changes noted above
> >
> > If this is correct, then I'm inclined to agree with Ismael's suggestion
> of
> > starting with incrementalAlterConfigs, and falling back on alterConfigs
> if
> > the former is not supported by the broker, and do not believe it's
> > necessary to wait for 4.0.0 or provide opt-in or opt-out flags to release
> > this change. This would also be similar to changes we made to
> MirrorMaker 2
> > in KIP-894 [1], where the default behavior for syncing topic configs is
> now
> > to start with incrementalAlterConfigs and fall back on alterConfigs if
> it's
> > not supported.
> >
> > If there are other, more significant changes to the user-facing semantics
> > for the CLI, then these should be called out here and in the KIP, and we
> > might consider a more cautious approach.
> >
> > [1] -
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
> >
> >
> > Also, regarding this part of the KIP:
> >
> >> incrementalAlterConfigs is more convenient especially for updating
> > configs of list data type, such as
> "leader.replication.throttled.replicas"
> >
> > While this is true for the Java admin client and the corresponding broker
> > APIs, it doesn't appear to be relevant to the kafka-configs.sh CLI tool.
> We
> > don't appear to support appending/subtracting from list properties via
> the
> > CLI for any other entity type right now, and there's nothing in the KIP
> > that leads me to believe we'd be adding it for broker configs.
> >
> > Cheers,
> >
> > Chris
> >
> > On Thu, Jan 4, 2024 at 10:12 PM ziming deng  >
> > wrote:
> >
> >> Hi Ismael,
> >> I added this automatically approach to “Rejected alternatives”
> concerning
> >> that we need to unify the semantics between alterConfigs and
> >> incrementalAlterConfigs, so I choose to give this privilege to users.
> >>
> >> After reviewing these code and doing some tests I found that they
> >> following the similar approach, I think the simplest way is to let the
> >> client choose the best method heuristically.
> >>
> >> Thank you for pointing out this, I will change the KIP later.
> >>
> >> Best,
> >> Ziming
> >>
> >>> On Jan 4, 2024, at 17:28, Ismael Juma  wrote:
> >>>
> >>> Hi Ziming,
> >>>
> >>> Why is the flag required at all? Can we use incremental and fallback
> >> automatically if it's not supported by the broker? At this point, the
> vast
> >> majority of clusters should support it.
> >>>
> >>> Ismael
> >>>
> >>> On Mon, Dec 18, 2023 at 7:58 PM ziming deng  >> > wrote:
> 
>  Hello, I want to start a discussion on KIP-1011, to make the broker
> >> config change path unified with that of user/topic/client-metrics and
> avoid
> >> some bugs.
> 
>  Here is the link:
> 
>  KIP-1011: Use incrementalAlterConfigs when updating 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Thank you for your clarification, Chris,

I have spent some time to review KIP-894 and I think it's automatic way is 
better and bring no side effect, and I will also adopt this way here.
As you mentioned, the changes in semantics is minor, the most important reason 
for this change is fixing bug brought by sensitive configs.


>  We
> don't appear to support appending/subtracting from list properties via the
> CLI for any other entity type right now,
You are right about this, I tried and found that we can’t subtract or append 
configs, I will change the KIP to "making way for appending/subtracting list 
properties"

--
Best,
Ziming

> On Jan 6, 2024, at 01:34, Chris Egerton  wrote:
> 
> Hi all,
> 
> Can we clarify any changes in the user-facing semantics for the CLI tool
> that would come about as a result of this KIP? I think the debate over the
> necessity of an opt-in flag, or waiting for 4.0.0, ultimately boils down to
> this.
> 
> My understanding is that the only changes in semantics are fairly minor
> (semantic versioning pun intended):
> 
> - Existing sensitive broker properties no longer have to be explicitly
> specified on the command line if they're not being changed
> - A small race condition is fixed where the broker config is updated by a
> separate operation in between when the CLI reads the existing broker config
> and writes the new broker config
> - Usage of a new broker API that has been supported since version 2.3.0,
> but which does not require any new ACLs and does not act any differently
> apart from the two small changes noted above
> 
> If this is correct, then I'm inclined to agree with Ismael's suggestion of
> starting with incrementalAlterConfigs, and falling back on alterConfigs if
> the former is not supported by the broker, and do not believe it's
> necessary to wait for 4.0.0 or provide opt-in or opt-out flags to release
> this change. This would also be similar to changes we made to MirrorMaker 2
> in KIP-894 [1], where the default behavior for syncing topic configs is now
> to start with incrementalAlterConfigs and fall back on alterConfigs if it's
> not supported.
> 
> If there are other, more significant changes to the user-facing semantics
> for the CLI, then these should be called out here and in the KIP, and we
> might consider a more cautious approach.
> 
> [1] -
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations
> 
> 
> Also, regarding this part of the KIP:
> 
>> incrementalAlterConfigs is more convenient especially for updating
> configs of list data type, such as "leader.replication.throttled.replicas"
> 
> While this is true for the Java admin client and the corresponding broker
> APIs, it doesn't appear to be relevant to the kafka-configs.sh CLI tool. We
> don't appear to support appending/subtracting from list properties via the
> CLI for any other entity type right now, and there's nothing in the KIP
> that leads me to believe we'd be adding it for broker configs.
> 
> Cheers,
> 
> Chris
> 
> On Thu, Jan 4, 2024 at 10:12 PM ziming deng  >
> wrote:
> 
>> Hi Ismael,
>> I added this automatically approach to “Rejected alternatives” concerning
>> that we need to unify the semantics between alterConfigs and
>> incrementalAlterConfigs, so I choose to give this privilege to users.
>> 
>> After reviewing these code and doing some tests I found that they
>> following the similar approach, I think the simplest way is to let the
>> client choose the best method heuristically.
>> 
>> Thank you for pointing out this, I will change the KIP later.
>> 
>> Best,
>> Ziming
>> 
>>> On Jan 4, 2024, at 17:28, Ismael Juma  wrote:
>>> 
>>> Hi Ziming,
>>> 
>>> Why is the flag required at all? Can we use incremental and fallback
>> automatically if it's not supported by the broker? At this point, the vast
>> majority of clusters should support it.
>>> 
>>> Ismael
>>> 
>>> On Mon, Dec 18, 2023 at 7:58 PM ziming deng > > wrote:
 
 Hello, I want to start a discussion on KIP-1011, to make the broker
>> config change path unified with that of user/topic/client-metrics and avoid
>> some bugs.
 
 Here is the link:
 
 KIP-1011: Use incrementalAlterConfigs when updating broker configs by
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation
 cwiki.apache.org 
 
 <
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh>KIP-1011:
>> Use incrementalAlterConfigs when updating broker configs by
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation <
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh
>>> 
 cwiki.apache.org  <
>> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Chris Egerton]

Hi all,

Can we clarify any changes in the user-facing semantics for the CLI tool
that would come about as a result of this KIP? I think the debate over the
necessity of an opt-in flag, or waiting for 4.0.0, ultimately boils down to
this.

My understanding is that the only changes in semantics are fairly minor
(semantic versioning pun intended):

- Existing sensitive broker properties no longer have to be explicitly
specified on the command line if they're not being changed
- A small race condition is fixed where the broker config is updated by a
separate operation in between when the CLI reads the existing broker config
and writes the new broker config
- Usage of a new broker API that has been supported since version 2.3.0,
but which does not require any new ACLs and does not act any differently
apart from the two small changes noted above

If this is correct, then I'm inclined to agree with Ismael's suggestion of
starting with incrementalAlterConfigs, and falling back on alterConfigs if
the former is not supported by the broker, and do not believe it's
necessary to wait for 4.0.0 or provide opt-in or opt-out flags to release
this change. This would also be similar to changes we made to MirrorMaker 2
in KIP-894 [1], where the default behavior for syncing topic configs is now
to start with incrementalAlterConfigs and fall back on alterConfigs if it's
not supported.

If there are other, more significant changes to the user-facing semantics
for the CLI, then these should be called out here and in the KIP, and we
might consider a more cautious approach.

[1] -
https://cwiki.apache.org/confluence/display/KAFKA/KIP-894%3A+Use+incrementalAlterConfigs+API+for+syncing+topic+configurations


Also, regarding this part of the KIP:

> incrementalAlterConfigs is more convenient especially for updating
configs of list data type, such as "leader.replication.throttled.replicas"

While this is true for the Java admin client and the corresponding broker
APIs, it doesn't appear to be relevant to the kafka-configs.sh CLI tool. We
don't appear to support appending/subtracting from list properties via the
CLI for any other entity type right now, and there's nothing in the KIP
that leads me to believe we'd be adding it for broker configs.

Cheers,

Chris

On Thu, Jan 4, 2024 at 10:12 PM ziming deng 
wrote:

> Hi Ismael,
> I added this automatically approach to “Rejected alternatives” concerning
> that we need to unify the semantics between alterConfigs and
> incrementalAlterConfigs, so I choose to give this privilege to users.
>
> After reviewing these code and doing some tests I found that they
> following the similar approach, I think the simplest way is to let the
> client choose the best method heuristically.
>
> Thank you for pointing out this, I will change the KIP later.
>
> Best,
> Ziming
>
> > On Jan 4, 2024, at 17:28, Ismael Juma  wrote:
> >
> > Hi Ziming,
> >
> > Why is the flag required at all? Can we use incremental and fallback
> automatically if it's not supported by the broker? At this point, the vast
> majority of clusters should support it.
> >
> > Ismael
> >
> > On Mon, Dec 18, 2023 at 7:58 PM ziming deng  > wrote:
> >>
> >> Hello, I want to start a discussion on KIP-1011, to make the broker
> config change path unified with that of user/topic/client-metrics and avoid
> some bugs.
> >>
> >> Here is the link:
> >>
> >> KIP-1011: Use incrementalAlterConfigs when updating broker configs by
> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> >> cwiki.apache.org
> >>
> >>  <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh>KIP-1011:
> Use incrementalAlterConfigs when updating broker configs by
> kafka-configs.sh - Apache Kafka - Apache Software Foundation <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh
> >
> >> cwiki.apache.org <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh>
><
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh
> >
> >>
> >> Best,
> >> Ziming.
>
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Hi Ismael,
I added this automatically approach to “Rejected alternatives” concerning that 
we need to unify the semantics between alterConfigs and 
incrementalAlterConfigs, so I choose to give this privilege to users.

After reviewing these code and doing some tests I found that they following the 
similar approach, I think the simplest way is to let the client choose the best 
method heuristically.

Thank you for pointing out this, I will change the KIP later.

Best,
Ziming

> On Jan 4, 2024, at 17:28, Ismael Juma  wrote:
> 
> Hi Ziming,
> 
> Why is the flag required at all? Can we use incremental and fallback 
> automatically if it's not supported by the broker? At this point, the vast 
> majority of clusters should support it.
> 
> Ismael
> 
> On Mon, Dec 18, 2023 at 7:58 PM ziming deng  > wrote:
>> 
>> Hello, I want to start a discussion on KIP-1011, to make the broker config 
>> change path unified with that of user/topic/client-metrics and avoid some 
>> bugs.
>> 
>> Here is the link: 
>> 
>> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation
>> cwiki.apache.org
>> 
>>  
>> KIP-1011:
>>  Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
>> 
>> cwiki.apache.org 
>> 
>>  
>> 
>> 
>> Best, 
>> Ziming.

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Ismael Juma]

Hi Ziming,

Why is the flag required at all? Can we use incremental and fallback
automatically if it's not supported by the broker? At this point, the vast
majority of clusters should support it.

Ismael

On Mon, Dec 18, 2023 at 7:58 PM ziming deng 
wrote:

>
> Hello, I want to start a discussion on KIP-1011, to make the broker config
> change path unified with that of user/topic/client-metrics and avoid some
> bugs.
>
> Here is the link:
>
> KIP-1011: Use incrementalAlterConfigs when updating broker configs by
> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> 
> cwiki.apache.org
> 
> [image: favicon.ico]
> 
> 
>
> Best,
> Ziming.
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Federico Valeri]

On Fri, Dec 29, 2023 at 8:49 AM ziming deng  wrote:
>
> Hello all,
>
> We have 2 people prefer using "--enable-incremental" and it makes sense to 
> move the incompatible process to 4.X, I changed the default way to still use 
> `alterConfigs` and use `incrementalAlterConfigs` only when 
> "--enable-incremental" is specified.
>

I agree with this change, much better. Thanks.

> I will initiate a vote process if no further opinions coming.
>
> Best,
> Ziming
>
>
> > On Dec 26, 2023, at 13:21, Kamal Chandraprakash 
> >  wrote:
> >
> > Hi Ziming,
> >
> > Thanks for the KIP! The proposal LGTM.
> >
> > I'm also inclined towards option 2 (i.e. add an explicit
> > --enable-incremental flag in 3.X version)
> > to avoid any incompatible change in v3.X. As mentioned in this thread, many
> > users might be
> > using external tools to do the topic rebalance and apply throttle config on
> > the brokers, it might
> > break for them when the tools are upgraded before the broker binary.
> >
> > cruise-control is currently using kafka-version: 3.1
> > https://github.com/linkedin/cruise-control/blob/migrate_to_kafka_2_5/gradle.properties#L5
> >
> > On Mon, Dec 25, 2023 at 7:15 PM Divij Vaidya 
> > wrote:
> >
> >> Thank you for the summary, Ziming.
> >>
> >> Personally, I would prefer the latter i.e. having the incompatible change
> >> in 4.x instead of 3.x. This is because a major version upgrade goes through
> >> additional scrutiny by the users and usually comes with inevitable code
> >> changes required on the client. Hence, this incompatibility will be part of
> >> one amongst many changes that users will perform to upgrade to 4.x. This is
> >> unlike a major version change from 3.7 to 3.8 where users expect a simple
> >> upgrade without any code changes.
> >>
> >> Let's wait and hear what others think about this.
> >>
> >> --
> >> Divij Vaidya
> >>
> >>
> >>
> >> On Mon, Dec 25, 2023 at 1:18 PM ziming deng 
> >> wrote:
> >>
> >>> Hello Divij Vaidya,
> >>>
> >>> You are right that users should update existing scripts to add
> >>> ‘—disable-incremental’, and you mentioned another upgrade path which is
> >>> similar, the summary of the 2 schemes are:
> >>> we change existing scripts to use `incrementalAlterConfigs` and add
> >>> "--disable-incremental" flag for old servers in Kafka 3.X, and remove it
> >> in
> >>> Kafka 4.X.
> >>> we keep existing scripts unchanged and add an "--enable-incremental" flag
> >>> for new servers in Kafka 3.X, and remove it in Kafka 4.X.
> >>>
> >>> I think there will always be an incompatible upgrade process to move
> >>> default behavior from `alterConfigs` to `incrementalConfigs`. In the
> >> first
> >>> scheme we are doing this incompatible upgrade in Kafka 3.X, and in the
> >>> second scheme we are moving it to 4.X, I think we should do it as soon as
> >>> possible if it's inevitable.
> >>> However, I will add this to , and I'm open to this
> >>> if more people think it's more suitable.
> >>>
> >>>
> >>> --,
> >>> Ziming
> >>>
>  On Dec 22, 2023, at 18:13, Divij Vaidya 
> >> wrote:
> 
>  Divij Vaidya
> >>>
> >>>
> >>
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Hello all,

We have 2 people prefer using "--enable-incremental" and it makes sense to move 
the incompatible process to 4.X, I changed the default way to still use 
`alterConfigs` and use `incrementalAlterConfigs` only when 
"--enable-incremental" is specified.

I will initiate a vote process if no further opinions coming.

Best,
Ziming


> On Dec 26, 2023, at 13:21, Kamal Chandraprakash 
>  wrote:
> 
> Hi Ziming,
> 
> Thanks for the KIP! The proposal LGTM.
> 
> I'm also inclined towards option 2 (i.e. add an explicit
> --enable-incremental flag in 3.X version)
> to avoid any incompatible change in v3.X. As mentioned in this thread, many
> users might be
> using external tools to do the topic rebalance and apply throttle config on
> the brokers, it might
> break for them when the tools are upgraded before the broker binary.
> 
> cruise-control is currently using kafka-version: 3.1
> https://github.com/linkedin/cruise-control/blob/migrate_to_kafka_2_5/gradle.properties#L5
> 
> On Mon, Dec 25, 2023 at 7:15 PM Divij Vaidya 
> wrote:
> 
>> Thank you for the summary, Ziming.
>> 
>> Personally, I would prefer the latter i.e. having the incompatible change
>> in 4.x instead of 3.x. This is because a major version upgrade goes through
>> additional scrutiny by the users and usually comes with inevitable code
>> changes required on the client. Hence, this incompatibility will be part of
>> one amongst many changes that users will perform to upgrade to 4.x. This is
>> unlike a major version change from 3.7 to 3.8 where users expect a simple
>> upgrade without any code changes.
>> 
>> Let's wait and hear what others think about this.
>> 
>> --
>> Divij Vaidya
>> 
>> 
>> 
>> On Mon, Dec 25, 2023 at 1:18 PM ziming deng 
>> wrote:
>> 
>>> Hello Divij Vaidya,
>>> 
>>> You are right that users should update existing scripts to add
>>> ‘—disable-incremental’, and you mentioned another upgrade path which is
>>> similar, the summary of the 2 schemes are:
>>> we change existing scripts to use `incrementalAlterConfigs` and add
>>> "--disable-incremental" flag for old servers in Kafka 3.X, and remove it
>> in
>>> Kafka 4.X.
>>> we keep existing scripts unchanged and add an "--enable-incremental" flag
>>> for new servers in Kafka 3.X, and remove it in Kafka 4.X.
>>> 
>>> I think there will always be an incompatible upgrade process to move
>>> default behavior from `alterConfigs` to `incrementalConfigs`. In the
>> first
>>> scheme we are doing this incompatible upgrade in Kafka 3.X, and in the
>>> second scheme we are moving it to 4.X, I think we should do it as soon as
>>> possible if it's inevitable.
>>> However, I will add this to , and I'm open to this
>>> if more people think it's more suitable.
>>> 
>>> 
>>> --,
>>> Ziming
>>> 
 On Dec 22, 2023, at 18:13, Divij Vaidya 
>> wrote:
 
 Divij Vaidya
>>> 
>>> 
>> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Kamal Chandraprakash]

Hi Ziming,

Thanks for the KIP! The proposal LGTM.

I'm also inclined towards option 2 (i.e. add an explicit
--enable-incremental flag in 3.X version)
to avoid any incompatible change in v3.X. As mentioned in this thread, many
users might be
using external tools to do the topic rebalance and apply throttle config on
the brokers, it might
break for them when the tools are upgraded before the broker binary.

cruise-control is currently using kafka-version: 3.1
https://github.com/linkedin/cruise-control/blob/migrate_to_kafka_2_5/gradle.properties#L5

On Mon, Dec 25, 2023 at 7:15 PM Divij Vaidya 
wrote:

> Thank you for the summary, Ziming.
>
> Personally, I would prefer the latter i.e. having the incompatible change
> in 4.x instead of 3.x. This is because a major version upgrade goes through
> additional scrutiny by the users and usually comes with inevitable code
> changes required on the client. Hence, this incompatibility will be part of
> one amongst many changes that users will perform to upgrade to 4.x. This is
> unlike a major version change from 3.7 to 3.8 where users expect a simple
> upgrade without any code changes.
>
> Let's wait and hear what others think about this.
>
> --
> Divij Vaidya
>
>
>
> On Mon, Dec 25, 2023 at 1:18 PM ziming deng 
> wrote:
>
> > Hello Divij Vaidya,
> >
> > You are right that users should update existing scripts to add
> > ‘—disable-incremental’, and you mentioned another upgrade path which is
> > similar, the summary of the 2 schemes are:
> > we change existing scripts to use `incrementalAlterConfigs` and add
> > "--disable-incremental" flag for old servers in Kafka 3.X, and remove it
> in
> > Kafka 4.X.
> > we keep existing scripts unchanged and add an "--enable-incremental" flag
> > for new servers in Kafka 3.X, and remove it in Kafka 4.X.
> >
> > I think there will always be an incompatible upgrade process to move
> > default behavior from `alterConfigs` to `incrementalConfigs`. In the
> first
> > scheme we are doing this incompatible upgrade in Kafka 3.X, and in the
> > second scheme we are moving it to 4.X, I think we should do it as soon as
> > possible if it's inevitable.
> > However, I will add this to , and I'm open to this
> > if more people think it's more suitable.
> >
> >
> > --,
> > Ziming
> >
> > > On Dec 22, 2023, at 18:13, Divij Vaidya 
> wrote:
> > >
> > > Divij Vaidya
> >
> >
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Divij Vaidya]

Thank you for the summary, Ziming.

Personally, I would prefer the latter i.e. having the incompatible change
in 4.x instead of 3.x. This is because a major version upgrade goes through
additional scrutiny by the users and usually comes with inevitable code
changes required on the client. Hence, this incompatibility will be part of
one amongst many changes that users will perform to upgrade to 4.x. This is
unlike a major version change from 3.7 to 3.8 where users expect a simple
upgrade without any code changes.

Let's wait and hear what others think about this.

--
Divij Vaidya



On Mon, Dec 25, 2023 at 1:18 PM ziming deng 
wrote:

> Hello Divij Vaidya,
>
> You are right that users should update existing scripts to add
> ‘—disable-incremental’, and you mentioned another upgrade path which is
> similar, the summary of the 2 schemes are:
> we change existing scripts to use `incrementalAlterConfigs` and add
> "--disable-incremental" flag for old servers in Kafka 3.X, and remove it in
> Kafka 4.X.
> we keep existing scripts unchanged and add an "--enable-incremental" flag
> for new servers in Kafka 3.X, and remove it in Kafka 4.X.
>
> I think there will always be an incompatible upgrade process to move
> default behavior from `alterConfigs` to `incrementalConfigs`. In the first
> scheme we are doing this incompatible upgrade in Kafka 3.X, and in the
> second scheme we are moving it to 4.X, I think we should do it as soon as
> possible if it's inevitable.
> However, I will add this to , and I'm open to this
> if more people think it's more suitable.
>
>
> --,
> Ziming
>
> > On Dec 22, 2023, at 18:13, Divij Vaidya  wrote:
> >
> > Divij Vaidya
>
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Hello Divij Vaidya,

You are right that users should update existing scripts to add 
‘—disable-incremental’, and you mentioned another upgrade path which is 
similar, the summary of the 2 schemes are: 
we change existing scripts to use `incrementalAlterConfigs` and add 
"--disable-incremental" flag for old servers in Kafka 3.X, and remove it in 
Kafka 4.X.
we keep existing scripts unchanged and add an "--enable-incremental" flag for 
new servers in Kafka 3.X, and remove it in Kafka 4.X.

I think there will always be an incompatible upgrade process to move default 
behavior from `alterConfigs` to `incrementalConfigs`. In the first scheme we 
are doing this incompatible upgrade in Kafka 3.X, and in the second scheme we 
are moving it to 4.X, I think we should do it as soon as possible if it's 
inevitable.
However, I will add this to , and I'm open to this if 
more people think it's more suitable. 


--,
Ziming

> On Dec 22, 2023, at 18:13, Divij Vaidya  wrote:
> 
> Divij Vaidya

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Divij Vaidya]

Hi folks

Am I right in understanding that if a user is currently using this CLI to
update old versions (< 2.3.0), they will have to update their scripts and
operational tools when they upgrade ops tools / scripts from Kafka 3.6 to
3.8 (assuming this releases in 3.8)? In that case, may I suggest keeping
the default behaviour "as is" and instead introducing a
"--enable-incremental" flag to create the new behaviour? I am suggesting
this because ideally, we don't want users to require making a code changes
or a change to their operational tooling to perform a minor upgrade on a
Kafka version. From 4.0, we will of course, default to using
incrementalAlterConfig.

--
Divij Vaidya



On Fri, Dec 22, 2023 at 6:54 AM ziming deng 
wrote:

> +1 for adding them to rejected alternatives, These kafka-ui tools should
> also evolve with the iterations of Kafka.
>
> > On Dec 21, 2023, at 16:58, Николай Ижиков  wrote:
> >
> >> In fact alterConfig and incrementalAlterConfig have different
> semantics, we should pass all configs when using alterConfig and we can
> update config  incrementally using incrementalAlterConfigs, and is’t not
> worth doing so since alterConfig has been deprecated for a long time.
> >
> > There can be third-party tools like `kafka-ui` or similar that suffer
> from the same bug as you fixing.
> > If we fix `alterConfig` itself then we fix all tools, scripts that still
> using alterConfig.
> >
> > Anyway, let’s add to the «Rejected alternatives» section reasons - why
> we keep buggy method as is and fixing only tools.
> >
> >> I think your suggestion is nice, it should be marked as deprecated and
> will be removed together with `AdminClient.alterConfigs()`
> >
> > Is it OK to introduce option that is deprecated from the beginning?
> >
> >
> >> 21 дек. 2023 г., в 06:03, ziming deng  > написал(а):
> >>
> >>> shouldn't we also introduce --disable-incremental as deprecated?
> >>
> >> I think your suggestion is nice, it should be marked as deprecated and
> will be removed together with `AdminClient.alterConfigs()`
> >>
> >>
> >>> On Dec 19, 2023, at 16:36, Federico Valeri 
> wrote:
> >>>
> >>> HI Ziming, thanks for the KIP. Looks good to me.
> >>>
> >>> Just on question: given that alterConfig is deprecated, shouldn't we
> >>> also introduce --disable-incremental as deprecated? That way we would
> >>> get rid of both in Kafka 4.0. Also see:
> >>> https://issues.apache.org/jira/browse/KAFKA-14705.
> >>>
> >>> On Tue, Dec 19, 2023 at 9:05 AM ziming deng  > wrote:
> 
>  Thank you for mention this Ismael,
> 
>  I added this to the motivation section, and I think we can still
> update configs in this case by passing all sensitive configs, which is
> weird and not friendly.
> 
>  --
>  Best,
>  Ziming
> 
> > On Dec 19, 2023, at 14:24, Ismael Juma  m...@ismaeljuma.com>> wrote:
> >
> > Thanks for the KIP. I think one of the main benefits of the change
> isn't listed: sensitive configs make it impossible to make updates with the
> current cli tool because sensitive config values are never returned.
> >
> > Ismael
> >
> > On Mon, Dec 18, 2023 at 7:58 PM ziming deng <
> dengziming1...@gmail.com  dengziming1...@gmail.com> > wrote:
> >>
> >> Hello, I want to start a discussion on KIP-1011, to make the broker
> config change path unified with that of user/topic/client-metrics and avoid
> some bugs.
> >>
> >> Here is the link:
> >>
> >> KIP-1011: Use incrementalAlterConfigs when updating broker configs
> by kafka-configs.sh - Apache Kafka - Apache Software Foundation
> >> cwiki.apache.org  <
> http://cwiki.apache.org/>
> >>
> >> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh>KIP-1011:
> Use incrementalAlterConfigs when updating broker configs by
> kafka-configs.sh - Apache Kafka - Apache Software Foundation <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh
> >
> >> cwiki.apache.org  <
> http://cwiki.apache.org/> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh>
><
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1011%3A+Use+incrementalAlterConfigs+when+updating+broker+configs+by+kafka-configs.sh
> >
> >>
> >> Best,
> >> Ziming.
>
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

+1 for adding them to rejected alternatives, These kafka-ui tools should also 
evolve with the iterations of Kafka.

> On Dec 21, 2023, at 16:58, Николай Ижиков  wrote:
> 
>> In fact alterConfig and incrementalAlterConfig have different semantics, we 
>> should pass all configs when using alterConfig and we can update config  
>> incrementally using incrementalAlterConfigs, and is’t not worth doing so 
>> since alterConfig has been deprecated for a long time.
> 
> There can be third-party tools like `kafka-ui` or similar that suffer from 
> the same bug as you fixing.
> If we fix `alterConfig` itself then we fix all tools, scripts that still 
> using alterConfig.
> 
> Anyway, let’s add to the «Rejected alternatives» section reasons - why we 
> keep buggy method as is and fixing only tools.
> 
>> I think your suggestion is nice, it should be marked as deprecated and will 
>> be removed together with `AdminClient.alterConfigs()`
> 
> Is it OK to introduce option that is deprecated from the beginning?
> 
> 
>> 21 дек. 2023 г., в 06:03, ziming deng > > написал(а):
>> 
>>> shouldn't we also introduce --disable-incremental as deprecated?
>> 
>> I think your suggestion is nice, it should be marked as deprecated and will 
>> be removed together with `AdminClient.alterConfigs()`
>> 
>> 
>>> On Dec 19, 2023, at 16:36, Federico Valeri  wrote:
>>> 
>>> HI Ziming, thanks for the KIP. Looks good to me.
>>> 
>>> Just on question: given that alterConfig is deprecated, shouldn't we
>>> also introduce --disable-incremental as deprecated? That way we would
>>> get rid of both in Kafka 4.0. Also see:
>>> https://issues.apache.org/jira/browse/KAFKA-14705.
>>> 
>>> On Tue, Dec 19, 2023 at 9:05 AM ziming deng >> > wrote:
 
 Thank you for mention this Ismael,
 
 I added this to the motivation section, and I think we can still update 
 configs in this case by passing all sensitive configs, which is weird and 
 not friendly.
 
 --
 Best,
 Ziming
 
> On Dec 19, 2023, at 14:24, Ismael Juma  > wrote:
> 
> Thanks for the KIP. I think one of the main benefits of the change isn't 
> listed: sensitive configs make it impossible to make updates with the 
> current cli tool because sensitive config values are never returned.
> 
> Ismael
> 
> On Mon, Dec 18, 2023 at 7:58 PM ziming deng   
> > wrote:
>> 
>> Hello, I want to start a discussion on KIP-1011, to make the broker 
>> config change path unified with that of user/topic/client-metrics and 
>> avoid some bugs.
>> 
>> Here is the link:
>> 
>> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation
>> cwiki.apache.org  
>> 
>> KIP-1011:
>>  Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
>> 
>> cwiki.apache.org   
>> 
>>  
>> 
>> 
>> Best,
>> Ziming.

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Николай Ижиков]

> In fact alterConfig and incrementalAlterConfig have different semantics, we 
> should pass all configs when using alterConfig and we can update config  
> incrementally using incrementalAlterConfigs, and is’t not worth doing so 
> since alterConfig has been deprecated for a long time.

There can be third-party tools like `kafka-ui` or similar that suffer from the 
same bug as you fixing.
If we fix `alterConfig` itself then we fix all tools, scripts that still using 
alterConfig.

Anyway, let’s add to the «Rejected alternatives» section reasons - why we keep 
buggy method as is and fixing only tools.

> I think your suggestion is nice, it should be marked as deprecated and will 
> be removed together with `AdminClient.alterConfigs()`

Is it OK to introduce option that is deprecated from the beginning?


> 21 дек. 2023 г., в 06:03, ziming deng  написал(а):
> 
>> shouldn't we also introduce --disable-incremental as deprecated?
> 
> I think your suggestion is nice, it should be marked as deprecated and will 
> be removed together with `AdminClient.alterConfigs()`
> 
> 
>> On Dec 19, 2023, at 16:36, Federico Valeri  wrote:
>> 
>> HI Ziming, thanks for the KIP. Looks good to me.
>> 
>> Just on question: given that alterConfig is deprecated, shouldn't we
>> also introduce --disable-incremental as deprecated? That way we would
>> get rid of both in Kafka 4.0. Also see:
>> https://issues.apache.org/jira/browse/KAFKA-14705.
>> 
>> On Tue, Dec 19, 2023 at 9:05 AM ziming deng > > wrote:
>>> 
>>> Thank you for mention this Ismael,
>>> 
>>> I added this to the motivation section, and I think we can still update 
>>> configs in this case by passing all sensitive configs, which is weird and 
>>> not friendly.
>>> 
>>> --
>>> Best,
>>> Ziming
>>> 
 On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
 
 Thanks for the KIP. I think one of the main benefits of the change isn't 
 listed: sensitive configs make it impossible to make updates with the 
 current cli tool because sensitive config values are never returned.
 
 Ismael
 
 On Mon, Dec 18, 2023 at 7:58 PM ziming deng >>>  > wrote:
> 
> Hello, I want to start a discussion on KIP-1011, to make the broker 
> config change path unified with that of user/topic/client-metrics and 
> avoid some bugs.
> 
> Here is the link:
> 
> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> cwiki.apache.org 
> 
> KIP-1011:
>  Use incrementalAlterConfigs when updating broker configs by 
> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
> 
> cwiki.apache.org  
> 
>  
> 
> 
> Best,
> Ziming.
> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

> shouldn't we also introduce --disable-incremental as deprecated? 

I think your suggestion is nice, it should be marked as deprecated and will be 
removed together with `AdminClient.alterConfigs()`


> On Dec 19, 2023, at 16:36, Federico Valeri  wrote:
> 
> HI Ziming, thanks for the KIP. Looks good to me.
> 
> Just on question: given that alterConfig is deprecated, shouldn't we
> also introduce --disable-incremental as deprecated? That way we would
> get rid of both in Kafka 4.0. Also see:
> https://issues.apache.org/jira/browse/KAFKA-14705.
> 
> On Tue, Dec 19, 2023 at 9:05 AM ziming deng  > wrote:
>> 
>> Thank you for mention this Ismael,
>> 
>> I added this to the motivation section, and I think we can still update 
>> configs in this case by passing all sensitive configs, which is weird and 
>> not friendly.
>> 
>> --
>> Best,
>> Ziming
>> 
>>> On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
>>> 
>>> Thanks for the KIP. I think one of the main benefits of the change isn't 
>>> listed: sensitive configs make it impossible to make updates with the 
>>> current cli tool because sensitive config values are never returned.
>>> 
>>> Ismael
>>> 
>>> On Mon, Dec 18, 2023 at 7:58 PM ziming deng >>  > wrote:
 
 Hello, I want to start a discussion on KIP-1011, to make the broker config 
 change path unified with that of user/topic/client-metrics and avoid some 
 bugs.
 
 Here is the link:
 
 KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
 kafka-configs.sh - Apache Kafka - Apache Software Foundation
 cwiki.apache.org 
 
 KIP-1011:
  Use incrementalAlterConfigs when updating broker configs by 
 kafka-configs.sh - Apache Kafka - Apache Software Foundation 
 
 cwiki.apache.org  
 
  
 
 
 Best,
 Ziming.

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Hello, thanks for mention this.

In fact alterConfig and incrementalAlterConfig have different semantics, we 
should pass all configs when using alterConfig and we can update config  
incrementally using incrementalAlterConfigs, and is’t not worth doing so since 
alterConfig has been deprecated for a long time.

--,
Best, 
Ziming

> On Dec 19, 2023, at 16:40, Николай Ижиков  wrote:
> 
> Hello.
> 
> Can we just forward all invocations of alterConfig to incrementalAlterConfig?
> Transform parameters and call correct method.
> 
>> 19 дек. 2023 г., в 11:36, Federico Valeri  написал(а):
>> 
>> HI Ziming, thanks for the KIP. Looks good to me.
>> 
>> Just on question: given that alterConfig is deprecated, shouldn't we
>> also introduce --disable-incremental as deprecated? That way we would
>> get rid of both in Kafka 4.0. Also see:
>> https://issues.apache.org/jira/browse/KAFKA-14705.
>> 
>> On Tue, Dec 19, 2023 at 9:05 AM ziming deng  wrote:
>>> 
>>> Thank you for mention this Ismael,
>>> 
>>> I added this to the motivation section, and I think we can still update 
>>> configs in this case by passing all sensitive configs, which is weird and 
>>> not friendly.
>>> 
>>> --
>>> Best,
>>> Ziming
>>> 
 On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
 
 Thanks for the KIP. I think one of the main benefits of the change isn't 
 listed: sensitive configs make it impossible to make updates with the 
 current cli tool because sensitive config values are never returned.
 
 Ismael
 
 On Mon, Dec 18, 2023 at 7:58 PM ziming deng >>> > wrote:
> 
> Hello, I want to start a discussion on KIP-1011, to make the broker 
> config change path unified with that of user/topic/client-metrics and 
> avoid some bugs.
> 
> Here is the link:
> 
> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> cwiki.apache.org
> 
> KIP-1011:
>  Use incrementalAlterConfigs when updating broker configs by 
> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
> 
> cwiki.apache.org 
> 
>  
> 
> 
> Best,
> Ziming.
>>> 
> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Николай Ижиков]

Hello.

Can we just forward all invocations of alterConfig to incrementalAlterConfig?
Transform parameters and call correct method.

> 19 дек. 2023 г., в 11:36, Federico Valeri  написал(а):
> 
> HI Ziming, thanks for the KIP. Looks good to me.
> 
> Just on question: given that alterConfig is deprecated, shouldn't we
> also introduce --disable-incremental as deprecated? That way we would
> get rid of both in Kafka 4.0. Also see:
> https://issues.apache.org/jira/browse/KAFKA-14705.
> 
> On Tue, Dec 19, 2023 at 9:05 AM ziming deng  wrote:
>> 
>> Thank you for mention this Ismael,
>> 
>> I added this to the motivation section, and I think we can still update 
>> configs in this case by passing all sensitive configs, which is weird and 
>> not friendly.
>> 
>> --
>> Best,
>> Ziming
>> 
>>> On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
>>> 
>>> Thanks for the KIP. I think one of the main benefits of the change isn't 
>>> listed: sensitive configs make it impossible to make updates with the 
>>> current cli tool because sensitive config values are never returned.
>>> 
>>> Ismael
>>> 
>>> On Mon, Dec 18, 2023 at 7:58 PM ziming deng >> > wrote:
 
 Hello, I want to start a discussion on KIP-1011, to make the broker config 
 change path unified with that of user/topic/client-metrics and avoid some 
 bugs.
 
 Here is the link:
 
 KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
 kafka-configs.sh - Apache Kafka - Apache Software Foundation
 cwiki.apache.org
 
 KIP-1011:
  Use incrementalAlterConfigs when updating broker configs by 
 kafka-configs.sh - Apache Kafka - Apache Software Foundation 
 
 cwiki.apache.org 
 
  
 
 
 Best,
 Ziming.
>> 

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Federico Valeri]

HI Ziming, thanks for the KIP. Looks good to me.

Just on question: given that alterConfig is deprecated, shouldn't we
also introduce --disable-incremental as deprecated? That way we would
get rid of both in Kafka 4.0. Also see:
https://issues.apache.org/jira/browse/KAFKA-14705.

On Tue, Dec 19, 2023 at 9:05 AM ziming deng  wrote:
>
> Thank you for mention this Ismael,
>
> I added this to the motivation section, and I think we can still update 
> configs in this case by passing all sensitive configs, which is weird and not 
> friendly.
>
> --
> Best,
> Ziming
>
> > On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
> >
> > Thanks for the KIP. I think one of the main benefits of the change isn't 
> > listed: sensitive configs make it impossible to make updates with the 
> > current cli tool because sensitive config values are never returned.
> >
> > Ismael
> >
> > On Mon, Dec 18, 2023 at 7:58 PM ziming deng  > > wrote:
> >>
> >> Hello, I want to start a discussion on KIP-1011, to make the broker config 
> >> change path unified with that of user/topic/client-metrics and avoid some 
> >> bugs.
> >>
> >> Here is the link:
> >>
> >> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
> >> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> >> cwiki.apache.org
> >>
> >>  
> >> KIP-1011:
> >>  Use incrementalAlterConfigs when updating broker configs by 
> >> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
> >> 
> >> cwiki.apache.org 
> >> 
> >>  
> >> 
> >>
> >> Best,
> >> Ziming.
>

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[ziming deng]

Thank you for mention this Ismael,

I added this to the motivation section, and I think we can still update configs 
in this case by passing all sensitive configs, which is weird and not friendly.

--
Best,
Ziming

> On Dec 19, 2023, at 14:24, Ismael Juma  wrote:
> 
> Thanks for the KIP. I think one of the main benefits of the change isn't 
> listed: sensitive configs make it impossible to make updates with the current 
> cli tool because sensitive config values are never returned.
> 
> Ismael
> 
> On Mon, Dec 18, 2023 at 7:58 PM ziming deng  > wrote:
>> 
>> Hello, I want to start a discussion on KIP-1011, to make the broker config 
>> change path unified with that of user/topic/client-metrics and avoid some 
>> bugs.
>> 
>> Here is the link: 
>> 
>> KIP-1011: Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation
>> cwiki.apache.org
>> 
>>  
>> KIP-1011:
>>  Use incrementalAlterConfigs when updating broker configs by 
>> kafka-configs.sh - Apache Kafka - Apache Software Foundation 
>> 
>> cwiki.apache.org 
>> 
>>  
>> 
>> 
>> Best, 
>> Ziming.

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

[Ismael Juma]

Thanks for the KIP. I think one of the main benefits of the change isn't
listed: sensitive configs make it impossible to make updates with the
current cli tool because sensitive config values are never returned.

Ismael

On Mon, Dec 18, 2023 at 7:58 PM ziming deng 
wrote:

>
> Hello, I want to start a discussion on KIP-1011, to make the broker config
> change path unified with that of user/topic/client-metrics and avoid some
> bugs.
>
> Here is the link:
>
> KIP-1011: Use incrementalAlterConfigs when updating broker configs by
> kafka-configs.sh - Apache Kafka - Apache Software Foundation
> 
> cwiki.apache.org
> 
> [image: favicon.ico]
> 
> 
>
> Best,
> Ziming.
>