from:"Konstantine Karantasis"

Thanks for raising this Colin.

https://issues.apache.org/jira/browse/KAFKA-13277

is a one line fix with a test now. The suggestion to include it in RC2
makes sense to me as well. I’ll make sure it’s in.

Konstantine

On Tue, Sep 7, 2021 at 11:42 PM Colin McCabe  wrote:

> Hi Konstantine,
>
> Given that we are making a new RC, I would suggest that we merge
> "KAFKA-13277; Fix size calculation for tagged string fields in message
> generator" to 3.0. What do you think?
>
> best,
> Colin
>
>
> On Tue, Sep 7, 2021, at 13:20, Konstantine Karantasis wrote:
> > Thanks David,
> >
> > Assuming we'll have the PRs you mention merged soon, I'll include them in
> > RC2 for 3.0.0 given their low risk and the fact that we need to generate
> a
> > new RC anyways.
> >
> > Konstantine
> >
> > On Tue, Sep 7, 2021 at 4:43 PM David Jacot 
> > wrote:
> >
> > > Hi Konstantine,
> > >
> > > I would like to propose https://github.com/apache/kafka/pull/11300 as
> a
> > > blocker
> > > as well. The PR fixes KAFKA-13258/13259/13260. There are all very
> small but
> > > annoying issues. The PR is trivial.
> > >
> > > Regarding KAFKA-13266, the fix is quite simple, so low risk in my
> opinion.
> > > Jason
> > > will review it soon.
> > >
> > > Best,
> > > David
> > >
> > > On Tue, Sep 7, 2021 at 3:27 PM Ismael Juma  wrote:
> > >
> > > > Hi Konstantine,
> > > >
> > > > I will remove the final modifier for now.
> > > >
> > > > I added it because the removal of  the deprecated `close` overload
> could
> > > > lead to weird behavior if the no-args `close` was overridden (the
> > > > implementation of the no-arg `close` delegated to the removed
> `close`,
> > > but
> > > > that's no longer the case). However, I didn't realize `MockConsumer`
> was
> > > a
> > > > public class (which is a mistake imo since we tweak its behavior
> > > regularly
> > > > and sometimes add methods without a KIP).
> > > >
> > > > Ismael
> > > >
> > > > On Tue, Sep 7, 2021 at 1:46 AM Konstantine Karantasis <
> > > > kkaranta...@apache.org> wrote:
> > > >
> > > > > Hi Gary,
> > > > >
> > > > > Regarding KAFKA-13262, this might need a more detailed and strong
> > > > > justification to be considered as a blocker at this point in the
> > > release.
> > > > > But we are already moving towards RC2 because of the blockers
> mentioned
> > > > > above, so I'd be in favor of a PR that would revert the change that
> > > made
> > > > > this method final if we could get one this week. I'm not sure why
> the
> > > > > MockConsumer::close method was marked as final in
> > > > > https://github.com/apache/kafka/pull/10438. The change is not
> included
> > > > in
> > > > > the description. Maybe it wasn't obvious that this class is used
> > > outside
> > > > > this project or maybe we thought we should discourage such use.
> > > > >
> > > > > Gary, is this the only change you are referring to?
> > > > > Ismael, I see you authored the PR originally. Any thoughts on the
> > > > suggested
> > > > > change?
> > > > >
> > > > > Konstantine
> > > > >
> > > > > On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
> > > > > kkaranta...@apache.org> wrote:
> > > > >
> > > > > > Hi Magnus,
> > > > > >
> > > > > > Thanks for reporting the numbers.
> > > > > >
> > > > > > I agree that what you observe warrants further investigation,
> not so
> > > > much
> > > > > > because these are clear performance regressions but because they
> > > might
> > > > be
> > > > > > indications of underlying issues, as you noted. However, from a
> major
> > > > > > release standpoint I'd say that such preliminary evidence
> probably
> > > > falls
> > > > > > within the range of acceptable differences in the presence of
> major
> > > > > changes
> > > > > > and can be optimized over time in subsequent releases.
> > > > > >
> > > > > > Having said that, I'd ta

Re: [VOTE] 3.0.0 RC1

Thanks David,

Assuming we'll have the PRs you mention merged soon, I'll include them in
RC2 for 3.0.0 given their low risk and the fact that we need to generate a
new RC anyways.

Konstantine

On Tue, Sep 7, 2021 at 4:43 PM David Jacot 
wrote:

> Hi Konstantine,
>
> I would like to propose https://github.com/apache/kafka/pull/11300 as a
> blocker
> as well. The PR fixes KAFKA-13258/13259/13260. There are all very small but
> annoying issues. The PR is trivial.
>
> Regarding KAFKA-13266, the fix is quite simple, so low risk in my opinion.
> Jason
> will review it soon.
>
> Best,
> David
>
> On Tue, Sep 7, 2021 at 3:27 PM Ismael Juma  wrote:
>
> > Hi Konstantine,
> >
> > I will remove the final modifier for now.
> >
> > I added it because the removal of  the deprecated `close` overload could
> > lead to weird behavior if the no-args `close` was overridden (the
> > implementation of the no-arg `close` delegated to the removed `close`,
> but
> > that's no longer the case). However, I didn't realize `MockConsumer` was
> a
> > public class (which is a mistake imo since we tweak its behavior
> regularly
> > and sometimes add methods without a KIP).
> >
> > Ismael
> >
> > On Tue, Sep 7, 2021 at 1:46 AM Konstantine Karantasis <
> > kkaranta...@apache.org> wrote:
> >
> > > Hi Gary,
> > >
> > > Regarding KAFKA-13262, this might need a more detailed and strong
> > > justification to be considered as a blocker at this point in the
> release.
> > > But we are already moving towards RC2 because of the blockers mentioned
> > > above, so I'd be in favor of a PR that would revert the change that
> made
> > > this method final if we could get one this week. I'm not sure why the
> > > MockConsumer::close method was marked as final in
> > > https://github.com/apache/kafka/pull/10438. The change is not included
> > in
> > > the description. Maybe it wasn't obvious that this class is used
> outside
> > > this project or maybe we thought we should discourage such use.
> > >
> > > Gary, is this the only change you are referring to?
> > > Ismael, I see you authored the PR originally. Any thoughts on the
> > suggested
> > > change?
> > >
> > > Konstantine
> > >
> > > On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
> > > kkaranta...@apache.org> wrote:
> > >
> > > > Hi Magnus,
> > > >
> > > > Thanks for reporting the numbers.
> > > >
> > > > I agree that what you observe warrants further investigation, not so
> > much
> > > > because these are clear performance regressions but because they
> might
> > be
> > > > indications of underlying issues, as you noted. However, from a major
> > > > release standpoint I'd say that such preliminary evidence probably
> > falls
> > > > within the range of acceptable differences in the presence of major
> > > changes
> > > > and can be optimized over time in subsequent releases.
> > > >
> > > > Having said that, I'd take Israel's comment one step further and say
> > that
> > > > in order to be able to give more definitive answers it'd be good to
> > have
> > > a
> > > > description of the setup that is reproducible. Briefly, that would
> > > include
> > > > both the benchmark setup (specs etc) as well as the benchmark suite
> and
> > > the
> > > > way to run it (links to benchmark code including configurations and
> > > > description of how to run and reproduce).
> > > >
> > > > Small note that the attached plot is missing units on the x-axis
> > (y-axis
> > > I
> > > > assume is milliseconds from your description).
> > > >
> > > > Konstantine
> > > >
> > > >
> > > > On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo 
> > > wrote:
> > > >
> > > >> Magnus,
> > > >>
> > > >> Please could you share the machine and network specs?
> > > >>
> > > >> How much CPU, RAM is available on each node?
> > > >>
> > > >> What JDK, JRE version are you using?
> > > >>
> > > >> What are your broker and client configuration values? Please could
> you
> > > >> share this info if possible?
> > > >>
> > > >> Thanks.
> > > >>
> > > &g

Re: [VOTE] 3.0.0 RC1

Thanks for clarifying Ismael.

In that case, your proposed change sounds good to include in RC2 as well.

Konstantine

On Tue, Sep 7, 2021 at 4:27 PM Ismael Juma  wrote:

> Hi Konstantine,
>
> I will remove the final modifier for now.
>
> I added it because the removal of  the deprecated `close` overload could
> lead to weird behavior if the no-args `close` was overridden (the
> implementation of the no-arg `close` delegated to the removed `close`, but
> that's no longer the case). However, I didn't realize `MockConsumer` was a
> public class (which is a mistake imo since we tweak its behavior regularly
> and sometimes add methods without a KIP).
>
> Ismael
>
> On Tue, Sep 7, 2021 at 1:46 AM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
> > Hi Gary,
> >
> > Regarding KAFKA-13262, this might need a more detailed and strong
> > justification to be considered as a blocker at this point in the release.
> > But we are already moving towards RC2 because of the blockers mentioned
> > above, so I'd be in favor of a PR that would revert the change that made
> > this method final if we could get one this week. I'm not sure why the
> > MockConsumer::close method was marked as final in
> > https://github.com/apache/kafka/pull/10438. The change is not included
> in
> > the description. Maybe it wasn't obvious that this class is used outside
> > this project or maybe we thought we should discourage such use.
> >
> > Gary, is this the only change you are referring to?
> > Ismael, I see you authored the PR originally. Any thoughts on the
> suggested
> > change?
> >
> > Konstantine
> >
> > On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
> > kkaranta...@apache.org> wrote:
> >
> > > Hi Magnus,
> > >
> > > Thanks for reporting the numbers.
> > >
> > > I agree that what you observe warrants further investigation, not so
> much
> > > because these are clear performance regressions but because they might
> be
> > > indications of underlying issues, as you noted. However, from a major
> > > release standpoint I'd say that such preliminary evidence probably
> falls
> > > within the range of acceptable differences in the presence of major
> > changes
> > > and can be optimized over time in subsequent releases.
> > >
> > > Having said that, I'd take Israel's comment one step further and say
> that
> > > in order to be able to give more definitive answers it'd be good to
> have
> > a
> > > description of the setup that is reproducible. Briefly, that would
> > include
> > > both the benchmark setup (specs etc) as well as the benchmark suite and
> > the
> > > way to run it (links to benchmark code including configurations and
> > > description of how to run and reproduce).
> > >
> > > Small note that the attached plot is missing units on the x-axis
> (y-axis
> > I
> > > assume is milliseconds from your description).
> > >
> > > Konstantine
> > >
> > >
> > > On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo 
> > wrote:
> > >
> > >> Magnus,
> > >>
> > >> Please could you share the machine and network specs?
> > >>
> > >> How much CPU, RAM is available on each node?
> > >>
> > >> What JDK, JRE version are you using?
> > >>
> > >> What are your broker and client configuration values? Please could you
> > >> share this info if possible?
> > >>
> > >> Thanks.
> > >>
> > >>
> > >>
> > >> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
> > >> wrote:
> > >>
> > >> > Hi Konstantine,
> > >> >
> > >> > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
> > >> >
> > >> > * Compaction seems to take slightly longer to kick in when segment
> > sizes
> > >> >   exceed their threshold. (Used to take less than 20 seconds, now
> > takes
> > >> > 20..30 seconds.)
> > >> >
> > >> > * CreateTopic seems to take slightly longer to propagate through the
> > >> > cluster,
> > >> >   e.g., before a new topic is available in metadata from other
> > brokers.
> > >> >
> > >> > * CreateTopics seems to take longer when the Admin request timeout
> is
> > >> set,
> > >> >   looks like a plateau at 10 seconds:
&

Re: [VOTE] 3.0.0 RC1

Regarding the documentation link which I forgot to answer above, docs are
currently not deployed during preview. The link is generated in this
templated format while running the release scripts.

Gary, for now, you may find the docs for AK 3.0 in:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/ as archives in:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/kafka_2.12-3.0.0-site-docs.tgz
and
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/kafka_2.13-3.0.0-site-docs.tgz


Konstantine


On Tue, Sep 7, 2021 at 11:46 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

>
> Hi Gary,
>
> Regarding KAFKA-13262, this might need a more detailed and strong
> justification to be considered as a blocker at this point in the release.
> But we are already moving towards RC2 because of the blockers mentioned
> above, so I'd be in favor of a PR that would revert the change that made
> this method final if we could get one this week. I'm not sure why the
> MockConsumer::close method was marked as final in
> https://github.com/apache/kafka/pull/10438. The change is not included in
> the description. Maybe it wasn't obvious that this class is used outside
> this project or maybe we thought we should discourage such use.
>
> Gary, is this the only change you are referring to?
> Ismael, I see you authored the PR originally. Any thoughts on the
> suggested change?
>
> Konstantine
>
> On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
>> Hi Magnus,
>>
>> Thanks for reporting the numbers.
>>
>> I agree that what you observe warrants further investigation, not so much
>> because these are clear performance regressions but because they might be
>> indications of underlying issues, as you noted. However, from a major
>> release standpoint I'd say that such preliminary evidence probably falls
>> within the range of acceptable differences in the presence of major changes
>> and can be optimized over time in subsequent releases.
>>
>> Having said that, I'd take Israel's comment one step further and say that
>> in order to be able to give more definitive answers it'd be good to have a
>> description of the setup that is reproducible. Briefly, that would include
>> both the benchmark setup (specs etc) as well as the benchmark suite and the
>> way to run it (links to benchmark code including configurations and
>> description of how to run and reproduce).
>>
>> Small note that the attached plot is missing units on the x-axis (y-axis
>> I assume is milliseconds from your description).
>>
>> Konstantine
>>
>>
>> On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo  wrote:
>>
>>> Magnus,
>>>
>>> Please could you share the machine and network specs?
>>>
>>> How much CPU, RAM is available on each node?
>>>
>>> What JDK, JRE version are you using?
>>>
>>> What are your broker and client configuration values? Please could you
>>> share this info if possible?
>>>
>>> Thanks.
>>>
>>>
>>>
>>> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
>>> wrote:
>>>
>>> > Hi Konstantine,
>>> >
>>> > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
>>> >
>>> > * Compaction seems to take slightly longer to kick in when segment
>>> sizes
>>> >   exceed their threshold. (Used to take less than 20 seconds, now takes
>>> > 20..30 seconds.)
>>> >
>>> > * CreateTopic seems to take slightly longer to propagate through the
>>> > cluster,
>>> >   e.g., before a new topic is available in metadata from other brokers.
>>> >
>>> > * CreateTopics seems to take longer when the Admin request timeout is
>>> set,
>>> >   looks like a plateau at 10 seconds:
>>> >   https://imgur.com/a/n6y76sj
>>> >
>>> > (This is a 3 broker cluster with identical configs between 2.8 and
>>> 3.0.0.)
>>> >
>>> > Nothing critical, but could be an indication of regression so I
>>> thought I'd
>>> > mention it.
>>> >
>>> > Regards,
>>> > Magnus
>>> >
>>> >
>>> > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
>>> > kkaranta...@apache.org>:
>>> >
>>> > > Small correction to my previous email.
>>> > > The actual link for public preview of the 3.0.0 blog post draft is:
>>> > >
>>> > >
>>> &g

Re: [VOTE] 3.0.0 RC1

Hi Tom,

It'd be good to avoid a breaking change in a subsequent release that is
related to the modernization of KafkaFuture. Avoiding to use a reference to
an non-public implementation class in the public interfaces seems to help
us with that, as you observed. I'm approving KAFKA-13276 to be included in
RC2 along with the other blockers.

Thanks,
Konstantine

On Tue, Sep 7, 2021 at 11:46 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

>
> Hi Gary,
>
> Regarding KAFKA-13262, this might need a more detailed and strong
> justification to be considered as a blocker at this point in the release.
> But we are already moving towards RC2 because of the blockers mentioned
> above, so I'd be in favor of a PR that would revert the change that made
> this method final if we could get one this week. I'm not sure why the
> MockConsumer::close method was marked as final in
> https://github.com/apache/kafka/pull/10438. The change is not included in
> the description. Maybe it wasn't obvious that this class is used outside
> this project or maybe we thought we should discourage such use.
>
> Gary, is this the only change you are referring to?
> Ismael, I see you authored the PR originally. Any thoughts on the
> suggested change?
>
> Konstantine
>
> On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
>> Hi Magnus,
>>
>> Thanks for reporting the numbers.
>>
>> I agree that what you observe warrants further investigation, not so much
>> because these are clear performance regressions but because they might be
>> indications of underlying issues, as you noted. However, from a major
>> release standpoint I'd say that such preliminary evidence probably falls
>> within the range of acceptable differences in the presence of major changes
>> and can be optimized over time in subsequent releases.
>>
>> Having said that, I'd take Israel's comment one step further and say that
>> in order to be able to give more definitive answers it'd be good to have a
>> description of the setup that is reproducible. Briefly, that would include
>> both the benchmark setup (specs etc) as well as the benchmark suite and the
>> way to run it (links to benchmark code including configurations and
>> description of how to run and reproduce).
>>
>> Small note that the attached plot is missing units on the x-axis (y-axis
>> I assume is milliseconds from your description).
>>
>> Konstantine
>>
>>
>> On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo  wrote:
>>
>>> Magnus,
>>>
>>> Please could you share the machine and network specs?
>>>
>>> How much CPU, RAM is available on each node?
>>>
>>> What JDK, JRE version are you using?
>>>
>>> What are your broker and client configuration values? Please could you
>>> share this info if possible?
>>>
>>> Thanks.
>>>
>>>
>>>
>>> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
>>> wrote:
>>>
>>> > Hi Konstantine,
>>> >
>>> > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
>>> >
>>> > * Compaction seems to take slightly longer to kick in when segment
>>> sizes
>>> >   exceed their threshold. (Used to take less than 20 seconds, now takes
>>> > 20..30 seconds.)
>>> >
>>> > * CreateTopic seems to take slightly longer to propagate through the
>>> > cluster,
>>> >   e.g., before a new topic is available in metadata from other brokers.
>>> >
>>> > * CreateTopics seems to take longer when the Admin request timeout is
>>> set,
>>> >   looks like a plateau at 10 seconds:
>>> >   https://imgur.com/a/n6y76sj
>>> >
>>> > (This is a 3 broker cluster with identical configs between 2.8 and
>>> 3.0.0.)
>>> >
>>> > Nothing critical, but could be an indication of regression so I
>>> thought I'd
>>> > mention it.
>>> >
>>> > Regards,
>>> > Magnus
>>> >
>>> >
>>> > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
>>> > kkaranta...@apache.org>:
>>> >
>>> > > Small correction to my previous email.
>>> > > The actual link for public preview of the 3.0.0 blog post draft is:
>>> > >
>>> > >
>>> >
>>> https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6
>>> > >
>>> > >

Re: [VOTE] 3.0.0 RC1

Hi Gary,

Regarding KAFKA-13262, this might need a more detailed and strong
justification to be considered as a blocker at this point in the release.
But we are already moving towards RC2 because of the blockers mentioned
above, so I'd be in favor of a PR that would revert the change that made
this method final if we could get one this week. I'm not sure why the
MockConsumer::close method was marked as final in
https://github.com/apache/kafka/pull/10438. The change is not included in
the description. Maybe it wasn't obvious that this class is used outside
this project or maybe we thought we should discourage such use.

Gary, is this the only change you are referring to?
Ismael, I see you authored the PR originally. Any thoughts on the suggested
change?

Konstantine

On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

> Hi Magnus,
>
> Thanks for reporting the numbers.
>
> I agree that what you observe warrants further investigation, not so much
> because these are clear performance regressions but because they might be
> indications of underlying issues, as you noted. However, from a major
> release standpoint I'd say that such preliminary evidence probably falls
> within the range of acceptable differences in the presence of major changes
> and can be optimized over time in subsequent releases.
>
> Having said that, I'd take Israel's comment one step further and say that
> in order to be able to give more definitive answers it'd be good to have a
> description of the setup that is reproducible. Briefly, that would include
> both the benchmark setup (specs etc) as well as the benchmark suite and the
> way to run it (links to benchmark code including configurations and
> description of how to run and reproduce).
>
> Small note that the attached plot is missing units on the x-axis (y-axis I
> assume is milliseconds from your description).
>
> Konstantine
>
>
> On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo  wrote:
>
>> Magnus,
>>
>> Please could you share the machine and network specs?
>>
>> How much CPU, RAM is available on each node?
>>
>> What JDK, JRE version are you using?
>>
>> What are your broker and client configuration values? Please could you
>> share this info if possible?
>>
>> Thanks.
>>
>>
>>
>> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
>> wrote:
>>
>> > Hi Konstantine,
>> >
>> > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
>> >
>> > * Compaction seems to take slightly longer to kick in when segment sizes
>> >   exceed their threshold. (Used to take less than 20 seconds, now takes
>> > 20..30 seconds.)
>> >
>> > * CreateTopic seems to take slightly longer to propagate through the
>> > cluster,
>> >   e.g., before a new topic is available in metadata from other brokers.
>> >
>> > * CreateTopics seems to take longer when the Admin request timeout is
>> set,
>> >   looks like a plateau at 10 seconds:
>> >   https://imgur.com/a/n6y76sj
>> >
>> > (This is a 3 broker cluster with identical configs between 2.8 and
>> 3.0.0.)
>> >
>> > Nothing critical, but could be an indication of regression so I thought
>> I'd
>> > mention it.
>> >
>> > Regards,
>> > Magnus
>> >
>> >
>> > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
>> > kkaranta...@apache.org>:
>> >
>> > > Small correction to my previous email.
>> > > The actual link for public preview of the 3.0.0 blog post draft is:
>> > >
>> > >
>> >
>> https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6
>> > >
>> > > (see also the email thread with title: [DISCUSS] Please review the
>> 3.0.0
>> > > blog post)
>> > >
>> > > Best,
>> > > Konstantine
>> > >
>> > > On Tue, Aug 31, 2021 at 6:34 PM Konstantine Karantasis <
>> > > kkaranta...@apache.org> wrote:
>> > >
>> > > >
>> > > > Hello Kafka users, developers and client-developers,
>> > > >
>> > > > This is the second release candidate for Apache Kafka 3.0.0.
>> > > > It corresponds to a major release that includes many new features,
>> > > > including:
>> > > >
>> > > > * The deprecation of support for Java 8 and Scala 2.12.
>> > > > * Kafka Raft support for snapshots of the metadata topic and
>> > > > other

Re: [VOTE] 3.0.0 RC1

Hi David,

Thanks for discovering the issue reported in KAFKA-13266. I'm tentatively
approving it if we can review the PR this week, since we are going to have
a new RC due to KAFKA-13270 anyways. This is not a change of a few lines of
code though and while most of it seems to be shuffling around existing code
and has additional test coverage, would you mind commenting on the risk of
this change at this point? I'll be checking the PR reviews as well. Again,
for now, this is approved for 3.0.0 unless we decide otherwise this week.

Konstantine

On Fri, Sep 3, 2021 at 3:45 PM David Jacot 
wrote:

> Hi Konstantine,
>
> I'd like to raise a potential blocker:
> https://issues.apache.org/jira/browse/KAFKA-13266.
>
> When this bug is hit, the partition remains failed until the broker is
> restated or
> the partition is updated (e.g. its epoch is bumped). It only affects the
> KRaft mode.
>
> The PR is ready: https://issues.apache.org/jira/browse/KAFKA-13266.
>
> Best,
> David
>
> On Fri, Sep 3, 2021 at 3:17 AM Ron Dagostino  wrote:
>
> > Hi Konstantine.  I have opened a probable blocker ticket
> > https://issues.apache.org/jira/browse/KAFKA-13270.  I will work on a PR
> > shortly.  The description on that ticket is as follows:
> >
> > The implementation of
> https://issues.apache.org/jira/browse/ZOOKEEPER-3593
> > in
> > ZooKeeper version 3.6.0 decreased the default value for the ZooKeeper
> > client's `jute.maxbuffer` configuration from 4MB to 1MB. This can cause a
> > problem if Kafka tries to retrieve a large amount of data across many
> > znodes – in such a case the ZooKeeper client will repeatedly emit a
> message
> > of the form "java.io.IOException: Packet len <> is out of range" and
> > the Kafka broker will never connect to ZooKeeper and fail to make
> progress
> > on the startup sequence. We can avoid the potential for this issue to
> occur
> > by explicitly setting the value to 4MB whenever we create a new ZooKeeper
> > client as long as no explicit value has been set via the `jute.maxbuffer`
> > system property.
> >
> > Ron
> >
> > On Thu, Sep 2, 2021 at 5:52 PM Israel Ekpo  wrote:
> >
> > > Magnus,
> > >
> > > Please could you share the machine and network specs?
> > >
> > > How much CPU, RAM is available on each node?
> > >
> > > What JDK, JRE version are you using?
> > >
> > > What are your broker and client configuration values? Please could you
> > > share this info if possible?
> > >
> > > Thanks.
> > >
> > >
> > >
> > > On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
> > > wrote:
> > >
> > > > Hi Konstantine,
> > > >
> > > > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
> > > >
> > > > * Compaction seems to take slightly longer to kick in when segment
> > sizes
> > > >   exceed their threshold. (Used to take less than 20 seconds, now
> takes
> > > > 20..30 seconds.)
> > > >
> > > > * CreateTopic seems to take slightly longer to propagate through the
> > > > cluster,
> > > >   e.g., before a new topic is available in metadata from other
> brokers.
> > > >
> > > > * CreateTopics seems to take longer when the Admin request timeout is
> > > set,
> > > >   looks like a plateau at 10 seconds:
> > > >   https://imgur.com/a/n6y76sj
> > > >
> > > > (This is a 3 broker cluster with identical configs between 2.8 and
> > > 3.0.0.)
> > > >
> > > > Nothing critical, but could be an indication of regression so I
> thought
> > > I'd
> > > > mention it.
> > > >
> > > > Regards,
> > > > Magnus
> > > >
> > > >
> > > > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
> > > > kkaranta...@apache.org>:
> > > >
> > > > > Small correction to my previous email.
> > > > > The actual link for public preview of the 3.0.0 blog post draft is:
> > > > >
> > > > >
> > > >
> > >
> >
> https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6
> > > > >
> > > > > (see also the email thread with title: [DISCUSS] Please review the
> > > 3.0.0
> > > > > blog post)
> > > > >
> > > > > Best,
> > > > > Konstantine
> > > > >
> > >

Re: [VOTE] 3.0.0 RC1

Hi Ron,

Thanks for reporting this issue. KAFKA-13270 is an obvious regression from
previous versions, so the fix - that I now see has been merged already - is
approved for inclusion in 3.0.0 and I'll add it to the next release
candidate that we'll need to generate for 3.0.0.

Konstantine


On Fri, Sep 3, 2021 at 4:24 AM Ron Dagostino  wrote:

> Hi Konstantine.  I have opened a probable blocker ticket
> https://issues.apache.org/jira/browse/KAFKA-13270.  I will work on a PR
> shortly.  The description on that ticket is as follows:
>
> The implementation of https://issues.apache.org/jira/browse/ZOOKEEPER-3593
> in
> ZooKeeper version 3.6.0 decreased the default value for the ZooKeeper
> client's `jute.maxbuffer` configuration from 4MB to 1MB. This can cause a
> problem if Kafka tries to retrieve a large amount of data across many
> znodes – in such a case the ZooKeeper client will repeatedly emit a message
> of the form "java.io.IOException: Packet len <> is out of range" and
> the Kafka broker will never connect to ZooKeeper and fail to make progress
> on the startup sequence. We can avoid the potential for this issue to occur
> by explicitly setting the value to 4MB whenever we create a new ZooKeeper
> client as long as no explicit value has been set via the `jute.maxbuffer`
> system property.
>
> Ron
>
> On Thu, Sep 2, 2021 at 5:52 PM Israel Ekpo  wrote:
>
> > Magnus,
> >
> > Please could you share the machine and network specs?
> >
> > How much CPU, RAM is available on each node?
> >
> > What JDK, JRE version are you using?
> >
> > What are your broker and client configuration values? Please could you
> > share this info if possible?
> >
> > Thanks.
> >
> >
> >
> > On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
> > wrote:
> >
> > > Hi Konstantine,
> > >
> > > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
> > >
> > > * Compaction seems to take slightly longer to kick in when segment
> sizes
> > >   exceed their threshold. (Used to take less than 20 seconds, now takes
> > > 20..30 seconds.)
> > >
> > > * CreateTopic seems to take slightly longer to propagate through the
> > > cluster,
> > >   e.g., before a new topic is available in metadata from other brokers.
> > >
> > > * CreateTopics seems to take longer when the Admin request timeout is
> > set,
> > >   looks like a plateau at 10 seconds:
> > >   https://imgur.com/a/n6y76sj
> > >
> > > (This is a 3 broker cluster with identical configs between 2.8 and
> > 3.0.0.)
> > >
> > > Nothing critical, but could be an indication of regression so I thought
> > I'd
> > > mention it.
> > >
> > > Regards,
> > > Magnus
> > >
> > >
> > > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
> > > kkaranta...@apache.org>:
> > >
> > > > Small correction to my previous email.
> > > > The actual link for public preview of the 3.0.0 blog post draft is:
> > > >
> > > >
> > >
> >
> https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6
> > > >
> > > > (see also the email thread with title: [DISCUSS] Please review the
> > 3.0.0
> > > > blog post)
> > > >
> > > > Best,
> > > > Konstantine
> > > >
> > > > On Tue, Aug 31, 2021 at 6:34 PM Konstantine Karantasis <
> > > > kkaranta...@apache.org> wrote:
> > > >
> > > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the second release candidate for Apache Kafka 3.0.0.
> > > > > It corresponds to a major release that includes many new features,
> > > > > including:
> > > > >
> > > > > * The deprecation of support for Java 8 and Scala 2.12.
> > > > > * Kafka Raft support for snapshots of the metadata topic and
> > > > > other improvements in the self-managed quorum.
> > > > > * Deprecation of message formats v0 and v1.
> > > > > * Stronger delivery guarantees for the Kafka producer enabled by
> > > default.
> > > > > * Optimizations in OffsetFetch and FindCoordinator requests.
> > > > > * More flexible Mirror Maker 2 configuration and deprecation of
> > > > > Mirror Maker 1.
> > > > > * Ability to rest

Re: [VOTE] 3.0.0 RC1

Hi Magnus,

Thanks for reporting the numbers.

I agree that what you observe warrants further investigation, not so much
because these are clear performance regressions but because they might be
indications of underlying issues, as you noted. However, from a major
release standpoint I'd say that such preliminary evidence probably falls
within the range of acceptable differences in the presence of major changes
and can be optimized over time in subsequent releases.

Having said that, I'd take Israel's comment one step further and say that
in order to be able to give more definitive answers it'd be good to have a
description of the setup that is reproducible. Briefly, that would include
both the benchmark setup (specs etc) as well as the benchmark suite and the
way to run it (links to benchmark code including configurations and
description of how to run and reproduce).

Small note that the attached plot is missing units on the x-axis (y-axis I
assume is milliseconds from your description).

Konstantine


On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo  wrote:

> Magnus,
>
> Please could you share the machine and network specs?
>
> How much CPU, RAM is available on each node?
>
> What JDK, JRE version are you using?
>
> What are your broker and client configuration values? Please could you
> share this info if possible?
>
> Thanks.
>
>
>
> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill 
> wrote:
>
> > Hi Konstantine,
> >
> > Some findings from running 3.0.0-RC1 with the librdkafka test suite:
> >
> > * Compaction seems to take slightly longer to kick in when segment sizes
> >   exceed their threshold. (Used to take less than 20 seconds, now takes
> > 20..30 seconds.)
> >
> > * CreateTopic seems to take slightly longer to propagate through the
> > cluster,
> >   e.g., before a new topic is available in metadata from other brokers.
> >
> > * CreateTopics seems to take longer when the Admin request timeout is
> set,
> >   looks like a plateau at 10 seconds:
> >   https://imgur.com/a/n6y76sj
> >
> > (This is a 3 broker cluster with identical configs between 2.8 and
> 3.0.0.)
> >
> > Nothing critical, but could be an indication of regression so I thought
> I'd
> > mention it.
> >
> > Regards,
> > Magnus
> >
> >
> > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis <
> > kkaranta...@apache.org>:
> >
> > > Small correction to my previous email.
> > > The actual link for public preview of the 3.0.0 blog post draft is:
> > >
> > >
> >
> https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6
> > >
> > > (see also the email thread with title: [DISCUSS] Please review the
> 3.0.0
> > > blog post)
> > >
> > > Best,
> > > Konstantine
> > >
> > > On Tue, Aug 31, 2021 at 6:34 PM Konstantine Karantasis <
> > > kkaranta...@apache.org> wrote:
> > >
> > > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the second release candidate for Apache Kafka 3.0.0.
> > > > It corresponds to a major release that includes many new features,
> > > > including:
> > > >
> > > > * The deprecation of support for Java 8 and Scala 2.12.
> > > > * Kafka Raft support for snapshots of the metadata topic and
> > > > other improvements in the self-managed quorum.
> > > > * Deprecation of message formats v0 and v1.
> > > > * Stronger delivery guarantees for the Kafka producer enabled by
> > default.
> > > > * Optimizations in OffsetFetch and FindCoordinator requests.
> > > > * More flexible Mirror Maker 2 configuration and deprecation of
> > > > Mirror Maker 1.
> > > > * Ability to restart a connector's tasks on a single call in Kafka
> > > Connect.
> > > > * Connector log contexts and connector client overrides are now
> enabled
> > > > by default.
> > > > * Enhanced semantics for timestamp synchronization in Kafka Streams.
> > > > * Revamped public API for Stream's TaskId.
> > > > * Default serde becomes null in Kafka Streams and several
> > > > other configuration changes.
> > > >
> > > > You may read and review a more detailed list of changes in the 3.0.0
> > blog
> > > > post draft here:
> > > >
> > > >
> > >
> >
> https://blogs.apache.org/roller-ui/authoring/preview/kafka/?previewEntry=what-s-new-in-apache6
> > > >
> > > &g

Re: [VOTE] 3.0.0 RC1

Small correction to my previous email.
The actual link for public preview of the 3.0.0 blog post draft is:

https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6

(see also the email thread with title: [DISCUSS] Please review the 3.0.0
blog post)

Best,
Konstantine

On Tue, Aug 31, 2021 at 6:34 PM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

>
> Hello Kafka users, developers and client-developers,
>
> This is the second release candidate for Apache Kafka 3.0.0.
> It corresponds to a major release that includes many new features,
> including:
>
> * The deprecation of support for Java 8 and Scala 2.12.
> * Kafka Raft support for snapshots of the metadata topic and
> other improvements in the self-managed quorum.
> * Deprecation of message formats v0 and v1.
> * Stronger delivery guarantees for the Kafka producer enabled by default.
> * Optimizations in OffsetFetch and FindCoordinator requests.
> * More flexible Mirror Maker 2 configuration and deprecation of
> Mirror Maker 1.
> * Ability to restart a connector's tasks on a single call in Kafka Connect.
> * Connector log contexts and connector client overrides are now enabled
> by default.
> * Enhanced semantics for timestamp synchronization in Kafka Streams.
> * Revamped public API for Stream's TaskId.
> * Default serde becomes null in Kafka Streams and several
> other configuration changes.
>
> You may read and review a more detailed list of changes in the 3.0.0 blog
> post draft here:
>
> https://blogs.apache.org/roller-ui/authoring/preview/kafka/?previewEntry=what-s-new-in-apache6
>
> Release notes for the 3.0.0 release:
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/RELEASE_NOTES.html
>
> *** Please download, test and vote by Wednesday, September 8, 2021 ***
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/javadoc/
>
> * Tag to be voted upon (off 3.0 branch) is the 3.0.0 tag:
> https://github.com/apache/kafka/releases/tag/3.0.0-rc1
>
> * Documentation:
> https://kafka.apache.org/30/documentation.html
>
> * Protocol:
> https://kafka.apache.org/30/protocol.html
>
> * Successful Jenkins builds for the 3.0 branch:
> Unit/integration tests:
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.0/121/pipeline/
> (only few flaky failures)
> System tests:
> https://jenkins.confluent.io/job/system-test-kafka/job/3.0/57/
>
> /**
>
> Thanks,
> Konstantine
>

[DISCUSS] Please review the 3.0.0 blog post

With the opportunity of the latest release candidate (RC1) I have also
published a draft of the 3.0.0 release announcement, available for public
preview, here:

https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6

This blog post is scheduled to be published in
https://blogs.apache.org/kafka/ soon after the release candidate gets
approved and 3.0.0 gets promoted to production.

With that in mind, I would greatly appreciate your reviews, ideally by
Wednesday, September 8, 2021, which is also the current target date to
download, test and vote for the current 3.0.0 release candidate.

Sincerely,
Konstantine

[VOTE] 3.0.0 RC1

Hello Kafka users, developers and client-developers,

This is the second release candidate for Apache Kafka 3.0.0.
It corresponds to a major release that includes many new features,
including:

* The deprecation of support for Java 8 and Scala 2.12.
* Kafka Raft support for snapshots of the metadata topic and
other improvements in the self-managed quorum.
* Deprecation of message formats v0 and v1.
* Stronger delivery guarantees for the Kafka producer enabled by default.
* Optimizations in OffsetFetch and FindCoordinator requests.
* More flexible Mirror Maker 2 configuration and deprecation of
Mirror Maker 1.
* Ability to restart a connector's tasks on a single call in Kafka Connect.
* Connector log contexts and connector client overrides are now enabled
by default.
* Enhanced semantics for timestamp synchronization in Kafka Streams.
* Revamped public API for Stream's TaskId.
* Default serde becomes null in Kafka Streams and several
other configuration changes.

You may read and review a more detailed list of changes in the 3.0.0 blog
post draft here:
https://blogs.apache.org/roller-ui/authoring/preview/kafka/?previewEntry=what-s-new-in-apache6

Release notes for the 3.0.0 release:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/RELEASE_NOTES.html

*** Please download, test and vote by Wednesday, September 8, 2021 ***

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/javadoc/

* Tag to be voted upon (off 3.0 branch) is the 3.0.0 tag:
https://github.com/apache/kafka/releases/tag/3.0.0-rc1

* Documentation:
https://kafka.apache.org/30/documentation.html

* Protocol:
https://kafka.apache.org/30/protocol.html

* Successful Jenkins builds for the 3.0 branch:
Unit/integration tests:
https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.0/121/pipeline/
(only few flaky failures)
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/3.0/57/

/**

Thanks,
Konstantine

Re: [VOTE] 3.0.0 RC0

There were a few issues during the staging of maven artifacts in the nexus
staging repository for RC0.

Closing the vote early here in favor of RC1 that has been staged
successfully and will be announced shortly.

Thanks,
Konstantine


On Fri, Aug 27, 2021 at 7:10 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 3.0.0.
> It is a major release that includes many new features, including:
>
> * The deprecation of support for Java 8 and Scala 2.12.
> * Kafka Raft support for snapshots of the metadata topic and other
> improvements in the self-managed quorum.
> * Deprecation of message formats v0 and v1.
> * Stronger delivery guarantees for the Kafka producer enabled by default.
> * Optimizations in OffsetFetch and FindCoordinator requests.
> * More flexible Mirror Maker 2 configuration and deprecation of Mirror
> Maker 1.
> * Ability to restart a connector's tasks on a single call in Kafka Connect.
> * Connector log contexts and connector client overrides are now enabled by
> default.
> * Enhanced semantics for timestamp synchronization in Kafka Streams.
> * Revamped public API for Stream's TaskId.
> * Default serde becomes null in Kafka Streams and several other
> configuration changes.
>
> Release notes for the 3.0.0 release:
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Friday, September 3, 2021 ***
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/javadoc/
>
> * Tag to be voted upon (off 3.0 branch) is the 3.0.0 tag:
> https://github.com/apache/kafka/releases/tag/3.0.0-rc0
>
> * Documentation:
> https://kafka.apache.org/30/documentation.html
>
> * Protocol:
> https://kafka.apache.org/30/protocol.html
>
> * Successful Jenkins builds for the 3.0 branch:
> Unit/integration tests:
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.0/118/pipeline/
> System tests:
> https://jenkins.confluent.io/job/system-test-kafka/job/3.0/53/
>
> /**
>
> Thanks,
> Konstantine
>

[VOTE] 3.0.0 RC0

2021-08-26 Thread Konstantine Karantasis

Hello Kafka users, developers and client-developers,

This is the first candidate for release of Apache Kafka 3.0.0.
It is a major release that includes many new features, including:

* The deprecation of support for Java 8 and Scala 2.12.
* Kafka Raft support for snapshots of the metadata topic and other
improvements in the self-managed quorum.
* Deprecation of message formats v0 and v1.
* Stronger delivery guarantees for the Kafka producer enabled by default.
* Optimizations in OffsetFetch and FindCoordinator requests.
* More flexible Mirror Maker 2 configuration and deprecation of Mirror
Maker 1.
* Ability to restart a connector's tasks on a single call in Kafka Connect.
* Connector log contexts and connector client overrides are now enabled by
default.
* Enhanced semantics for timestamp synchronization in Kafka Streams.
* Revamped public API for Stream's TaskId.
* Default serde becomes null in Kafka Streams and several other
configuration changes.

Release notes for the 3.0.0 release:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/RELEASE_NOTES.html

*** Please download, test and vote by Friday, September 3, 2021 ***

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~kkarantasis/kafka-3.0.0-rc0/javadoc/

* Tag to be voted upon (off 3.0 branch) is the 3.0.0 tag:
https://github.com/apache/kafka/releases/tag/3.0.0-rc0

* Documentation:
https://kafka.apache.org/30/documentation.html

* Protocol:
https://kafka.apache.org/30/protocol.html

* Successful Jenkins builds for the 3.0 branch:
Unit/integration tests:
https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.0/118/pipeline/
System tests: https://jenkins.confluent.io/job/system-test-kafka/job/3.0/53/

/**

Thanks,
Konstantine

[jira] [Resolved] (KAFKA-12929) KIP-750: Deprecate support for Java 8 in Kafka 3.0

2021-08-25 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12929.

Resolution: Fixed

Resolving to unblock the RC for 3.0.0 and will keep a note to add to the 
downloads page



> KIP-750: Deprecate support for Java 8 in Kafka 3.0
> --
>
> Key: KAFKA-12929
> URL: https://issues.apache.org/jira/browse/KAFKA-12929
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Ismael Juma
>Assignee: Ismael Juma
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12930) KIP-751: Deprecate support for Scala 2.12 in Kafka 3.0

2021-08-25 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12930.

Resolution: Fixed

Resolving to unblock the RC for 3.0.0 and will keep a note to add to the 
downloads page

> KIP-751: Deprecate support for Scala 2.12 in Kafka 3.0
> --
>
> Key: KAFKA-12930
> URL: https://issues.apache.org/jira/browse/KAFKA-12930
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Ismael Juma
>Assignee: Ismael Juma
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-08-20 Thread Konstantine Karantasis

Thanks for the brief discussion Matthias and Israel.

Since there's no option but to merge a change, at this late stage in the
release process I'd agree with Matthias's suggestion to move forward with a
partial reversion of changes, which seems to be the safest of the two
options. I'll port the changes to the release plan and the blogpost I'm
preparing.

Hi Ron. Thanks for reporting the new issue. Indeed, we are quite picky
about what constitutes a blocker at this point. I was about to start
preparing the first RC earlier today.
The issue you report seems to meet our bar. Given that it's low risk, I'm
in favor of including it if we can merge the PR you prepared some time
soon.

Thank you all,
Konstantine

On Thu, Aug 19, 2021 at 6:09 PM Ron Dagostino  wrote:

> Hi Konstantine.  A potential 3.0 blocker was discovered today,
> https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-13219.  The
> BrokerState metric is not working for KRaft clusters -- it always indicates
> that the broker is in the "not running" state.  A PR is available at
> https://github.com/apache/kafka/pull/11239.  The risk is low, and there is
> no workaround, but I also understand the bar for a blocker is pretty high
> at this point.
>
> Ron
>
> On Thu, Aug 19, 2021 at 5:54 PM Israel Ekpo  wrote:
>
> > Konstantine,
> >
> > After a brief chat with Sophie, I just updated the name for KIP-633 to be
> > more descriptive of what is actually happening in the implementation
> >
> > It is changed on the KIP page and the JIRA task title is also updated to
> be
> > more descriptive
> > https://cwiki.apache.org/confluence/x/Ho2NCg
> > https://issues.apache.org/jira/browse/KAFKA-8613
> >
> > However, I could not update the page for the 3.0 release plan. I do not
> > have the permissions for that update.
> > https://cwiki.apache.org/confluence/x/woONCg
> >
> > When you have a moment, please could you update the release plan for 3.0
> to
> > reflect the update name for the KIP?
> >
> > Thank you very much
> >
> > Sincerely,
> > Israel
> >
> >
> > On Thu, Aug 19, 2021 at 5:50 PM Israel Ekpo 
> wrote:
> >
> > > Hi Matthias, it is possible to still deprecate JoinWindows.of(size)
> even
> > > though the new join semantics is disabled.
> > >
> > > I just pre-recorded a talk for Kafka Summit Americas where I am
> > > recommending a switch to the new APIs instead of the deprecated one
> > > starting from 3.0
> > >
> > > I would love to be involved in the discussion for the fix so that we
> try
> > > to honor the expectations of the KIP as much as possible.
> > >
> > > So far, it looks like the PR will still honor the expectations of the
> KIP
> > >
> > > https://github.com/apache/kafka/pull/11235
> > >
> > > Thanks for sharing this to create awareness.
> > >
> > > Sincerely,
> > > Israel
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 18, 2021 at 8:57 PM Matthias J. Sax 
> > wrote:
> > >
> > >> Hi,
> > >>
> > >> we discovered a potential blocker for 3.0.0 today:
> > >> https://issues.apache.org/jira/browse/KAFKA-13216
> > >>
> > >> We are still evaluating a potential fix. If we cannot fix it quickly,
> > >> the fall-back would be to partially roll-back KIP-633, to disable the
> > >> new join semantics such that people cannot hit this bug.
> > >>
> > >> Thoughts?
> > >>
> > >>
> > >> -Matthias
> > >>
> > >>
> > >> On 8/17/21 1:29 PM, Konstantine Karantasis wrote:
> > >> > Hi Rajini.
> > >> >
> > >> > Approved, given its low risk and the lack of convenient workarounds.
> > >> >
> > >> > Konstantine
> > >> >
> > >> >
> > >> > On Tue, Aug 17, 2021 at 11:00 AM Rajini Sivaram <
> > >> rajinisiva...@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> Hi Konstantine,
> > >> >>
> > >> >> We found an issue with replication with IBP 2.7:
> > >> >> https://issues.apache.org/jira/browse/KAFKA-13207. The fix is
> small
> > >> and
> > >> >> low
> > >> >> risk and has been merged to trunk. Can we include this in 3.0
> branch
> > >> since
> > >> >> it can result in IllegalStateException during replication?
> > >> &g

[jira] [Created] (KAFKA-13215) Flaky test org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectEndOffsetInformation

2021-08-18 Thread Konstantine Karantasis (Jira)

Konstantine Karantasis created KAFKA-13215:
--

 Summary: Flaky test 
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectEndOffsetInformation
 Key: KAFKA-13215
 URL: https://issues.apache.org/jira/browse/KAFKA-13215
 Project: Kafka
  Issue Type: Bug
  Components: streams
Reporter: Bruno Cadonna
Assignee: Walker Carlson
 Fix For: 3.1.0


Integration test {{test 
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation()}}
 sometimes fails with

{code:java}
java.lang.AssertionError: only one task
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
at 
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.getTaskMetadata(TaskMetadataIntegrationTest.java:162)
at 
org.apache.kafka.streams.integration.TaskMetadataIntegrationTest.shouldReportCorrectCommittedOffsetInformation(TaskMetadataIntegrationTest.java:117)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-08-17 Thread Konstantine Karantasis

Hi Rajini.

Approved, given its low risk and the lack of convenient workarounds.

Konstantine


On Tue, Aug 17, 2021 at 11:00 AM Rajini Sivaram 
wrote:

> Hi Konstantine,
>
> We found an issue with replication with IBP 2.7:
> https://issues.apache.org/jira/browse/KAFKA-13207. The fix is small and
> low
> risk and has been merged to trunk. Can we include this in 3.0 branch since
> it can result in IllegalStateException during replication?
>
> Thank you,
>
> Rajini
>
>
> On Thu, Aug 5, 2021 at 6:21 PM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
> > Jose, thanks for the heads up on the 3 new blocker candidates.
> >
> > I read the tickets and they have clear descriptions and implementation
> > details.
> > However, at this stage to be able to make a call and approve new blockers
> > I'd appreciate it if we could get some insight regarding the risk and the
> > necessity of a fix. A rough ETA would also be helpful.
> >
> > Having said that, based on the descriptions and the existence of a few
> > other blockers, I'm tentatively approving KAFKA-13161, KAFKA-13165, and
> > KAFKA-13168 and we might have to make a new assessment if these are the
> > only blockers in the next few days or if we notice a regression during
> > testing.
> >
> > Konstantine
> >
> >
> >
> > On Thu, Aug 5, 2021 at 10:04 AM Konstantine Karantasis <
> > kkaranta...@apache.org> wrote:
> >
> > >
> > > Thanks for reporting this new issue Ryan,
> > >
> > > It's important and this issue seems to have clearly regressed dynamic
> > > default configs in the 3.0 branch.
> > > So, it's approved.
> > >
> > > Konstantine
> > >
> > >
> > > On Wed, Aug 4, 2021 at 4:34 PM José Armando García Sancio
> > >  wrote:
> > >
> > >> Hey all,
> > >>
> > >> For the KIP-500 work for 3.0 we would like to propose the following
> > >> Jiras as blockers:
> > >>
> > >> 1. https://issues.apache.org/jira/browse/KAFKA-13168
> > >> 2. https://issues.apache.org/jira/browse/KAFKA-13165
> > >> 3. https://issues.apache.org/jira/browse/KAFKA-13161
> > >>
> > >> The description for each Jira should have more details.
> > >>
> > >> Thanks,
> > >> -Jose
> > >>
> > >> On Tue, Aug 3, 2021 at 12:14 PM Ryan Dielhenn
> > >>  wrote:
> > >> >
> > >> > Hi Konstantine,
> > >> >
> > >> > I would like to report another bug in KRaft.
> > >> >
> > >> > The ConfigHandler that processes dynamic broker config deltas in
> KRaft
> > >> > expects that the default resource name for dynamic broker configs is
> > the
> > >> > old default entity name used in ZK: "". Since dynamic
> default
> > >> > broker configs are persisted as empty string in the quorum instead
> of
> > >> > "", the brokers are not updating the their default
> > >> configuration
> > >> > when they see empty string as a resource name in the config delta
> and
> > >> are
> > >> > throwing a NumberFormatException when they try to parse the resource
> > >> name
> > >> > to process it as a per-broker configuration.
> > >> >
> > >> > I filed a JIRA: https://issues.apache.org/jira/browse/KAFKA-13160
> > >> >
> > >> > I also have a PR to fix this:
> > >> https://github.com/apache/kafka/pull/11168
> > >> >
> > >> > I think that this should be a blocker for 3.0 because dynamic
> default
> > >> > broker configs will not be usable in KRaft otherwise.
> > >> >
> > >> > Best,
> > >> > Ryan Dielhenn
> > >> >
> > >> > On Sat, Jul 31, 2021 at 10:42 AM Konstantine Karantasis <
> > >> > kkaranta...@apache.org> wrote:
> > >> >
> > >> > > Thanks Ryan,
> > >> > >
> > >> > > Approved. Seems also like a low risk fix.
> > >> > > With that opportunity, let's make sure there are no other configs
> > that
> > >> > > would need a similar validation.
> > >> > >
> > >> > > Konstantine
> > >> > >
> > >> > > On Fri, Jul 30, 2021 at 8:33 AM Ryan Dielhenn
> > >> > >  wrote:
> > >> >

Re: [DISCUSS] Apache Kafka 2.8.1 release

2021-08-15 Thread Konstantine Karantasis

Thanks for volunteering David! +1.

Konstantine

On Thu, Aug 12, 2021 at 1:00 AM David Jacot 
wrote:

> Hi,
>
> I'd like to volunteer to be the release manager for the next bugfix
> release, 2.8.1.
>
> I created the release plan on the wiki:
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.8.1
>
> Best,
> David
>

[jira] [Resolved] (KAFKA-13165) Validate node id, process role and quorum voters

2021-08-11 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-13165.

Resolution: Fixed

> Validate node id, process role and quorum voters
> 
>
> Key: KAFKA-13165
> URL: https://issues.apache.org/jira/browse/KAFKA-13165
> Project: Kafka
>  Issue Type: Sub-task
>  Components: kraft
>Reporter: Jose Armando Garcia Sancio
>Assignee: Ryan Dielhenn
>Priority: Blocker
>  Labels: kip-500
> Fix For: 3.0.0
>
>
> Under certain configuration is possible for the Kafka Server to boot up as a 
> broker only but be the cluster metadata quorum leader. We should validate the 
> configuration to avoid this case.
>  # If the {{process.roles}} contains {{controller}} then the {{node.id}} 
> needs to be in the {{controller.quorum.voters}}
>  # If the {{process.roles}} doesn't contain {{controller}} then the 
> {{node.id}} cannot be in the {{controller.quorum.voters}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-08-05 Thread Konstantine Karantasis

Jose, thanks for the heads up on the 3 new blocker candidates.

I read the tickets and they have clear descriptions and implementation
details.
However, at this stage to be able to make a call and approve new blockers
I'd appreciate it if we could get some insight regarding the risk and the
necessity of a fix. A rough ETA would also be helpful.

Having said that, based on the descriptions and the existence of a few
other blockers, I'm tentatively approving KAFKA-13161, KAFKA-13165, and
KAFKA-13168 and we might have to make a new assessment if these are the
only blockers in the next few days or if we notice a regression during
testing.

Konstantine



On Thu, Aug 5, 2021 at 10:04 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

>
> Thanks for reporting this new issue Ryan,
>
> It's important and this issue seems to have clearly regressed dynamic
> default configs in the 3.0 branch.
> So, it's approved.
>
> Konstantine
>
>
> On Wed, Aug 4, 2021 at 4:34 PM José Armando García Sancio
>  wrote:
>
>> Hey all,
>>
>> For the KIP-500 work for 3.0 we would like to propose the following
>> Jiras as blockers:
>>
>> 1. https://issues.apache.org/jira/browse/KAFKA-13168
>> 2. https://issues.apache.org/jira/browse/KAFKA-13165
>> 3. https://issues.apache.org/jira/browse/KAFKA-13161
>>
>> The description for each Jira should have more details.
>>
>> Thanks,
>> -Jose
>>
>> On Tue, Aug 3, 2021 at 12:14 PM Ryan Dielhenn
>>  wrote:
>> >
>> > Hi Konstantine,
>> >
>> > I would like to report another bug in KRaft.
>> >
>> > The ConfigHandler that processes dynamic broker config deltas in KRaft
>> > expects that the default resource name for dynamic broker configs is the
>> > old default entity name used in ZK: "". Since dynamic default
>> > broker configs are persisted as empty string in the quorum instead of
>> > "", the brokers are not updating the their default
>> configuration
>> > when they see empty string as a resource name in the config delta and
>> are
>> > throwing a NumberFormatException when they try to parse the resource
>> name
>> > to process it as a per-broker configuration.
>> >
>> > I filed a JIRA: https://issues.apache.org/jira/browse/KAFKA-13160
>> >
>> > I also have a PR to fix this:
>> https://github.com/apache/kafka/pull/11168
>> >
>> > I think that this should be a blocker for 3.0 because dynamic default
>> > broker configs will not be usable in KRaft otherwise.
>> >
>> > Best,
>> > Ryan Dielhenn
>> >
>> > On Sat, Jul 31, 2021 at 10:42 AM Konstantine Karantasis <
>> > kkaranta...@apache.org> wrote:
>> >
>> > > Thanks Ryan,
>> > >
>> > > Approved. Seems also like a low risk fix.
>> > > With that opportunity, let's make sure there are no other configs that
>> > > would need a similar validation.
>> > >
>> > > Konstantine
>> > >
>> > > On Fri, Jul 30, 2021 at 8:33 AM Ryan Dielhenn
>> > >  wrote:
>> > >
>> > > > Hey Konstantine,
>> > > >
>> > > > Thanks for the question. If these configs are not validated the
>> user's
>> > > > experience will be affected and upgrades from 3.0 will be harder.
>> > > >
>> > > > Best,
>> > > > Ryan Dielhenn
>> > > >
>> > > > On Thu, Jul 29, 2021 at 3:59 PM Konstantine Karantasis <
>> > > > kkaranta...@apache.org> wrote:
>> > > >
>> > > > > Thanks for reporting this issue Ryan.
>> > > > >
>> > > > > I believe what you mention corresponds to the ticket you created
>> here:
>> > > > > https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-13151
>> > > > >
>> > > > > What happens if the configurations are present but the broker
>> doesn't
>> > > > fail
>> > > > > at startup when configured to run in KRaft mode?
>> > > > > Asking to see if we have any workarounds in our availability.
>> > > > >
>> > > > > Thanks,
>> > > > > Konstantine
>> > > > >
>> > > > > On Thu, Jul 29, 2021 at 2:51 PM Ryan Dielhenn
>> > > > >  wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Disregard log

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-08-05 Thread Konstantine Karantasis

Thanks for reporting this new issue Ryan,

It's important and this issue seems to have clearly regressed dynamic
default configs in the 3.0 branch.
So, it's approved.

Konstantine


On Wed, Aug 4, 2021 at 4:34 PM José Armando García Sancio
 wrote:

> Hey all,
>
> For the KIP-500 work for 3.0 we would like to propose the following
> Jiras as blockers:
>
> 1. https://issues.apache.org/jira/browse/KAFKA-13168
> 2. https://issues.apache.org/jira/browse/KAFKA-13165
> 3. https://issues.apache.org/jira/browse/KAFKA-13161
>
> The description for each Jira should have more details.
>
> Thanks,
> -Jose
>
> On Tue, Aug 3, 2021 at 12:14 PM Ryan Dielhenn
>  wrote:
> >
> > Hi Konstantine,
> >
> > I would like to report another bug in KRaft.
> >
> > The ConfigHandler that processes dynamic broker config deltas in KRaft
> > expects that the default resource name for dynamic broker configs is the
> > old default entity name used in ZK: "". Since dynamic default
> > broker configs are persisted as empty string in the quorum instead of
> > "", the brokers are not updating the their default configuration
> > when they see empty string as a resource name in the config delta and are
> > throwing a NumberFormatException when they try to parse the resource name
> > to process it as a per-broker configuration.
> >
> > I filed a JIRA: https://issues.apache.org/jira/browse/KAFKA-13160
> >
> > I also have a PR to fix this: https://github.com/apache/kafka/pull/11168
> >
> > I think that this should be a blocker for 3.0 because dynamic default
> > broker configs will not be usable in KRaft otherwise.
> >
> > Best,
> > Ryan Dielhenn
> >
> > On Sat, Jul 31, 2021 at 10:42 AM Konstantine Karantasis <
> > kkaranta...@apache.org> wrote:
> >
> > > Thanks Ryan,
> > >
> > > Approved. Seems also like a low risk fix.
> > > With that opportunity, let's make sure there are no other configs that
> > > would need a similar validation.
> > >
> > > Konstantine
> > >
> > > On Fri, Jul 30, 2021 at 8:33 AM Ryan Dielhenn
> > >  wrote:
> > >
> > > > Hey Konstantine,
> > > >
> > > > Thanks for the question. If these configs are not validated the
> user's
> > > > experience will be affected and upgrades from 3.0 will be harder.
> > > >
> > > > Best,
> > > > Ryan Dielhenn
> > > >
> > > > On Thu, Jul 29, 2021 at 3:59 PM Konstantine Karantasis <
> > > > kkaranta...@apache.org> wrote:
> > > >
> > > > > Thanks for reporting this issue Ryan.
> > > > >
> > > > > I believe what you mention corresponds to the ticket you created
> here:
> > > > > https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-13151
> > > > >
> > > > > What happens if the configurations are present but the broker
> doesn't
> > > > fail
> > > > > at startup when configured to run in KRaft mode?
> > > > > Asking to see if we have any workarounds in our availability.
> > > > >
> > > > > Thanks,
> > > > > Konstantine
> > > > >
> > > > > On Thu, Jul 29, 2021 at 2:51 PM Ryan Dielhenn
> > > > >  wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Disregard log.clean.policy being included in this blocker.
> > > > > >
> > > > > > Best,
> > > > > > Ryan Dielhenn
> > > > > >
> > > > > > On Thu, Jul 29, 2021 at 2:38 PM Ryan Dielhenn <
> > > rdielh...@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey Konstantine,
> > > > > > >
> > > > > > > I'd like to report another bug in KRaft.
> > > > > > >
> > > > > > > log.cleanup.policy, alter.config.policy.class.name, and
> > > > > > > create.topic.policy.class.name are all unsupported by KRaft
> but
> > > > KRaft
> > > > > > > servers allow them to be configured. I believe this should be
> > > > > considered
> > > > > > a
> > > > > > > blocker and that KRaft servers should fail startup if any of
> these
> > > > are
> > > > > > > configured. I do not have a PR yet but will soon.
> > > &

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-31 Thread Konstantine Karantasis

Thanks Ryan,

Approved. Seems also like a low risk fix.
With that opportunity, let's make sure there are no other configs that
would need a similar validation.

Konstantine

On Fri, Jul 30, 2021 at 8:33 AM Ryan Dielhenn
 wrote:

> Hey Konstantine,
>
> Thanks for the question. If these configs are not validated the user's
> experience will be affected and upgrades from 3.0 will be harder.
>
> Best,
> Ryan Dielhenn
>
> On Thu, Jul 29, 2021 at 3:59 PM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
> > Thanks for reporting this issue Ryan.
> >
> > I believe what you mention corresponds to the ticket you created here:
> > https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-13151
> >
> > What happens if the configurations are present but the broker doesn't
> fail
> > at startup when configured to run in KRaft mode?
> > Asking to see if we have any workarounds in our availability.
> >
> > Thanks,
> > Konstantine
> >
> > On Thu, Jul 29, 2021 at 2:51 PM Ryan Dielhenn
> >  wrote:
> >
> > > Hi,
> > >
> > > Disregard log.clean.policy being included in this blocker.
> > >
> > > Best,
> > > Ryan Dielhenn
> > >
> > > On Thu, Jul 29, 2021 at 2:38 PM Ryan Dielhenn 
> > > wrote:
> > >
> > > > Hey Konstantine,
> > > >
> > > > I'd like to report another bug in KRaft.
> > > >
> > > > log.cleanup.policy, alter.config.policy.class.name, and
> > > > create.topic.policy.class.name are all unsupported by KRaft but
> KRaft
> > > > servers allow them to be configured. I believe this should be
> > considered
> > > a
> > > > blocker and that KRaft servers should fail startup if any of these
> are
> > > > configured. I do not have a PR yet but will soon.
> > > >
> > > > On another note, I have a PR for the dynamic broker configuration fix
> > > > here: https://github.com/apache/kafka/pull/11141
> > > >
> > > > Best,
> > > > Ryan Dielhenn
> > > >
> > > > On Wed, May 26, 2021 at 2:48 PM Konstantine Karantasis
> > > >  wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> Please find below the updated release plan for the Apache Kafka
> 3.0.0
> > > >> release.
> > > >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177046466
> > > >>
> > > >> New suggested dates for the release are as follows:
> > > >>
> > > >> KIP Freeze is 09 June 2021 (same date as in the initial plan)
> > > >> Feature Freeze is 30 June 2021 (new date, extended by two weeks)
> > > >> Code Freeze is 14 July 2021 (new date, extended by two weeks)
> > > >>
> > > >> At least two weeks of stabilization will follow Code Freeze.
> > > >>
> > > >> The release plan is up to date and currently includes all the
> approved
> > > >> KIPs
> > > >> that are targeting 3.0.0.
> > > >>
> > > >> Please let me know if you have any objections with the recent
> > extension
> > > of
> > > >> Feature Freeze and Code Freeze or any other concerns.
> > > >>
> > > >> Regards,
> > > >> Konstantine
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-29 Thread Konstantine Karantasis

Thanks for reporting this issue Ryan.

I believe what you mention corresponds to the ticket you created here:
https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-13151

What happens if the configurations are present but the broker doesn't fail
at startup when configured to run in KRaft mode?
Asking to see if we have any workarounds in our availability.

Thanks,
Konstantine

On Thu, Jul 29, 2021 at 2:51 PM Ryan Dielhenn
 wrote:

> Hi,
>
> Disregard log.clean.policy being included in this blocker.
>
> Best,
> Ryan Dielhenn
>
> On Thu, Jul 29, 2021 at 2:38 PM Ryan Dielhenn 
> wrote:
>
> > Hey Konstantine,
> >
> > I'd like to report another bug in KRaft.
> >
> > log.cleanup.policy, alter.config.policy.class.name, and
> > create.topic.policy.class.name are all unsupported by KRaft but KRaft
> > servers allow them to be configured. I believe this should be considered
> a
> > blocker and that KRaft servers should fail startup if any of these are
> > configured. I do not have a PR yet but will soon.
> >
> > On another note, I have a PR for the dynamic broker configuration fix
> > here: https://github.com/apache/kafka/pull/11141
> >
> > Best,
> > Ryan Dielhenn
> >
> > On Wed, May 26, 2021 at 2:48 PM Konstantine Karantasis
> >  wrote:
> >
> >> Hi all,
> >>
> >> Please find below the updated release plan for the Apache Kafka 3.0.0
> >> release.
> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177046466
> >>
> >> New suggested dates for the release are as follows:
> >>
> >> KIP Freeze is 09 June 2021 (same date as in the initial plan)
> >> Feature Freeze is 30 June 2021 (new date, extended by two weeks)
> >> Code Freeze is 14 July 2021 (new date, extended by two weeks)
> >>
> >> At least two weeks of stabilization will follow Code Freeze.
> >>
> >> The release plan is up to date and currently includes all the approved
> >> KIPs
> >> that are targeting 3.0.0.
> >>
> >> Please let me know if you have any objections with the recent extension
> of
> >> Feature Freeze and Code Freeze or any other concerns.
> >>
> >> Regards,
> >> Konstantine
> >>
> >
>

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-27 Thread Konstantine Karantasis

Thanks Ryan and Ron for reporting these new issues.

KAFKA-13142 is approved as an issue with no good workaround (and keeping in
mind that we have a few open blockers still) and
KAFKA-13137 is approved as a blocker because it's a regression.

Konstantine




On Tue, Jul 27, 2021 at 5:22 PM Ron Dagostino  wrote:

> Hi Konstantine. I've opened KAFKA-13137 as a potential blocker.  An
> approved PR is available at
> https://github.com/apache/kafka/pull/11131.  The kafka.controller
> metrics that the KRaft controllers expose have the wrong MBean names.
>
> Ron
>
> > On Jul 27, 2021, at 8:13 PM, Ryan Dielhenn 
> wrote:
> >
> > Hello,
>
> >
> > I would like to report a bug in KRaft.
> >
> > Dynamic broker configs are not being validated on the brokers before
> being
> > forwarded to the controller and persisted in the metadata quorum. This
> is a
> > blocker because a core requirement of KRaft mode in 3.0 is that it should
> > support upgrades from 3.0. If invalid dynamic configs are persisted to
> > metadata then they will still be there after an upgrade.
> >
> > I believe that this should be considered as a blocker for 3.0 here is the
> > JIRA: https://issues.apache.org/jira/browse/KAFKA-13142. I will be
> working
> > on a PR this week.
> >
> > Best Regards,
> > Ryan
> >
> >> On Wed, May 26, 2021 at 2:48 PM Konstantine Karantasis
> >>  wrote:
> >>
> >> Hi all,
> >>
> >> Please find below the updated release plan for the Apache Kafka 3.0.0
> >> release.
> >>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177046466
> >>
> >> New suggested dates for the release are as follows:
> >>
> >> KIP Freeze is 09 June 2021 (same date as in the initial plan)
> >> Feature Freeze is 30 June 2021 (new date, extended by two weeks)
> >> Code Freeze is 14 July 2021 (new date, extended by two weeks)
> >>
> >> At least two weeks of stabilization will follow Code Freeze.
> >>
> >> The release plan is up to date and currently includes all the approved
> KIPs
> >> that are targeting 3.0.0.
> >>
> >> Please let me know if you have any objections with the recent extension
> of
> >> Feature Freeze and Code Freeze or any other concerns.
> >>
> >> Regards,
> >> Konstantine
> >>
>

[jira] [Resolved] (KAFKA-13139) Empty response after requesting to restart a connector without the tasks results in NPE

2021-07-27 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-13139.

Resolution: Fixed

> Empty response after requesting to restart a connector without the tasks 
> results in NPE
> ---
>
> Key: KAFKA-13139
> URL: https://issues.apache.org/jira/browse/KAFKA-13139
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 3.0.0
>    Reporter: Konstantine Karantasis
>Assignee: Konstantine Karantasis
>Priority: Blocker
> Fix For: 3.0.0
>
>
> After https://issues.apache.org/jira/browse/KAFKA-4793 a response to restart 
> only the connector (without any tasks) returns OK with an empty body. 
> As system test runs revealed, this causes an NPE in 
> [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L135]
> We should return 204 (NO_CONTENT) instead. 
> This is a regression from previous behavior, therefore the ticket is marked 
> as a blocker candidate for 3.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-26 Thread Konstantine Karantasis

Hi all,

I'd like to report a bug for Connect this time.

System test runs revealed an issue when restarting a connector but not the
tasks.
You may find a description in:
https://issues.apache.org/jira/browse/KAFKA-13139
The underlying issue is related to the implementation of KIP-745.

This makes it a regression for 3.0 and given that it doesn't seem to have a
viable workaround other than not using this specific functionality, I'm
willing to consider it as a blocker for 3.0.0 if there are no objections.
A PR is up for review: https://github.com/apache/kafka/pull/11132

Regards,
Konstantine

On Mon, Jul 26, 2021 at 9:15 AM Konstantine Karantasis <
kkaranta...@apache.org> wrote:

>
> Hi Justine,
>
> Thanks for reporting this issue. Sounds like a regression in 3.0.
> Therefore, it would make sense to try to address it before we release
> 3.0.0.
>
> I marked the jira issue as a Blocker targeting 3.0. Hopefully your PR will
> fix the issue but let me know if we need to make further adjustments.
>
> Konstantine
>
>
> On Sun, Jul 25, 2021 at 7:55 PM Justine Olshan
>  wrote:
>
>> Hi Konstantine,
>> I've discovered a bug with topic IDs that can be encountered when
>> upgrading
>> from IBP versions below 2.8.
>>
>> Since 2.8, when handling leader and isr requests, the request topic IDs
>> are
>> compared to the log IDs and partitions with inconsistent IDs are skipped.
>>
>> With a change introduced in 3.0, in some upgrade scenarios, topic IDs for
>> existing topics will not be assigned to the logs. For topics affected with
>> the issue, we will not be able to check the topic ID. We could potentially
>> handle partitions with inconsistent topic IDs incorrectly.
>>
>> I believe this should be considered a blocker for 3.0. JIRA is here
>> <https://issues.apache.org/jira/browse/KAFKA-13132> and PR to fix the
>> issue
>> here <https://github.com/apache/kafka/pull/11126>.
>>
>> Thanks,
>> Justine
>>
>> On Fri, Jul 23, 2021 at 10:51 AM Konstantine Karantasis <
>> kkaranta...@apache.org> wrote:
>>
>> > Thanks for the PR and the follow up Sophie.
>> >
>> > We can still get this in and there's no risk to do so, given the
>> proposed
>> > changes.
>> > Therefore, I agree to cherry-pick to 3.0 since the PR is about to get
>> > merged.
>> >
>> > Konstantine
>> >
>> > On Thu, Jul 22, 2021 at 9:12 PM Sophie Blee-Goldman
>> >  wrote:
>> >
>> > > Hey Konstantine,
>> > >
>> > > A javadocs ticket of ours was demoted to a non-blocker earlier this
>> week
>> > > due to lack of action,
>> > > but I now have a PR ready and under review. It's picking up some
>> > essential
>> > > followup that was
>> > > missed during the implementation of KIP-633 and is pretty essential. I
>> > > tagged you on the PR,
>> > > it's technically touching on a few things that aren't just docs, but
>> only
>> > > to add a handful of checks
>> > > that already existed on the old APIs and just got missed on the new
>> APIs.
>> > > Anything beyond that
>> > > I left as a TODO to follow up on after 3.0.
>> > >
>> > > KAFKA-13021 <https://issues.apache.org/jira/browse/KAFKA-13021> ---
>> > > https://github.com/apache/kafka/pull/4
>> > >
>> > > I think we should be able to get it merged by tomorrow. Assuming we
>> do,
>> > can
>> > > I promote it back
>> > > to blocker status and pick the fix to the 3.0 branch?
>> > >
>> > > Thanks!
>> > > Sophie
>> > >
>> > > On Thu, Jul 22, 2021 at 4:29 PM Konstantine Karantasis
>> > >  wrote:
>> > >
>> > > > Thanks for raising this John.
>> > > >
>> > > > While we are working to eliminate the existing blockers I think it
>> > would
>> > > be
>> > > > great to use this time in order to test the upgrade path that you
>> > > mention.
>> > > >
>> > > > Before we approve a release candidate (once such a RC is generated)
>> we
>> > > > should confirm that the upgrade works as expected.
>> > > > So, I agree with you that this is not an RC generation blocker per
>> se
>> > but
>> > > > it's a release blocker overall.
>> > > >
>> > > > Konstantine
>> > > >
>> > > >
>> > > >

[jira] [Created] (KAFKA-13139) Empty response after requesting to restart a connector without the tasks results in NPE

2021-07-26 Thread Konstantine Karantasis (Jira)

Konstantine Karantasis created KAFKA-13139:
--

 Summary: Empty response after requesting to restart a connector 
without the tasks results in NPE
 Key: KAFKA-13139
 URL: https://issues.apache.org/jira/browse/KAFKA-13139
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Affects Versions: 3.0.0
Reporter: Konstantine Karantasis
Assignee: Konstantine Karantasis
 Fix For: 3.0.0


After https://issues.apache.org/jira/browse/KAFKA-4793 a response to restart 
only the connector (without any tasks) returns OK with an empty body. 

As system test runs revealed, this causes an NPE in 
[https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L135]

We should return 204 (NO_CONTENT) instead. 

This is a regression from previous behavior, therefore the ticket is marked as 
a blocker candidate for 3.0



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-26 Thread Konstantine Karantasis

Hi Justine,

Thanks for reporting this issue. Sounds like a regression in 3.0.
Therefore, it would make sense to try to address it before we release
3.0.0.

I marked the jira issue as a Blocker targeting 3.0. Hopefully your PR will
fix the issue but let me know if we need to make further adjustments.

Konstantine


On Sun, Jul 25, 2021 at 7:55 PM Justine Olshan 
wrote:

> Hi Konstantine,
> I've discovered a bug with topic IDs that can be encountered when upgrading
> from IBP versions below 2.8.
>
> Since 2.8, when handling leader and isr requests, the request topic IDs are
> compared to the log IDs and partitions with inconsistent IDs are skipped.
>
> With a change introduced in 3.0, in some upgrade scenarios, topic IDs for
> existing topics will not be assigned to the logs. For topics affected with
> the issue, we will not be able to check the topic ID. We could potentially
> handle partitions with inconsistent topic IDs incorrectly.
>
> I believe this should be considered a blocker for 3.0. JIRA is here
> <https://issues.apache.org/jira/browse/KAFKA-13132> and PR to fix the
> issue
> here <https://github.com/apache/kafka/pull/11126>.
>
> Thanks,
> Justine
>
> On Fri, Jul 23, 2021 at 10:51 AM Konstantine Karantasis <
> kkaranta...@apache.org> wrote:
>
> > Thanks for the PR and the follow up Sophie.
> >
> > We can still get this in and there's no risk to do so, given the proposed
> > changes.
> > Therefore, I agree to cherry-pick to 3.0 since the PR is about to get
> > merged.
> >
> > Konstantine
> >
> > On Thu, Jul 22, 2021 at 9:12 PM Sophie Blee-Goldman
> >  wrote:
> >
> > > Hey Konstantine,
> > >
> > > A javadocs ticket of ours was demoted to a non-blocker earlier this
> week
> > > due to lack of action,
> > > but I now have a PR ready and under review. It's picking up some
> > essential
> > > followup that was
> > > missed during the implementation of KIP-633 and is pretty essential. I
> > > tagged you on the PR,
> > > it's technically touching on a few things that aren't just docs, but
> only
> > > to add a handful of checks
> > > that already existed on the old APIs and just got missed on the new
> APIs.
> > > Anything beyond that
> > > I left as a TODO to follow up on after 3.0.
> > >
> > > KAFKA-13021 <https://issues.apache.org/jira/browse/KAFKA-13021> ---
> > > https://github.com/apache/kafka/pull/4
> > >
> > > I think we should be able to get it merged by tomorrow. Assuming we do,
> > can
> > > I promote it back
> > > to blocker status and pick the fix to the 3.0 branch?
> > >
> > > Thanks!
> > > Sophie
> > >
> > > On Thu, Jul 22, 2021 at 4:29 PM Konstantine Karantasis
> > >  wrote:
> > >
> > > > Thanks for raising this John.
> > > >
> > > > While we are working to eliminate the existing blockers I think it
> > would
> > > be
> > > > great to use this time in order to test the upgrade path that you
> > > mention.
> > > >
> > > > Before we approve a release candidate (once such a RC is generated)
> we
> > > > should confirm that the upgrade works as expected.
> > > > So, I agree with you that this is not an RC generation blocker per se
> > but
> > > > it's a release blocker overall.
> > > >
> > > > Konstantine
> > > >
> > > >
> > > > On Thu, Jul 22, 2021 at 4:21 PM John Roesler 
> > > wrote:
> > > >
> > > > > Hello Konstantine,
> > > > >
> > > > > Someone just called to my attention that KAFKA-12724 had not
> > > > > been marked as a 3.0 blocker. We never added 2.8 to the
> > > > > Streams upgrade system test suite. This isn't a blocker in
> > > > > that it is a problem, but we should make sure that Streams
> > > > > is actually upgradable before releasing 3.0.
> > > > >
> > > > > I'm sorry for the oversight. For what it's worth, I think we
> > > > > could proceed with a release candidate while we continue to
> > > > > address the missing system test.
> > > > >
> > > > > Thanks,
> > > > > -John
> > > > >
> > > > > https://issues.apache.org/jira/browse/KAFKA-12724
> > > > >
> > > > > On Wed, 2021-07-21 at 14:00 -0700, Konstantine Karantasis
> > > > > wrote:
> > >

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-23 Thread Konstantine Karantasis

Thanks for the PR and the follow up Sophie.

We can still get this in and there's no risk to do so, given the proposed
changes.
Therefore, I agree to cherry-pick to 3.0 since the PR is about to get
merged.

Konstantine

On Thu, Jul 22, 2021 at 9:12 PM Sophie Blee-Goldman
 wrote:

> Hey Konstantine,
>
> A javadocs ticket of ours was demoted to a non-blocker earlier this week
> due to lack of action,
> but I now have a PR ready and under review. It's picking up some essential
> followup that was
> missed during the implementation of KIP-633 and is pretty essential. I
> tagged you on the PR,
> it's technically touching on a few things that aren't just docs, but only
> to add a handful of checks
> that already existed on the old APIs and just got missed on the new APIs.
> Anything beyond that
> I left as a TODO to follow up on after 3.0.
>
> KAFKA-13021 <https://issues.apache.org/jira/browse/KAFKA-13021> ---
> https://github.com/apache/kafka/pull/4
>
> I think we should be able to get it merged by tomorrow. Assuming we do, can
> I promote it back
> to blocker status and pick the fix to the 3.0 branch?
>
> Thanks!
> Sophie
>
> On Thu, Jul 22, 2021 at 4:29 PM Konstantine Karantasis
>  wrote:
>
> > Thanks for raising this John.
> >
> > While we are working to eliminate the existing blockers I think it would
> be
> > great to use this time in order to test the upgrade path that you
> mention.
> >
> > Before we approve a release candidate (once such a RC is generated) we
> > should confirm that the upgrade works as expected.
> > So, I agree with you that this is not an RC generation blocker per se but
> > it's a release blocker overall.
> >
> > Konstantine
> >
> >
> > On Thu, Jul 22, 2021 at 4:21 PM John Roesler 
> wrote:
> >
> > > Hello Konstantine,
> > >
> > > Someone just called to my attention that KAFKA-12724 had not
> > > been marked as a 3.0 blocker. We never added 2.8 to the
> > > Streams upgrade system test suite. This isn't a blocker in
> > > that it is a problem, but we should make sure that Streams
> > > is actually upgradable before releasing 3.0.
> > >
> > > I'm sorry for the oversight. For what it's worth, I think we
> > > could proceed with a release candidate while we continue to
> > > address the missing system test.
> > >
> > > Thanks,
> > > -John
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-12724
> > >
> > > On Wed, 2021-07-21 at 14:00 -0700, Konstantine Karantasis
> > > wrote:
> > > > Thanks for the heads up Colin.
> > > >
> > > > KAFKA-13112 seems important and of course relevant to what we ship
> with
> > > > 3.0.
> > > > Same for the test failures captured by KAFKA-13095 and KAFKA-12851.
> > > Fixing
> > > > those will increase the stability of our builds.
> > > >
> > > > Therefore, considering these tickets as blockers currently makes
> sense
> > to
> > > > me.
> > > >
> > > > Konstantine
> > > >
> > > >
> > > > On Wed, Jul 21, 2021 at 11:46 AM Colin McCabe 
> > > wrote:
> > > >
> > > > > Hi Konstantine,
> > > > >
> > > > > Thanks for your work on this release! We discovered three blocker
> > bugs
> > > > > which are worth bringing up here:
> > > > >
> > > > > 1. KAFKA-13112: Controller's committed offset get out of sync with
> > raft
> > > > > client listener context
> > > > > 2. KAFKA-13095: TransactionsTest is failing in kraft mode
> > > > > 3. KAFKA-12851: Flaky Test
> > > > > RaftEventSimulationTest.canMakeProgressIfMajorityIsReachable
> > > > >
> > > > > There are two subtasks for #1 which we are working on. We suspect
> > that
> > > #3
> > > > > has been fixed by a previous fix we made... we're looking into it.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > > On Mon, Jul 19, 2021, at 20:23, Konstantine Karantasis wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > Since last week, we have reached the stage of Code Freeze for the
> > > 3.0.0
> > > > > > Apache Kafka release.
> > > > > >
> > > > > > From this point forward and until the official release of 3.0.0,
> > only
> > &g

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-22 Thread Konstantine Karantasis

Thanks for raising this John.

While we are working to eliminate the existing blockers I think it would be
great to use this time in order to test the upgrade path that you mention.

Before we approve a release candidate (once such a RC is generated) we
should confirm that the upgrade works as expected.
So, I agree with you that this is not an RC generation blocker per se but
it's a release blocker overall.

Konstantine


On Thu, Jul 22, 2021 at 4:21 PM John Roesler  wrote:

> Hello Konstantine,
>
> Someone just called to my attention that KAFKA-12724 had not
> been marked as a 3.0 blocker. We never added 2.8 to the
> Streams upgrade system test suite. This isn't a blocker in
> that it is a problem, but we should make sure that Streams
> is actually upgradable before releasing 3.0.
>
> I'm sorry for the oversight. For what it's worth, I think we
> could proceed with a release candidate while we continue to
> address the missing system test.
>
> Thanks,
> -John
>
> https://issues.apache.org/jira/browse/KAFKA-12724
>
> On Wed, 2021-07-21 at 14:00 -0700, Konstantine Karantasis
> wrote:
> > Thanks for the heads up Colin.
> >
> > KAFKA-13112 seems important and of course relevant to what we ship with
> > 3.0.
> > Same for the test failures captured by KAFKA-13095 and KAFKA-12851.
> Fixing
> > those will increase the stability of our builds.
> >
> > Therefore, considering these tickets as blockers currently makes sense to
> > me.
> >
> > Konstantine
> >
> >
> > On Wed, Jul 21, 2021 at 11:46 AM Colin McCabe 
> wrote:
> >
> > > Hi Konstantine,
> > >
> > > Thanks for your work on this release! We discovered three blocker bugs
> > > which are worth bringing up here:
> > >
> > > 1. KAFKA-13112: Controller's committed offset get out of sync with raft
> > > client listener context
> > > 2. KAFKA-13095: TransactionsTest is failing in kraft mode
> > > 3. KAFKA-12851: Flaky Test
> > > RaftEventSimulationTest.canMakeProgressIfMajorityIsReachable
> > >
> > > There are two subtasks for #1 which we are working on. We suspect that
> #3
> > > has been fixed by a previous fix we made... we're looking into it.
> > >
> > > best,
> > > Colin
> > >
> > > On Mon, Jul 19, 2021, at 20:23, Konstantine Karantasis wrote:
> > > > Hi all,
> > > >
> > > > Since last week, we have reached the stage of Code Freeze for the
> 3.0.0
> > > > Apache Kafka release.
> > > >
> > > > From this point forward and until the official release of 3.0.0, only
> > > > critical fixes for blocker issues should be merged to the 3.0 release
> > > > branch.
> > > >
> > > > The release plan currently includes ten (10) such known blockers.
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0
> > > >
> > > > Besides these issues, any new issue that potentially gets discovered
> will
> > > > need to be reported on dev@kafka.apache.org (under this thread) and
> be
> > > > evaluated as a release blocker. At this point, the bar for such
> issues is
> > > > high; they need to be regressions or critical issues without an
> > > acceptable
> > > > workaround to be considered as release blockers.
> > > >
> > > > Exceptions of changes that may be merged to the 3.0 branch without a
> > > > mention on the dev mailing list are fixes for test failures that will
> > > help
> > > > stabilize the build and small documentation changes.
> > > >
> > > > If by the end of this week we are down to zero blockers and have
> green
> > > > builds and passing system tests, I will attempt to generate the first
> > > > Release Candidate (RC) for 3.0.0 on Friday.
> > > >
> > > > Thank you all for the hard work so far.
> > > > Konstantine
> > > >
> > > >
> > > > On Mon, Jul 12, 2021 at 1:59 PM Konstantine Karantasis
> > > >  wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > This is a reminder that Code Freeze for Apache Kafka 3.0 is coming
> up
> > > this
> > > > > week and is set to take place by the end of day Wednesday, July
> 14th.
> > > > >
> > > > > Currently in the project we have 22 blocker issues for 3.0, out of
> 41
> > > total
> > > > > tickets targeting 3.0.
> > > &

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-21 Thread Konstantine Karantasis

Thanks for the heads up Colin.

KAFKA-13112 seems important and of course relevant to what we ship with
3.0.
Same for the test failures captured by KAFKA-13095 and KAFKA-12851. Fixing
those will increase the stability of our builds.

Therefore, considering these tickets as blockers currently makes sense to
me.

Konstantine


On Wed, Jul 21, 2021 at 11:46 AM Colin McCabe  wrote:

> Hi Konstantine,
>
> Thanks for your work on this release! We discovered three blocker bugs
> which are worth bringing up here:
>
> 1. KAFKA-13112: Controller's committed offset get out of sync with raft
> client listener context
> 2. KAFKA-13095: TransactionsTest is failing in kraft mode
> 3. KAFKA-12851: Flaky Test
> RaftEventSimulationTest.canMakeProgressIfMajorityIsReachable
>
> There are two subtasks for #1 which we are working on. We suspect that #3
> has been fixed by a previous fix we made... we're looking into it.
>
> best,
> Colin
>
> On Mon, Jul 19, 2021, at 20:23, Konstantine Karantasis wrote:
> > Hi all,
> >
> > Since last week, we have reached the stage of Code Freeze for the 3.0.0
> > Apache Kafka release.
> >
> > From this point forward and until the official release of 3.0.0, only
> > critical fixes for blocker issues should be merged to the 3.0 release
> > branch.
> >
> > The release plan currently includes ten (10) such known blockers.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0
> >
> > Besides these issues, any new issue that potentially gets discovered will
> > need to be reported on dev@kafka.apache.org (under this thread) and be
> > evaluated as a release blocker. At this point, the bar for such issues is
> > high; they need to be regressions or critical issues without an
> acceptable
> > workaround to be considered as release blockers.
> >
> > Exceptions of changes that may be merged to the 3.0 branch without a
> > mention on the dev mailing list are fixes for test failures that will
> help
> > stabilize the build and small documentation changes.
> >
> > If by the end of this week we are down to zero blockers and have green
> > builds and passing system tests, I will attempt to generate the first
> > Release Candidate (RC) for 3.0.0 on Friday.
> >
> > Thank you all for the hard work so far.
> > Konstantine
> >
> >
> > On Mon, Jul 12, 2021 at 1:59 PM Konstantine Karantasis
> >  wrote:
> >
> > > Hi all,
> > >
> > > This is a reminder that Code Freeze for Apache Kafka 3.0 is coming up
> this
> > > week and is set to take place by the end of day Wednesday, July 14th.
> > >
> > > Currently in the project we have 22 blocker issues for 3.0, out of 41
> total
> > > tickets targeting 3.0.
> > >
> > > You may find the list of open issues in the release plan page:
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0
> > >
> > > Thanks for all the hard work so far and for reducing the number of open
> > > issues in the recent days.
> > > Please take another look and help us resolve all the blockers for this
> > > upcoming major release.
> > >
> > > Best,
> > > Konstantine
> > >
> > > On Mon, Jul 12, 2021 at 1:57 PM Konstantine Karantasis <
> > > konstant...@confluent.io> wrote:
> > >
> > > >
> > > > Thanks for the update Levani,
> > > >
> > > > KIP-708 is now on the list of postponed KIPs.
> > > >
> > > > Konstantine
> > > >
> > > > On Thu, Jul 1, 2021 at 10:48 PM Levani Kokhreidze <
> > > levani.co...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi Konstantine,
> > > >>
> > > >> FYI, I don’t think we will be able to have KIP-708 ready on time.
> > > >> Feel free to remove it from the release plan.
> > > >>
> > > >> Best,
> > > >> Levani
> > > >>
> > > >> > On 1. Jul 2021, at 01:27, Konstantine Karantasis <
> > > >> konstant...@confluent.io.INVALID> wrote:
> > > >> >
> > > >> > Hi all,
> > > >> >
> > > >> > Today we have reached the Feature Freeze milestone for Apache
> Kafka
> > > 3.0.
> > > >> > Exciting!
> > > >> >
> > > >> > I'm going to allow for any pending changes to settle within the
> next
> > > >> couple
> > > >> > of days.

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-19 Thread Konstantine Karantasis

Hi all,

Since last week, we have reached the stage of Code Freeze for the 3.0.0
Apache Kafka release.

>From this point forward and until the official release of 3.0.0, only
critical fixes for blocker issues should be merged to the 3.0 release
branch.

The release plan currently includes ten (10) such known blockers.

https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0

Besides these issues, any new issue that potentially gets discovered will
need to be reported on dev@kafka.apache.org (under this thread) and be
evaluated as a release blocker. At this point, the bar for such issues is
high; they need to be regressions or critical issues without an acceptable
workaround to be considered as release blockers.

Exceptions of changes that may be merged to the 3.0 branch without a
mention on the dev mailing list are fixes for test failures that will help
stabilize the build and small documentation changes.

If by the end of this week we are down to zero blockers and have green
builds and passing system tests, I will attempt to generate the first
Release Candidate (RC) for 3.0.0 on Friday.

Thank you all for the hard work so far.
Konstantine


On Mon, Jul 12, 2021 at 1:59 PM Konstantine Karantasis
 wrote:

> Hi all,
>
> This is a reminder that Code Freeze for Apache Kafka 3.0 is coming up this
> week and is set to take place by the end of day Wednesday, July 14th.
>
> Currently in the project we have 22 blocker issues for 3.0, out of 41 total
> tickets targeting 3.0.
>
> You may find the list of open issues in the release plan page:
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0
>
> Thanks for all the hard work so far and for reducing the number of open
> issues in the recent days.
> Please take another look and help us resolve all the blockers for this
> upcoming major release.
>
> Best,
> Konstantine
>
> On Mon, Jul 12, 2021 at 1:57 PM Konstantine Karantasis <
> konstant...@confluent.io> wrote:
>
> >
> > Thanks for the update Levani,
> >
> > KIP-708 is now on the list of postponed KIPs.
> >
> > Konstantine
> >
> > On Thu, Jul 1, 2021 at 10:48 PM Levani Kokhreidze <
> levani.co...@gmail.com>
> > wrote:
> >
> >> Hi Konstantine,
> >>
> >> FYI, I don’t think we will be able to have KIP-708 ready on time.
> >> Feel free to remove it from the release plan.
> >>
> >> Best,
> >> Levani
> >>
> >> > On 1. Jul 2021, at 01:27, Konstantine Karantasis <
> >> konstant...@confluent.io.INVALID> wrote:
> >> >
> >> > Hi all,
> >> >
> >> > Today we have reached the Feature Freeze milestone for Apache Kafka
> 3.0.
> >> > Exciting!
> >> >
> >> > I'm going to allow for any pending changes to settle within the next
> >> couple
> >> > of days.
> >> > I trust that we all approve and merge adopted features and changes
> >> which we
> >> > consider to be in good shape for 3.0.
> >> >
> >> > Given the 4th of July holiday in the US, the 3.0 release branch will
> >> appear
> >> > sometime on Tuesday, July 6th.
> >> > Until then, please keep merging to trunk only the changes you intend
> to
> >> > include in Apache Kafka 3.0.
> >> >
> >> > Regards,
> >> > Konstantine
> >> >
> >> >
> >> > On Wed, Jun 30, 2021 at 3:25 PM Konstantine Karantasis <
> >> > konstant...@confluent.io> wrote:
> >> >
> >> >>
> >> >> Done. Thanks Luke!
> >> >>
> >> >> On Tue, Jun 29, 2021 at 6:39 PM Luke Chen  wrote:
> >> >>
> >> >>> Hi Konstantine,
> >> >>> We've decided that the KIP-726 will be released in V3.1, not V3.0.
> >> >>> KIP-726: Make the "cooperative-sticky, range" as the default
> assignor
> >> >>>
> >> >>> Could you please remove this KIP from the 3.0 release plan wiki
> page?
> >> >>>
> >> >>> Thank you.
> >> >>> Luke
> >> >>>
> >> >>> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis
> >> >>>  wrote:
> >> >>>
> >> >>>> Thanks for the update Colin.
> >> >>>> They are now both in the release plan.
> >> >>>>
> >> >>>> Best,
> >> >>>> Konstantine
> >> >>>>
> >> >>>> On Tue, Jun 29, 2021 at 2:55 PM Co

[jira] [Resolved] (KAFKA-12803) Support reassigning partitions when in KRaft mode

2021-07-19 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12803.

Resolution: Fixed

Resolving as Fixed given that the PR got merged and cherry-picked to 3.0:
https://github.com/apache/kafka/pull/10753

> Support reassigning partitions when in KRaft mode
> -
>
> Key: KAFKA-12803
> URL: https://issues.apache.org/jira/browse/KAFKA-12803
> Project: Kafka
>  Issue Type: Improvement
>  Components: controller
>Affects Versions: 2.8.0
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Major
>  Labels: kip-500
> Fix For: 3.0.0
>
>
> Support reassigning partitions when in KRaft mode



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-13090) Improve cluster snapshot integration test

2021-07-19 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-13090.

Resolution: Fixed

Resolving given that the PR got merged and cherry-picked to 3.0: 
https://github.com/apache/kafka/pull/11054

> Improve cluster snapshot integration test
> -
>
> Key: KAFKA-13090
> URL: https://issues.apache.org/jira/browse/KAFKA-13090
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Major
>  Labels: kip-500
> Fix For: 3.0.0
>
>
> Extends the test in RaftClusterSnapshotTest to verify that both the 
> controllers and brokers are generating snapshots.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-13073) Simulation test fails due to inconsistency in MockLog's implementation

2021-07-14 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-13073.

Resolution: Fixed

> Simulation test fails due to inconsistency in MockLog's implementation
> --
>
> Key: KAFKA-13073
> URL: https://issues.apache.org/jira/browse/KAFKA-13073
> Project: Kafka
>  Issue Type: Bug
>  Components: controller, replication
>Affects Versions: 3.0.0
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Major
>  Labels: kip-500
> Fix For: 3.0.0
>
>
> We are getting the following error on trunk
> {code:java}
> RaftEventSimulationTest > canRecoverAfterAllNodesKilled STANDARD_OUT
> timestamp = 2021-07-12T16:26:55.663, 
> RaftEventSimulationTest:canRecoverAfterAllNodesKilled =
>   java.lang.RuntimeException:
> Uncaught exception during poll of node 1  
> |---jqwik---
> tries = 25| # of calls to property
> checks = 25   | # of not rejected calls
> generation = RANDOMIZED   | parameters are randomly generated
> after-failure = PREVIOUS_SEED | use the previous seed
> when-fixed-seed = ALLOW   | fixing the random seed is allowed
> edge-cases#mode = MIXIN   | edge cases are mixed in
> edge-cases#total = 108| # of all combined edge cases
> edge-cases#tried = 4  | # of edge cases tried in current run
> seed = 8079861963960994566| random seed to reproduce generated values 
>Sample
> --
>   arg0: 4002
>   arg1: 2
>   arg2: 4{code}
> I think there are a couple of issues here:
>  # The {{ListenerContext}} for {{KafkaRaftClient}} uses the value returned by 
> {{ReplicatedLog::startOffset()}} to determined the log start and when to load 
> a snapshot while the {{MockLog}} implementation uses {{logStartOffset}} which 
> could be a different value.
>  # {{MockLog}} doesn't implement {{ReplicatedLog::maybeClean}} so the log 
> start offset is always 0.
>  # The snapshot id validation for {{MockLog}} and {{KafkaMetadataLog}}'s 
> {{createNewSnapshot}} throws an exception when the snapshot id is less than 
> the log start offset.
> Solutions:
> Fix the error quoted above we only need to fix bullet point 3. but I think we 
> should fix all of the issues enumerated in this Jira.
> For 1. we should change the {{MockLog}} implementation so that it uses 
> {{startOffset}} both externally and internally.
> For 2. I will file another issue to track this implementation.
> For 3. I think this validation is too strict. I think it is safe to simply 
> ignore any attempt by the state machine to create an snapshot with an id less 
> that the log start offset. We should return a {{Optional.empty()}}when the 
> snapshot id is less than the log start offset. This tells the user that it 
> doesn't need to generate a snapshot for that offset. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12308) ConfigDef.parseType deadlock

2021-07-14 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12308.

Resolution: Fixed

> ConfigDef.parseType deadlock
> 
>
> Key: KAFKA-12308
> URL: https://issues.apache.org/jira/browse/KAFKA-12308
> Project: Kafka
>  Issue Type: Bug
>  Components: config, KafkaConnect
>Affects Versions: 2.5.0
> Environment: kafka 2.5.0
> centos7
> java version "1.8.0_231"
>Reporter: cosmozhu
>Priority: Major
> Attachments: deadlock.log
>
>
> hi,
>  the problem was found, when I restarted *ConnectDistributed*
> I restart ConnectDistributed in the single node for the test, with not delete 
> connectors.
>  sometimes the process stopped when creating connectors.
> I add some logger and found it had a deadlock in `ConfigDef.parseType`.My 
> connectors always have the same transforms. I guess when connector startup 
> (in startAndStopExecutor which default 8 threads) and load the same class 
> file it has something wrong.
> I attached the jstack log file.
> thanks for any help.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-7421) Deadlock in Kafka Connect during class loading

2021-07-14 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-7421.
---
Resolution: Fixed

> Deadlock in Kafka Connect during class loading
> --
>
> Key: KAFKA-7421
> URL: https://issues.apache.org/jira/browse/KAFKA-7421
> Project: Kafka
>  Issue Type: Bug
>  Components: KafkaConnect
>Affects Versions: 2.0.0
>Reporter: Maciej Bryński
>    Assignee: Konstantine Karantasis
>Priority: Major
> Fix For: 3.0.0
>
>
> I'm getting this deadlock on half of Kafka Connect runs when having two 
> different types connectors (in this configuration it's debezium and hdfs).
> Thread 1:
> {code}
> "pool-22-thread-2@4748" prio=5 tid=0x4d nid=NA waiting for monitor entry
>   java.lang.Thread.State: BLOCKED
>waiting for pool-22-thread-1@4747 to release lock on <0x1423> (a 
> org.apache.kafka.connect.runtime.isolation.PluginClassLoader)
> at 
> org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:91)
> at 
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.loadClass(DelegatingClassLoader.java:367)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Class.java:-1)
> at java.lang.Class.forName(Class.java:348)
> at 
> org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:715)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.enrich(ConnectorConfig.java:295)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.(ConnectorConfig.java:200)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.(ConnectorConfig.java:194)
> at 
> org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:233)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:916)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1300(DistributedHerder.java:111)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$15.call(DistributedHerder.java:932)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder$15.call(DistributedHerder.java:928)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Thread 2:
> {code}
> "pool-22-thread-1@4747" prio=5 tid=0x4c nid=NA waiting for monitor entry
>   java.lang.Thread.State: BLOCKED
>blocks pool-22-thread-2@4748
>waiting for pool-22-thread-2@4748 to release lock on <0x1421> (a 
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:406)
> at 
> org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.loadClass(DelegatingClassLoader.java:358)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
> - locked <0x1424> (a java.lang.Object)
> at 
> org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
> - locked <0x1423> (a 
> org.apache.kafka.connect.runtime.isolation.PluginClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at 
> io.debezium.transforms.ByLogicalTableRouter.(ByLogicalTableRouter.java:57)
> at java.lang.Class.forName0(Class.java:-1)
> at java.lang.Class.forName(Class.java:348)
> at 
> org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:715)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.enrich(ConnectorConfig.java:295)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.(ConnectorConfig.java:200)
> at 
> org.apache.kafka.connect.runtime.ConnectorConfig.(ConnectorConfig.java:194)
> at 
> org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:233)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:916)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1300(DistributedHerder.java:111)
> at 
> org.apache.kafka.connect.runtime.distributed.DistributedHe

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-12 Thread Konstantine Karantasis

Hi all,

This is a reminder that Code Freeze for Apache Kafka 3.0 is coming up this
week and is set to take place by the end of day Wednesday, July 14th.

Currently in the project we have 22 blocker issues for 3.0, out of 41 total
tickets targeting 3.0.

You may find the list of open issues in the release plan page:
https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0

Thanks for all the hard work so far and for reducing the number of open
issues in the recent days.
Please take another look and help us resolve all the blockers for this
upcoming major release.

Best,
Konstantine

On Mon, Jul 12, 2021 at 1:57 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:

>
> Thanks for the update Levani,
>
> KIP-708 is now on the list of postponed KIPs.
>
> Konstantine
>
> On Thu, Jul 1, 2021 at 10:48 PM Levani Kokhreidze 
> wrote:
>
>> Hi Konstantine,
>>
>> FYI, I don’t think we will be able to have KIP-708 ready on time.
>> Feel free to remove it from the release plan.
>>
>> Best,
>> Levani
>>
>> > On 1. Jul 2021, at 01:27, Konstantine Karantasis <
>> konstant...@confluent.io.INVALID> wrote:
>> >
>> > Hi all,
>> >
>> > Today we have reached the Feature Freeze milestone for Apache Kafka 3.0.
>> > Exciting!
>> >
>> > I'm going to allow for any pending changes to settle within the next
>> couple
>> > of days.
>> > I trust that we all approve and merge adopted features and changes
>> which we
>> > consider to be in good shape for 3.0.
>> >
>> > Given the 4th of July holiday in the US, the 3.0 release branch will
>> appear
>> > sometime on Tuesday, July 6th.
>> > Until then, please keep merging to trunk only the changes you intend to
>> > include in Apache Kafka 3.0.
>> >
>> > Regards,
>> > Konstantine
>> >
>> >
>> > On Wed, Jun 30, 2021 at 3:25 PM Konstantine Karantasis <
>> > konstant...@confluent.io> wrote:
>> >
>> >>
>> >> Done. Thanks Luke!
>> >>
>> >> On Tue, Jun 29, 2021 at 6:39 PM Luke Chen  wrote:
>> >>
>> >>> Hi Konstantine,
>> >>> We've decided that the KIP-726 will be released in V3.1, not V3.0.
>> >>> KIP-726: Make the "cooperative-sticky, range" as the default assignor
>> >>>
>> >>> Could you please remove this KIP from the 3.0 release plan wiki page?
>> >>>
>> >>> Thank you.
>> >>> Luke
>> >>>
>> >>> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis
>> >>>  wrote:
>> >>>
>> >>>> Thanks for the update Colin.
>> >>>> They are now both in the release plan.
>> >>>>
>> >>>> Best,
>> >>>> Konstantine
>> >>>>
>> >>>> On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe 
>> >>> wrote:
>> >>>>
>> >>>>> Hi Konstantine,
>> >>>>>
>> >>>>> Can you please add two KIPs to the 3.0 release plan wiki page?
>> >>>>>
>> >>>>> I'm thinking of:
>> >>>>>KIP-630: Kafka Raft Snapshots
>> >>>>>KIP-746: Revise KRaft Metadata Records
>> >>>>>
>> >>>>> These are marked as 3.0 on the KIP page but I guess we don't have
>> >>> them on
>> >>>>> the page yet.
>> >>>>>
>> >>>>> Many thanks.
>> >>>>> Colin
>> >>>>>
>> >>>>>
>> >>>>> On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote:
>> >>>>>> Hi there,
>> >>>>>>
>> >>>>>> As the feature freeze date is approaching, I just wanted to kindly
>> >>> ask
>> >>>>> for
>> >>>>>> some reviews on the already submitted PR (
>> >>>>>> https://github.com/apache/kafka/pull/10840) that implements the
>> >>>> approved
>> >>>>>> KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR has
>> >>>> been
>> >>>>>> ready for review for 2 weeks, and I simply want to make sure there
>> >>> is
>> >>>>>> enough time to address any possible changes that might be
>> requested.
>> >>>>>>
>> >

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-07-12 Thread Konstantine Karantasis

Thanks for the update Levani,

KIP-708 is now on the list of postponed KIPs.

Konstantine

On Thu, Jul 1, 2021 at 10:48 PM Levani Kokhreidze 
wrote:

> Hi Konstantine,
>
> FYI, I don’t think we will be able to have KIP-708 ready on time.
> Feel free to remove it from the release plan.
>
> Best,
> Levani
>
> > On 1. Jul 2021, at 01:27, Konstantine Karantasis <
> konstant...@confluent.io.INVALID> wrote:
> >
> > Hi all,
> >
> > Today we have reached the Feature Freeze milestone for Apache Kafka 3.0.
> > Exciting!
> >
> > I'm going to allow for any pending changes to settle within the next
> couple
> > of days.
> > I trust that we all approve and merge adopted features and changes which
> we
> > consider to be in good shape for 3.0.
> >
> > Given the 4th of July holiday in the US, the 3.0 release branch will
> appear
> > sometime on Tuesday, July 6th.
> > Until then, please keep merging to trunk only the changes you intend to
> > include in Apache Kafka 3.0.
> >
> > Regards,
> > Konstantine
> >
> >
> > On Wed, Jun 30, 2021 at 3:25 PM Konstantine Karantasis <
> > konstant...@confluent.io> wrote:
> >
> >>
> >> Done. Thanks Luke!
> >>
> >> On Tue, Jun 29, 2021 at 6:39 PM Luke Chen  wrote:
> >>
> >>> Hi Konstantine,
> >>> We've decided that the KIP-726 will be released in V3.1, not V3.0.
> >>> KIP-726: Make the "cooperative-sticky, range" as the default assignor
> >>>
> >>> Could you please remove this KIP from the 3.0 release plan wiki page?
> >>>
> >>> Thank you.
> >>> Luke
> >>>
> >>> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis
> >>>  wrote:
> >>>
> >>>> Thanks for the update Colin.
> >>>> They are now both in the release plan.
> >>>>
> >>>> Best,
> >>>> Konstantine
> >>>>
> >>>> On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe 
> >>> wrote:
> >>>>
> >>>>> Hi Konstantine,
> >>>>>
> >>>>> Can you please add two KIPs to the 3.0 release plan wiki page?
> >>>>>
> >>>>> I'm thinking of:
> >>>>>KIP-630: Kafka Raft Snapshots
> >>>>>KIP-746: Revise KRaft Metadata Records
> >>>>>
> >>>>> These are marked as 3.0 on the KIP page but I guess we don't have
> >>> them on
> >>>>> the page yet.
> >>>>>
> >>>>> Many thanks.
> >>>>> Colin
> >>>>>
> >>>>>
> >>>>> On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote:
> >>>>>> Hi there,
> >>>>>>
> >>>>>> As the feature freeze date is approaching, I just wanted to kindly
> >>> ask
> >>>>> for
> >>>>>> some reviews on the already submitted PR (
> >>>>>> https://github.com/apache/kafka/pull/10840) that implements the
> >>>> approved
> >>>>>> KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR has
> >>>> been
> >>>>>> ready for review for 2 weeks, and I simply want to make sure there
> >>> is
> >>>>>> enough time to address any possible changes that might be requested.
> >>>>>>
> >>>>>> Thanks in advance and sorry for any inconvenience caused,
> >>>>>> --
> >>>>>> Josep
> >>>>>> On Mon, Jun 21, 2021 at 11:54 PM Konstantine Karantasis
> >>>>>>  wrote:
> >>>>>>
> >>>>>>> Thanks for the update Bruno.
> >>>>>>> I've moved KIP-698 to the list of postponed KIPs in the plan.
> >>>>>>>
> >>>>>>> Konstantine
> >>>>>>>
> >>>>>>> On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna  >>>>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Konstantine,
> >>>>>>>>
> >>>>>>>> The implementation of
> >>>>>>>>
> >>>>>>>> KIP-698: Add Explicit User Initialization of Broker-side State
> >>> to
> >>>>> Kafka
> >>>>>>>> Streams
&g

[jira] [Resolved] (KAFKA-10046) Deprecated PartitionGrouper config is ignored

2021-07-09 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-10046.

Resolution: Fixed

> Deprecated PartitionGrouper config is ignored
> -
>
> Key: KAFKA-10046
> URL: https://issues.apache.org/jira/browse/KAFKA-10046
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: John Roesler
>Priority: Blocker
> Fix For: 3.0.0
>
>
> It looks like at some point, we accidentally broke the chain that lets the 
> user-provided PartitionGrouper config actually get to the Consumer-managed 
> StreamsPartitionAssignor. The effect is that any configured value would just 
> be ignored.
> Investigation is needed to determine when this regression took place and 
> whether we should fix it or just remove the config.
> See the discussion in [https://github.com/apache/kafka/pull/8716/] . The 
> TaskAssignorIntegrationTest in that PR can be used to verify this regression.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12419) Remove Deprecated APIs of Kafka Streams in 3.0

2021-07-09 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12419.

Resolution: Fixed

> Remove Deprecated APIs of Kafka Streams in 3.0
> --
>
> Key: KAFKA-12419
> URL: https://issues.apache.org/jira/browse/KAFKA-12419
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams, streams-test-utils
>Reporter: Guozhang Wang
>Assignee: Tomasz Nguyen
>Priority: Blocker
>  Labels: needs-kip
> Fix For: 3.0.0
>
>
> Here's a list of deprecated APIs that we have accumulated in the past, we can 
> consider removing them in 3.0:
>  * KIP-198: "--zookeeper" flag from StreamsResetter (1.0)
>  * KIP-171: "–execute" flag from StreamsResetter (1.1)
>  * KIP-233: overloaded "StreamsBuilder#addGlobalStore" (1.1)
>  * KIP-251: overloaded "ProcessorContext#forward" (2.0)
>  * KIP-276: "StreamsConfig#getConsumerConfig" (2.0)
>  * KIP-319: "WindowBytesStoreSupplier#segments" (2.1)
>  * KIP-321: "TopologyDescription.Source#topics" (2.1)
>  * KIP-328: "Windows#until/segmentInterval/maintainMS" (2.1)
>  * KIP-358: "Windows/Materialized" overloaded functions with `long` (2.1)
>  * KIP-365/366: Implicit Scala Apis (2.1)
>  * KIP-372: overloaded "KStream#groupBy" (2.1)
>  * KIP-307: "Joined#named" (2.3)
>  * KIP-345: Broker config "group.initial.rebalance.delay.ms" (2.3)
>  * KIP-429: "PartitionAssignor" interface (2.4)
>  * KIP-470: "TopologyTestDriver#pipeInput" (2.4)
>  * KIP-476: overloaded "KafkaClientSupplier#getAdminClient" (2.4)
>  * KIP-479: overloaded "KStream#join" (2.4)
>  * KIP-530: old "UsePreviousTimeOnInvalidTimeStamp" (2.5)
>  * KIP-535 / 562: overloaded "KafkaStreams#metadataForKey" and 
> "KafkaStreams#store" (2.5)
> And here's a list of already filed JIRAs for removing deprecated APIs
>  * KAFKA-10434
>  * KAFKA-7785
>  * KAFKA-12796
>  * KAFKA-12809
>  * KAFKA-12808
>  * KAFKA-12813



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12499) Adjust transaction timeout according to commit interval on Streams EOS

2021-07-09 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12499.

Resolution: Fixed

> Adjust transaction timeout according to commit interval on Streams EOS
> --
>
> Key: KAFKA-12499
> URL: https://issues.apache.org/jira/browse/KAFKA-12499
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Reporter: Boyang Chen
>Assignee: Boyang Chen
>Priority: Blocker
> Fix For: 3.0.0
>
>
> The transaction timeout is set to 1 minute by default on Producer today, 
> while the commit interval on the other hand could be set to a very large 
> value, which makes the stream always hit transaction timeout and drop into 
> rebalance. We should increase the transaction timeout correspondingly when 
> commit interval is large.
> On the other hand, broker could have a limit on the max transaction timeout 
> to be set. If we scale up client transaction timeout over the limit, stream 
> will fail due to  INVALID_TRANSACTION_TIMEOUT. To alleviate this problem, 
> user could define their own customized transaction timeout to avoid hitting 
> the limit, so we should still respect what user configures in the override.
> The new rule for configuring transaction timeout should look like:
> 1. If transaction timeout is set in streams config, use it
> 2. if not, transaction_timeout = max(default_transaction_timeout, 10 * 
> commit_interval) 
> Additionally if INVALID_TRANSACTION_TIMEOUT was thrown on Streams when 
> calling initTransaction(), we should wrap the exception to inform user that 
> their setting for commit interval could potentially be too high and should 
> adjust.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12686) Race condition in AlterIsr response handling

2021-07-08 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12686.

Resolution: Fixed

> Race condition in AlterIsr response handling
> 
>
> Key: KAFKA-12686
> URL: https://issues.apache.org/jira/browse/KAFKA-12686
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 2.7.0, 2.8.0
>Reporter: David Arthur
>Assignee: David Arthur
>Priority: Minor
> Fix For: 3.0.0
>
>
> In Partition.scala, there is a race condition between the handling of an 
> AlterIsrResponse and a LeaderAndIsrRequest. This is a pretty rare scenario 
> and would involve the AlterIsrResponse being delayed for some time, but it is 
> possible. This was observed in a test environment when lots of ISR and 
> leadership changes were happening due to broker restarts.
> When the leader handles the LeaderAndIsr, it calls Partition#makeLeader which 
> overrides the {{isrState}} variable and clears the pending ISR items via 
> {{AlterIsrManager#clearPending(TopicPartition)}}. 
> The bug is that AlterIsrManager does not check its inflight state before 
> clearing pending items. The way AlterIsrManager is designed, it retains 
> inflight items in the pending items collection until the response is 
> processed (to allow for retries). The result is that an inflight item is 
> inadvertently removed from this collection.
> Since the inflight item is cleared from the collection, AlterIsrManager 
> allows for new AlterIsrItem-s to be enqueued for this partition even though 
> it has an inflight AlterIsrItem. By allowing an update to be enqueued, 
> Partition will transition its {{isrState}} to one of the inflight states 
> (PendingIsrExpand, PendingIsrShrink, etc). Once the inflight partition's 
> response is handled, it will fail to update the {{isrState}} due to detecting 
> changes since the request was sent (which is by design). However, after the 
> response callback is run, AlterIsrManager will clear the partitions that it 
> saw in the response from the unsent items collection. This includes the newly 
> added (and unsent) update.
> The result is that Partition has a "inflight" isrState but AlterIsrManager 
> does not have an unsent item for this partition. This prevents any further 
> ISR updates on the partition until the next leader election (when 
> {{isrState}} is reset).
> If this bug is encountered, the workaround is to force a leader election 
> which will reset the partition's state.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12792) Fix metrics bug and introduce TimelineInteger

2021-07-08 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12792.

Resolution: Fixed

> Fix metrics bug and introduce TimelineInteger
> -
>
> Key: KAFKA-12792
> URL: https://issues.apache.org/jira/browse/KAFKA-12792
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Major
> Fix For: 3.0.0
>
>
> Introduce a TimelineInteger class which represents a single integer value 
> which can be changed while maintaining snapshot consistency.  Fix a case 
> where a metric value would be corrupted after a snapshot restore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-10587) Rename kafka-mirror-maker CLI command line arguments for KIP-629

2021-07-08 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-10587.

Resolution: Fixed

> Rename kafka-mirror-maker CLI command line arguments for KIP-629
> 
>
> Key: KAFKA-10587
> URL: https://issues.apache.org/jira/browse/KAFKA-10587
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Xavier Léauté
>Assignee: Omnia Ibrahim
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-10292) fix flaky streams/streams_broker_bounce_test.py

2021-07-08 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-10292.

Resolution: Fixed

> fix flaky streams/streams_broker_bounce_test.py
> ---
>
> Key: KAFKA-10292
> URL: https://issues.apache.org/jira/browse/KAFKA-10292
> Project: Kafka
>  Issue Type: Sub-task
>  Components: streams, system tests
>Reporter: Chia-Ping Tsai
>Assignee: Bruno Cadonna
>Priority: Major
> Fix For: 3.0.0
>
>
> {quote}
> Module: kafkatest.tests.streams.streams_broker_bounce_test
> Class:  StreamsBrokerBounceTest
> Method: test_broker_type_bounce
> Arguments:
> \{
>   "broker_type": "leader",
>   "failure_mode": "clean_bounce",
>   "num_threads": 1,
>   "sleep_time_secs": 120
> \}
> {quote}
> {quote}
> Module: kafkatest.tests.streams.streams_broker_bounce_test
> Class:  StreamsBrokerBounceTest
> Method: test_broker_type_bounce
> Arguments:
> \{
>   "broker_type": "controller",
>   "failure_mode": "hard_shutdown",
>   "num_threads": 3,
>   "sleep_time_secs": 120
> \}
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-12931) KIP-746: Revise KRaft Metadata Records

2021-07-08 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-12931.

Resolution: Fixed

> KIP-746: Revise KRaft Metadata Records
> --
>
> Key: KAFKA-12931
> URL: https://issues.apache.org/jira/browse/KAFKA-12931
> Project: Kafka
>  Issue Type: Improvement
>  Components: controller, kraft
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-10619) Producer will enable EOS by default

2021-07-07 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-10619.

Resolution: Fixed

The [PR|https://github.com/apache/kafka/pull/9497] has been merged and will be 
included with AK 3.0

> Producer will enable EOS by default
> ---
>
> Key: KAFKA-10619
> URL: https://issues.apache.org/jira/browse/KAFKA-10619
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Cheng Tan
>Assignee: Cheng Tan
>Priority: Major
>  Labels: needs-kip
> Fix For: 3.0.0
>
>
> This is an after-work for KIP-185. 
> In the producer config,
>  # the default value of `acks` will change to `all`
>  # `enable.idempotence` will change to `true`
> [An analysis of the impact of max.in.flight.requests.per.connection and acks 
> on Producer 
> performance|https://cwiki.apache.org/confluence/display/KAFKA/An+analysis+of+the+impact+of+max.in.flight.requests.per.connection+and+acks+on+Producer+performance]
>  indicates that changing `acks` from `1` to `all` won't increase the latency 
> and decrease the throughput in a significant way.
>  
> KIP: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-6987) Reimplement KafkaFuture with CompletableFuture

2021-07-07 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-6987.
---
Resolution: Fixed

> Reimplement KafkaFuture with CompletableFuture
> --
>
> Key: KAFKA-6987
> URL: https://issues.apache.org/jira/browse/KAFKA-6987
> Project: Kafka
>  Issue Type: Improvement
>  Components: clients
>Affects Versions: 2.0.0
>Reporter: Andras Beni
>Assignee: Tom Bentley
>Priority: Minor
>  Labels: kip, needs-review
> Fix For: 3.0.0
>
>
> KafkaFuture documentation states:
> {{This will eventually become a thin shim on top of Java 8's 
> CompletableFuture.}}
> With Java 7 support dropped in 2.0, it is time to get rid of custom code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

New release branch 3.0

2021-07-06 Thread Konstantine Karantasis

Hi Kafka developers and friends,

The release branch for Apache Kafka 3.0 (with version 3.0.0) has been
created
(https://github.com/apache/kafka/tree/3.0).

The trunk branch is about to be bumped to 3.1.0-SNAPSHOT via
https://github.com/apache/kafka/pull/10981

At this point, I'll be reviewing the open JIRA tickets to move every
non-blocker from this release to the next one.

Going forward, most changes should land only on the trunk branch.

Blockers (existing and new that we discover while testing the release) will
be double-committed.
Please discuss with your reviewers whether your PR should go to trunk or to
trunk as well as 3.0 so they can merge accordingly.

The 3.0 branch going live is an excellent opportunity to focus on testing
the features you aim to include in the upcoming release and making sure
that tests are stable, dependable and offer good coverage.

Please help us test the release and make sure it gets out the door stable
and with high quality.

Sincerely,
Konstantine

[jira] [Resolved] (KAFKA-4793) Kafka Connect: POST /connectors/(string: name)/restart doesn't start failed tasks

2021-06-30 Thread Konstantine Karantasis (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantine Karantasis resolved KAFKA-4793.
---
Resolution: Fixed

> Kafka Connect: POST /connectors/(string: name)/restart doesn't start failed 
> tasks
> -
>
> Key: KAFKA-4793
> URL: https://issues.apache.org/jira/browse/KAFKA-4793
> Project: Kafka
>  Issue Type: Improvement
>  Components: KafkaConnect
>Reporter: Gwen Shapira
>Assignee: Kalpesh Patel
>Priority: Major
>  Labels: needs-kip
> Fix For: 3.0.0
>
>
> Sometimes tasks stop due to repeated failures. Users will want to restart the 
> connector and have it retry after fixing an issue. 
> We expected "POST /connectors/(string: name)/restart" to cause retry of 
> failed tasks, but this doesn't appear to be the case.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-30 Thread Konstantine Karantasis

Hi all,

Today we have reached the Feature Freeze milestone for Apache Kafka 3.0.
Exciting!

I'm going to allow for any pending changes to settle within the next couple
of days.
I trust that we all approve and merge adopted features and changes which we
consider to be in good shape for 3.0.

Given the 4th of July holiday in the US, the 3.0 release branch will appear
sometime on Tuesday, July 6th.
Until then, please keep merging to trunk only the changes you intend to
include in Apache Kafka 3.0.

Regards,
Konstantine


On Wed, Jun 30, 2021 at 3:25 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:

>
> Done. Thanks Luke!
>
> On Tue, Jun 29, 2021 at 6:39 PM Luke Chen  wrote:
>
>> Hi Konstantine,
>> We've decided that the KIP-726 will be released in V3.1, not V3.0.
>> KIP-726: Make the "cooperative-sticky, range" as the default assignor
>>
>> Could you please remove this KIP from the 3.0 release plan wiki page?
>>
>> Thank you.
>> Luke
>>
>> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis
>>  wrote:
>>
>> > Thanks for the update Colin.
>> > They are now both in the release plan.
>> >
>> > Best,
>> > Konstantine
>> >
>> > On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe 
>> wrote:
>> >
>> > > Hi Konstantine,
>> > >
>> > > Can you please add two KIPs to the 3.0 release plan wiki page?
>> > >
>> > > I'm thinking of:
>> > > KIP-630: Kafka Raft Snapshots
>> > > KIP-746: Revise KRaft Metadata Records
>> > >
>> > > These are marked as 3.0 on the KIP page but I guess we don't have
>> them on
>> > > the page yet.
>> > >
>> > > Many thanks.
>> > > Colin
>> > >
>> > >
>> > > On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote:
>> > > > Hi there,
>> > > >
>> > > > As the feature freeze date is approaching, I just wanted to kindly
>> ask
>> > > for
>> > > > some reviews on the already submitted PR (
>> > > > https://github.com/apache/kafka/pull/10840) that implements the
>> > approved
>> > > > KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR has
>> > been
>> > > > ready for review for 2 weeks, and I simply want to make sure there
>> is
>> > > > enough time to address any possible changes that might be requested.
>> > > >
>> > > > Thanks in advance and sorry for any inconvenience caused,
>> > > > --
>> > > > Josep
>> > > > On Mon, Jun 21, 2021 at 11:54 PM Konstantine Karantasis
>> > > >  wrote:
>> > > >
>> > > > > Thanks for the update Bruno.
>> > > > > I've moved KIP-698 to the list of postponed KIPs in the plan.
>> > > > >
>> > > > > Konstantine
>> > > > >
>> > > > > On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna > >
>> > > wrote:
>> > > > >
>> > > > > > Hi Konstantine,
>> > > > > >
>> > > > > > The implementation of
>> > > > > >
>> > > > > > KIP-698: Add Explicit User Initialization of Broker-side State
>> to
>> > > Kafka
>> > > > > > Streams
>> > > > > >
>> > > > > > will not be ready for 3.0, so you can remove it from the list.
>> > > > > >
>> > > > > > Best,
>> > > > > > Bruno
>> > > > > >
>> > > > > > On 15.06.21 07:33, Konstantine Karantasis wrote:
>> > > > > > > Done. Moved it into the table of Adopted KIPs targeting 3.0.0
>> and
>> > > to
>> > > > > the
>> > > > > > > release plan of course.
>> > > > > > > Thanks for catching this Israel.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Konstantine
>> > > > > > >
>> > > > > > > On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo <
>> > israele...@gmail.com>
>> > > > > > wrote:
>> > > > > > >
>> > > > > > >> Konstantine,
>> > > > > > >>
>> > > > > > >> One of mine is missing from this list
>> > >

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-30 Thread Konstantine Karantasis

Done. Thanks Luke!

On Tue, Jun 29, 2021 at 6:39 PM Luke Chen  wrote:

> Hi Konstantine,
> We've decided that the KIP-726 will be released in V3.1, not V3.0.
> KIP-726: Make the "cooperative-sticky, range" as the default assignor
>
> Could you please remove this KIP from the 3.0 release plan wiki page?
>
> Thank you.
> Luke
>
> On Wed, Jun 30, 2021 at 8:23 AM Konstantine Karantasis
>  wrote:
>
> > Thanks for the update Colin.
> > They are now both in the release plan.
> >
> > Best,
> > Konstantine
> >
> > On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe  wrote:
> >
> > > Hi Konstantine,
> > >
> > > Can you please add two KIPs to the 3.0 release plan wiki page?
> > >
> > > I'm thinking of:
> > > KIP-630: Kafka Raft Snapshots
> > > KIP-746: Revise KRaft Metadata Records
> > >
> > > These are marked as 3.0 on the KIP page but I guess we don't have them
> on
> > > the page yet.
> > >
> > > Many thanks.
> > > Colin
> > >
> > >
> > > On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote:
> > > > Hi there,
> > > >
> > > > As the feature freeze date is approaching, I just wanted to kindly
> ask
> > > for
> > > > some reviews on the already submitted PR (
> > > > https://github.com/apache/kafka/pull/10840) that implements the
> > approved
> > > > KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR has
> > been
> > > > ready for review for 2 weeks, and I simply want to make sure there is
> > > > enough time to address any possible changes that might be requested.
> > > >
> > > > Thanks in advance and sorry for any inconvenience caused,
> > > > --
> > > > Josep
> > > > On Mon, Jun 21, 2021 at 11:54 PM Konstantine Karantasis
> > > >  wrote:
> > > >
> > > > > Thanks for the update Bruno.
> > > > > I've moved KIP-698 to the list of postponed KIPs in the plan.
> > > > >
> > > > > Konstantine
> > > > >
> > > > > On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna 
> > > wrote:
> > > > >
> > > > > > Hi Konstantine,
> > > > > >
> > > > > > The implementation of
> > > > > >
> > > > > > KIP-698: Add Explicit User Initialization of Broker-side State to
> > > Kafka
> > > > > > Streams
> > > > > >
> > > > > > will not be ready for 3.0, so you can remove it from the list.
> > > > > >
> > > > > > Best,
> > > > > > Bruno
> > > > > >
> > > > > > On 15.06.21 07:33, Konstantine Karantasis wrote:
> > > > > > > Done. Moved it into the table of Adopted KIPs targeting 3.0.0
> and
> > > to
> > > > > the
> > > > > > > release plan of course.
> > > > > > > Thanks for catching this Israel.
> > > > > > >
> > > > > > > Best,
> > > > > > > Konstantine
> > > > > > >
> > > > > > > On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo <
> > israele...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > >> Konstantine,
> > > > > > >>
> > > > > > >> One of mine is missing from this list
> > > > > > >>
> > > > > > >> KIP-633: Drop 24 hour default of grace period in Streams
> > > > > > >> Please could you include it?
> > > > > > >>
> > > > > > >> Voting has already concluded a long time ago
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Mon, Jun 14, 2021 at 6:08 PM Konstantine Karantasis
> > > > > > >>  wrote:
> > > > > > >>
> > > > > > >>> Hi all.
> > > > > > >>>
> > > > > > >>> KIP Freeze for the next major release of Apache Kafka was
> > reached
> > > > > last
> > > > > > >>> week.
> > > > > > >>>
> > > > > > >>> As of now, 36 KIPs have concluded their voting process and
> have

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-29 Thread Konstantine Karantasis

Thanks for the update Colin.
They are now both in the release plan.

Best,
Konstantine

On Tue, Jun 29, 2021 at 2:55 PM Colin McCabe  wrote:

> Hi Konstantine,
>
> Can you please add two KIPs to the 3.0 release plan wiki page?
>
> I'm thinking of:
> KIP-630: Kafka Raft Snapshots
> KIP-746: Revise KRaft Metadata Records
>
> These are marked as 3.0 on the KIP page but I guess we don't have them on
> the page yet.
>
> Many thanks.
> Colin
>
>
> On Tue, Jun 22, 2021, at 06:29, Josep Prat wrote:
> > Hi there,
> >
> > As the feature freeze date is approaching, I just wanted to kindly ask
> for
> > some reviews on the already submitted PR (
> > https://github.com/apache/kafka/pull/10840) that implements the approved
> > KIP-744 (https://cwiki.apache.org/confluence/x/XIrOCg). The PR has been
> > ready for review for 2 weeks, and I simply want to make sure there is
> > enough time to address any possible changes that might be requested.
> >
> > Thanks in advance and sorry for any inconvenience caused,
> > --
> > Josep
> > On Mon, Jun 21, 2021 at 11:54 PM Konstantine Karantasis
> >  wrote:
> >
> > > Thanks for the update Bruno.
> > > I've moved KIP-698 to the list of postponed KIPs in the plan.
> > >
> > > Konstantine
> > >
> > > On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna 
> wrote:
> > >
> > > > Hi Konstantine,
> > > >
> > > > The implementation of
> > > >
> > > > KIP-698: Add Explicit User Initialization of Broker-side State to
> Kafka
> > > > Streams
> > > >
> > > > will not be ready for 3.0, so you can remove it from the list.
> > > >
> > > > Best,
> > > > Bruno
> > > >
> > > > On 15.06.21 07:33, Konstantine Karantasis wrote:
> > > > > Done. Moved it into the table of Adopted KIPs targeting 3.0.0 and
> to
> > > the
> > > > > release plan of course.
> > > > > Thanks for catching this Israel.
> > > > >
> > > > > Best,
> > > > > Konstantine
> > > > >
> > > > > On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo 
> > > > wrote:
> > > > >
> > > > >> Konstantine,
> > > > >>
> > > > >> One of mine is missing from this list
> > > > >>
> > > > >> KIP-633: Drop 24 hour default of grace period in Streams
> > > > >> Please could you include it?
> > > > >>
> > > > >> Voting has already concluded a long time ago
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Mon, Jun 14, 2021 at 6:08 PM Konstantine Karantasis
> > > > >>  wrote:
> > > > >>
> > > > >>> Hi all.
> > > > >>>
> > > > >>> KIP Freeze for the next major release of Apache Kafka was reached
> > > last
> > > > >>> week.
> > > > >>>
> > > > >>> As of now, 36 KIPs have concluded their voting process and have
> been
> > > > >>> adopted.
> > > > >>> These KIPs are targeting 3.0 (unless it's noted otherwise in the
> > > > release
> > > > >>> plan) and their inclusion as new features will be finalized right
> > > after
> > > > >>> Feature Freeze.
> > > > >>>
> > > > >>> At the high level, out of these 36 KIPs, 11 have been implemented
> > > > already
> > > > >>> and 25 are open or in progress.
> > > > >>> Here is the full list of adopted KIPs:
> > > > >>>
> > > > >>> KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in
> 3.0)
> > > > >>> KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)
> > > > >>> KIP-746: Revise KRaft Metadata Records
> > > > >>> KIP-745: Connect API to restart connector and tasks
> > > > >>> KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface
> with
> > > > >>> internal implementation
> > > > >>> KIP-743: Remove config value 0.10.0-2.4 of Streams built-in
> metrics
> > > > >> version
> > > > >>> config
> > > > >>> KIP-741: Change default serde to be null
> > > > >>> K

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-21 Thread Konstantine Karantasis

Thanks for the update Bruno.
I've moved KIP-698 to the list of postponed KIPs in the plan.

Konstantine

On Mon, Jun 21, 2021 at 2:30 AM Bruno Cadonna  wrote:

> Hi Konstantine,
>
> The implementation of
>
> KIP-698: Add Explicit User Initialization of Broker-side State to Kafka
> Streams
>
> will not be ready for 3.0, so you can remove it from the list.
>
> Best,
> Bruno
>
> On 15.06.21 07:33, Konstantine Karantasis wrote:
> > Done. Moved it into the table of Adopted KIPs targeting 3.0.0 and to the
> > release plan of course.
> > Thanks for catching this Israel.
> >
> > Best,
> > Konstantine
> >
> > On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo 
> wrote:
> >
> >> Konstantine,
> >>
> >> One of mine is missing from this list
> >>
> >> KIP-633: Drop 24 hour default of grace period in Streams
> >> Please could you include it?
> >>
> >> Voting has already concluded a long time ago
> >>
> >>
> >>
> >> On Mon, Jun 14, 2021 at 6:08 PM Konstantine Karantasis
> >>  wrote:
> >>
> >>> Hi all.
> >>>
> >>> KIP Freeze for the next major release of Apache Kafka was reached last
> >>> week.
> >>>
> >>> As of now, 36 KIPs have concluded their voting process and have been
> >>> adopted.
> >>> These KIPs are targeting 3.0 (unless it's noted otherwise in the
> release
> >>> plan) and their inclusion as new features will be finalized right after
> >>> Feature Freeze.
> >>>
> >>> At the high level, out of these 36 KIPs, 11 have been implemented
> already
> >>> and 25 are open or in progress.
> >>> Here is the full list of adopted KIPs:
> >>>
> >>> KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in 3.0)
> >>> KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)
> >>> KIP-746: Revise KRaft Metadata Records
> >>> KIP-745: Connect API to restart connector and tasks
> >>> KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface with
> >>> internal implementation
> >>> KIP-743: Remove config value 0.10.0-2.4 of Streams built-in metrics
> >> version
> >>> config
> >>> KIP-741: Change default serde to be null
> >>> KIP-740: Clean up public API in TaskId and fix TaskMetadata#taskId()
> >>> KIP-738: Removal of Connect's internal converter properties
> >>> KIP-734: Improve AdminClient.listOffsets to return timestamp and offset
> >> for
> >>> the record with the largest timestamp
> >>> KIP-733: Change Kafka Streams default replication factor config
> >>> KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2
> >>> KIP-730: Producer ID generation in KRaft mode
> >>> KIP-726: Make the "cooperative-sticky, range" as the default assignor
> >>> KIP-725: Streamlining configurations for WindowedSerializer and
> >>> WindowedDeserializer.
> >>> KIP-724: Drop support for message formats v0 and v1
> >>> KIP-722: Enable connector client overrides by default
> >>> KIP-721: Enable connector log contexts in Connect Log4j configuration
> >>> KIP-720: Deprecate MirrorMaker v1
> >>> KIP-716: Allow configuring the location of the offset-syncs topic with
> >>> MirrorMaker2
> >>> KIP-715: Expose Committed offset in streams
> >>> KIP-709: Extend OffsetFetch requests to accept multiple group ids.
> >>> KIP-708: Rack awareness for Kafka Streams
> >>> KIP-707: The future of KafkaFuture
> >>> KIP-699: Update FindCoordinator to resolve multiple Coordinators at a
> >> time
> >>> KIP-698: Add Explicit User Initialization of Broker-side State to Kafka
> >>> Streams
> >>> KIP-695: Further Improve Kafka Streams Timestamp Synchronization
> >>> KIP-691: Enhance Transactional Producer Exception Handling
> >>> KIP-679: Producer will enable the strongest delivery guarantee by
> default
> >>> KIP-666: Add Instant-based methods to ReadOnlySessionStore
> >>> KIP-653: Upgrade log4j to log4j2
> >>> KIP-623: Add "internal-topics" option to streams application reset tool
> >>> KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to
> >>> ProcessorContext
> >>> KIP-466: Add support for List serialization and deserialization
> >>> KIP-405: Kafka Tiered Storage
> >>> KIP-3

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Konstantine Karantasis

Thank you Mickael and everyone for the kind replies. I'm honored and
excited to help the project from this role as well.

Best,
Konstantine

On Mon, Jun 21, 2021 at 1:35 PM Bruno Cadonna  wrote:

> Congrats Konstantine!
>
> Best,
> Bruno
>
> On 21.06.21 20:10, Randall Hauch wrote:
> > Congratulations, Konstantine!
> >
> > On Mon, Jun 21, 2021 at 12:39 PM Walker Carlson
> >  wrote:
> >
> >> Congratulations!
> >>
> >> On Mon, Jun 21, 2021 at 12:25 PM Dhruvil Shah
>  >>>
> >> wrote:
> >>
> >>> Congratulations Konstantine! Well deserved!
> >>>
> >>> On Mon, Jun 21, 2021 at 10:20 AM Boyang Chen <
> reluctanthero...@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Congratulations Konstantine!
> >>>>
> >>>> On Mon, Jun 21, 2021 at 10:16 AM Matthias J. Sax 
> >>> wrote:
> >>>>
> >>>>> Congrats!
> >>>>>
> >>>>> On 6/21/21 12:57 PM, Raymond Ng wrote:
> >>>>>> Congrats Konstantine!
> >>>>>>
> >>>>>> /Ray
> >>>>>>
> >>>>>> On Mon, Jun 21, 2021 at 9:45 AM Guozhang Wang 
> >>>>> wrote:
> >>>>>>
> >>>>>>> Congratulations Konstantine!
> >>>>>>>
> >>>>>>> On Mon, Jun 21, 2021 at 9:37 AM Tom Bentley 
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Congratulations Konstantine!
> >>>>>>>>
> >>>>>>>> On Mon, Jun 21, 2021 at 5:33 PM David Jacot
> >>>>>  >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Congrats, Konstantine. Well deserved!
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> David
> >>>>>>>>>
> >>>>>>>>> On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan <
> >>>>>>> ramesh.154...@gmail.com
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Congrats Konstantine
> >>>>>>>>>>
> >>>>>>>>>> On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison <
> >>>> mimai...@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> It's my pleasure to announce that Konstantine Karantasis is
> >> now
> >>> a
> >>>>>>>>>>> member of the Kafka PMC.
> >>>>>>>>>>>
> >>>>>>>>>>> Konstantine has been a Kafka committer since Feb 2020. He has
> >>>>>>>> remained
> >>>>>>>>>>> active in the community since becoming a committer.
> >>>>>>>>>>>
> >>>>>>>>>>> Congratulations Konstantine!
> >>>>>>>>>>>
> >>>>>>>>>>> Mickael, on behalf of the Apache Kafka PMC
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> -- Guozhang
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-14 Thread Konstantine Karantasis

Done. Moved it into the table of Adopted KIPs targeting 3.0.0 and to the
release plan of course.
Thanks for catching this Israel.

Best,
Konstantine

On Mon, Jun 14, 2021 at 7:40 PM Israel Ekpo  wrote:

> Konstantine,
>
> One of mine is missing from this list
>
> KIP-633: Drop 24 hour default of grace period in Streams
> Please could you include it?
>
> Voting has already concluded a long time ago
>
>
>
> On Mon, Jun 14, 2021 at 6:08 PM Konstantine Karantasis
>  wrote:
>
> > Hi all.
> >
> > KIP Freeze for the next major release of Apache Kafka was reached last
> > week.
> >
> > As of now, 36 KIPs have concluded their voting process and have been
> > adopted.
> > These KIPs are targeting 3.0 (unless it's noted otherwise in the release
> > plan) and their inclusion as new features will be finalized right after
> > Feature Freeze.
> >
> > At the high level, out of these 36 KIPs, 11 have been implemented already
> > and 25 are open or in progress.
> > Here is the full list of adopted KIPs:
> >
> > KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in 3.0)
> > KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)
> > KIP-746: Revise KRaft Metadata Records
> > KIP-745: Connect API to restart connector and tasks
> > KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface with
> > internal implementation
> > KIP-743: Remove config value 0.10.0-2.4 of Streams built-in metrics
> version
> > config
> > KIP-741: Change default serde to be null
> > KIP-740: Clean up public API in TaskId and fix TaskMetadata#taskId()
> > KIP-738: Removal of Connect's internal converter properties
> > KIP-734: Improve AdminClient.listOffsets to return timestamp and offset
> for
> > the record with the largest timestamp
> > KIP-733: Change Kafka Streams default replication factor config
> > KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2
> > KIP-730: Producer ID generation in KRaft mode
> > KIP-726: Make the "cooperative-sticky, range" as the default assignor
> > KIP-725: Streamlining configurations for WindowedSerializer and
> > WindowedDeserializer.
> > KIP-724: Drop support for message formats v0 and v1
> > KIP-722: Enable connector client overrides by default
> > KIP-721: Enable connector log contexts in Connect Log4j configuration
> > KIP-720: Deprecate MirrorMaker v1
> > KIP-716: Allow configuring the location of the offset-syncs topic with
> > MirrorMaker2
> > KIP-715: Expose Committed offset in streams
> > KIP-709: Extend OffsetFetch requests to accept multiple group ids.
> > KIP-708: Rack awareness for Kafka Streams
> > KIP-707: The future of KafkaFuture
> > KIP-699: Update FindCoordinator to resolve multiple Coordinators at a
> time
> > KIP-698: Add Explicit User Initialization of Broker-side State to Kafka
> > Streams
> > KIP-695: Further Improve Kafka Streams Timestamp Synchronization
> > KIP-691: Enhance Transactional Producer Exception Handling
> > KIP-679: Producer will enable the strongest delivery guarantee by default
> > KIP-666: Add Instant-based methods to ReadOnlySessionStore
> > KIP-653: Upgrade log4j to log4j2
> > KIP-623: Add "internal-topics" option to streams application reset tool
> > KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to
> > ProcessorContext
> > KIP-466: Add support for List serialization and deserialization
> > KIP-405: Kafka Tiered Storage
> > KIP-390: Support Compression Level
> >
> > If you notice that a KIP is missing from the list above and should be
> part
> > of the release plan for 3.0, please reply below.
> > The single source of truth remains the official release plan for 3.0,
> which
> > you may read at any time here:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0
> >
> > Kind reminder that for all the adopted KIPs any required changes to the
> > documentation are also part of their respective feature.
> >
> > For the KIPs that are still in progress, please work closely with your
> > reviewers to make sure that the features are stable and land on time for
> > Feature Freeze.
> >
> > The next milestone for the Apache Kafka 3.0 release is Feature Freeze on
> > June 30th, 2021.
> >
> > Best regards,
> > Konstantine
> >
> > On Fri, Jun 4, 2021 at 3:47 PM Konstantine Karantasis <
> > konstant...@confluent.io> wrote:
> >
> > >
> > > Hi all,
> > >
> > > Just a quick reminder that KIP Freeze is next We

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates

2021-06-14 Thread Konstantine Karantasis

Hi all.

KIP Freeze for the next major release of Apache Kafka was reached last
week.

As of now, 36 KIPs have concluded their voting process and have been
adopted.
These KIPs are targeting 3.0 (unless it's noted otherwise in the release
plan) and their inclusion as new features will be finalized right after
Feature Freeze.

At the high level, out of these 36 KIPs, 11 have been implemented already
and 25 are open or in progress.
Here is the full list of adopted KIPs:

KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in 3.0)
KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)
KIP-746: Revise KRaft Metadata Records
KIP-745: Connect API to restart connector and tasks
KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface with
internal implementation
KIP-743: Remove config value 0.10.0-2.4 of Streams built-in metrics version
config
KIP-741: Change default serde to be null
KIP-740: Clean up public API in TaskId and fix TaskMetadata#taskId()
KIP-738: Removal of Connect's internal converter properties
KIP-734: Improve AdminClient.listOffsets to return timestamp and offset for
the record with the largest timestamp
KIP-733: Change Kafka Streams default replication factor config
KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2
KIP-730: Producer ID generation in KRaft mode
KIP-726: Make the "cooperative-sticky, range" as the default assignor
KIP-725: Streamlining configurations for WindowedSerializer and
WindowedDeserializer.
KIP-724: Drop support for message formats v0 and v1
KIP-722: Enable connector client overrides by default
KIP-721: Enable connector log contexts in Connect Log4j configuration
KIP-720: Deprecate MirrorMaker v1
KIP-716: Allow configuring the location of the offset-syncs topic with
MirrorMaker2
KIP-715: Expose Committed offset in streams
KIP-709: Extend OffsetFetch requests to accept multiple group ids.
KIP-708: Rack awareness for Kafka Streams
KIP-707: The future of KafkaFuture
KIP-699: Update FindCoordinator to resolve multiple Coordinators at a time
KIP-698: Add Explicit User Initialization of Broker-side State to Kafka
Streams
KIP-695: Further Improve Kafka Streams Timestamp Synchronization
KIP-691: Enhance Transactional Producer Exception Handling
KIP-679: Producer will enable the strongest delivery guarantee by default
KIP-666: Add Instant-based methods to ReadOnlySessionStore
KIP-653: Upgrade log4j to log4j2
KIP-623: Add "internal-topics" option to streams application reset tool
KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext
KIP-466: Add support for List serialization and deserialization
KIP-405: Kafka Tiered Storage
KIP-390: Support Compression Level

If you notice that a KIP is missing from the list above and should be part
of the release plan for 3.0, please reply below.
The single source of truth remains the official release plan for 3.0, which
you may read at any time here:

https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.0.0

Kind reminder that for all the adopted KIPs any required changes to the
documentation are also part of their respective feature.

For the KIPs that are still in progress, please work closely with your
reviewers to make sure that the features are stable and land on time for
Feature Freeze.

The next milestone for the Apache Kafka 3.0 release is Feature Freeze on
June 30th, 2021.

Best regards,
Konstantine

On Fri, Jun 4, 2021 at 3:47 PM Konstantine Karantasis <
konstant...@confluent.io> wrote:

>
> Hi all,
>
> Just a quick reminder that KIP Freeze is next Wednesday, June 9th.
> A vote thread needs to be open for at least 72 hours, so to everyone that
> is working hard on proposals targeting 3.0.0, please make sure that your
> [VOTE] threads are started on time.
>
> Best,
> Konstantine
>
>
> On Wed, May 26, 2021 at 8:10 PM Israel Ekpo  wrote:
>
>> +1 on the new schedule.
>>
>> On Wed, May 26, 2021 at 8:14 PM Sophie Blee-Goldman
>>  wrote:
>>
>> > Ah ok, thanks Konstantine. I won't bug you about every new KIP that
>> comes
>> > in between now and KIP Freeze :P
>> >
>> > +1 on the scheduling changes as well
>> >
>> > On Wed, May 26, 2021 at 4:00 PM David Arthur  wrote:
>> >
>> > > The new schedule looks good to me, +1
>> > >
>> > > On Wed, May 26, 2021 at 6:29 PM Ismael Juma 
>> wrote:
>> > >
>> > > > Thanks Konstantine, +1 from me.
>> > > >
>> > > > Ismael
>> > > >
>> > > > On Wed, May 26, 2021 at 2:48 PM Konstantine Karantasis
>> > > >  wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > Please find below the updated release plan for the Apache Kafka
>> 3.0.0
>> > > > >

Re: [VOTE] KIP-716: Allow configuring the location of the offset-syncs topic with MirrorMaker2

2021-06-11 Thread Konstantine Karantasis

Providing this flexibility makes sense. Thanks Mickael.

+1 (binding)

Konstantine

On Fri, Jun 11, 2021 at 1:31 AM Tom Bentley  wrote:

> Hi Mickael,
>
> Thanks for the KIP, +1 (binding).
>
> Cheers,
>
> Tom
>
> On Tue, Apr 13, 2021 at 4:56 PM Mickael Maison 
> wrote:
>
> > Hi,
> >
> > It has been a few weeks since I opened this vote and I have not seen
> > any replies yet.
> > Let me know if you have any feedback.
> >
> > Thanks
> >
> > On Wed, Mar 24, 2021 at 7:04 PM Mickael Maison  >
> > wrote:
> > >
> > > Hi,
> > >
> > > I'd like to start a vote on KIP-716: Allow configuring the location of
> > > the offset-syncs topic with MirrorMaker2.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-716%3A+Allow+configuring+the+location+of+the+offset-syncs+topic+with+MirrorMaker2
> > >
> > > Thanks
> >
> >
>

Re: [VOTE] KIP-719: Add Log4J2 Appender

2021-06-11 Thread Konstantine Karantasis

+1 (binding)

Konstantine

On Fri, Jun 11, 2021 at 7:57 AM Boojapho O  wrote:

> +1 (non-binding)
>
> On 2021/06/09 15:31:13, Dongjin Lee  wrote:
> > Bumping up the voting thread.
> >
> > Please note that today is the KIP freeze day.
> >
> > Thanks,
> > Dongjin
> >
> > On Mon, Jun 7, 2021 at 9:28 PM Dongjin Lee  wrote:
> >
> > > Bumping up the voting thread.
> > >
> > >
> > > As a reminder: Please note that without the Log4j2 appender, we can't
> > > complete KIP-653: Upgrade log4j to log4j2
> > > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2>
> (accepted)
> > > since we can't entirely remove the log4j artifact from the classpath
> for
> > > the tools dependency.
> > >
> > >
> > > Best,
> > >
> > > Dongjin
> > >
> > > On Tue, May 25, 2021 at 10:45 PM Dongjin Lee 
> wrote:
> > >
> > >> Hi Kafka dev,
> > >>
> > >> I'd like to kick-off the voting for KIP-719: Add Log4J2 Appender.
> > >>
> > >>
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender
> > >>
> > >> Best,
> > >> Dongjin
> > >>
> > >> --
> > >> *Dongjin Lee*
> > >>
> > >> *A hitchhiker in the mathematical world.*
> > >>
> > >>
> > >>
> > >> *github:  github.com/dongjinleekr
> > >> keybase:
> https://keybase.io/dongjinleekr
> > >> linkedin:
> kr.linkedin.com/in/dongjinleekr
> > >> speakerdeck:
> speakerdeck.com/dongjin
> > >> *
> > >>
> > >
> > >
> > > --
> > > *Dongjin Lee*
> > >
> > > *A hitchhiker in the mathematical world.*
> > >
> > >
> > >
> > > *github:  github.com/dongjinleekr
> > > keybase:
> https://keybase.io/dongjinleekr
> > > linkedin:
> kr.linkedin.com/in/dongjinleekr
> > > speakerdeck:
> speakerdeck.com/dongjin
> > > *
> > >
> >
> >
> > --
> > *Dongjin Lee*
> >
> > *A hitchhiker in the mathematical world.*
> >
> >
> >
> > *github:  github.com/dongjinleekr
> > keybase:
> https://keybase.io/dongjinleekr
> > linkedin:
> kr.linkedin.com/in/dongjinleekr
> > speakerdeck:
> speakerdeck.com/dongjin
> > *
> >
>

Re: [VOTE] KIP-390: Support Compression Level (rebooted)

2021-06-10 Thread Konstantine Karantasis

Makes sense. Looks like a good improvement. Thanks for including the
evaluation in the proposal Dongjin.

+1 (binding)

Konstantine

On Wed, Jun 9, 2021 at 6:59 PM Dongjin Lee  wrote:

> Thanks Ismel, Tom and Ryanne,
>
> I am now updating the KIP about the further works. Sure, You won't be
> disappointed.
>
> As of Present:
>
> - binding: +2 (Ismael, Tom)
> - non-binding: +1 (Ryanne)
>
> Anyone else?
>
> Best,
> Dongjin
>
> On Thu, Jun 10, 2021 at 2:03 AM Tom Bentley  wrote:
>
> > Hi Dongjin,
> >
> > Thanks for the KIP, +1 (binding).
> >
> > Kind regards,
> >
> > Tom
> >
> > On Wed, Jun 9, 2021 at 5:16 PM Ismael Juma  wrote:
> >
> > > I'm +1 on the proposed change. As I stated in the discuss thread, I
> don't
> > > think we should rule out the buffer size config, but we could list that
> > as
> > > future work vs rejected alternatives.
> > >
> > > Ismael
> > >
> > > On Sat, Jun 5, 2021 at 2:37 PM Dongjin Lee  wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'd like to open a voting thread for KIP-390: Support Compression
> Level
> > > > (rebooted):
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-390%3A+Support+Compression+Level
> > > >
> > > > Best,
> > > > Dongjin
> > > >
> > > > --
> > > > *Dongjin Lee*
> > > >
> > > > *A hitchhiker in the mathematical world.*
> > > >
> > > >
> > > >
> > > > *github:  github.com/dongjinleekr
> > > > keybase:
> > > https://keybase.io/dongjinleekr
> > > > linkedin:
> > > kr.linkedin.com/in/dongjinleekr
> > > > speakerdeck:
> > > > speakerdeck.com/dongjin
> > > > *
> > > >
> > >
> >
>
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
>
>
> *github:  github.com/dongjinleekr
> keybase: https://keybase.io/dongjinleekr
> linkedin: kr.linkedin.com/in/dongjinleekr
> speakerdeck:
> speakerdeck.com/dongjin
> *
>

Re: [VOTE] KIP-726: Make the "cooperative-sticky, range" as the default assignor

2021-06-09 Thread Konstantine Karantasis

Thanks for the KIP Luke.

+1 (binding)

Konstantine


On Wed, Jun 9, 2021 at 6:36 AM Luke Chen  wrote:

> Hi all,
> Bump this thread to see if there's anyone wants to vote or have questions.
>
> Thank you.
> Luke
>
> Sophie Blee-Goldman  於 2021年6月8日 週二 上午4:40
> 寫道：
>
> > +1 (binding)
> >
> > Thanks Luke
> >
> > On Mon, Jun 7, 2021 at 3:26 AM Luke Chen  wrote:
> >
> > > Hi Ismael,
> > > Thanks for your comments. I updated the KIP for the "Compatibility,
> > Upgrade
> > > path" section.
> > > Simply put, no special upgrade path is necessary.
> > >
> > > Thank you.
> > > Luke
> > >
> > >
> > > On Mon, Jun 7, 2021 at 4:16 PM Ismael Juma  wrote:
> > >
> > > > Hi Luke,
> > > >
> > > > The KIP is a bit unclear with regards to whether two rolling bounces
> > are
> > > > required or not. What exactly is being proposed?
> > > >
> > > > Ismael
> > > >
> > > > On Wed, Jun 2, 2021, 8:16 PM Luke Chen  wrote:
> > > >
> > > > > Hi all,
> > > > > I'd like to call for a vote on KIP-726: Make the
> "cooperative-sticky,
> > > > > range" as the default assignor.
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177048248
> > > > >
> > > > > This KIP is still waiting for the prerequisite stories to get
> > > completed,
> > > > > but it should be soon. Hopefully this can be put into V3.0 since
> > > > > cooperative rebalancing is a major
> > > > > improvement to the life of a consumer group (and its operator).
> > > > >
> > > > > The discussion thread can be found here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ref63417ea84a58c9068ea025b3ad38ca2cc340f5818ac07cd83452b7%40%3Cdev.kafka.apache.org%3E
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-494 Connect REST Endpoint for Transformations (SMTs) and other Plugins

2021-06-08 Thread Konstantine Karantasis

That's a valid point based on what the implementation is today. But this
new endpoint aims to be an extension and a superset in terms of
functionality of the existing `/connector-plugins` endpoint.

This last discussion and the fact that the endpoint `/plugins/connector`
that you suggest is almost synonymous to `/connector-plugins' makes me
think that we might have not thought through how this new endpoint will
evolve and look as a top level endpoint for the various plugin types.

Konstantine



On Mon, Jun 7, 2021 at 5:02 PM Cyrus Vafadari  wrote:

> Since it is just two endpoints, I agree it's fine to move them over.
>
> However, the endpoint you suggest would imply that someone could attempt to
> validate the configs of an SMT or Header Converter, since it doesn't
> specify the plugin's type. So if you agree I'll update the proposal to move
> the PUT endpoint to
>
> PUT /plugins/connector/{connector-type}/config/validate
>
> so we don't have to return 400s for all the other types for which
> validation is not supported.
>
> Does that sound good to you?
>
>
> On Mon, Jun 7, 2021 at 4:49 PM Konstantine Karantasis
>  wrote:
>
> > Hi Cyrus,
> >
> > Given that `/connector-plugins` is a top level endpoint, I think the
> > suggestion to deprecate `/connector-plugins` should affect all the nested
> > endpoints.
> >
> > This should be straightforward here (in terms of KIP changes but also
> > implementation), because the endpoints are only two:
> >
> > GET /connector-plugins
> > and
> > PUT /connector-plugins/{connector-type}/config/validate
> >
> > I find deprecating both of them in favor of:
> > GET /plugins
> > and
> > PUT /plugins/{connector-type}/config/validate
> >
> > more clear and aligned with how we want the new endpoint to look like.
> >
> > Wdyt?
> >
> > Konstantine
> >
> >
> > On Mon, Jun 7, 2021 at 4:18 PM Cyrus Vafadari 
> wrote:
> >
> > > I wasn't trying to suggest deprecating all verbs and endpoints prefixed
> > > with `/connector-plugins`, just the `GET: /connector-plugins` as I
> > detailed
> > > in the KIP.   Maybe I'm not understanding your recommendation, or
> > there's a
> > > way I can make the KIP text clearer? Or do you disagree this is the
> right
> > > move tactically? I thought my text was sufficient: "it will be
> redundant
> > to
> > > this new, more generally useful endpoint."
> > >
> > > On Mon, Jun 7, 2021 at 12:56 PM Konstantine Karantasis
> > >  wrote:
> > >
> > > > Thanks for the response Cyrus.
> > > >
> > > > If we are suggesting deprecation of `/connector-plugins` in favor of
> > > > `/plugins` I think we should mention explicitly that the new endpoint
> > > > covers the existing functionality precisely, including the validation
> > > > endpoint for connector plugins.
> > > > I'd recommend extending the spec to refer to this, adding an example
> to
> > > > make this clear and mentioning that the two endpoints will be
> > equivalent
> > > > with respect to existing functionality in the upgrade section.
> > > >
> > > > Konstantine
> > > >
> > > > On Mon, Jun 7, 2021 at 10:57 AM Cyrus Vafadari 
> > > > wrote:
> > > >
> > > > > I will move to vote
> > > > >
> > > > > On Mon, Jun 7, 2021 at 10:44 AM Cyrus Vafadari <
> cvafad...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Thanks for the feedback,
> > > > > >
> > > > > > I added an example request/response for SMTs, and I thought about
> > > your
> > > > > > suggestion re:deprecation and am now explicitly proposing to mark
> > the
> > > > > > existing endpoint as deprecated, though I don't anticipate the
> need
> > > to
> > > > > > remove it will arise any time soon!
> > > > > >
> > > > > > Cyrus
> > > > > >
> > > > > > On Fri, Jun 4, 2021 at 7:35 PM Konstantine Karantasis
> > > > > >  wrote:
> > > > > >
> > > > > >> Hi Cyrus.
> > > > > >>
> > > > > >> The proposal looks good and I like the API spec definition the
> way
> > > > it's
> > > > > >> presented.
> > > > > >>
> > > > > >> Having said that, a few examples that wou

Re: [DISCUSS] KIP-726: Make the CooperativeStickyAssignor as the default assignor

2021-06-08 Thread Konstantine Karantasis

Thanks for the KIP Luke.

Looks good overall. Just a few minor suggestions:

1. How about replacing
"Note that this change will also propagate to Connect." with -> "Note that
this change will also automatically be inherited by sink connectors, like
any other application that uses Kafka consumers, as long as a consumer
assignor is not explicitly defined in their configuration."

The current sentence is a bit general and it would be good to avoid any
confusion with Connect's rebalancing protocol for tasks, which already
supports a single bounce upgrade given that rebalancing protocols in
Connect have a linear lineage until today.

2. Let's remove the placeholder text from the Rejected Alternatives and
simply state that there aren't any. Unless something is worth mentioning in
that section.

Thanks,
Konstantine

On Tue, Jun 8, 2021 at 5:20 AM Luke Chen  wrote:

> Hi Ryan,
> Thanks for the comments. The KAFKA-12896
>  is already set as
> blocker for V3.0, which means, V3.0 won't be released before KAFKA-12896
>  is fixed. I think this
> KIP and KAFKA-12896 
> and
> work in parallel for V3.0 without conflict.
>
> Thank you.
> Luke
>
> On Tue, Jun 8, 2021 at 11:30 AM Ryan Leslie (BLOOMBERG/ 919 3RD A) <
> rles...@bloomberg.net> wrote:
>
> > Hey guys,
> >
> > Should open bugs concerning cooperative-sticky also be considered
> blockers
> > to making it the default? For example, KAFKA-12896 is perhaps still being
> > investigated:
> >
> > https://issues.apache.org/jira/browse/KAFKA-12896
> >
> > Thanks,
> >
> > Ryan
> >
> > From: dev@kafka.apache.org At: 06/07/21 19:37:45 UTC-4:00To:
> > dev@kafka.apache.org
> > Subject: Re: [DISCUSS] KIP-726: Make the CooperativeStickyAssignor as the
> > default assignor
> >
> > Thanks Luke. We may as well get this KIP in to 3.0 so that we can fully
> > enable cooperative rebalancing
> > by default in 3.0 if we have KAFKA-12477 done in time, and if we don't
> then
> > there's no harm as it's
> > not going to change the behavior.
> >
> > On Wed, Jun 2, 2021 at 7:28 PM Luke Chen  wrote:
> >
> > > Hi Sophie,
> > > Thanks for the reminder. Yes, I was thinking this KIP doesn't have to
> be
> > > put into a major release since it will be fully backward compatible, so
> > no
> > > need to push it. Currently, if we want to work on this KIP, we need
> > > KAFKA-12477 and KAFKA-12487. But you're right, we can at least try our
> > best
> > > to see if we can make it into V3.0 since cooperative rebalancing is a
> > major
> > > improvement. I'll kick off a vote later.
> > >
> > > Thank you.
> > > Luke
> > >
> > > On Thu, Jun 3, 2021 at 7:08 AM Sophie Blee-Goldman
> > >  wrote:
> > >
> > > > Hey Luke,
> > > >
> > > > It's been a while since the last update on this, which is mostly my
> > fault
> > > > for picking up
> > > > other things in the meantime. I'm planning to get back to work
> > > > on KAFKA-12477 next
> > > > week but there are minimal changes to the current implementation
> given
> > > the
> > > > proposal
> > > > I put forth earlier in this KIP discussion, so I think we're good to
> > go.
> > > >
> > > > Although this KIP no longer requires a major release since it should
> be
> > > > fully compatible, I
> > > > still hope we can get it in to 3.0 since cooperative rebalancing is a
> > > major
> > > > improvement to
> > > > the life of a consumer group (and its operator). Can we make sure the
> > KIP
> > > > reflects the latest
> > > > and then kick off a vote by next Monday at the latest so we can make
> > KIP
> > > > freeze?
> > > >
> > > > Thanks!
> > > > Sophie
> > > >
> > > > On Fri, Apr 16, 2021 at 2:33 PM Guozhang Wang 
> > > wrote:
> > > >
> > > > > 1) From user's perspective, it is always possible that a commit
> > within
> > > > > onPartitionsRevoked throw in practice (e.g. if the member missed
> the
> > > > > previous rebalance where its assigned partitions are already
> > > re-assigned)
> > > > > -- and the onPartitionsLost was introduced for that exact reason,
> > i.e.
> > > it
> > > > > is primarily for optimizations, but not for correctness guarantees
> --
> > > on
> > > > > the other hand, it would be surprising to users to see the commit
> > > returns
> > > > > and then later found it not going through. Given that, I'd suggest
> we
> > > > still
> > > > > throw the exception right away. Regarding the flag itself though, I
> > > agree
> > > > > that keeping it set until the next succeeded join group makes sense
> > to
> > > be
> > > > > safer.
> > > > >
> > > > > 2) That's crystal, thank you for the clarification.
> > > > >
> > > > > On Wed, Apr 14, 2021 at 6:46 PM Sophie Blee-Goldman
> > > > >  wrote:
> > > > >
> > > > > > 1) Once the short-circuit is triggered, the member will downgrade
> > to
> > > > the
> > > > > > EAGER protocol, but
> > > > > > won't necessarily try to rejoin the group right away.
> > > > > >

Re: [DISCUSS] KIP-494 Connect REST Endpoint for Transformations (SMTs) and other Plugins

Hi Cyrus,

Given that `/connector-plugins` is a top level endpoint, I think the
suggestion to deprecate `/connector-plugins` should affect all the nested
endpoints.

This should be straightforward here (in terms of KIP changes but also
implementation), because the endpoints are only two:

GET /connector-plugins
and
PUT /connector-plugins/{connector-type}/config/validate

I find deprecating both of them in favor of:
GET /plugins
and
PUT /plugins/{connector-type}/config/validate

more clear and aligned with how we want the new endpoint to look like.

Wdyt?

Konstantine


On Mon, Jun 7, 2021 at 4:18 PM Cyrus Vafadari  wrote:

> I wasn't trying to suggest deprecating all verbs and endpoints prefixed
> with `/connector-plugins`, just the `GET: /connector-plugins` as I detailed
> in the KIP.   Maybe I'm not understanding your recommendation, or there's a
> way I can make the KIP text clearer? Or do you disagree this is the right
> move tactically? I thought my text was sufficient: "it will be redundant to
> this new, more generally useful endpoint."
>
> On Mon, Jun 7, 2021 at 12:56 PM Konstantine Karantasis
>  wrote:
>
> > Thanks for the response Cyrus.
> >
> > If we are suggesting deprecation of `/connector-plugins` in favor of
> > `/plugins` I think we should mention explicitly that the new endpoint
> > covers the existing functionality precisely, including the validation
> > endpoint for connector plugins.
> > I'd recommend extending the spec to refer to this, adding an example to
> > make this clear and mentioning that the two endpoints will be equivalent
> > with respect to existing functionality in the upgrade section.
> >
> > Konstantine
> >
> > On Mon, Jun 7, 2021 at 10:57 AM Cyrus Vafadari 
> > wrote:
> >
> > > I will move to vote
> > >
> > > On Mon, Jun 7, 2021 at 10:44 AM Cyrus Vafadari 
> > > wrote:
> > >
> > > > Thanks for the feedback,
> > > >
> > > > I added an example request/response for SMTs, and I thought about
> your
> > > > suggestion re:deprecation and am now explicitly proposing to mark the
> > > > existing endpoint as deprecated, though I don't anticipate the need
> to
> > > > remove it will arise any time soon!
> > > >
> > > > Cyrus
> > > >
> > > > On Fri, Jun 4, 2021 at 7:35 PM Konstantine Karantasis
> > > >  wrote:
> > > >
> > > >> Hi Cyrus.
> > > >>
> > > >> The proposal looks good and I like the API spec definition the way
> > it's
> > > >> presented.
> > > >>
> > > >> Having said that, a few examples that would list the request type
> and
> > > >> body,
> > > >> the returned status and the json response would be nice too,
> following
> > > the
> > > >> tradition of other KIPs.
> > > >>
> > > >> See
> > > >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-745%3A+Connect+API+to+restart+connector+and+tasks
> > > >> for a recent example.
> > > >>
> > > >> I also see there's no mention regarding the future of the current
> > > >> `/connector-plugins` endpoint or any deprecation plans.
> > > >>
> > > >> I think we should make our intentions clear in the KIP itself
> > > >> which introduces the new endpoint as a superset of the old one,
> beyond
> > > the
> > > >> discussion in this thread, for future readers.
> > > >> Also, I think it's useful to keep in mind that deprecation doesn't
> > > >> necessarily mean imminent removal. The next opportunity to remove
> this
> > > end
> > > >> point would be the next major release at the earliest.
> > > >>
> > > >> Regards,
> > > >> Konstantine
> > > >>
> > > >>
> > > >> On Mon, Apr 19, 2021 at 10:13 AM Cyrus Vafadari <
> cvafad...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Thanks! Anyone else from the community with final thoughts before
> > > going
> > > >> to
> > > >> > vote?
> > > >> >
> > > >> > On Mon, Apr 19, 2021 at 4:16 AM Tom Bentley 
> > > >> wrote:
> > > >> >
> > > >> > > Hi Cyrus,
> > > >> > >
> > > >> > > That seems reasonable to me.
> > > >&

Re: [VOTE] KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in 3.0)

+1 (binding)

Konstantine

On Mon, Jun 7, 2021 at 6:51 AM Josep Prat 
wrote:

> Hi Ismael,
>
> Good KIP, it's a +1 (non binding) from my side.
>
> Best,
>
> ———
> Josep Prat
>
> Aiven Deutschland GmbH
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> m: +491715557497
>
> w: aiven.io
>
> e: josep.p...@aiven.io
>
> On Mon, Jun 7, 2021, 15:49 Ismael Juma  wrote:
>
> > Since the goal is to provide ample warning regarding the future removal
> of
> > Scala 2.12 and the change we're proposing for 3.0 is simply a
> documentation
> > update, I am starting the vote:
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181308218
> >
> > If there are concerns or objections, feel free to point them out in this
> > thread or the discuss thread.
> >
> > Ismael
> >
>

Re: [VOTE] KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)

Thanks for the KIP Ismael. I agree that giving an early enough deprecation
notice for an upcoming change like this one and doing so in a major release
is best in this case.

+1 (binding)

Konstantine

On Mon, Jun 7, 2021 at 7:30 AM Ryanne Dolan  wrote:

> +1 (non-binding)
>
> Ryanne
>
> On Mon, Jun 7, 2021, 9:26 AM Satish Duggana 
> wrote:
>
> > +1 (non-binding)
> >
> > On Mon, 7 Jun 2021 at 19:30, Dongjin Lee  wrote:
> >
> > > +1 (non-binding)
> > >
> > > As a note: I think the exact removal schedule may be changed.
> > >
> > > Best,
> > > Dongjin
> > >
> > > On Mon, Jun 7, 2021, 10:56 PM Josep Prat 
> > > wrote:
> > >
> > > > Thanks Ismael, it's a +1 (non-binding) from my side,
> > > >
> > > >
> > > > Best,
> > > >
> > > > ———
> > > > Josep Prat
> > > >
> > > > Aiven Deutschland GmbH
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > m: +491715557497
> > > >
> > > > w: aiven.io
> > > >
> > > > e: josep.p...@aiven.io
> > > >
> > > > On Mon, Jun 7, 2021, 15:51 Ismael Juma  wrote:
> > > >
> > > > > Since the goal is to provide ample warning regarding the future
> > removal
> > > > of
> > > > > Java 8 support and the change we're proposing for 3.0 is simply a
> > > > > documentation update, I am starting the vote:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181308223
> > > > >
> > > > > If there are concerns or objections, feel free to point them out in
> > > this
> > > > > thread or the discuss thread.
> > > > >
> > > > > Ismael
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] KIP-745: Connect API to restart connector and tasks

Thanks Randall.

+1 (binding)

Konstantine

On Mon, Jun 7, 2021 at 12:47 PM Randall Hauch  wrote:

> Hello all,
>
> I would like to start a vote on KIP-745:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-745%3A+Connect+API+to+restart+connector+and+tasks
>
> +1 (binding) from myself.
>
> Thanks, and best regards!
>
> Randall
>

Re: [DISCUSS] KIP-494 Connect REST Endpoint for Transformations (SMTs) and other Plugins

Thanks for the response Cyrus.

If we are suggesting deprecation of `/connector-plugins` in favor of
`/plugins` I think we should mention explicitly that the new endpoint
covers the existing functionality precisely, including the validation
endpoint for connector plugins.
I'd recommend extending the spec to refer to this, adding an example to
make this clear and mentioning that the two endpoints will be equivalent
with respect to existing functionality in the upgrade section.

Konstantine

On Mon, Jun 7, 2021 at 10:57 AM Cyrus Vafadari  wrote:

> I will move to vote
>
> On Mon, Jun 7, 2021 at 10:44 AM Cyrus Vafadari 
> wrote:
>
> > Thanks for the feedback,
> >
> > I added an example request/response for SMTs, and I thought about your
> > suggestion re:deprecation and am now explicitly proposing to mark the
> > existing endpoint as deprecated, though I don't anticipate the need to
> > remove it will arise any time soon!
> >
> > Cyrus
> >
> > On Fri, Jun 4, 2021 at 7:35 PM Konstantine Karantasis
> >  wrote:
> >
> >> Hi Cyrus.
> >>
> >> The proposal looks good and I like the API spec definition the way it's
> >> presented.
> >>
> >> Having said that, a few examples that would list the request type and
> >> body,
> >> the returned status and the json response would be nice too, following
> the
> >> tradition of other KIPs.
> >>
> >> See
> >>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-745%3A+Connect+API+to+restart+connector+and+tasks
> >> for a recent example.
> >>
> >> I also see there's no mention regarding the future of the current
> >> `/connector-plugins` endpoint or any deprecation plans.
> >>
> >> I think we should make our intentions clear in the KIP itself
> >> which introduces the new endpoint as a superset of the old one, beyond
> the
> >> discussion in this thread, for future readers.
> >> Also, I think it's useful to keep in mind that deprecation doesn't
> >> necessarily mean imminent removal. The next opportunity to remove this
> end
> >> point would be the next major release at the earliest.
> >>
> >> Regards,
> >> Konstantine
> >>
> >>
> >> On Mon, Apr 19, 2021 at 10:13 AM Cyrus Vafadari 
> >> wrote:
> >>
> >> > Thanks! Anyone else from the community with final thoughts before
> going
> >> to
> >> > vote?
> >> >
> >> > On Mon, Apr 19, 2021 at 4:16 AM Tom Bentley 
> >> wrote:
> >> >
> >> > > Hi Cyrus,
> >> > >
> >> > > That seems reasonable to me.
> >> > >
> >> > > On Thu, Apr 15, 2021 at 6:44 PM Cyrus Vafadari  >
> >> > > wrote:
> >> > >
> >> > > > What do you think of "type" field remaining in, and returning
> >> > > > "transformation", "converter" or whatever type it is. This way the
> >> > schema
> >> > > > remains consistent, and you can programmatically understand what
> >> > plugins
> >> > > > are returned on the holistic "GET /plugins" endpoint? It will be
> >> > slightly
> >> > > > redundant in the case where you specify the plugin-type as a path
> >> > > > parameter.
> >> > > >
> >> > > > On Thu, Apr 15, 2021 at 1:13 PM Tom Bentley 
> >> > wrote:
> >> > > >
> >> > > > > Hi Cyrus,
> >> > > > >
> >> > > > > Re 2: A very minor thing but while type=source|sink for a
> >> connector,
> >> > it
> >> > > > > doesn't makes sense for the other plugin types, but so the json
> >> for
> >> > > those
> >> > > > > plugins should omit that property rather than have type=null.
> >> > > > >
> >> > > > > Apart from that it seems reasonable to me. Thanks again,
> >> > > > >
> >> > > > > Tom
> >> > > > >
> >> > > > > On Thu, Apr 15, 2021 at 3:49 PM Cyrus Vafadari <
> >> cvafad...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Tom,
> >> > > > > >
> >> > > > > > Thanks for taking the time to respond with thoughtful
> questions:
> >> > > > >

Re: [DISCUSS] KIP-745: Connect API to restart connector and tasks

Thanks Randall. I reply inline as well.


On Sat, Jun 5, 2021 at 1:57 PM Randall Hauch  wrote:

> Thanks for the review and feedback, Konstantine. I think I've addressed all
> of your concerns below.
>
> On Fri, Jun 4, 2021 at 7:55 PM Konstantine Karantasis
>  wrote:
>
> > Thanks for the KIP Randall!
> > Really happy to see this feature coming along finally. Will indeed
> resolve
> > an unintuitive aspect of Connect's REST API.
> >
> > I'm in favor of the current approach overall. Just a few comments below
> > that could use a brief discussion:
> >
> > 1) Is there a specific reason why you choose the config topic to persist
> > the restart requests?
> > I know it doesn't really matter which internal topic is used for
> backwards
> > compatibility, because the worker ignores records that does not
> understand.
> >
>
> I think there's value in the restart request record being ordered along
> with the other config topic records, including configuration records,
> target state change records, and session key updates. This also aligns with
> how the distributed herder will process the restart requests within its
> `tick()` method, again making sure there are no ongoing starts and stops
> from rebalances or new connectors while the restarts are taking place. Plus
> there is already a built-in config topic listener mechanism in the
> distributed herder.
>
> I think those are good reasons to use the config topic. But there also is
> not a viable alternative: the status topic has no existing listener
> mechanism in the StatusBackingStore, and the offset topic is clearly not
> applicable.
>

These are good points that in practice make the config topic preferrable.
Even though some sound more like implementation details, I think saying
explicitly why the config topic gives us a good tradeoff (you mostly say
it) and why the status topic is not preferred (some info is missing here,
especially regarding the listener) is valuable.
Also a bit of a nit but mentioning the "tick()" method might not be clear
for the uninitiated with the internals of the Connect framework.
Calling this the main thread loop of the workers might be simpler.


>
> > But are there any implications upon worker restart? I read you
> specifically
> > mention that a worker will read restart requests upon its own restart.
> But
> > is this really desirable?
> >
>
> It will read them, just like all of the config records, target state change
> records, and session key updates. However, upon worker restart the herder
> will ignore any restart requests, just like it ignores session key update
> and task state change records.
>
>
> > The config topic offers a way to persist configurations of the latest
> > running connectors and upon cluster restart to bring these connectors
> back
> > up.
> > However, I'm not sure we want to persist recent restart attempts, the
> same
> > way we persist configurations. Additionally configurations might get
> > additional backups in order to allow for the config topic to be recovered
> > for disaster recovery reasons. To that respect, I'm not sure that the
> > restart requests require the same guarantees.
> >
>
> This is not the first or only non-config record written to this topic, so
> there's already precedent.
>

I acknowledge the precedent but we might need to draw a line at some point
and make sure we maintain more permanent information in the config topic
and more transient in the status topic, modulo any other concerns or
trade-offs.
As discussed above, I agree that the config topic is a better option here.


>
> > Given the above, would the status topic offer a better alternative in
> order
> > to broadcast the restart requests across the workers? Haven't looked
> > closely what would be the implications of writing these new records to
> the
> > status topic, but this topic tends to be more transient.
> > Or is it that you've preferred the config topic for the workers to be
> aware
> > of the interleaving between restart requests and requests for
> > reconfigurations? If that's the case an explicit call out might be
> > worthwhile in the KIP itself.
> >
>
> I guess I thought I already tried to do that. But I can add a bit more
> detail to the KIP to call this out even more.
>

+1. Indeed there's some explanation on that already. Maybe a subsection
header and a juxtaposition with the status topic could help to highlight
the reason behind this selection in a more clear way.


>
>
> >
> > 2) You mention in the KIP the status STOPPED. But this is not currently
> > part of the task or connector states that get persisted to the status
> >

Re: [DISCUSS] KIP-494 Connect REST Endpoint for Transformations (SMTs) and other Plugins

Hi Cyrus.

The proposal looks good and I like the API spec definition the way it's
presented.

Having said that, a few examples that would list the request type and body,
the returned status and the json response would be nice too, following the
tradition of other KIPs.

See
https://cwiki.apache.org/confluence/display/KAFKA/KIP-745%3A+Connect+API+to+restart+connector+and+tasks
for a recent example.

I also see there's no mention regarding the future of the current
`/connector-plugins` endpoint or any deprecation plans.

I think we should make our intentions clear in the KIP itself
which introduces the new endpoint as a superset of the old one, beyond the
discussion in this thread, for future readers.
Also, I think it's useful to keep in mind that deprecation doesn't
necessarily mean imminent removal. The next opportunity to remove this end
point would be the next major release at the earliest.

Regards,
Konstantine


On Mon, Apr 19, 2021 at 10:13 AM Cyrus Vafadari  wrote:

> Thanks! Anyone else from the community with final thoughts before going to
> vote?
>
> On Mon, Apr 19, 2021 at 4:16 AM Tom Bentley  wrote:
>
> > Hi Cyrus,
> >
> > That seems reasonable to me.
> >
> > On Thu, Apr 15, 2021 at 6:44 PM Cyrus Vafadari 
> > wrote:
> >
> > > What do you think of "type" field remaining in, and returning
> > > "transformation", "converter" or whatever type it is. This way the
> schema
> > > remains consistent, and you can programmatically understand what
> plugins
> > > are returned on the holistic "GET /plugins" endpoint? It will be
> slightly
> > > redundant in the case where you specify the plugin-type as a path
> > > parameter.
> > >
> > > On Thu, Apr 15, 2021 at 1:13 PM Tom Bentley 
> wrote:
> > >
> > > > Hi Cyrus,
> > > >
> > > > Re 2: A very minor thing but while type=source|sink for a connector,
> it
> > > > doesn't makes sense for the other plugin types, but so the json for
> > those
> > > > plugins should omit that property rather than have type=null.
> > > >
> > > > Apart from that it seems reasonable to me. Thanks again,
> > > >
> > > > Tom
> > > >
> > > > On Thu, Apr 15, 2021 at 3:49 PM Cyrus Vafadari 
> > > > wrote:
> > > >
> > > > > Hi Tom,
> > > > >
> > > > > Thanks for taking the time to respond with thoughtful questions:
> > > > >
> > > > > 1. I propose HTTP-400, will update the KIP to reflect this proposal
> > > > > 2. I think the schema should be of the same format. It is quite
> > > minimal,
> > > > so
> > > > > there is room to add fields in the future without breaking
> > > compatibility.
> > > > > I will update the KIP to specify.
> > > > > 3. Yes, that's right.
> > > > > 4. I personally don't see the value in deprecating since it's a
> > simple
> > > > > alias (interface and impl will both be simple changes). I would be
> > > > > comfortable kicking that can down the road if/when there is a need
> > for
> > > an
> > > > > actual breaking change. That way we can keep the scope and diff
> here
> > > > tight.
> > > > > But iff the community feels strongly that deprecating is the right
> > > thing
> > > > to
> > > > > do, I'm happy to update the KIP to propose deprecating.
> > > > >
> > > > > Thanks again!
> > > > >
> > > > > On Wed, Apr 14, 2021 at 9:38 AM Tom Bentley 
> > > wrote:
> > > > >
> > > > > > Hi Cyrus,
> > > > > >
> > > > > > Thanks for the KIP. A few questions:
> > > > > >
> > > > > > 1. What status code does the /plugins/{plugin-type} endpoint
> return
> > > > when
> > > > > an
> > > > > > unknown type is given?
> > > > > > 2. The result of /connector-plugins is a list of objects with
> > > 'class',
> > > > > > 'type' and 'version' properties. Presumably /plugins/connector is
> > the
> > > > > same,
> > > > > > but what is the schema for the other plugin types?
> > > > > > 3. You're not proposing an equivalent of the
> > > > > > /connector-plugins/{connectorType}/config/validate endpoint for
> > > > > > non-connector types, which I think makes sense, because you would
> > > > > validate
> > > > > > an SMT's config via its used by a connector, so the existing
> > endpoint
> > > > > > suffices, right?
> > > > > > 4. Would /connector-plugins eventually be deprecated in favour of
> > > > > > /plugins/connector, or do we expect it to remain in the API
> > > > indefinitely
> > > > > > and accept that /connector-plugins and /plugins/connector provide
> > > > > identical
> > > > > > responses? If we had the intention to deprecate in the future
> maybe
> > > we
> > > > > > should add a /plugins/connector/config/validate endpoint now?
> > > > > >
> > > > > > Many thanks,
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > On Tue, Apr 13, 2021 at 6:46 PM Cyrus Vafadari <
> > cvafad...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello all,
> > > > > > >
> > > > > > > As the number of connect plugins, SMT's, etc have grown, I
> wanted
> > > to
> > > > > bump
> > > > > > > this thread to see if there is more interest in adding a
> Connect
> > > REST
> > > > > > > endpoint to

Re: [VOTE] KIP-738: Removal of Connect's internal converter properties

+1 (binding)  but also with the note that we don't tend to require a KIP in
order to remove configurations that have been deprecated in previous KIPs.

A jira ticket with the right release as a target should suffice. In future
KIPs adding a note that says that a feature is deprecated and may be
removed in future major releases can help make this clear in the future.

Konstantine.


On Thu, Jun 3, 2021 at 10:47 AM Tom Bentley  wrote:

> +1 (binding).
>
> Thanks Chris!
>
> On Wed, Jun 2, 2021 at 11:00 PM Randall Hauch  wrote:
>
> > Thanks for putting this together, Chris.
> >
> > Technically, we don't need a new KIP to explicitly remove an API, config,
> > etc. that was previously deprecated under an earlier approved KIP. But
> > KIP-174 could have been a bit more explicit (note taken for future KIPs)
> > that deprecation means "deprecation and future removal", and KIP-738 does
> > add a nice migration path for anyone stuck using a non-default internal
> > converter.
> >
> > So I'm +1 (binding) to just wrap this up and remove these configs.
> >
> > I'd also point out there is a wiki page [1] and dev discussion thread [2]
> > highlighting all Connect-related KIPs, including all deprecations. I'll
> > update that page to include KIP-736, but it'd be great to have more
> > discussion there.
> >
> > Randall
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177047362
> > [2]
> >
> >
> https://lists.apache.org/thread.html/r01e9fa52998a337e435e5d0effca02e74b0552bdec271c1eeca39cd2%40%3Cdev.kafka.apache.org%3E
> >
> > On Tue, May 18, 2021 at 11:26 AM Ryanne Dolan 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Thanks!
> > >
> > > Ryanne
> > >
> > > On Tue, May 18, 2021, 6:38 AM Chris Egerton
>  > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'd like to call for a vote on KIP-738:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-738%3A+Removal+of+Connect%27s+internal+converter+properties
> > > >
> > > > The discussion thread (which was originally titled with "KIP-736")
> can
> > be
> > > > found here:
> > > >
> > > >
> > >
> >
> https://mail-archives.apache.org/mod_mbox/kafka-dev/202105.mbox/%3CCAMdOrUV0bqqs-ry7Q6FkNNn21ZhODTrg2d61zE5WZJw1MpQvSQ%40mail.gmail.com%3E
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > >
> >
>

Re: [VOTE] KIP-722: Enable connector client overrides by default

Good to flip the switch on this useful feature with the opportunity of the
major release here as well.
I don't see any disadvantages in doing so.

+1 (binding)

Konstantine

On Thu, May 6, 2021 at 6:07 AM Ryanne Dolan  wrote:

> +1 (non-binding) Thanks!
>
> Ryanne
>
> On Wed, May 5, 2021, 4:04 PM Randall Hauch  wrote:
>
> > I'd like to start a vote on KIP-722:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-722%3A+Enable+connector+client+overrides+by+default
> >
> > +1 (binding) from myself.
> >
> > Thanks, and best regards!
> >
> > Randall
> >
>

Re: [VOTE] KIP-721: Enable connector log contexts in Connect Log4j configuration

I agree 3.0.0 is a good opportunity to enable this useful feature by
default.

+1 (binding)

Konstantine

On Fri, May 7, 2021 at 6:33 PM Dongjin Lee  wrote:

> +1 (Non-binding)
>
> Thanks,
> Dongjin
>
> On Thu, May 6, 2021 at 3:19 AM Randall Hauch  wrote:
>
> > I'd like to start a vote on KIP-721:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-721%3A+Enable+connector+log+contexts+in+Connect+Log4j+configuration
> >
> > +1 (binding) from myself.
> >
> > Thanks, and best regards!
> >
> > Randall
> >
>
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
>
>
> *github:  github.com/dongjinleekr
> keybase: https://keybase.io/dongjinleekr
> linkedin: kr.linkedin.com/in/dongjinleekr
> speakerdeck:
> speakerdeck.com/dongjin
> *
>

Re: [DISCUSS] KIP-745: Connect API to restart connector and tasks

Thanks for the KIP Randall!
Really happy to see this feature coming along finally. Will indeed resolve
an unintuitive aspect of Connect's REST API.

I'm in favor of the current approach overall. Just a few comments below
that could use a brief discussion:

1) Is there a specific reason why you choose the config topic to persist
the restart requests?
I know it doesn't really matter which internal topic is used for backwards
compatibility, because the worker ignores records that does not understand.

But are there any implications upon worker restart? I read you specifically
mention that a worker will read restart requests upon its own restart. But
is this really desirable?
The config topic offers a way to persist configurations of the latest
running connectors and upon cluster restart to bring these connectors back
up.
However, I'm not sure we want to persist recent restart attempts, the same
way we persist configurations. Additionally configurations might get
additional backups in order to allow for the config topic to be recovered
for disaster recovery reasons. To that respect, I'm not sure that the
restart requests require the same guarantees.

Given the above, would the status topic offer a better alternative in order
to broadcast the restart requests across the workers? Haven't looked
closely what would be the implications of writing these new records to the
status topic, but this topic tends to be more transient.
Or is it that you've preferred the config topic for the workers to be aware
of the interleaving between restart requests and requests for
reconfigurations? If that's the case an explicit call out might be
worthwhile in the KIP itself.

2) You mention in the KIP the status STOPPED. But this is not currently
part of the task or connector states that get persisted to the status
topic, as these are defined in AbstractStatus#State enum. Should we remove
any reference to a STOPPED state to avoid confusion? The only new state
seems to be the RESTARTING state.

And a few minor comments.
a) The sentence "We need to keep the existing behavior of the method to
just restart the Connector object must be retained for backward
compatibility," doesn't read well for me.
b) Subsection title: "Use REST API for Worker-to-Worker Communication and"
is possibly an unfinished sentence.
c) In the paragraph right below, the last sentence "especially when the
number of workers " might also be missing something.

Best,
Konstantine

On Thu, Jun 3, 2021 at 8:42 AM Kalpesh Patel 
wrote:

> Agreed, this would be a great enhancement.
>
> On Thu, Jun 3, 2021 at 9:26 AM Nikita Kretov  wrote:
>
> > Hello! It's really not intuitive that you need to restart connect and
> > tasks separately! I'd like to see this KIP landed in 3.0.0 release!
> >
> > On 6/2/21 7:40 PM, Randall Hauch wrote:
> > > Hello all,
> > >
> > > Many users struggle with the connector restart REST API only restarting
> > the
> > > Connector instance rather than everything they associated with a
> "named"
> > > connector. I've created a KIP that attempts to solve this problem via
> new
> > > query parameters on an existing REST API method, though by default it
> > > remains backward compatible with older behavior:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-745%3A+Connect+API+to+restart+connector+and+tasks
> > >
> > > Please take a look and respond here with questions. I'd love to get
> this
> > > into AK 3.0.0, and the KIP freeze is coming up soon.
> > >
> > > Best regards,
> > >
> > > Randall
> > >
> >
>

Re: [DISCUSS] Apache Kafka 3.0.0 release plan with new updated dates