Re: [ANNOUNCE] New committer: Luke Chen

[Tom Bentley]

Congratulations Luke!

On Thu, 10 Feb 2022 at 06:41, Josep Prat 
wrote:

> Congrats Luke!
>
> ———
> Josep Prat
>
> Aiven Deutschland GmbH
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> m: +491715557497
>
> w: aiven.io
>
> e: josep.p...@aiven.io
>
> On Thu, Feb 10, 2022, 07:07 Randall Hauch  wrote:
>
> > Congratulations, Luke!
> >
> > On Wed, Feb 9, 2022 at 11:02 PM Matthias J. Sax 
> wrote:
> >
> > > Congratulations! Glad to have you onboard, Luke!
> > >
> > > -Matthias
> > >
> > > On 2/9/22 16:37, Bill Bejeck wrote:
> > > > Congrats Luke! Well deserved.
> > > >
> > > > -Bill
> > > >
> > > > On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo 
> > wrote:
> > > >
> > > >> Congratulations Luke!
> > > >>
> > > >> Thank you for your service
> > > >>
> > > >> On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang 
> > > wrote:
> > > >>
> > > >>> The PMC for Apache Kafka has invited Luke Chen (showuon) as a
> > committer
> > > >> and
> > > >>> we are pleased to announce that he has accepted!
> > > >>>
> > > >>> Luke has been actively contributing to Kafka since early 2020. He
> has
> > > >>> made more than 120 commits on various components of Kafka, with
> > notable
> > > >>> contributions to the rebalance protocol in Consumer and Streams
> > > (KIP-766,
> > > >>> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few),
> as
> > > >> well
> > > >>> as making an impact on improving test stability of the project.
> Aside
> > > >> from
> > > >>> all his code contributions, Luke has been a great participant in
> > > >>> discussions across the board, a very active and helpful reviewer of
> > > other
> > > >>> contributors' works, all of which are super valuable and highly
> > > >> appreciated
> > > >>> by the community.
> > > >>>
> > > >>>
> > > >>> Thanks for all of your contributions Luke. Congratulations!
> > > >>>
> > > >>> -- Guozhang, on behalf of the Apache Kafka PMC
> > > >>>
> > > >> --
> > > >> Israel Ekpo
> > > >> Lead Instructor, IzzyAcademy.com
> > > >> https://www.youtube.com/c/izzyacademy
> > > >> https://izzyacademy.com/
> > > >>
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Luke Chen

[Josep Prat]

Congrats Luke!

———
Josep Prat

Aiven Deutschland GmbH

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

m: +491715557497

w: aiven.io

e: josep.p...@aiven.io

On Thu, Feb 10, 2022, 07:07 Randall Hauch  wrote:

> Congratulations, Luke!
>
> On Wed, Feb 9, 2022 at 11:02 PM Matthias J. Sax  wrote:
>
> > Congratulations! Glad to have you onboard, Luke!
> >
> > -Matthias
> >
> > On 2/9/22 16:37, Bill Bejeck wrote:
> > > Congrats Luke! Well deserved.
> > >
> > > -Bill
> > >
> > > On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo 
> wrote:
> > >
> > >> Congratulations Luke!
> > >>
> > >> Thank you for your service
> > >>
> > >> On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang 
> > wrote:
> > >>
> > >>> The PMC for Apache Kafka has invited Luke Chen (showuon) as a
> committer
> > >> and
> > >>> we are pleased to announce that he has accepted!
> > >>>
> > >>> Luke has been actively contributing to Kafka since early 2020. He has
> > >>> made more than 120 commits on various components of Kafka, with
> notable
> > >>> contributions to the rebalance protocol in Consumer and Streams
> > (KIP-766,
> > >>> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
> > >> well
> > >>> as making an impact on improving test stability of the project. Aside
> > >> from
> > >>> all his code contributions, Luke has been a great participant in
> > >>> discussions across the board, a very active and helpful reviewer of
> > other
> > >>> contributors' works, all of which are super valuable and highly
> > >> appreciated
> > >>> by the community.
> > >>>
> > >>>
> > >>> Thanks for all of your contributions Luke. Congratulations!
> > >>>
> > >>> -- Guozhang, on behalf of the Apache Kafka PMC
> > >>>
> > >> --
> > >> Israel Ekpo
> > >> Lead Instructor, IzzyAcademy.com
> > >> https://www.youtube.com/c/izzyacademy
> > >> https://izzyacademy.com/
> > >>
> > >
> >
>

Re: [ANNOUNCE] New committer: Luke Chen

[Randall Hauch]

Congratulations, Luke!

On Wed, Feb 9, 2022 at 11:02 PM Matthias J. Sax  wrote:

> Congratulations! Glad to have you onboard, Luke!
>
> -Matthias
>
> On 2/9/22 16:37, Bill Bejeck wrote:
> > Congrats Luke! Well deserved.
> >
> > -Bill
> >
> > On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo  wrote:
> >
> >> Congratulations Luke!
> >>
> >> Thank you for your service
> >>
> >> On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang 
> wrote:
> >>
> >>> The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
> >> and
> >>> we are pleased to announce that he has accepted!
> >>>
> >>> Luke has been actively contributing to Kafka since early 2020. He has
> >>> made more than 120 commits on various components of Kafka, with notable
> >>> contributions to the rebalance protocol in Consumer and Streams
> (KIP-766,
> >>> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
> >> well
> >>> as making an impact on improving test stability of the project. Aside
> >> from
> >>> all his code contributions, Luke has been a great participant in
> >>> discussions across the board, a very active and helpful reviewer of
> other
> >>> contributors' works, all of which are super valuable and highly
> >> appreciated
> >>> by the community.
> >>>
> >>>
> >>> Thanks for all of your contributions Luke. Congratulations!
> >>>
> >>> -- Guozhang, on behalf of the Apache Kafka PMC
> >>>
> >> --
> >> Israel Ekpo
> >> Lead Instructor, IzzyAcademy.com
> >> https://www.youtube.com/c/izzyacademy
> >> https://izzyacademy.com/
> >>
> >
>

Re: [DISCUSS] Should we automatically close stale PRs?

[Matthias J. Sax]

Nikolay,

thanks for helping out!


First, I thought it’s an author job to keep KIP status up to date.
But, it can be tricky to determine actual KIP status because of lack of feedback from committers 


Yes, it is the author's task, but it's also the author's task to keep 
the discussion alive (what -- to be fair -- can be hard). We had some 
great contributions thought that took very long, but the KIP author kept 
following up and thus signaling that they still have interest. Just 
going silent and effectively dropping a KIP is not the best way (even if 
I can understand that it sometime frustrating and some people just walk 
away).




Second - the other issue is determine - what KIP just wait for a hero to 
implement it, and what just wrong idea or something similar.


As pointed out on the KIP wiki page, if somebody is not willing to 
implement the KIP, they should not even start it. Getting a KIP voted 
but not finishing it, is not really helping the project.


About "just the wrong idea": this also happens, but usually it's clear 
quite quickly if people raise concerns about an idea.



-Matthias


On 2/9/22 12:13, Nikolay Izhikov wrote:

Thanks for that list, Nikolay,


Thank, John.

I made a second round of digging through abandoned PR’s.
Next pack, that should be closed:

https://github.com/apache/kafka/pull/1291
https://github.com/apache/kafka/pull/1323
https://github.com/apache/kafka/pull/1412
https://github.com/apache/kafka/pull/1757
https://github.com/apache/kafka/pull/1741
https://github.com/apache/kafka/pull/1715
https://github.com/apache/kafka/pull/1668
https://github.com/apache/kafka/pull/1666
https://github.com/apache/kafka/pull/1661
https://github.com/apache/kafka/pull/1626
https://github.com/apache/kafka/pull/1624
https://github.com/apache/kafka/pull/1608
https://github.com/apache/kafka/pull/1606
https://github.com/apache/kafka/pull/1582
https://github.com/apache/kafka/pull/1522
https://github.com/apache/kafka/pull/1516
https://github.com/apache/kafka/pull/1493
https://github.com/apache/kafka/pull/1473
https://github.com/apache/kafka/pull/1870
https://github.com/apache/kafka/pull/1883
https://github.com/apache/kafka/pull/1893
https://github.com/apache/kafka/pull/1894
https://github.com/apache/kafka/pull/1912
https://github.com/apache/kafka/pull/1933
https://github.com/apache/kafka/pull/1983
https://github.com/apache/kafka/pull/1984
https://github.com/apache/kafka/pull/2017
https://github.com/apache/kafka/pull/2018


9 февр. 2022 г., в 22:37, John Roesler  написал(а):

Thanks for that list, Nikolay,

I've just closed them all.

And thanks to you all for working to keep Kafka development
healthy!

-John

On Wed, 2022-02-09 at 14:19 +0300, Nikolay Izhikov wrote:

Hello, guys.

I made a quick search throw oldest PRs.
Looks like the following list can be safely closed.

Committers, can you, please, push the actual «close» button for this list of 
PRs?

https://github.com/apache/kafka/pull/560
https://github.com/apache/kafka/pull/200
https://github.com/apache/kafka/pull/62
https://github.com/apache/kafka/pull/719
https://github.com/apache/kafka/pull/735
https://github.com/apache/kafka/pull/757
https://github.com/apache/kafka/pull/824
https://github.com/apache/kafka/pull/880
https://github.com/apache/kafka/pull/907
https://github.com/apache/kafka/pull/983
https://github.com/apache/kafka/pull/1035
https://github.com/apache/kafka/pull/1078
https://github.com/apache/kafka/pull/
https://github.com/apache/kafka/pull/1135
https://github.com/apache/kafka/pull/1147
https://github.com/apache/kafka/pull/1150
https://github.com/apache/kafka/pull/1244
https://github.com/apache/kafka/pull/1269
https://github.com/apache/kafka/pull/1415
https://github.com/apache/kafka/pull/1468


7 февр. 2022 г., в 20:04, Mickael Maison  написал(а):

Hi David,

I agree with you, I think we should close stale PRs.

Overall, I think we should also see if there are other Github actions
that may ease the work for reviewers and/or give more visibility of
the process to PR authors.
I'm thinking things like:
- code coverage changes
- better view on results from the build, for example if it's failing
checkstyle, the author could be notified first
- check whether public API are touched and it requires a KIP

For example, see some actions/integration used by other Apache projects:
- Flink: https://github.com/apache/flink/pull/18638#issuecomment-1030709579
- Beam: https://github.com/apache/beam/pull/16746#issue-1124656975
- Pinot: https://github.com/apache/pinot/pull/8139#issuecomment-1030701265

Finally, as several people have mentioned already, what can we do to
increase the impact of contributors that are not (yet?) committers?
Currently, our long delays in reviewing PRs and KIPs is hurting the
project and we're for sure missing out some fixes and potential
contributors. I think Josep's idea is interesting and finding ways to
engage more people and share some responsibilities better will improve
the project. Currently the 

[jira] [Resolved] (KAFKA-13655) Cannot edit clients page

[Matthias J. Sax (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias J. Sax resolved KAFKA-13655.
-
Resolution: Not A Bug

> Cannot edit clients page
> 
>
> Key: KAFKA-13655
> URL: https://issues.apache.org/jira/browse/KAFKA-13655
> Project: Kafka
>  Issue Type: Bug
>  Components: documentation
>Reporter: Mario Mastrodicasa
>Priority: Major
>
> Dear administrator,
> I want to add references to a new .NET Kafka Client, but I'm not able to find 
> a button, or link, to edit the page. The page reports an unrestricted access, 
> but I can't edit it.
> The page I want to edit is 
> [https://cwiki.apache.org/confluence/display/KAFKA/Clients] and the client is 
> under GitHub at this link [https://github.com/masesgroup/KafkaBridge]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [ANNOUNCE] New committer: Luke Chen

[Matthias J. Sax]

Congratulations! Glad to have you onboard, Luke!

-Matthias

On 2/9/22 16:37, Bill Bejeck wrote:

Congrats Luke! Well deserved.

-Bill

On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo  wrote:


Congratulations Luke!

Thank you for your service

On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang  wrote:


The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer

and

we are pleased to announce that he has accepted!

Luke has been actively contributing to Kafka since early 2020. He has
made more than 120 commits on various components of Kafka, with notable
contributions to the rebalance protocol in Consumer and Streams (KIP-766,
KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as

well

as making an impact on improving test stability of the project. Aside

from

all his code contributions, Luke has been a great participant in
discussions across the board, a very active and helpful reviewer of other
contributors' works, all of which are super valuable and highly

appreciated

by the community.


Thanks for all of your contributions Luke. Congratulations!

-- Guozhang, on behalf of the Apache Kafka PMC


--
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/

Re: [DISCUSS] KIP-818: Introduce cache-size-bytes-total Task Level Metric

[Sagar]

Hi Guozhang,

Sure. I will add it to the KIP.

Thanks!
Sagar.

On Mon, Feb 7, 2022 at 6:22 AM Guozhang Wang  wrote:

> Since the PR is reopened and we are going to re-merged the fixed PRs, what
> about just adding that as part of the KIP as the addendum?
>
> On Fri, Feb 4, 2022 at 2:13 AM Sagar  wrote:
>
> > Thanks Sophie/Guozhang.
> >
> > Yeah I could have amended the KIP but it slipped my mind when Guozhang
> > proposed this in the PR. Later on, the PR was merged and KIP was marked
> as
> > adopted so I thought I will write a new one. I know the PR had been
> > reopened now :p . I dont have much preference on a new KIP v/s the
> original
> > one so anything is ok with me as well.
> >
> > I agree with the INFO part. I will make that change.
> >
> > Regarding task level, from my understanding, since every task's
> > buffer/cache size might be different so if a certain task might be
> > overshooting the limits then the task level metric might help people to
> > infer this. Also, thanks for the explanation Guozhang on why this should
> be
> > a task level metric. What are your thoughts on this @Sophie?
> >
> > Thanks!
> > Sagar.
> >
> >
> > On Fri, Feb 4, 2022 at 4:47 AM Guozhang Wang  wrote:
> >
> > > Thanks Sagar for proposing the KIP, and Sophie for sharing your
> thoughts.
> > > Here're my 2c:
> > >
> > > I think I agree with Sophie for making the two metrics (both the added
> > and
> > > the newly proposed) on INFO level since we are always calculating them
> > > anyways. Regarding the level of the cache-size though, I'm thinking a
> bit
> > > different with you two: today we do not actually keep that caches on
> the
> > > per-store level, but rather on the per-thread level, i.e. when the
> cache
> > is
> > > full we would flush not only on the triggering state store but also
> > > potentially on other state stores as well of the task that thread owns.
> > > This mechanism, in hindsight, is a bit weird and we have some
> discussions
> > > about refactoring that in the future already. Personally I'd like to
> make
> > > this new metric to be aligned with whatever our future design will be.
> > >
> > > In the long run if we would not have a static assignment from tasks to
> > > threads, it may not make sense to keep a dedicated cache pool per
> thread.
> > > Instead all tasks will be dynamically sharing the globally configured
> max
> > > cache size (dynamically here means, we would not just divide the total
> > size
> > > by the num.tasks and then assign that to each task), and when a cache
> put
> > > triggers the flushing because the sum now exceeds the global configured
> > > value, we would potentially flush all the cached records for that task.
> > If
> > > this is the end stage, then I think keeping this metric at the task
> level
> > > is good.
> > >
> > >
> > >
> > > Guozhang
> > >
> > >
> > >
> > >
> > > On Thu, Feb 3, 2022 at 10:15 AM Sophie Blee-Goldman
> > >  wrote:
> > >
> > > > Hey Sagar,  thanks for the KIP!
> > > >
> > > > And yes, all metrics are considered part of the public API and thus
> > > require
> > > > a KIP to add (or modify, etc...) Although in this particular case,
> you
> > > > could probably make a good case for just considering it as an update
> to
> > > the
> > > > original KIP which added the analogous metric
> > `input-buffer-bytes-total`.
> > > > For  things like this that weren't considered during the KIP proposal
> > but
> > > > came up during the implementation or review, and are small changes
> that
> > > > would have made sense to include in that KIP had they been thought
> of,
> > > you
> > > > can just send an update to the existing KIP's discussion and.or
> voting
> > > > thread that explains what you want to add or modify and maybe a brief
> > > > description why.
> > > >
> > > > It's always ok to make a new KIP when in doubt, but there are some
> > cases
> > > > where an update email is sufficient. If there are any concerns or
> > > > suggestions that significantly expand the scope of the update, you
> can
> > > > always go create a new KIP and move the discussion there.
> > > >
> > > > I'd say you can feel free to proceed in whichever way you'd prefer
> for
> > > this
> > > > new proposal -- it just needs to appear in some KIP somewhere, and
> have
> > > > given the community thew opportunity to discuss and provide feedback
> > on.
> > > >
> > > > On that note, I do have two suggestions:
> > > >
> > > > 1)  since we need to measure the size of the cache (and the input
> > > buffer(s)
> > > > anyways, we may as well make `cache-size-bytes-total` -- and also the
> > new
> > > > input-buffer-bytes-total -- an INFO level metric. In general the more
> > > > metrics the merrier, the only real reason for disabling some are if
> > they
> > > > have a performance impact or other cost that not everyone will want
> to
> > > pay.
> > > > In this case we're already computing the value of these metrics, so
> why
> > > not
> > > > expose it to the user as an INFO metric
> 

Re: [ANNOUNCE] New committer: Luke Chen

[Sagar]

Congrats Luke!

Thanks!
Sagar.

On Thu, Feb 10, 2022 at 8:12 AM deng ziming 
wrote:

> Congratulations, Luke!
>
> Thanks,
> Ziming Deng
>
> > On Feb 10, 2022, at 9:39 AM, John Roesler  wrote:
> >
> > Congratulations, Luke!
> > -John
> >
> > On Wed, Feb 9, 2022, at 19:33, Mayuresh Gharat wrote:
> >> Congratulations Luke!
> >>
> >> Thanks,
> >>
> >> Mayuresh
> >>
> >> On Wed, Feb 9, 2022, 5:24 PM Ismael Juma  wrote:
> >>
> >>> Congratulations Luke!
> >>>
> >>> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang 
> wrote:
> >>>
>  The PMC for Apache Kafka has invited Luke Chen (showuon) as a
> committer
> >>> and
>  we are pleased to announce that he has accepted!
> 
>  Luke has been actively contributing to Kafka since early 2020. He has
>  made more than 120 commits on various components of Kafka, with
> notable
>  contributions to the rebalance protocol in Consumer and Streams
> (KIP-766,
>  KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
> >>> well
>  as making an impact on improving test stability of the project. Aside
> >>> from
>  all his code contributions, Luke has been a great participant in
>  discussions across the board, a very active and helpful reviewer of
> other
>  contributors' works, all of which are super valuable and highly
> >>> appreciated
>  by the community.
> 
> 
>  Thanks for all of your contributions Luke. Congratulations!
> 
>  -- Guozhang, on behalf of the Apache Kafka PMC
> 
> >>>
>
>

[GitHub] [kafka-site] showuon merged pull request #397: MINOR: add Luke as committer

[GitBox]

showuon merged pull request #397:
URL: https://github.com/apache/kafka-site/pull/397


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [ANNOUNCE] New committer: Luke Chen

[deng ziming]

Congratulations, Luke!

Thanks,
Ziming Deng

> On Feb 10, 2022, at 9:39 AM, John Roesler  wrote:
> 
> Congratulations, Luke!
> -John 
> 
> On Wed, Feb 9, 2022, at 19:33, Mayuresh Gharat wrote:
>> Congratulations Luke!
>> 
>> Thanks,
>> 
>> Mayuresh
>> 
>> On Wed, Feb 9, 2022, 5:24 PM Ismael Juma  wrote:
>> 
>>> Congratulations Luke!
>>> 
>>> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang  wrote:
>>> 
 The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
>>> and
 we are pleased to announce that he has accepted!
 
 Luke has been actively contributing to Kafka since early 2020. He has
 made more than 120 commits on various components of Kafka, with notable
 contributions to the rebalance protocol in Consumer and Streams (KIP-766,
 KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
>>> well
 as making an impact on improving test stability of the project. Aside
>>> from
 all his code contributions, Luke has been a great participant in
 discussions across the board, a very active and helpful reviewer of other
 contributors' works, all of which are super valuable and highly
>>> appreciated
 by the community.
 
 
 Thanks for all of your contributions Luke. Congratulations!
 
 -- Guozhang, on behalf of the Apache Kafka PMC
 
>>> 

[GitHub] [kafka-site] showuon opened a new pull request #397: MINOR: add Luke as committer

[GitBox]

showuon opened a new pull request #397:
URL: https://github.com/apache/kafka-site/pull/397


   Add Luke Chen into committer page. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [ANNOUNCE] New committer: Luke Chen

[John Roesler]

Congratulations, Luke!
-John 

On Wed, Feb 9, 2022, at 19:33, Mayuresh Gharat wrote:
> Congratulations Luke!
>
> Thanks,
>
> Mayuresh
>
> On Wed, Feb 9, 2022, 5:24 PM Ismael Juma  wrote:
>
>> Congratulations Luke!
>>
>> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang  wrote:
>>
>> > The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
>> and
>> > we are pleased to announce that he has accepted!
>> >
>> > Luke has been actively contributing to Kafka since early 2020. He has
>> > made more than 120 commits on various components of Kafka, with notable
>> > contributions to the rebalance protocol in Consumer and Streams (KIP-766,
>> > KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
>> well
>> > as making an impact on improving test stability of the project. Aside
>> from
>> > all his code contributions, Luke has been a great participant in
>> > discussions across the board, a very active and helpful reviewer of other
>> > contributors' works, all of which are super valuable and highly
>> appreciated
>> > by the community.
>> >
>> >
>> > Thanks for all of your contributions Luke. Congratulations!
>> >
>> > -- Guozhang, on behalf of the Apache Kafka PMC
>> >
>>

Re: [ANNOUNCE] New committer: Luke Chen

[Mayuresh Gharat]

Congratulations Luke!

Thanks,

Mayuresh

On Wed, Feb 9, 2022, 5:24 PM Ismael Juma  wrote:

> Congratulations Luke!
>
> On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang  wrote:
>
> > The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
> and
> > we are pleased to announce that he has accepted!
> >
> > Luke has been actively contributing to Kafka since early 2020. He has
> > made more than 120 commits on various components of Kafka, with notable
> > contributions to the rebalance protocol in Consumer and Streams (KIP-766,
> > KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
> well
> > as making an impact on improving test stability of the project. Aside
> from
> > all his code contributions, Luke has been a great participant in
> > discussions across the board, a very active and helpful reviewer of other
> > contributors' works, all of which are super valuable and highly
> appreciated
> > by the community.
> >
> >
> > Thanks for all of your contributions Luke. Congratulations!
> >
> > -- Guozhang, on behalf of the Apache Kafka PMC
> >
>

Re: [ANNOUNCE] New committer: Luke Chen

[Ismael Juma]

Congratulations Luke!

On Wed, Feb 9, 2022 at 3:22 PM Guozhang Wang  wrote:

> The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer and
> we are pleased to announce that he has accepted!
>
> Luke has been actively contributing to Kafka since early 2020. He has
> made more than 120 commits on various components of Kafka, with notable
> contributions to the rebalance protocol in Consumer and Streams (KIP-766,
> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as well
> as making an impact on improving test stability of the project. Aside from
> all his code contributions, Luke has been a great participant in
> discussions across the board, a very active and helpful reviewer of other
> contributors' works, all of which are super valuable and highly appreciated
> by the community.
>
>
> Thanks for all of your contributions Luke. Congratulations!
>
> -- Guozhang, on behalf of the Apache Kafka PMC
>

Re: [ANNOUNCE] New committer: Luke Chen

[Dongjin Lee]

Congratulations, Luke! You deserve it!

Best,
Dongjin

On Thu, Feb 10, 2022 at 8:23 AM Guozhang Wang  wrote:

> The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer and
> we are pleased to announce that he has accepted!
>
> Luke has been actively contributing to Kafka since early 2020. He has
> made more than 120 commits on various components of Kafka, with notable
> contributions to the rebalance protocol in Consumer and Streams (KIP-766,
> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as well
> as making an impact on improving test stability of the project. Aside from
> all his code contributions, Luke has been a great participant in
> discussions across the board, a very active and helpful reviewer of other
> contributors' works, all of which are super valuable and highly appreciated
> by the community.
>
>
> Thanks for all of your contributions Luke. Congratulations!
>
> -- Guozhang, on behalf of the Apache Kafka PMC
>


-- 
*Dongjin Lee*

*A hitchhiker in the mathematical world.*



*github:  github.com/dongjinleekr
keybase: https://keybase.io/dongjinleekr
linkedin: kr.linkedin.com/in/dongjinleekr
speakerdeck: speakerdeck.com/dongjin
*

Re: [ANNOUNCE] New committer: Luke Chen

[Bill Bejeck]

Congrats Luke! Well deserved.

-Bill

On Wed, Feb 9, 2022 at 7:25 PM Israel Ekpo  wrote:

> Congratulations Luke!
>
> Thank you for your service
>
> On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang  wrote:
>
> > The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer
> and
> > we are pleased to announce that he has accepted!
> >
> > Luke has been actively contributing to Kafka since early 2020. He has
> > made more than 120 commits on various components of Kafka, with notable
> > contributions to the rebalance protocol in Consumer and Streams (KIP-766,
> > KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as
> well
> > as making an impact on improving test stability of the project. Aside
> from
> > all his code contributions, Luke has been a great participant in
> > discussions across the board, a very active and helpful reviewer of other
> > contributors' works, all of which are super valuable and highly
> appreciated
> > by the community.
> >
> >
> > Thanks for all of your contributions Luke. Congratulations!
> >
> > -- Guozhang, on behalf of the Apache Kafka PMC
> >
> --
> Israel Ekpo
> Lead Instructor, IzzyAcademy.com
> https://www.youtube.com/c/izzyacademy
> https://izzyacademy.com/
>

Re: [ANNOUNCE] New committer: Luke Chen

[Israel Ekpo]

Congratulations Luke!

Thank you for your service

On Wed, Feb 9, 2022 at 6:22 PM Guozhang Wang  wrote:

> The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer and
> we are pleased to announce that he has accepted!
>
> Luke has been actively contributing to Kafka since early 2020. He has
> made more than 120 commits on various components of Kafka, with notable
> contributions to the rebalance protocol in Consumer and Streams (KIP-766,
> KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as well
> as making an impact on improving test stability of the project. Aside from
> all his code contributions, Luke has been a great participant in
> discussions across the board, a very active and helpful reviewer of other
> contributors' works, all of which are super valuable and highly appreciated
> by the community.
>
>
> Thanks for all of your contributions Luke. Congratulations!
>
> -- Guozhang, on behalf of the Apache Kafka PMC
>
-- 
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/

[ANNOUNCE] New committer: Luke Chen

[Guozhang Wang]

The PMC for Apache Kafka has invited Luke Chen (showuon) as a committer and
we are pleased to announce that he has accepted!

Luke has been actively contributing to Kafka since early 2020. He has
made more than 120 commits on various components of Kafka, with notable
contributions to the rebalance protocol in Consumer and Streams (KIP-766,
KIP-726, KIP-591, KAFKA-12675 and KAFKA12464, to just name a few), as well
as making an impact on improving test stability of the project. Aside from
all his code contributions, Luke has been a great participant in
discussions across the board, a very active and helpful reviewer of other
contributors' works, all of which are super valuable and highly appreciated
by the community.


Thanks for all of your contributions Luke. Congratulations!

-- Guozhang, on behalf of the Apache Kafka PMC

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

[Guozhang Wang]

I'm +1 on John's point 3) for punctuations.

And I think if people are on the same page that a reference equality check
per record is not a huge overhead, I think doing that enforcement is better
than documentations and hand-wavy undefined behaviors.


Guozhang

On Wed, Feb 9, 2022 at 11:27 AM John Roesler  wrote:

> Thanks for the KIP Jorge,
>
> I'm in support of your proposal.
>
> 1)
> I do agree with Guozhang's point (1). I think the cleanest
> approach. I think it's cleaner and better to keep the
> enforcement internal to the framework than to introduce a
> public API or context wrapper for processors to use
> explicitly.
>
> 2) I tend to agree with you on this one; I think the
> equality check ought to be fast enough in practice.
>
> 3) I think this is implicit, but should be explicit in the
> KIP: For the `processValues` API, because the framework sets
> the key on the context before calling `process` and then
> unsets it afterwards, there will always be no key set during
> task puctuation. Therefore, while processors may still
> register punctuators, they will not be able to forward
> anything from them.
>
> This is functionally equivalent to the existing
> transformers, by the way, that are also forbidden to forward
> anything during punctuation.
>
> For what it's worth, I think this is the best tradeoff.
>
> The only alternative I see is not to place any restriction
> on forwarded keys at all and just document that if users
> don't maintain proper partitioning, they'll get undefined
> behavior. That might be more powerful, but it's also a
> usability problem.
>
> Thanks,
> -John
>
> On Wed, 2022-02-09 at 11:34 +, Jorge Esteban Quilcate
> Otoya wrote:
> > Thanks Guozhang.
> >
> > > Does `ValueProcessorContext` have to be a public API? It seems to me
> > that this can be completely abstracted away from user interfaces as an
> > internal class
> >
> > Totally agree. No intention to add these as public APIs. Will update the
> > KIP to reflect this.
> >
> > > in the past the rationale for enforcing it at the
> > interface layer rather than do runtime checks is that it is more
> efficient.
> > > I'm not sure how much overhead it may incur to check if the key did not
> > change: if it is just a reference equality check maybe it's okay. What's
> > your take on this?
> >
> > Agree, reference equality should cover this validation and the overhead
> > impact should not be meaningful.
> > Will update the KIP to reflect this as well.
> >
> >
> > On Tue, 8 Feb 2022 at 19:05, Guozhang Wang  wrote:
> >
> > > Hello Jorge,
> > >
> > > Thanks for bringing this KIP! I think this is a nice idea to consider
> using
> > > a single overloaded function name for #process, just a couple quick
> > > questions after reading the proposal:
> > >
> > > 1) Does `ValueProcessorContext` have to be a public API? It seems to me
> > > that this can be completely abstracted away from user interfaces as an
> > > internal class, and we call the `setKey` before calling
> user-instantiated
> > > `process` function, and then in its overridden `forward` it can just
> check
> > > if the key changes or not.
> > > 2) Related to 1) above, in the past the rationale for enforcing it at
> the
> > > interface layer rather than do runtime checks is that it is more
> efficient.
> > > I'm not sure how much overhead it may incur to check if the key did not
> > > change: if it is just a reference equality check maybe it's okay.
> What's
> > > your take on this?
> > >
> > >
> > > Guozhang
> > >
> > > On Tue, Feb 8, 2022 at 5:17 AM Jorge Esteban Quilcate Otoya <
> > > quilcate.jo...@gmail.com> wrote:
> > >
> > > > Hi Dev team,
> > > >
> > > > I'd like to start a new discussion thread on Kafka Streams KIP-820:
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-820%3A+Extend+KStream+process+with+new+Processor+API
> > > >
> > > > This KIP is aimed to extend the current `KStream#process` API to
> return
> > > > output values that could be chained across the topology, as well as
> > > > introducing a new `KStream#processValues` to use processor while
> > > validating
> > > > keys haven't change and repartition is not required.
> > > >
> > > > Looking forward to your feedback.
> > > >
> > > > Regards,
> > > > Jorge.
> > > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
>
>

-- 
-- Guozhang

[jira] [Created] (KAFKA-13662) Migrate DeserializationExceptionHandler to latest ProcessorContext API

[Jorge Esteban Quilcate Otoya (Jira)]

Jorge Esteban Quilcate Otoya created KAFKA-13662:


 Summary: Migrate DeserializationExceptionHandler to latest 
ProcessorContext API
 Key: KAFKA-13662
 URL: https://issues.apache.org/jira/browse/KAFKA-13662
 Project: Kafka
  Issue Type: Task
Reporter: Jorge Esteban Quilcate Otoya


DeserializationExceptionHandler depends on old ProcessorContext API. This API 
has been deprecated and migrating to the latest one requires a public API that 
hasn't been considered as part of KIP-478.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #674

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 517298 lines...]
[2022-02-09T20:55:12.019Z] 
[2022-02-09T20:55:12.019Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterLeft[caching enabled = true] STARTED
[2022-02-09T20:55:19.028Z] 
[2022-02-09T20:55:19.028Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterLeft[caching enabled = true] PASSED
[2022-02-09T20:55:19.028Z] 
[2022-02-09T20:55:19.028Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInner[caching enabled = true] STARTED
[2022-02-09T20:55:26.048Z] 
[2022-02-09T20:55:26.048Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInner[caching enabled = true] PASSED
[2022-02-09T20:55:26.048Z] 
[2022-02-09T20:55:26.048Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuter[caching enabled = true] STARTED
[2022-02-09T20:55:26.162Z] 
[2022-02-09T20:55:26.162Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeftOuter[caching enabled = false] PASSED
[2022-02-09T20:55:26.162Z] 
[2022-02-09T20:55:26.162Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeftLeft[caching enabled = false] STARTED
[2022-02-09T20:55:33.057Z] 
[2022-02-09T20:55:33.057Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuter[caching enabled = true] PASSED
[2022-02-09T20:55:33.057Z] 
[2022-02-09T20:55:33.057Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeft[caching enabled = true] STARTED
[2022-02-09T20:55:38.804Z] 
[2022-02-09T20:55:38.804Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeft[caching enabled = true] PASSED
[2022-02-09T20:55:38.804Z] 
[2022-02-09T20:55:38.804Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerInner[caching enabled = true] STARTED
[2022-02-09T20:55:40.438Z] 
[2022-02-09T20:55:40.438Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeftLeft[caching enabled = false] PASSED
[2022-02-09T20:55:40.438Z] 
[2022-02-09T20:55:40.438Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterLeft[caching enabled = false] STARTED
[2022-02-09T20:55:45.813Z] 
[2022-02-09T20:55:45.813Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerInner[caching enabled = true] PASSED
[2022-02-09T20:55:45.813Z] 
[2022-02-09T20:55:45.813Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerOuter[caching enabled = true] STARTED
[2022-02-09T20:55:51.741Z] 
[2022-02-09T20:55:51.741Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerOuter[caching enabled = true] PASSED
[2022-02-09T20:55:51.741Z] 
[2022-02-09T20:55:51.741Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerLeft[caching enabled = true] STARTED
[2022-02-09T20:55:56.350Z] 
[2022-02-09T20:55:56.350Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterLeft[caching enabled = false] PASSED
[2022-02-09T20:55:56.350Z] 
[2022-02-09T20:55:56.350Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInner[caching enabled = false] STARTED
[2022-02-09T20:55:58.750Z] 
[2022-02-09T20:55:58.750Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInnerLeft[caching enabled = true] PASSED
[2022-02-09T20:55:58.750Z] 
[2022-02-09T20:55:58.750Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterInner[caching enabled = true] STARTED
[2022-02-09T20:56:04.500Z] 
[2022-02-09T20:56:04.500Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterInner[caching enabled = true] PASSED
[2022-02-09T20:56:04.500Z] 
[2022-02-09T20:56:04.500Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterOuter[caching enabled = true] STARTED
[2022-02-09T20:56:10.342Z] 
[2022-02-09T20:56:10.342Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testInner[caching enabled = false] PASSED
[2022-02-09T20:56:10.342Z] 
[2022-02-09T20:56:10.342Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuter[caching enabled = false] STARTED
[2022-02-09T20:56:11.509Z] 
[2022-02-09T20:56:11.509Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testOuterOuter[caching enabled = true] PASSED
[2022-02-09T20:56:11.509Z] 
[2022-02-09T20:56:11.509Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeftInner[caching enabled = false] STARTED
[2022-02-09T20:56:18.521Z] 
[2022-02-09T20:56:18.521Z] 
org.apache.kafka.streams.integration.TableTableJoinIntegrationTest > 
testLeftInner[caching enabled = false] PASSED
[2022-02-09T20:56:18.522Z] 

[jira] [Resolved] (KAFKA-3790) Default options when removing ACLs do not comply with documentation

[Jira]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sébastien Launay resolved KAFKA-3790.
-
Resolution: Won't Fix

> Default options when removing ACLs do not comply with documentation
> ---
>
> Key: KAFKA-3790
> URL: https://issues.apache.org/jira/browse/KAFKA-3790
> Project: Kafka
>  Issue Type: Bug
>  Components: documentation, security
>Affects Versions: 0.9.0.1, 0.10.0.0
>Reporter: Sébastien Launay
>Priority: Minor
>
> When removing ACLs without providing options like principal, host or 
> operation, we got a prompt for removing all the matching ACLs but when 
> executing the command none get removed.
> The following commands can be used to reproduce the inconsistency:
> {noformat}
> $ ./bin/kafka-acls.sh --authorizer-properties 
> zookeeper.connect=localhost:2181 -list -topic test
> Current ACLs for resource `Topic:test`: 
> $ ./bin/kafka-acls.sh --authorizer-properties 
> zookeeper.connect=localhost:2181 --add --allow-principal User:Alice 
> --operation Write --topic test --allow-host 1.2.3.4
> Adding ACLs for resource `Topic:test`: 
>   User:Alice has Allow permission for operations: Write from hosts: 
> 1.2.3.4 
> Current ACLs for resource `Topic:test`: 
>   User:Alice has Allow permission for operations: Write from hosts: 
> 1.2.3.4 
> $ ./bin/kafka-acls.sh --authorizer-properties 
> zookeeper.connect=localhost:2181 --remove --allow-principal User:Alice 
> --topic test 
> Are you sure you want to remove ACLs: 
>   User:Alice has Allow permission for operations: All from hosts: * 
>  from resource `Topic:test`? (y/n)
> y
> Current ACLs for resource `Topic:test`: 
>   User:Alice has Allow permission for operations: Write from hosts: 
> 1.2.3.4 
> {noformat}
> *The Current ACLs for resource {{Topic:test}} is expected to be empty after 
> the last command.*
> Only a specific ACL (when all options mentioned above are provided) or else 
> all the ACLs for a given resource (none of the options mentioned above are 
> provided) can get removed as shown by the following code snippets:
> {noformat}
>   // AclCommand.scala
>   ...
>   private def removeAcl(opts: AclCommandOptions) {
> withAuthorizer(opts) { authorizer =>
>   val resourceToAcl = getResourceToAcls(opts)
>   for ((resource, acls) <- resourceToAcl) {
> if (acls.isEmpty) {
>   if (confirmAction(opts, s"Are you sure you want to delete all ACLs 
> for resource `${resource}`? (y/n)"))
> authorizer.removeAcls(resource)
> } else {
>   if (confirmAction(opts, s"Are you sure you want to remove ACLs: 
> $Newline ${acls.map("\t" + _).mkString(Newline)} $Newline from resource 
> `${resource}`? (y/n)"))
> authorizer.removeAcls(acls, resource)
> }
>   }
>   listAcl(opts)
> }
>   }
> ...
>   // SimpleAclAuthorizer.scala
> ...
>   override def removeAcls(aclsTobeRemoved: Set[Acl], resource: Resource): 
> Boolean = {
>  inWriteLock(lock) {
>updateResourceAcls(resource) { currentAcls =>
> currentAcls -- aclsTobeRemoved
>}
>  }
>}
> {noformat}
> A workaround consists of listing the ACL in order to know which exact one to 
> remove which make the automation of ACL management trickier.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] Should we automatically close stale PRs?

[Nikolay Izhikov]

> Thanks for that list, Nikolay,

Thank, John.

I made a second round of digging through abandoned PR’s.
Next pack, that should be closed:

https://github.com/apache/kafka/pull/1291
https://github.com/apache/kafka/pull/1323
https://github.com/apache/kafka/pull/1412
https://github.com/apache/kafka/pull/1757
https://github.com/apache/kafka/pull/1741
https://github.com/apache/kafka/pull/1715
https://github.com/apache/kafka/pull/1668
https://github.com/apache/kafka/pull/1666
https://github.com/apache/kafka/pull/1661
https://github.com/apache/kafka/pull/1626
https://github.com/apache/kafka/pull/1624
https://github.com/apache/kafka/pull/1608
https://github.com/apache/kafka/pull/1606
https://github.com/apache/kafka/pull/1582
https://github.com/apache/kafka/pull/1522
https://github.com/apache/kafka/pull/1516
https://github.com/apache/kafka/pull/1493
https://github.com/apache/kafka/pull/1473
https://github.com/apache/kafka/pull/1870
https://github.com/apache/kafka/pull/1883
https://github.com/apache/kafka/pull/1893
https://github.com/apache/kafka/pull/1894
https://github.com/apache/kafka/pull/1912
https://github.com/apache/kafka/pull/1933
https://github.com/apache/kafka/pull/1983
https://github.com/apache/kafka/pull/1984
https://github.com/apache/kafka/pull/2017
https://github.com/apache/kafka/pull/2018

> 9 февр. 2022 г., в 22:37, John Roesler  написал(а):
> 
> Thanks for that list, Nikolay,
> 
> I've just closed them all.
> 
> And thanks to you all for working to keep Kafka development
> healthy!
> 
> -John
> 
> On Wed, 2022-02-09 at 14:19 +0300, Nikolay Izhikov wrote:
>> Hello, guys.
>> 
>> I made a quick search throw oldest PRs.
>> Looks like the following list can be safely closed.
>> 
>> Committers, can you, please, push the actual «close» button for this list of 
>> PRs?
>> 
>> https://github.com/apache/kafka/pull/560
>> https://github.com/apache/kafka/pull/200
>> https://github.com/apache/kafka/pull/62
>> https://github.com/apache/kafka/pull/719
>> https://github.com/apache/kafka/pull/735
>> https://github.com/apache/kafka/pull/757
>> https://github.com/apache/kafka/pull/824
>> https://github.com/apache/kafka/pull/880
>> https://github.com/apache/kafka/pull/907
>> https://github.com/apache/kafka/pull/983
>> https://github.com/apache/kafka/pull/1035
>> https://github.com/apache/kafka/pull/1078
>> https://github.com/apache/kafka/pull/
>> https://github.com/apache/kafka/pull/1135
>> https://github.com/apache/kafka/pull/1147
>> https://github.com/apache/kafka/pull/1150
>> https://github.com/apache/kafka/pull/1244
>> https://github.com/apache/kafka/pull/1269
>> https://github.com/apache/kafka/pull/1415
>> https://github.com/apache/kafka/pull/1468
>> 
>>> 7 февр. 2022 г., в 20:04, Mickael Maison  
>>> написал(а):
>>> 
>>> Hi David,
>>> 
>>> I agree with you, I think we should close stale PRs.
>>> 
>>> Overall, I think we should also see if there are other Github actions
>>> that may ease the work for reviewers and/or give more visibility of
>>> the process to PR authors.
>>> I'm thinking things like:
>>> - code coverage changes
>>> - better view on results from the build, for example if it's failing
>>> checkstyle, the author could be notified first
>>> - check whether public API are touched and it requires a KIP
>>> 
>>> For example, see some actions/integration used by other Apache projects:
>>> - Flink: https://github.com/apache/flink/pull/18638#issuecomment-1030709579
>>> - Beam: https://github.com/apache/beam/pull/16746#issue-1124656975
>>> - Pinot: https://github.com/apache/pinot/pull/8139#issuecomment-1030701265
>>> 
>>> Finally, as several people have mentioned already, what can we do to
>>> increase the impact of contributors that are not (yet?) committers?
>>> Currently, our long delays in reviewing PRs and KIPs is hurting the
>>> project and we're for sure missing out some fixes and potential
>>> contributors. I think Josep's idea is interesting and finding ways to
>>> engage more people and share some responsibilities better will improve
>>> the project. Currently the investment to become a committer is pretty
>>> high. This could provide a stepping stone (or an intermediary role)
>>> for some people in the community.
>>> 
>>> Thanks,
>>> Mickael
>>> 
>>> 
>>> On Mon, Feb 7, 2022 at 12:51 PM Josep Prat  
>>> wrote:
 
 Hi,
 
 It seems like a great idea. I agree with you that we should use this as a
 means to notify contributors and reviewers that there is some work to be
 done.
 
 Regarding labels, a couple of things, first one is that PR participants
 won't get notified when a label is applied. So probably it would be best to
 apply a label and add a comment.
 Secondly, GitHub offers better fine-grained roles for contributors: read,
 triage, write, maintain, admin (further reading here:
 https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-roles-for-an-organization#permissions-for-each-role).

Re: [DISCUSS] KIP-634: Complementary support for headers in Kafka Streams DSL

[John Roesler]

Hello Jorge,

Thanks for bringing this up again!

I've just read over the current version of the KIP.

1) I wonder if we really need RecordValue, since we now have
Record, and they are almost the same, both in API and in
purpose. What do you think about instead adding topic and
partition to Record?

2) I find the name "mapRecordValue" to be a bit confusing
because it seems like it would map the value of a record.
What do you think about "mapValueToRecord" instead?

3) I agree with adding the serde explicitly. However, it
would be good to state whether and when we'll automatically
wrap a value serde. For example, if the value serde is known
(or if we're using a default serde from the config), will
Streams automatically wrap it downstream of the record-
mapping operator?

Otherwise, your proposal looks good to me!

Thanks,
-John

On Tue, 2022-02-08 at 18:06 +, Jorge Esteban Quilcate
Otoya wrote:
> Hi Dev team,
> 
> I'd like to revamp the KIP again:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-634%3A+Complementary+support+for+headers+and+record+metadata+in+Kafka+Streams+DSL
> 
> - Reference implementation is now using the latest `Processor` API from
> KIP-478: https://github.com/apache/kafka/pull/10265/files for both
> Processors backing changes on the KStream API.
> - It is proposing to still extend `To` class for backwards compatibility.
> 
> Looking forward to your feedback.
> 
> Regards,
> Jorge.
> 
> On Thu, 4 Mar 2021 at 18:38, Jorge Esteban Quilcate Otoya <
> quilcate.jo...@gmail.com> wrote:
> 
> > Hi everyone!
> > 
> > I'd like to revamp this KIP. I have made some significant changes on the
> > scope:
> > - Added `mapRecordValue` to map not only headers, but other record
> > metadata: topic name, partition, offset, and timestamp into a new type
> > `RecordValue`.
> > - Added a serde for `RecordValue` to support stateful operations.
> > - Added `setRecordHeaders` to apply headers to record crossing the stream.
> > - Added headers to `To` to update headers via `context.forward(k, v, to)`.
> > 
> > New link:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-634%3A+Complementary+support+for+headers+and+record+metadata+in+Kafka+Streams+DSL
> > 
> > Looking forward to your feedback,
> > 
> > Cheers and stay safe,
> > Jorge.
> > 
> > On Thu, Oct 29, 2020 at 12:33 AM Jorge Esteban Quilcate Otoya <
> > quilcate.jo...@gmail.com> wrote:
> > 
> > > Thanks Sophie! Haven't followed KIP-478 but sounds great.
> > > I'll be happy to help on that migration to the new PAPI if it's still an
> > > open issue. We can bump this KIP after that.
> > > 
> > > Cheers,
> > > Jorge.
> > > 
> > > On Wed, Oct 28, 2020 at 7:00 PM Sophie Blee-Goldman 
> > > wrote:
> > > 
> > > > I *think* that the `To` Matthias was referring to was not KStream#to but
> > > > the To class
> > > > which is accepted as a possible parameter of ProcessorContext#forward
> > > > (correct
> > > > me if wrong).
> > > > 
> > > > This was on the old ProcessorContext interface, which has now been
> > > > replaced with
> > > > the new api.ProcessorContext in KIP-478. In the new interface we've 
> > > > moved
> > > > away
> > > > from the forward signatures that accept a separate key or value or
> > > > timestamp or To,
> > > > and wrapped all of these into a single Record class. This new Record
> > > > class
> > > > has the
> > > > headers as a field, so it seems like KIP-478 has happened to solve the
> > > > lack
> > > > of support
> > > > for Headers in the PAPI along the way.
> > > > 
> > > > This is all somewhat recent, and probably wasn't yet sorted out at the
> > > > time
> > > > of Matthias'
> > > > last reply. But given how this worked out it seems like we can just 
> > > > focus
> > > > on adding
> > > > support for Headers in the DSL in this KIP by building off of the
> > > > groundwork of
> > > > KIP-478? It doesn't seem necessary to go back and add support for 
> > > > headers
> > > > in the old
> > > > PAPI, since this will (or already has?) been deprecated.
> > > > 
> > > > The one challenge is that this will presumably require that we migrate
> > > > all
> > > > DSL operators
> > > > to the new PAPI before adding header support for those operators. But
> > > > that
> > > > definitely
> > > > sounds achievable here
> > > > 
> > > > On Wed, Oct 28, 2020 at 11:10 AM Jorge Esteban Quilcate Otoya <
> > > > quilcate.jo...@gmail.com> wrote:
> > > > 
> > > > > Hi Matthias,
> > > > > 
> > > > > Sorry for the late reply.
> > > > > 
> > > > > I like the proposal. Just to check if I got it right:
> > > > > 
> > > > > We can extend the `kstream.to()` function to support setting headers.
> > > > > e.g.:
> > > > > 
> > > > > ```
> > > > > void to(final String topic,
> > > > > final Produced produced,
> > > > > final HeadersExtractor headersExtractor);
> > > > > ```
> > > > > 
> > > > > where `HeadersExtractor`:
> > > > > 
> > > > > ```
> > > > > public interface HeadersExtractor {
> > > > > Headers 

Re: [DISCUSS] Should we automatically close stale PRs?

[John Roesler]

Thanks for that list, Nikolay,

I've just closed them all.

And thanks to you all for working to keep Kafka development
healthy!

-John

On Wed, 2022-02-09 at 14:19 +0300, Nikolay Izhikov wrote:
> Hello, guys.
> 
> I made a quick search throw oldest PRs.
> Looks like the following list can be safely closed.
> 
> Committers, can you, please, push the actual «close» button for this list of 
> PRs?
> 
> https://github.com/apache/kafka/pull/560
> https://github.com/apache/kafka/pull/200
> https://github.com/apache/kafka/pull/62
> https://github.com/apache/kafka/pull/719
> https://github.com/apache/kafka/pull/735
> https://github.com/apache/kafka/pull/757
> https://github.com/apache/kafka/pull/824
> https://github.com/apache/kafka/pull/880
> https://github.com/apache/kafka/pull/907
> https://github.com/apache/kafka/pull/983
> https://github.com/apache/kafka/pull/1035
> https://github.com/apache/kafka/pull/1078
> https://github.com/apache/kafka/pull/
> https://github.com/apache/kafka/pull/1135
> https://github.com/apache/kafka/pull/1147
> https://github.com/apache/kafka/pull/1150
> https://github.com/apache/kafka/pull/1244
> https://github.com/apache/kafka/pull/1269
> https://github.com/apache/kafka/pull/1415
> https://github.com/apache/kafka/pull/1468
> 
> > 7 февр. 2022 г., в 20:04, Mickael Maison  
> > написал(а):
> > 
> > Hi David,
> > 
> > I agree with you, I think we should close stale PRs.
> > 
> > Overall, I think we should also see if there are other Github actions
> > that may ease the work for reviewers and/or give more visibility of
> > the process to PR authors.
> > I'm thinking things like:
> > - code coverage changes
> > - better view on results from the build, for example if it's failing
> > checkstyle, the author could be notified first
> > - check whether public API are touched and it requires a KIP
> > 
> > For example, see some actions/integration used by other Apache projects:
> > - Flink: https://github.com/apache/flink/pull/18638#issuecomment-1030709579
> > - Beam: https://github.com/apache/beam/pull/16746#issue-1124656975
> > - Pinot: https://github.com/apache/pinot/pull/8139#issuecomment-1030701265
> > 
> > Finally, as several people have mentioned already, what can we do to
> > increase the impact of contributors that are not (yet?) committers?
> > Currently, our long delays in reviewing PRs and KIPs is hurting the
> > project and we're for sure missing out some fixes and potential
> > contributors. I think Josep's idea is interesting and finding ways to
> > engage more people and share some responsibilities better will improve
> > the project. Currently the investment to become a committer is pretty
> > high. This could provide a stepping stone (or an intermediary role)
> > for some people in the community.
> > 
> > Thanks,
> > Mickael
> > 
> > 
> > On Mon, Feb 7, 2022 at 12:51 PM Josep Prat  
> > wrote:
> > > 
> > > Hi,
> > > 
> > > It seems like a great idea. I agree with you that we should use this as a
> > > means to notify contributors and reviewers that there is some work to be
> > > done.
> > > 
> > > Regarding labels, a couple of things, first one is that PR participants
> > > won't get notified when a label is applied. So probably it would be best 
> > > to
> > > apply a label and add a comment.
> > > Secondly, GitHub offers better fine-grained roles for contributors: read,
> > > triage, write, maintain, admin (further reading here:
> > > https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-roles-for-an-organization#permissions-for-each-role).
> > > One thing that might make sense to do maybe is to add frequent 
> > > contributors
> > > with the "triage" role, so they could label PRs they reviewed and they can
> > > be taken by committers for a further review and potential merge. What do
> > > you think?
> > > 
> > > Best,
> > > 
> > > On Mon, Feb 7, 2022 at 12:16 PM Nikolay Izhikov  
> > > wrote:
> > > 
> > > > > We do not have a separate list of PRs that need pre-reviews.
> > > > 
> > > > Ok.
> > > > What should I do if PR need to be to closed found?
> > > > Who can I tag to do actual close?
> > > > 
> > > > > 7 февр. 2022 г., в 13:53, Bruno Cadonna  
> > > > > написал(а):
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > Thank you David for bringing this up!
> > > > > 
> > > > > I am in favour of automatically closing stale PRs. I agree with 
> > > > > Guozhang
> > > > that notifications of staleness to authors would be better than silently
> > > > closing them. I assume the notification happens automatically when the
> > > > label "Stale" is added to the PR.
> > > > > 
> > > > > +1 for Matthias' proposal of non-committers doing a pre-review. That
> > > > would definitely save some time for committer reviews.
> > > > > 
> > > > > Nikolay, great that you are willing to do reviews. We do not have a
> > > > separate list of PRs that need pre-reviews. You can consult the list of 
> > > > PRs
> > > > of Apache Kafka 

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

[John Roesler]

Thanks for the KIP Jorge,

I'm in support of your proposal.

1)
I do agree with Guozhang's point (1). I think the cleanest
approach. I think it's cleaner and better to keep the
enforcement internal to the framework than to introduce a
public API or context wrapper for processors to use
explicitly.

2) I tend to agree with you on this one; I think the
equality check ought to be fast enough in practice.

3) I think this is implicit, but should be explicit in the
KIP: For the `processValues` API, because the framework sets
the key on the context before calling `process` and then
unsets it afterwards, there will always be no key set during
task puctuation. Therefore, while processors may still
register punctuators, they will not be able to forward
anything from them.

This is functionally equivalent to the existing
transformers, by the way, that are also forbidden to forward
anything during punctuation.

For what it's worth, I think this is the best tradeoff.

The only alternative I see is not to place any restriction
on forwarded keys at all and just document that if users
don't maintain proper partitioning, they'll get undefined
behavior. That might be more powerful, but it's also a
usability problem.

Thanks,
-John

On Wed, 2022-02-09 at 11:34 +, Jorge Esteban Quilcate
Otoya wrote:
> Thanks Guozhang.
> 
> > Does `ValueProcessorContext` have to be a public API? It seems to me
> that this can be completely abstracted away from user interfaces as an
> internal class
> 
> Totally agree. No intention to add these as public APIs. Will update the
> KIP to reflect this.
> 
> > in the past the rationale for enforcing it at the
> interface layer rather than do runtime checks is that it is more efficient.
> > I'm not sure how much overhead it may incur to check if the key did not
> change: if it is just a reference equality check maybe it's okay. What's
> your take on this?
> 
> Agree, reference equality should cover this validation and the overhead
> impact should not be meaningful.
> Will update the KIP to reflect this as well.
> 
> 
> On Tue, 8 Feb 2022 at 19:05, Guozhang Wang  wrote:
> 
> > Hello Jorge,
> > 
> > Thanks for bringing this KIP! I think this is a nice idea to consider using
> > a single overloaded function name for #process, just a couple quick
> > questions after reading the proposal:
> > 
> > 1) Does `ValueProcessorContext` have to be a public API? It seems to me
> > that this can be completely abstracted away from user interfaces as an
> > internal class, and we call the `setKey` before calling user-instantiated
> > `process` function, and then in its overridden `forward` it can just check
> > if the key changes or not.
> > 2) Related to 1) above, in the past the rationale for enforcing it at the
> > interface layer rather than do runtime checks is that it is more efficient.
> > I'm not sure how much overhead it may incur to check if the key did not
> > change: if it is just a reference equality check maybe it's okay. What's
> > your take on this?
> > 
> > 
> > Guozhang
> > 
> > On Tue, Feb 8, 2022 at 5:17 AM Jorge Esteban Quilcate Otoya <
> > quilcate.jo...@gmail.com> wrote:
> > 
> > > Hi Dev team,
> > > 
> > > I'd like to start a new discussion thread on Kafka Streams KIP-820:
> > > 
> > > 
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-820%3A+Extend+KStream+process+with+new+Processor+API
> > > 
> > > This KIP is aimed to extend the current `KStream#process` API to return
> > > output values that could be chained across the topology, as well as
> > > introducing a new `KStream#processValues` to use processor while
> > validating
> > > keys haven't change and repartition is not required.
> > > 
> > > Looking forward to your feedback.
> > > 
> > > Regards,
> > > Jorge.
> > > 
> > 
> > 
> > --
> > -- Guozhang
> > 

[jira] [Resolved] (KAFKA-13646) Implement KIP-801: KRaft authorizer

[Jason Gustafson (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-13646.
-
Fix Version/s: 3.2.0
   Resolution: Fixed

> Implement KIP-801: KRaft authorizer
> ---
>
> Key: KAFKA-13646
> URL: https://issues.apache.org/jira/browse/KAFKA-13646
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Major
>  Labels: kip-500, kip-801
> Fix For: 3.2.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #673

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 292528 lines...]
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[2022-02-09T18:26:27.455Z] 
[2022-02-09T18:26:27.455Z] 
org.apache.kafka.streams.integration.StoreUpgradeIntegrationTest > 
shouldMigrateInMemoryWindowStoreToTimestampedWindowStoreUsingPapi PASSED
[2022-02-09T18:26:27.455Z] 
[2022-02-09T18:26:27.455Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testInner[caching enabled = true] STARTED
[2022-02-09T18:26:30.675Z] 
[2022-02-09T18:26:30.675Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testInner[caching enabled = true] PASSED
[2022-02-09T18:26:30.675Z] 
[2022-02-09T18:26:30.675Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testLeft[caching enabled = true] STARTED
[2022-02-09T18:26:33.846Z] 
[2022-02-09T18:26:33.846Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testLeft[caching enabled = true] PASSED
[2022-02-09T18:26:33.846Z] 
[2022-02-09T18:26:33.846Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testInner[caching enabled = false] STARTED
[2022-02-09T18:26:38.351Z] 
[2022-02-09T18:26:38.351Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testInner[caching enabled = false] PASSED
[2022-02-09T18:26:38.351Z] 
[2022-02-09T18:26:38.351Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testLeft[caching enabled = false] STARTED
[2022-02-09T18:26:41.593Z] 
[2022-02-09T18:26:41.593Z] 
org.apache.kafka.streams.integration.StreamTableJoinIntegrationTest > 
testLeft[caching enabled = false] PASSED
[2022-02-09T18:26:41.593Z] 
[2022-02-09T18:26:41.593Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldReplaceThreads STARTED
[2022-02-09T18:26:42.529Z] 
[2022-02-09T18:26:42.529Z] Exception: java.lang.AssertionError thrown from the 
UncaughtExceptionHandler in thread 
"appId_StreamsUncaughtExceptionHandlerIntegrationTestshouldReplaceThreads-68bb1508-5149-422a-a94b-7e9d94029758-StreamThread-1"
[2022-02-09T18:26:53.146Z] 
[2022-02-09T18:26:53.146Z] Exception: java.lang.AssertionError thrown from the 
UncaughtExceptionHandler in thread 
"appId_StreamsUncaughtExceptionHandlerIntegrationTestshouldReplaceThreads-68bb1508-5149-422a-a94b-7e9d94029758-StreamThread-2"
[2022-02-09T18:27:03.155Z] 
[2022-02-09T18:27:03.155Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldReplaceThreads PASSED
[2022-02-09T18:27:03.155Z] 
[2022-02-09T18:27:03.155Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownClientWhenIllegalStateException STARTED
[2022-02-09T18:27:05.493Z] 
[2022-02-09T18:27:05.493Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownClientWhenIllegalStateException PASSED
[2022-02-09T18:27:05.493Z] 
[2022-02-09T18:27:05.493Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownThreadUsingOldHandler STARTED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownThreadUsingOldHandler PASSED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutDownClientIfGlobalStreamThreadWantsToReplaceThread STARTED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] Exception: java.lang.AssertionError thrown from the 
UncaughtExceptionHandler in thread 
"appId_StreamsUncaughtExceptionHandlerIntegrationTestshouldShutDownClientIfGlobalStreamThreadWantsToReplaceThread-ce4fa289-dd19-429d-b8be-5a2ea91a989b-GlobalStreamThread"
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutDownClientIfGlobalStreamThreadWantsToReplaceThread PASSED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownClientWhenIllegalArgumentException STARTED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldShutdownClientWhenIllegalArgumentException PASSED
[2022-02-09T18:27:21.725Z] 
[2022-02-09T18:27:21.725Z] 
org.apache.kafka.streams.integration.StreamsUncaughtExceptionHandlerIntegrationTest
 > shouldReplaceSingleThread STARTED
[2022-02-09T18:27:21.725Z] 

[jira] [Created] (KAFKA-13661) KRaft uses the wrong permission for creating partitions

[Jason Gustafson (Jira)]

Jason Gustafson created KAFKA-13661:
---

 Summary: KRaft uses the wrong permission for creating partitions
 Key: KAFKA-13661
 URL: https://issues.apache.org/jira/browse/KAFKA-13661
 Project: Kafka
  Issue Type: Bug
Affects Versions: 3.0.0, 3.1.0
Reporter: Jason Gustafson
Assignee: Jason Gustafson


[~cmccabe] caught this as part of KAFKA-13646. KRaft currently checks CREATE on 
the topic resource. It should be ALTER. This will be fixed in trunk as part of 
KAFKA-13646, but it would be good to fix for 3.0 and 3.1 as well.

Note this does not affect zookeeper-based clusters.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (KAFKA-13660) Replace log4j with reload4j

[Mike Lothian (Jira)]

Mike Lothian created KAFKA-13660:


 Summary: Replace log4j with reload4j
 Key: KAFKA-13660
 URL: https://issues.apache.org/jira/browse/KAFKA-13660
 Project: Kafka
  Issue Type: Bug
  Components: logging
Affects Versions: 3.0.0, 2.4.0
Reporter: Mike Lothian


Kafka is using a known vulnerable version of log4j, the reload4j project was 
created by the code's original authors to address those issues. It is designed 
as a drop in replacement without any api changes

 

I've raised a merge request, replacing log4j with reload4j, slf4j-log4j12 with 
slf4j-reload4j and bumping the slf4j version

 

this is my first time contributing to the Kafka project and I'm not too 
familiar with the process, I'll go back and amend my PR with this issue number



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

RE: [DISCUSS] KIP-510: Metrics library upgrade

[Shannon Carey]

Hi Mario,

If you rename a metric named “yammer-metrics-count”, I think you should change 
it to just “metrics-count”, not to another implementation-specific name.

It might be worth considering using https://micrometer.io/ as it supports a 
wider array of more modern metrics reporting targets. For example, you can 
report timers as bucketed histograms to Prometheus which allows you to 
visualize them as heat maps and calculate arbitrary percentiles across all 
reporting instances.

Taking inspiration from some other open source projects, Flink for example 
provides a metrics.reporter..class configuration option 
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/metric_reporters/
 which allows you to specify one or more reporters and their parameters. Spring 
Boot also has support for configuring various reporters 
https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics
 as well as configuring various aspects of specific metrics 
https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics.customizing

In any case, I support your KIP.

-Shannon

On 2019/08/21 13:57:14 Mario Molina wrote:
> Hi there,
>
> I've written KIP-510
> 
> proposing the upgrade of the metrics library which Kafka is using.
>
> Please, have a look and let me know what you think,
> Thanks,
> Mario
>

[jira] [Created] (KAFKA-13659) MM2 should read all offset syncs at start up and should not set consumer offset higher than the end offset

[Kanalas Vidor (Jira)]

Kanalas Vidor created KAFKA-13659:
-

 Summary: MM2 should read all offset syncs at start up and should 
not set consumer offset higher than the end offset
 Key: KAFKA-13659
 URL: https://issues.apache.org/jira/browse/KAFKA-13659
 Project: Kafka
  Issue Type: Improvement
  Components: mirrormaker
Reporter: Kanalas Vidor


- MirrorCheckpointTask uses OffsetSyncStore, and does not check whether 
OffsetSyncStore managed to read to the "end" of the offset-syncs topic. 
OffsetSyncStore should fetch the endoffset of the topic at startup, and set a 
flag when it finally reaches the endoffset in consumption. 
MirrorCheckpointTask.poll should wait for this flag to be true before doing any 
in-memory updates and group offset management.
 - MirrorCheckpointTask can create checkpoints which point into the "future" - 
meaning it sometimes translates consumer offsets in a way that the target 
offset is greater than the endoffset of the replica topic partition. 
MirrorCheckpointTask should fetch the endoffsets of the affected topics, and 
make sure that it does not try to set the consumer offset to anything higher 
than the endoffset.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

[Jorge Esteban Quilcate Otoya]

Thanks Guozhang.

> Does `ValueProcessorContext` have to be a public API? It seems to me
that this can be completely abstracted away from user interfaces as an
internal class

Totally agree. No intention to add these as public APIs. Will update the
KIP to reflect this.

> in the past the rationale for enforcing it at the
interface layer rather than do runtime checks is that it is more efficient.
> I'm not sure how much overhead it may incur to check if the key did not
change: if it is just a reference equality check maybe it's okay. What's
your take on this?

Agree, reference equality should cover this validation and the overhead
impact should not be meaningful.
Will update the KIP to reflect this as well.


On Tue, 8 Feb 2022 at 19:05, Guozhang Wang  wrote:

> Hello Jorge,
>
> Thanks for bringing this KIP! I think this is a nice idea to consider using
> a single overloaded function name for #process, just a couple quick
> questions after reading the proposal:
>
> 1) Does `ValueProcessorContext` have to be a public API? It seems to me
> that this can be completely abstracted away from user interfaces as an
> internal class, and we call the `setKey` before calling user-instantiated
> `process` function, and then in its overridden `forward` it can just check
> if the key changes or not.
> 2) Related to 1) above, in the past the rationale for enforcing it at the
> interface layer rather than do runtime checks is that it is more efficient.
> I'm not sure how much overhead it may incur to check if the key did not
> change: if it is just a reference equality check maybe it's okay. What's
> your take on this?
>
>
> Guozhang
>
> On Tue, Feb 8, 2022 at 5:17 AM Jorge Esteban Quilcate Otoya <
> quilcate.jo...@gmail.com> wrote:
>
> > Hi Dev team,
> >
> > I'd like to start a new discussion thread on Kafka Streams KIP-820:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-820%3A+Extend+KStream+process+with+new+Processor+API
> >
> > This KIP is aimed to extend the current `KStream#process` API to return
> > output values that could be chained across the topology, as well as
> > introducing a new `KStream#processValues` to use processor while
> validating
> > keys haven't change and repartition is not required.
> >
> > Looking forward to your feedback.
> >
> > Regards,
> > Jorge.
> >
>
>
> --
> -- Guozhang
>

Re: [DISCUSS] Should we automatically close stale PRs?

[Nikolay Izhikov]

Hello, guys.

I made a quick search throw oldest PRs.
Looks like the following list can be safely closed.

Committers, can you, please, push the actual «close» button for this list of 
PRs?

https://github.com/apache/kafka/pull/560
https://github.com/apache/kafka/pull/200
https://github.com/apache/kafka/pull/62
https://github.com/apache/kafka/pull/719
https://github.com/apache/kafka/pull/735
https://github.com/apache/kafka/pull/757
https://github.com/apache/kafka/pull/824
https://github.com/apache/kafka/pull/880
https://github.com/apache/kafka/pull/907
https://github.com/apache/kafka/pull/983
https://github.com/apache/kafka/pull/1035
https://github.com/apache/kafka/pull/1078
https://github.com/apache/kafka/pull/
https://github.com/apache/kafka/pull/1135
https://github.com/apache/kafka/pull/1147
https://github.com/apache/kafka/pull/1150
https://github.com/apache/kafka/pull/1244
https://github.com/apache/kafka/pull/1269
https://github.com/apache/kafka/pull/1415
https://github.com/apache/kafka/pull/1468

> 7 февр. 2022 г., в 20:04, Mickael Maison  
> написал(а):
> 
> Hi David,
> 
> I agree with you, I think we should close stale PRs.
> 
> Overall, I think we should also see if there are other Github actions
> that may ease the work for reviewers and/or give more visibility of
> the process to PR authors.
> I'm thinking things like:
> - code coverage changes
> - better view on results from the build, for example if it's failing
> checkstyle, the author could be notified first
> - check whether public API are touched and it requires a KIP
> 
> For example, see some actions/integration used by other Apache projects:
> - Flink: https://github.com/apache/flink/pull/18638#issuecomment-1030709579
> - Beam: https://github.com/apache/beam/pull/16746#issue-1124656975
> - Pinot: https://github.com/apache/pinot/pull/8139#issuecomment-1030701265
> 
> Finally, as several people have mentioned already, what can we do to
> increase the impact of contributors that are not (yet?) committers?
> Currently, our long delays in reviewing PRs and KIPs is hurting the
> project and we're for sure missing out some fixes and potential
> contributors. I think Josep's idea is interesting and finding ways to
> engage more people and share some responsibilities better will improve
> the project. Currently the investment to become a committer is pretty
> high. This could provide a stepping stone (or an intermediary role)
> for some people in the community.
> 
> Thanks,
> Mickael
> 
> 
> On Mon, Feb 7, 2022 at 12:51 PM Josep Prat  
> wrote:
>> 
>> Hi,
>> 
>> It seems like a great idea. I agree with you that we should use this as a
>> means to notify contributors and reviewers that there is some work to be
>> done.
>> 
>> Regarding labels, a couple of things, first one is that PR participants
>> won't get notified when a label is applied. So probably it would be best to
>> apply a label and add a comment.
>> Secondly, GitHub offers better fine-grained roles for contributors: read,
>> triage, write, maintain, admin (further reading here:
>> https://docs.github.com/en/organizations/managing-access-to-your-organizations-repositories/repository-roles-for-an-organization#permissions-for-each-role).
>> One thing that might make sense to do maybe is to add frequent contributors
>> with the "triage" role, so they could label PRs they reviewed and they can
>> be taken by committers for a further review and potential merge. What do
>> you think?
>> 
>> Best,
>> 
>> On Mon, Feb 7, 2022 at 12:16 PM Nikolay Izhikov  wrote:
>> 
 We do not have a separate list of PRs that need pre-reviews.
>>> 
>>> Ok.
>>> What should I do if PR need to be to closed found?
>>> Who can I tag to do actual close?
>>> 
 7 февр. 2022 г., в 13:53, Bruno Cadonna  написал(а):
 
 Hi,
 
 Thank you David for bringing this up!
 
 I am in favour of automatically closing stale PRs. I agree with Guozhang
>>> that notifications of staleness to authors would be better than silently
>>> closing them. I assume the notification happens automatically when the
>>> label "Stale" is added to the PR.
 
 +1 for Matthias' proposal of non-committers doing a pre-review. That
>>> would definitely save some time for committer reviews.
 
 Nikolay, great that you are willing to do reviews. We do not have a
>>> separate list of PRs that need pre-reviews. You can consult the list of PRs
>>> of Apache Kafka (https://github.com/apache/kafka/pulls) and choose from
>>> there. I think that is the simplest way to start reviewing. Maybe Luke has
>>> some tips here since he does an excellent job in reviewing as a
>>> non-committer.
 
 Best,
 Bruno
 
 On 07.02.22 08:24, Nikolay Izhikov wrote:
> Hello, Matthias, Luke.
>> I agree with Matthias that contributors could and should help do more
>>> "pre-review" PRs.
> I, personally, ready to do the initial review of PRs. Do we have some
>>> recipe to filter PRs that has potential to land in 

[jira] [Resolved] (KAFKA-13616) Log4j 1.X CVE-2022-23302/5/7 vulnerabilities

[Dongjin Lee (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dongjin Lee resolved KAFKA-13616.
-
Resolution: Duplicate

> Log4j 1.X CVE-2022-23302/5/7 vulnerabilities
> 
>
> Key: KAFKA-13616
> URL: https://issues.apache.org/jira/browse/KAFKA-13616
> Project: Kafka
>  Issue Type: Bug
>Reporter: Dominique Mongelli
>Priority: Major
>
> Some log4j 1.x vulnerabilities have been disclosed recently:   
>  * CVE-2022-23302: https://nvd.nist.gov/vuln/detail/CVE-2022-23302    
>  * CVE-2022-23305 : https://nvd.nist.gov/vuln/detail/CVE-2022-23305    
>  * CVE-2022-23307 : [https://nvd.nist.gov/vuln/detail/CVE-2022-23307]
> We would like to know if kafka is affected by these vulnerabilities ?
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #672

[Apache Jenkins Server]

See