Build failed in Jenkins: kafka-trunk-jdk8 #3915

See Changes: [cmccabe] MINOR: the code generator should be able to set the java package (#7355) [manikumar] KAFKA-8892: Display the sorted configs in Kafka Configs Help Command. --

Build failed in Jenkins: kafka-trunk-jdk11 #827

See Changes: [github] KAFKA-8609: Add consumer rebalance metrics (#7347) -- [...truncated 2.64 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldPunctuateIf

Build failed in Jenkins: kafka-trunk-jdk11 #826

See Changes: [manikumar] KAFKA-8892: Display the sorted configs in Kafka Configs Help Command. -- [...truncated 2.63 MB...] org.apache.kafka.streams.TopologyTestDriverTe

[jira] [Resolved] (KAFKA-7990) Flaky Test KafkaStreamsTest#shouldCleanupOldStateDirs

[ https://issues.apache.org/jira/browse/KAFKA-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7990. -- Fix Version/s: 2.3.0 2.2.1 Resolution: Fixed > Flaky Test KafkaStream

[jira] [Reopened] (KAFKA-8319) Flaky Test KafkaStreamsTest.statefulTopologyShouldCreateStateDirectory

[ https://issues.apache.org/jira/browse/KAFKA-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-8319: -- > Flaky Test KafkaStreamsTest.statefulTopologyShouldCreateStateDirectory >

Re: [VOTE] KIP-482: The Kafka Protocol should Support Optional Tagged Fields

Hi, Colin, Thanks for the KIP. Overall, looks good to me too. A couple of minor comments. 100. Does the tag number need to be 31-bit? It seems that 15-bit could be enough. 101. We already use varInt in the message format. I assume that the protocol uses the same varInt representation? Jun O

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

+1 (non-binding) Definitely needed this before as it would have saved me some time from trying to guess a client's version from api version/source code. Thanks for the KIP! Regards, Kevin On Fri, Sep 20, 2019 at 4:29 PM Jason Gustafson wrote: > +1 from me. This is a clever solution. Kind of a

Re: [VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

Nice job on the KIP Konstantine. This is a great improvement. +1 (binding) -Bill On Fri, Sep 20, 2019 at 5:07 PM Mitchell wrote: > +1 (non-binding). > > Love this change. > > On Fri, Sep 20, 2019, 4:37 PM Randall Hauch wrote: > > > +1 (binding) > > > > Great work, Konstantine. > > > > On Thu,

Re: [VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

+1 (non-binding). Love this change. On Fri, Sep 20, 2019, 4:37 PM Randall Hauch wrote: > +1 (binding) > > Great work, Konstantine. > > On Thu, Sep 19, 2019 at 11:50 AM Gwen Shapira wrote: > > > +1 (binding) > > > > On Thu, Sep 19, 2019 at 8:53 AM Konstantine Karantasis > > wrote: > > > > > >

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

+1 from me. This is a clever solution. Kind of a pity we couldn't work flexible version support into the response, but I understand why it is difficult. One minor nitpick: the INVALID_REQUEST error already exists. Are you intending to reuse it? Thanks, Jason On Fri, Sep 20, 2019 at 3:50 PM Konst

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

Quite useful KIP from an operational standpoint and I also like it in its most recent revised form. +1 (non-binding). Konstantine On Fri, Sep 20, 2019 at 9:55 AM Gwen Shapira wrote: > +1 (binding) > > Thanks for the KIP, David and for the help with the design, Colin. I > think it looks great

Re: [VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

+1 (binding) Great work, Konstantine. On Thu, Sep 19, 2019 at 11:50 AM Gwen Shapira wrote: > +1 (binding) > > On Thu, Sep 19, 2019 at 8:53 AM Konstantine Karantasis > wrote: > > > > I'd like to start the vote on KIP-521. > > > > The proposal seems straightforward and no major concerns came up

RE: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

The KIP now says: We believe that making SSLEngine creation pluggable is worth to allow SSL experts to write their own implementation having the SSL domain knowledge and keep them free of knowing much about Kafka's reconfigurability. The other custom reconfigurable extension points don't seem t

Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

+1 (binding). Thanks, Rajini. best, Colin On Fri, Sep 20, 2019, at 00:43, Rajini Sivaram wrote: > Hi all, > > I would like to start vote on KIP-525 to return configs in CreateTopics > response. This is a minor KIP that returns additional data in the response > without breaking compatibility. >

Re: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

Thanks Clement for your thoughts. According to my current experience rewriting the code twice I would say I did what you suggest in the last point - " We must make an attempt, if only to explain why it fails in the Rejected Alternatives section of the KIP." In the rejected alternatives I already me

Re: [VOTE] KIP-517: Add consumer metrics to observe user poll behavior

Hi All, Thanks for the votes! I have to point out that Jason brought up a good point about making the " *poll-idle-ratio*" measurement consistent with existing broker idle metrics so we made a small change. Previously it was calculated as: > poll-idle-ratio = time-between-poll / time-inside-pol

Re: [VOTE] KIP-517: Add consumer metrics to observe user poll behavior

+1 (binding) Thanks for the KIP! -Bill On Fri, Sep 20, 2019 at 1:28 PM Jason Gustafson wrote: > +1 Thanks! > > On Thu, Sep 19, 2019 at 11:22 PM Tom Bentley wrote: > > > +1 (non-binding). > > > > On Fri, Sep 20, 2019 at 7:00 AM Maulin Vasavada < > maulin.vasav...@gmail.com > > > > > wrote: > >

Re: [VOTE] KIP-517: Add consumer metrics to observe user poll behavior

+1 Thanks! On Thu, Sep 19, 2019 at 11:22 PM Tom Bentley wrote: > +1 (non-binding). > > On Fri, Sep 20, 2019 at 7:00 AM Maulin Vasavada > > wrote: > > > +1 (non-binding). Thanks for the KIP. > > > > On Thu, Sep 19, 2019 at 10:38 PM Manikumar > > wrote: > > > > > +1 (binding), Thanks for the KIP

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Kevin, I'm coming in a little late, but thanks for the KIP, this will be very useful. It's a +1 for me. -Bill On Fri, Sep 20, 2019 at 11:32 AM Kevin Lu wrote: > Hi Jason, > > That is a good point in making the ratio metric consistent with the broker > idle metrics. I have updated the KIP to

Re: [DISCUSS] KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics

I've updated the 'Proposed Changes' to include two new producer configuration variables: topic.expiry.ms and topic.refresh.ms. Please take a look. Thanks, Brian On Tue, Sep 17, 2019 at 12:59 PM Brian Byrne wrote: > Dev team, > > Requesting discussion for improvement to the producer when dealing

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Jason, That is a good point in making the ratio metric consistent with the broker idle metrics. I have updated the KIP to calculate the poll idle ratio as you suggested. Thanks for pointing this out. Regards, Kevin On Fri, Sep 20, 2019 at 10:09 AM Jason Gustafson wrote: > Hi Kevin, > > For

[jira] [Resolved] (KAFKA-8892) Display the sorted configs in Kafka Configs Help Command.

[ https://issues.apache.org/jira/browse/KAFKA-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-8892. -- Fix Version/s: 2.4.0 Resolution: Fixed Issue resolved by pull request 7319 [https://github.com/a

Re: [VOTE] KIP-528: Deprecate PartitionGrouper configuration and interface

+1 (binding) On Thu, Sep 19, 2019 at 4:28 PM Matthias J. Sax wrote: > > Hi, > > I would like to propose a small KIP to deprecate some public APIs that > are considered "dangerous" to use, with the goal to remove them in the > next major release. > > Because the KIP is straight forward and to make

Re: [VOTE] KIP-528: Deprecate PartitionGrouper configuration and interface

Hi Matthias, +1 (non-binding) Best, Bruno On Fri, Sep 20, 2019 at 10:02 AM Matthias J. Sax wrote: > > Sure -- don't think we need to cover it in the KIP though. > > Btw: an implicit deprecation is implied by this KIP for the currently > public default implementation of the `PartitionGrouper` in

Re: [DISCUSS] KIP-522: Update BrokerApiVersionsCommand to use AdminClient

Thank you Colin for the context. Before opening the KIP I had a quick look and found the JIRAs you linked but saw no traces of the concerns you mentioned. I also found the PR but could not find an associated KIP. The example I provided in the KIP (incrementalAlterConfigs/alterConfigs) is one of t

Re: [VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

+1 (non-binding) Thanks for your efforts Best regards Patrik > Am 20.09.2019 um 19:09 schrieb Bill Bejeck : > > Thanks for the KIP! > > +1(binding) > > -Bill > >> On Fri, Sep 20, 2019 at 10:06 AM Guozhang Wang wrote: >> >> +1 (binding), Thanks! >> >> On Fri, Sep 20, 2019 at 9:56 AM Matthia

Re: [VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

Thanks for the KIP! +1(binding) -Bill On Fri, Sep 20, 2019 at 10:06 AM Guozhang Wang wrote: > +1 (binding), Thanks! > > On Fri, Sep 20, 2019 at 9:56 AM Matthias J. Sax > wrote: > > > Thanks for the KIP Jukka! > > > > +1 (binding) > > > > -Matthias > > > > On 9/20/19 9:44 AM, Paul Whalen wrote

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Kevin, For the computation of the idle ratio, can we make it consistent with the idle ratios on the broker? Basically we use the following: idle ratio = idle time / total time So when the consumer is idle (i.e waiting for records), then the idle ratio approaches 1. When the application is bus

Re: [VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

+1 (binding), Thanks! On Fri, Sep 20, 2019 at 9:56 AM Matthias J. Sax wrote: > Thanks for the KIP Jukka! > > +1 (binding) > > -Matthias > > On 9/20/19 9:44 AM, Paul Whalen wrote: > > +1 (non-binding). I haven’t contributed to the discussion but I’ve been > following - it’ll definitely make my te

Re: [VOTE] KIP-528: Deprecate PartitionGrouper configuration and interface

Sure -- don't think we need to cover it in the KIP though. Btw: an implicit deprecation is implied by this KIP for the currently public default implementation of the `PartitionGrouper` interface. When we finally remove the public interface, we can move the default implementation into an internal p

Re: [VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

Thanks for the KIP Jukka! +1 (binding) -Matthias On 9/20/19 9:44 AM, Paul Whalen wrote: > +1 (non-binding). I haven’t contributed to the discussion but I’ve been > following - it’ll definitely make my team’s life easier. > >> On Sep 20, 2019, at 11:36 AM, Jukka Karvanen >> wrote: >> >> Hi a

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

+1 (binding) Thanks for the KIP, David and for the help with the design, Colin. I think it looks great now. On Fri, Sep 20, 2019 at 9:42 AM Colin McCabe wrote: > > On Fri, Sep 20, 2019, at 01:45, David Jacot wrote: > > > Thanks for the clarification. The proposed behavior sounds reasonable. > >

[jira] [Created] (KAFKA-8928) Logged producer config does not always match actual config values

Chris Pettitt created KAFKA-8928: Summary: Logged producer config does not always match actual config values Key: KAFKA-8928 URL: https://issues.apache.org/jira/browse/KAFKA-8928 Project: Kafka

Re: [VOTE] KIP-528: Deprecate PartitionGrouper configuration and interface

+1 (non-binding). Thanks for bringing this up. Should we also log a deprecation warning on startup if the config is used? It would be just one extra opportunity to learn of the deprecation in case you missed the compiler's warning. -John On Thu, Sep 19, 2019 at 4:49 PM Bill Bejeck wrote: > > Th

Re: [VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

+1 (non-binding). I haven’t contributed to the discussion but I’ve been following - it’ll definitely make my team’s life easier. > On Sep 20, 2019, at 11:36 AM, Jukka Karvanen > wrote: > > Hi all, > > I would like to start vote on KIP-470: > https://cwiki.apache.org/confluence/display/KAFKA/

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

On Fri, Sep 20, 2019, at 01:45, David Jacot wrote: > > Thanks for the clarification. The proposed behavior sounds reasonable. > > Can you add a note about the implementation on the client? The client > > needs to be prepared to handle > a response that doesn't include the > > versions, as well, s

[jira] [Created] (KAFKA-8929) MM2 system tests

Ryanne Dolan created KAFKA-8929: --- Summary: MM2 system tests Key: KAFKA-8929 URL: https://issues.apache.org/jira/browse/KAFKA-8929 Project: Kafka Issue Type: Improvement Components: mi

[VOTE] KIP-470: TopologyTestDriver test input and output usability improvements

Hi all, I would like to start vote on KIP-470: https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements Regards, Jukka

[jira] [Created] (KAFKA-8930) MM2 documentation

Ryanne Dolan created KAFKA-8930: --- Summary: MM2 documentation Key: KAFKA-8930 URL: https://issues.apache.org/jira/browse/KAFKA-8930 Project: Kafka Issue Type: Improvement Components: d

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

Hi All, I modified wiki page to replace Serdes with Serializer / Deserializer. I will start the vote. Jukka pe 20. syysk. 2019 klo 4.21 John Roesler (j...@confluent.io) kirjoitti: > Hey, all, > > For what it's worth, I agree with Matthias. While it's true that > you're likely to have Serdes on

Re: Status of user Admin API for quotas

Hi Tom, As you said, there were a few KIPs, but they seem to have become inactive. It's kind of a tough problem-- probably at least as complex as the admin interface for ACLs. There's also the headache of how reassignment quotas should work. We probably want to change that quota to actually t

Re: KAFKA-8584: Support of ByteBuffer for bytes field implemented[Convert Kafka RPCs to use automatically generated code]

Hi Nikolay, Thanks for working on this. I think everyone agrees that we should have byte buffer support in the generator. We just haven't had a lot of time for reviewing it lately. I don't really mind which PR we use :) I will take a look at your PR today and see if we can get it into shap

Re: KAFKA-8584: Support of ByteBuffer for bytes field implemented[Convert Kafka RPCs to use automatically generated code]

Hello, all. Any feedback on this? Do we need support of ByteBuffer in RPC generated code? Which PR should be reviwed and merged? В Чт, 19/09/2019 в 10:11 +0300, Nikolay Izhikov пишет: > Hello, guys. > > Looks like we have duplicate tickets and PR's here. > > One from me: > > KAFKA-8584: Suppo

Re: [VOTE] KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local rule

Hi All, The vote has passed with 3 binding votes (Harsha, Rajini, Manikumar). Thanks for the votes. Thanks, Manikumar On Fri, Sep 20, 2019 at 8:55 PM Manikumar wrote: > > +1 (binding) from me > > Thanks, > > On Fri, Sep 20, 2019 at 4:39 PM Rajini Sivaram > wrote: > >> Thanks for KIP, Manikuma

Re: [VOTE] KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local rule

+1 (binding) from me Thanks, On Fri, Sep 20, 2019 at 4:39 PM Rajini Sivaram wrote: > Thanks for KIP, Manikumar! > > +1 (binding) > > Regards, > > Rajini > > On Mon, Sep 16, 2019 at 11:11 PM Harsha Chintalapani > wrote: > > > +1 (binding). > > Thanks, > > Harsha > > > > > > On Mon, Sep 16, 2019

RE: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

There will be chaperon code in the base class of the channel builders. The arguments you gave me are emotional not technical. The SSL extension point is reconfigurable hence it should extend Reconfigurable. We will encounter issues when we try to prototype it that way. We will solve the issues or

Jenkins build is back to normal : kafka-trunk-jdk11 #824

See

Status of user Admin API for quotas

Hi, I was wondering, what is the current status of efforts to add an AdminClient API for managing user quotas? My trawl of the list archives didn't turn up anything current (KIP-248 was rejected KIP-422 was discussed), but perhaps I missed something. Many thanks, Tom

Re: [VOTE] KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local rule

Thanks for KIP, Manikumar! +1 (binding) Regards, Rajini On Mon, Sep 16, 2019 at 11:11 PM Harsha Chintalapani wrote: > +1 (binding). > Thanks, > Harsha > > > On Mon, Sep 16, 2019 at 4:21 AM, Manikumar > wrote: > > > Hi all, > > > > I would like to start vote on this trivial KIP-309: https://c

Re: [VOTE] KIP-496: Administrative API to delete consumer offsets

Great. I have updated the KIP. Thanks, David On Thu, Sep 19, 2019 at 10:56 PM Colin McCabe wrote: > Sounds good to me. It makes sense to add this functionality to the > command line. > > best, > Colin > > On Wed, Sep 18, 2019, at 11:26, David Jacot wrote: > > Indeed, I have forgotten to add th

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

> Thanks for the clarification. The proposed behavior sounds reasonable. Can you add a note about the implementation on the client? The client needs to be prepared to handle > a response that doesn't include the versions, as well, since v1 did not. I have added a note about the implementation in

[VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

Hi all, I would like to start vote on KIP-525 to return configs in CreateTopics response. This is a minor KIP that returns additional data in the response without breaking compatibility. - https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+Crea