Kafka Performance Producer - Extension

Hi All, This is in regard to extend few features support for java kafka performance producer. We have a kafka producer application written in java with maven build. Attaching the documentation which gives detailed description of features supported by this application as of now. Would like to

[jira] [Created] (KAFKA-7607) NetworkClientUtils.sendAndReceive can take a long time to return during shutdown

Bob Barrett created KAFKA-7607: -- Summary: NetworkClientUtils.sendAndReceive can take a long time to return during shutdown Key: KAFKA-7607 URL: https://issues.apache.org/jira/browse/KAFKA-7607 Project:

Re: [DISCUSS] KIP-380: Detect outdated control requests and bounced brokers using broker generation

Hi, In this KIP, we are also going to add a new exception and a new error code "STALE_BROKER_EPOCH" in order to allow the broker to respond back the right error when it sees outdated broker epoch in the control requests. Since adding a new exception and error code is also considered as public

Re: [DISCUSS] KIP-388 Add observer interface to record request and response

I know we have a lot of use cases for this type of functionality at my enterprise deployment. I think it's helpful for maintaining reliability of the cluster especially and identifying clients that are not properly tuned and therefore applying excessive load to the brokers. Additionally, there is

Jenkins build is back to normal : kafka-2.0-jdk8 #185

See

[jira] [Created] (KAFKA-7609) Add Protocol Generator for Kafka

Colin P. McCabe created KAFKA-7609: -- Summary: Add Protocol Generator for Kafka Key: KAFKA-7609 URL: https://issues.apache.org/jira/browse/KAFKA-7609 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-388 Add observer interface to record request and response

another downside to client instrumentation (beyond the number of client codebases one would need to cover) is that in a large environments you'll have a very long tail of applications using older clients to upgrade - it would be a long and disruptive process (as opposed to updating broker-side

[jira] [Created] (KAFKA-7606) Remove deprecated --zookeeper option from StreamsResetter

Matthias J. Sax created KAFKA-7606: -- Summary: Remove deprecated --zookeeper option from StreamsResetter Key: KAFKA-7606 URL: https://issues.apache.org/jira/browse/KAFKA-7606 Project: Kafka

Re: [DISCUSS] KIP-388 Add observer interface to record request and response

bump. I think the proposed API (Observer) is useful for any sort of multi-tenant environment for chargeback and reporting purposes. if no one wants to comment, can we initiate a vote? On Mon, Nov 5, 2018 at 6:31 PM Lincong Li wrote: > > Hi everyone. Here >

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

+1 (binding) Built from src and ran tests, verified the artifacts, signatures, Ran core quick start. Thanks, Manikumar On Thu, Nov 8, 2018 at 12:15 AM Rajini Sivaram wrote: > +1 (binding) > > Checked source build and unit tests. Ran quickstart with source and binary. > > Thank you for managing

[RESULTS] [VOTE] Release Kafka version 2.0.1

This vote passes with 12 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jun Rao * Gwen Shapira * Rajini Sivaram Committers: * Harsha * Ewen Cheslack-Postava * Manikumar Community: * Magnus Edenhill * Eno Thereska * Vahid Hashemian * Jakub Scholz * Satish Duggana * Mickael

[VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

Hey all, thanks so much for all the inputs on KIP-345 so far. The original proposal has enhanced a lot with your help. To make sure the implementation go smoothly without back and forth, I would like to start a vote on the final design agreement now:

Re: [VOTE] - KIP-213 Support non-key joining in KTable

Hey Adam, I'm sorry, I haven't been plugged in to your KIP lately. I'll have to read it over again. Would it be discouraging if I wound up responding to your discussion thread? Thanks, -John On Wed, Nov 7, 2018 at 3:25 PM Adam Bellemare wrote: > Bumping this thread, as per convention - 1 > >

Re: [VOTE] KIP-386: Standardize on Min/Avg/Max metrics' default values

Hi, Stanislav, Thanks for the KIP. +1. I guess this only covers the Kafka metrics (not the Yammer metrics). It would be useful to make this clear. Jun On Tue, Nov 6, 2018 at 1:00 AM, Stanislav Kozlovski wrote: > Hey everybody, > > I'm starting a vote thread on KIP-386: Standardize on

Re: [VOTE] 2.0.1 RC0

Manikumar, looks like you have the necessary votes. :) Ismael On Wed, Nov 7, 2018, 10:45 AM Rajini Sivaram +1 (binding) > > Checked source build and unit tests. Ran quickstart with source and binary. > > Thank you for managing the release, Manikumar! > > Regards, > > Rajini > > On Wed, Nov 7,

Re: [VOTE] - KIP-213 Support non-key joining in KTable

Hi John By all means, reply how you wish! I am interested in ensuring that everything has been addressed. Thanks > On Nov 8, 2018, at 4:57 PM, John Roesler wrote: > > Hey Adam, > > I'm sorry, I haven't been plugged in to your KIP lately. I'll have to read > it over again. Would it be

[jira] [Resolved] (KAFKA-6641) Consider auto repartitioning for Stream.transform() API

[ https://issues.apache.org/jira/browse/KAFKA-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-6641. Resolution: Duplicate > Consider auto repartitioning for Stream.transform() API >

Re: [VOTE] KIP-386: Standardize on Min/Avg/Max metrics' default values

Thanks for the KIP Stanislav. +1 (binding) On Thu, Nov 8, 2018 at 2:40 PM Jun Rao wrote: > Hi, Stanislav, > > Thanks for the KIP. +1. I guess this only covers the Kafka metrics (not the > Yammer metrics). It would be useful to make this clear. > > Jun > > On Tue, Nov 6, 2018 at 1:00 AM,

Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

Adding discussion thread here: https://lists.apache.org/thread.html/9868b07692960d7ab2924d2c66ad6ce7e42e492c35af9cbb7c9981ae@%3Cdev.kafka.apache.org%3E From: Boyang Chen Sent: Friday, November 9, 2018 6:35 AM To: dev@kafka.apache.org Subject: [VOTE] KIP-345:

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

Yes, I was about to send closing mail. On Thu, Nov 8, 2018 at 9:39 PM Ismael Juma wrote: > Manikumar, looks like you have the necessary votes. :) > > Ismael > > On Wed, Nov 7, 2018, 10:45 AM Rajini Sivaram wrote: > >> +1 (binding) >> >> Checked source build and unit tests. Ran quickstart with

Build failed in Jenkins: kafka-trunk-jdk8 #3188

See Changes: [junrao] KAFKA-7165: Retry the BrokerInfo registration into ZooKeeper (#5575) -- [...truncated 2.73 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Re: [DISCUSS] KIP-388 Add observer interface to record request and response

Hi Lincong Li, I agree that server-side instrumentation is helpful. However, I don't think this is the right approach. The problem is that RequestChannel.Request and AbstractResponse are internal classes that should not be exposed. These are implementation details that we may change in the

[jira] [Resolved] (KAFKA-7412) Bug prone response from producer.send(ProducerRecord, Callback) if Kafka broker is not running

[ https://issues.apache.org/jira/browse/KAFKA-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7412. Resolution: Fixed Fix Version/s: 2.2.0 Clarified the javadoc in producer callback. Metadata is not

Build failed in Jenkins: kafka-trunk-jdk11 #86

See Changes: [junrao] KAFKA-7412: clarify the doc for producer callback (#5798) -- [...truncated 2.33 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: Kafka Performance Producer - Extension

On Thu, Nov 8, 2018, at 01:38, Srinivas, Kaushik (Nokia - IN/Bangalore) wrote:> Hi All, > > This is in regard to extend few features support for java kafka > performance producer.> Hi Kaushik, Have you created a pull request against the Kafka repo at https://github.com/apache/kafka/ ?

Re: [VOTE] KIP-374: Add '--help' option to all available Kafka CLI commands

+1 (binding) On Wed, Oct 31, 2018, at 05:42, Srinivas Reddy wrote: > Hi All, > > I would like to call for a vote on KIP-374: > https://cwiki.apache.org/confluence/x/FgSQBQ > > Summary: > Currently, the '--help' option is recognized by some Kafka commands > but not all. To provide a consistent

Jenkins build is back to normal : kafka-trunk-jdk8 #3189

See

Jenkins build is back to normal : kafka-trunk-jdk11 #85

See

Re: [DISCUSS] KIP-388 Add observer interface to record request and response

I agree, the current KIP doesn't discuss the public API that we would be exposing and it's extensive if the normal usage would allow for casting AbstractRequest into the various subclasses and potentially even accessing Records and related for produce request. There are many use cases where this

[VOTE] KIP-351: Add --under-min-isr option to describe TopicCommand

Hi All, I'm starting the vote thread for KIP-351: Add --under-min-isr option to describe topics command. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-351%3A+Add+--under-min-isr+option+to+describe+topics+command Discussion thread:

[jira] [Created] (KAFKA-7608) A Kafka Streams DSL transform or processor call should trigger a repartition like a join

Andy Bryant created KAFKA-7608: -- Summary: A Kafka Streams DSL transform or processor call should trigger a repartition like a join Key: KAFKA-7608 URL: https://issues.apache.org/jira/browse/KAFKA-7608

Build failed in Jenkins: kafka-trunk-jdk11 #84

See Changes: [rajinisivaram] KAFKA-7604; Fix flaky unit test -- [...truncated 2.33 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

RE: Kafka Performance Producer - Extension

Hi Colin, I haven’t yet created the pull request. Wanted initial feedback for the same. -kaushik -Original Message- From: Colin McCabe Sent: Friday, November 09, 2018 7:27 AM To: dev@kafka.apache.org Subject: Re: Kafka Performance Producer - Extension On Thu, Nov 8, 2018, at 01:38,

RE: Kafka Performance Producer - Extension

Attaching the pdf file with summary of features implemented in our kafka producer application. Kindly review and provide the feedback. -kaushik -Original Message- From: Srinivas, Kaushik (Nokia - IN/Bangalore) Sent: Friday, November 09, 2018 11:00 AM To: dev@kafka.apache.org;

Jenkins build is back to normal : kafka-trunk-jdk8 #3187

See