Build failed in Jenkins: kafka-trunk-jdk11 #813

See Changes: [wangguoz] KAFKA-8755: Fix state restore for standby tasks with optimized topology [jason] KAFKA-8474; Use HTML lists for config layout (#6870) [jason] KAFKA-8730; Add API to delete consumer

Build failed in Jenkins: kafka-trunk-jdk8 #3903

See Changes: [cmccabe] MINOR: Add UUID type to Kafka API code generation (#7291) -- [...truncated 8.35 MB...]

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

Hi, In many cases you have already serde for the stream definition. So I see natural to use it also for the tests. So you can write: public void setup() { testDriver = new TopologyTestDriver(TestStream.getTopology(), TestStream.getConfig()); inputTopic =

Build failed in Jenkins: kafka-2.3-jdk8 #106

See -- [...truncated 1.48 MB...] kafka.zk.KafkaZkClientTest > testSetGetAndDeletePartitionReassignment STARTED kafka.zk.KafkaZkClientTest > testSetGetAndDeletePartitionReassignment PASSED

Re: [DISCUSS] 2.2.2 Bug Fix Release

Thanks, Matthias. I'll get things ready locally but won't cut a release candidate until everyone is ready. On Fri, Sep 13, 2019 at 4:13 PM Matthias J. Sax wrote: > Thanks Randall! > > Overall SGTM, however, we need to resolve the open question about the > 2.2.1 regression before we can release

[VOTE] 2.3.1 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 2.3.1 which includes many bug fixes for Apache Kafka 2.3. Release notes for the 2.3.1 release: https://home.apache.org/~davidarthur/kafka-2.3.1-rc0/RELEASE_NOTES.html *** Please

[jira] [Resolved] (KAFKA-8474) Improve configuration layout on website

[ https://issues.apache.org/jira/browse/KAFKA-8474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8474. Resolution: Fixed > Improve configuration layout on website >

[jira] [Resolved] (KAFKA-8755) Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store

[ https://issues.apache.org/jira/browse/KAFKA-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8755. -- Resolution: Fixed > Stand-by Task of an Optimized Source Table Does Not Write Anything to its

Build failed in Jenkins: kafka-2.1-jdk8 #229

See Changes: [rhauch] HOTFIX: fix compile error in 2.1 branch (#7328) -- [...truncated 316.00 KB...]

Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Late here, but I am a big +1. Great to see this finally happening. -Jason On Fri, Sep 13, 2019 at 11:43 AM Colin McCabe wrote: > Hi all, > > With 3 binding +1 votes from Gwen Shapira, Ismael Juma, and Bill Bejeck > and 5 non-binding +1 votes from Tom Bentley, Ron Dagostino, David Jacot, >

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Radai & Jason, Thanks for the support and suggestion. 1. I think ratio is a good additional metric since the current proposed metrics are only absolute times which may not be useful in all scenarios. I have added this to the KIP: *- poll-idle-ratio*: The fraction of time the consumer

[DISCUSS] KIP-525 - Return topic metadata and configs in CreateTopics response

Hi all, I would like to start discussion on KIP-525 to return topic configs in CreateTopics response: - https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response When validateOnly=false, this will be the actual configs of the

Re: [DISCUSS] 2.2.2 Bug Fix Release

Thanks Randall! Overall SGTM, however, we need to resolve the open question about the 2.2.1 regression before we can release 2.2.2. I sent an email about this (subject `[DISCUSS] Streams-Broker compatibility regression in 2.2.1 release`) couple of days ago. -Matthias On 9/12/19 4:03 PM,

[jira] [Created] (KAFKA-8907) Return topic configs in CreateTopics response

Rajini Sivaram created KAFKA-8907: - Summary: Return topic configs in CreateTopics response Key: KAFKA-8907 URL: https://issues.apache.org/jira/browse/KAFKA-8907 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-2.3-jdk8 #105

See -- [...truncated 1.59 MB...] kafka.zk.KafkaZkClientTest > testSetGetAndDeletePartitionReassignment STARTED kafka.zk.KafkaZkClientTest > testSetGetAndDeletePartitionReassignment PASSED

Build failed in Jenkins: kafka-2.0-jdk8 #293

See Changes: [rhauch] HOTFIX: fix compile error in 2.0 branch (#7329) -- [...truncated 438.91 KB...] kafka.zk.ReassignPartitionsZNodeTest > testDecodeValidJson PASSED

Re: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

Hi Clement/Rajini I've gone through the code to understand how reconfigruation work currently. I sent both of you a note separately also. Let us reconvene next week and proceed. Thanks Maulin On Thu, Sep 12, 2019 at 12:25 PM Pellerin, Clement wrote: > You are proposing a different design

Re: Kafka SSH Tunnel Connection without editing hostfile

That is not the requirement. We want the communication via ssh tunnel. On Fri, Sep 13, 2019 at 4:50 PM M. Manna wrote: > why not try using internal vs external traffic > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic > > > if you set

Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Hi all, With 3 binding +1 votes from Gwen Shapira, Ismael Juma, and Bill Bejeck and 5 non-binding +1 votes from Tom Bentley, Ron Dagostino, David Jacot, Dhruvil Shah, Mickael Maison, the vote passes. Thanks, everyone! best, Colin On Fri, Sep 13, 2019, at 01:14, Mickael Maison wrote: > +1

Re: [VOTE] KIP-520: Augment Consumer.committed(partition) to allow multiple partitions

Thanks everyone for your votes. +1 from myself (binding). I'm closing this voting thread now with the following tally: binding +1: 4 (Bill, Matthias, Jason, Guozhang) non-binding +1: 6 (Boyang, Mickael, Kamal, Bruno, Manna, Tom) Thanks, Guozhang On Thu, Sep 12, 2019 at 11:02 AM Jason

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi Kevin, This looks reasonable to me. I'd also +1 Radai's suggestion if you're willing. Something like an idle ratio for the consumer would be helpful. Thanks, Jason On Fri, Sep 13, 2019 at 10:08 AM radai wrote: > while youre at it another metric that we have found to be useful is % > time

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hi All, While working on the implementation of KIP-479, some issues came to light that the KIP as written won't work. I have updated the KIP with a solution I believe will solve the original problem as well as address the impediment to the initial approach. This update is a significant change,

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

while youre at it another metric that we have found to be useful is % time spent in user code vs time spent in poll() (so time between poll calls / time inside poll calls) - the higher the % value the more indicative of user code being the cause of performance bottlenecks. On Fri, Sep 13, 2019 at

Re: [DISCUSS] KIP-514: Add a bounded flush() API to Kafka Producer

we have a lot of processes that need a time-bounded checkpoint logic. the standard use case is some consume-process-produce logic of the following form: while (alive) { data = consumer.poll() output = process(data) producer.send(output) if (System.millis() > nextCheckpoint) {

Re: [VOTE] KIP-496: Administrative API to delete consumer offsets

Hi David, Sounds good. best, Colin On Fri, Sep 13, 2019, at 04:45, David Jacot wrote: > Hi all, > > I would like to do another modification to the proposal. In the proposal, > the OffsetDeleteResponse > doesn't have a top level error field so I would like to add one. Many > errors concern the

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi Lucas, Thanks for tackling this. Topic IDs are a great idea, and this is a really good writeup. For /brokers/topics/[topic], the schema version should be bumped to version 3, rather than 2. KIP-455 bumped the version of this znode to 2 already :) Given that we're going to be seeing these

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

Maybe one follow up question: Why do we pass `Serdes` into `createInputTopic` and createOutputTopic` -- seems `Serializer` (for input) and `Deserialized` (for output) should be sufficient? -Matthias On 9/12/19 4:59 PM, Matthias J. Sax wrote: > Thanks for updating the KIP. > > I agree with

Re: [DISCUSS] KIP-517: Add consumer metric indicating time between poll calls

Hi All, Happy Friday! Bumping this. Any thoughts? Thanks. Regards, Kevin On Thu, Sep 5, 2019 at 9:35 AM Kevin Lu wrote: > Hi All, > > I'd like to propose a new consumer metric that measures the time between > calls to poll() for use in issues related to hitting max.poll.interval.ms > due to

Re: PR Broken builds

Hi Daniyar, I run `./gradlew :clients:spotbugsMain` with your PR locally and the following spotBugs warning was reported. org.apache.kafka.common.serialization.ListDeserializer.deserialize(String, byte[]) ignores result of java.io.DataInputStream.read(byte[]) This seems related. Best, Bruno

[DISCUSS] KIP-524: Allow users to choose config source when describing configs

Hi All, I have a minor KIP to improve the config tool: https://cwiki.apache.org/confluence/display/KAFKA/KIP-524%3A+Allow+users+to+choose+config+source+when+describing+configs. Let me know what you think. -Jason

[Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

Hello, Starting this thread to discuss KIP-532: wiki link : https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658 There has been some discussion around the use-case of this KIP in the

Re: PR Broken builds

Hi Daniyar, The jenkins logs were not accessible anymore. I restarted your builds. Let's see what the issues are. Best, Bruno On Thu, Sep 12, 2019 at 7:42 PM Development wrote: > > Hi, > > I’ve been working on KAFKA-8326 PR (https://github.com/apache/kafka/pull/6592 >

Re: [VOTE] KIP-496: Administrative API to delete consumer offsets

Hi all, I would like to do another modification to the proposal. In the proposal, the OffsetDeleteResponse doesn't have a top level error field so I would like to add one. Many errors concern the whole group (e.g. GROUP_ID_NOT_FOUND) so it would be great to have a way to communicate them back to

Re: Kafka SSH Tunnel Connection without editing hostfile

why not try using internal vs external traffic https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic if you set EXTERNAL enndpoints and map it to SSL - you clients should only receive EXTERNAL endpoints for comms. Does this sound okay for you?

[jira] [Created] (KAFKA-8906) Metrics for Key-Value, Window, and Session Stores are Inconsistent

Bruno Cadonna created KAFKA-8906: Summary: Metrics for Key-Value, Window, and Session Stores are Inconsistent Key: KAFKA-8906 URL: https://issues.apache.org/jira/browse/KAFKA-8906 Project: Kafka

Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

+1 non-binding Thanks Colin On Fri, Sep 13, 2019 at 5:21 AM Dhruvil Shah wrote: > > This is exciting! +1 (non-binding) > > - Dhruvil > > On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck wrote: > > > Thanks for the KIP! > > > > +1 (binding) > > > > -Bill > > > > On Thu, Sep 12, 2019 at 3:26 PM

[jira] [Created] (KAFKA-8905) Stream DSL: tasks should take serdes from upstream tasks

Eduard Wirch created KAFKA-8905: --- Summary: Stream DSL: tasks should take serdes from upstream tasks Key: KAFKA-8905 URL: https://issues.apache.org/jira/browse/KAFKA-8905 Project: Kafka Issue