Build failed in Jenkins: kafka-trunk-jdk8 #3817

See Changes: [cmccabe] KAFKA-8345: KIP-455 Protocol changes (part 1) (#7114) -- [...truncated 6.47 MB...] org.apache.kafka.connect.data.SchemaProjectorTest >

[jira] [Created] (KAFKA-8731) InMemorySessionStore throws NullPointerException on startup

Jonathan Gordon created KAFKA-8731: -- Summary: InMemorySessionStore throws NullPointerException on startup Key: KAFKA-8731 URL: https://issues.apache.org/jira/browse/KAFKA-8731 Project: Kafka

Re: [VOTE] KIP-492 Add java security providers in Kafka Security config

+1 (non-binding) Thanks, Satish. On Tue, Jul 30, 2019 at 5:18 AM Harsha Chintalapani wrote: > > Thanks for the KIP Sandeep. > > +1 (binding) > > Thanks, > Harsha > On Jul 29, 2019, 12:22 PM -0700, Sandeep Mopuri , wrote: > > Hi all, after some good discussion > >

[jira] [Created] (KAFKA-8730) Add API to delete consumer offsets (KIP-496)

Jason Gustafson created KAFKA-8730: -- Summary: Add API to delete consumer offsets (KIP-496) Key: KAFKA-8730 URL: https://issues.apache.org/jira/browse/KAFKA-8730 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-497: Add inter-broker API to alter ISR

Hi Jun, Thanks for the comments. 10. My intent was to rely only on the LeaderAndIsr request. It seemed simpler if there is just one way that ISR state is updated on leaders (e.g. the request log is all you need to see the changes), but I'd be interested if you can think of a reason to prefer

Re: [VOTE] KIP-492 Add java security providers in Kafka Security config

Thanks for the KIP Sandeep. +1 (binding) Thanks, Harsha On Jul 29, 2019, 12:22 PM -0700, Sandeep Mopuri , wrote: > Hi all, after some good discussion > about the > KIP >

Re: [DISCUSS] KIP-497: Add inter-broker API to alter ISR

Hi, Jason, Thanks for the KIP. Looks good overall. A couple of comments below. 10. It's not very clear to me when the partition state (i.e., leader, isr, leaderEpoch and zkVersion) is updated on the leader. My initial understanding is that the leader updates the partition state on receiving a

Re: [DISCUSS] KIP-496: Administrative API to delete consumer offsets

Sounds good. Just pointing out that the metrics scope likely overlap with https://issues.apache.org/jira/browse/KAFKA-3556, so may better document that in the wiki page. Guozhang On Mon, Jul 29, 2019 at 2:48 PM Jason Gustafson wrote: > Hi Guozhang, > > I have added metrics to the KIP. Please

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

What is the status of this KIP? -Matthias On 7/11/19 4:00 PM, Matthias J. Sax wrote: > Ivan, > > did you see my last reply? What do you think about my proposal to mix > both approaches and try to get best-of-both worlds? > > > -Matthias > > On 6/11/19 3:56 PM, Matthias J. Sax wrote: >>

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Any update on this KIP Vito? On 7/11/19 4:26 PM, Matthias J. Sax wrote: > Thanks Vito! I think the KIP shapes out nicely! > > > To answer the open question you raised (I also adjust my answers based > on the latest KIP update) > > > > About `StreamThreadNotStartedException`: I understand

Re: [VOTE] KIP-479 Add Materialized to Join

+1 (binding) On Thu, Jul 25, 2019 at 7:39 PM Matthias J. Sax wrote: > +1 (binding) > > On 7/25/19 1:05 PM, Bill Bejeck wrote: > > All, > > > > After a great discussion on KIP-479 ( > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-479%3A+Add+Materialized+to+Join > ) > > I'd > > like

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

+1 (binding) On 7/29/19 11:59 AM, Bill Bejeck wrote: > Thanks for the KIP. > > +1 (binding) > > -Bill > > On Wed, Jul 24, 2019 at 12:12 PM Guozhang Wang wrote: > >> Yeah I think I agree with you. >> >> +1 (binding) from me. >> >> >> Guozhang >> >> >> On Wed, Jul 24, 2019 at 7:43 AM John

Re: [DISCUSS] KIP-478 Strongly Typed Processor API

Thanks for the details! Also talked to Guozhang about a potential upgrade path. This KIP seems not to put us into an bad position to provide a clean upgrade path if we change the `ProcessorContext` in the future. Thus, I think we can move forward. -Matthias On 7/24/19 3:32 PM, John Roesler

Re: [DISCUSS] KIP-496: Administrative API to delete consumer offsets

Hi Guozhang, I have added metrics to the KIP. Please take a look. This gave me an excuse to also add a metric for the group rebalance rate, which probably would have made detecting KAFKA-8653 easier. Since this is a relatively straightforward KIP, I will go ahead and start a vote later this week

Re: [DISCUSS] KIP-497: Add inter-broker API to alter ISR

2. Sounds good. Thanks! Guozhang On Mon, Jul 29, 2019 at 11:43 AM Jason Gustafson wrote: > @Colin > > Yeah, that's a good question. If the version is higher, I think it's not > safe for the leader to use the new version until it has received the full > LeaderAndIsr state. For example, there

[VOTE] KIP-492 Add java security providers in Kafka Security config

Hi all, after some good discussion about the KIP , I'm starting the voting. This KIP proposes adding new security

Kafka connect task assignment Improvement ( New Feature )

Hi , This is Manjeet here working in operative media . I have been working on confluent kafka for almost 4 years and have made many customized changes for kafka connect sink and source connectors . I have made changes in kafka code base as well for our requirement. There is one feature I have

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

Thanks for the KIP. +1 (binding) -Bill On Wed, Jul 24, 2019 at 12:12 PM Guozhang Wang wrote: > Yeah I think I agree with you. > > +1 (binding) from me. > > > Guozhang > > > On Wed, Jul 24, 2019 at 7:43 AM John Roesler wrote: > > > Hi Guozhang, > > > > Thanks! I just replied in the discuss

Re: [DISCUSS] KIP-497: Add inter-broker API to alter ISR

@Colin Yeah, that's a good question. If the version is higher, I think it's not safe for the leader to use the new version until it has received the full LeaderAndIsr state. For example, there may be a reassignment in progress which could alter the leader's expected state change. At least knowing

[jira] [Resolved] (KAFKA-8727) Control over standby tasks host assignment

[ https://issues.apache.org/jira/browse/KAFKA-8727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Levani Kokhreidze resolved KAFKA-8727. -- Resolution: Duplicate  Duplicate of KAFKA-6718 > Control over standby tasks host

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

Currently the way it is implemented, the broker auto-creation configuration takes precedence. The producer will not use the CreateTopics request. (Technically it can--but the topic will already be created through the broker, so it will never try to create the topic.) It is possible to change this

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

Hi Justine, Thanks for the KIP! In case auto creation is enabled on both the client and server, will the producer still use the AdminClient (CreateTopics request) to create topics? and not rely on the broker auto create. I'm guessing the answer is yes but can you make it explicit. Thank you On

[jira] [Created] (KAFKA-8729) Augment ProduceResponse error messaging for specific culprit records

Guozhang Wang created KAFKA-8729: Summary: Augment ProduceResponse error messaging for specific culprit records Key: KAFKA-8729 URL: https://issues.apache.org/jira/browse/KAFKA-8729 Project: Kafka

Re: [VOTE] KIP-467: Augment ProduceResponse error messaging

+1 from myself as well (binding). I'm closing this vote thread with the following votes: binding +1s: 4 (Guozhang, Jun, Jason, Bill). Thanks everyone who've reviewed and voted on the KIP! Guozhang On Mon, Jul 29, 2019 at 9:30 AM Guozhang Wang wrote: > Yeah my thinking is that changing the

Re: [VOTE] KIP-467: Augment ProduceResponse error messaging

Yeah my thinking is that changing the return error code away from CORRUPTED_RECORD is really a bug fix, so we should just do it anyways without considering compatibility. I like returning INVALID_REQUEST too, would change it in the wiki. Guozhang On Fri, Jul 26, 2019 at 4:40 PM Jason Gustafson

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

I'm mostly happy with your current suggestion (two configs, one for serialization and one for deserialization) and your implementation suggestion. One thing to note: > We should _always_ be able to deserialize a standard JSON > number as a decimal I was doing some research into decimals and

Re: [DISCUSS] KIP-497: Add inter-broker API to alter ISR

Hi Jason, Thanks for the KIP. It looks good to me overall. 1. Just clarifying the "CurrentVersion" field in the newly proposed protocol. Does it contains the same value as zkVersion that controller get from ZK? 2. As for the comment in the KIP: "We can either await the update or we can send a

Re: [DISCUSS] KIP-496: Administrative API to delete consumer offsets

Thanks for the replies Jason! 2. No I do not see any problems, just trying to understand how restrict we are applying this rule :) Piggy-backing on the existing background thread and check interval mechanism means we are not "eagerly" expiring either, but I think this is fine. Guozhang On

[jira] [Created] (KAFKA-8728) Flaky Test KTableSourceTopicRestartIntegrationTest #shouldRestoreAndProgressWhenTopicWrittenToDuringRestorationWithEosDisabled

Matthias J. Sax created KAFKA-8728: -- Summary: Flaky Test KTableSourceTopicRestartIntegrationTest #shouldRestoreAndProgressWhenTopicWrittenToDuringRestorationWithEosDisabled Key: KAFKA-8728 URL:

[jira] [Created] (KAFKA-8727) Control over standby tasks host assignment

Levani Kokhreidze created KAFKA-8727: Summary: Control over standby tasks host assignment Key: KAFKA-8727 URL: https://issues.apache.org/jira/browse/KAFKA-8727 Project: Kafka Issue Type:

Kafka Group Coordinator Failover

Hello There, I am using Kafka and Zookeeper in a cluster setup, wherein having two nodes of Kafka and three nodes of Zookeeper with my clustered Spring Boot Application. The issue I am facing is when I stop both the Kafka nodes and starts one of them (which is not a group coordinator), the

Re: [VOTE] KIP-467: Augment ProduceResponse error messaging

Hi Guozang, Thanks for the KIP, looks like a great improvement. +1 for me. -Bill On Fri, Jul 26, 2019 at 8:18 PM Jun Rao wrote: > Hi, Guozhang, > > Thanks for the KIP. +1 > > Jun > > On Fri, Jul 26, 2019 at 4:40 PM Jason Gustafson > wrote: > > > Hi Guozhang, > > > > I agree it is misleading

Re: [DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

Hey Jose, Thanks for sharing your use cases. >From my experience, it is uncommon to run with a retention.ms setting small enough that it can make you lose messages when your consumers can't catch up. If you are concerned with data loss, I think the cost investment into hardware is generally worth

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

The way I see it, we need to control two seperate things: 1. How do we _deserialize_ a decimal type if we encounter a text node in the JSON?(We should _always_ be able to deserialize a standard JSON number as a decimal). 2. How do we chose how we want decimals to be _serialized_. This looks

[jira] [Created] (KAFKA-8726) Producer can't abort a transaction aftersome send errors

Mattia Barbon created KAFKA-8726: Summary: Producer can't abort a transaction aftersome send errors Key: KAFKA-8726 URL: https://issues.apache.org/jira/browse/KAFKA-8726 Project: Kafka Issue

[jira] [Created] (KAFKA-8725) Improve LogCleaner error handling when failing to grab the filthiest log

Stanislav Kozlovski created KAFKA-8725: -- Summary: Improve LogCleaner error handling when failing to grab the filthiest log Key: KAFKA-8725 URL: https://issues.apache.org/jira/browse/KAFKA-8725

Re: Request to become a contributor

Hi, Thanks for your interest. Added you to the contributor list. Thanks, On Mon, Jul 29, 2019 at 11:35 AM Amrutha Shanbhag < amrutha.shanb...@instaclustr.com> wrote: > Hi team, > > Request you to add me to the contributor list. > JIRA username: amrutha.shanbhag > > Thanks, > > > *Amrutha

Request to become a contributor

Hi team, Request you to add me to the contributor list. JIRA username: amrutha.shanbhag Thanks, *Amrutha Shanbhag**Senior Software Engineer* Read our latest technical