[jira] [Created] (KAFKA-9888) REST extensions can mutate connector configs in worker config state snapshot

2020-04-17 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-9888: Summary: REST extensions can mutate connector configs in worker config state snapshot Key: KAFKA-9888 URL: https://issues.apache.org/jira/browse/KAFKA-9888 Project:

Build failed in Jenkins: kafka-trunk-jdk8 #4445

2020-04-17 Thread Apache Jenkins Server
See Changes: [github] KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence [github] MINOR: reduce impact of trace logging in replica hot path (#8468) [github] KAFKA-9818: Fix flaky test in

Build failed in Jenkins: kafka-trunk-jdk11 #1364

2020-04-17 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9881: Convert integration test to verify measurements from RocksDB [github] KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence [github] MINOR: reduce impact

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

2020-04-17 Thread Ivan Ponomarev
Hi everyone! Let me revive the discussion of this KIP. I'm very sorry for stopping my participation in the discussion in June 2019. My project work was very intensive then and it didn't leave me spare time. But I think I must finish this, because we invested substantial effort into this

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Guozhang Wang
Sounds good to me. Thanks Boyang. On Fri, Apr 17, 2020 at 3:32 PM Boyang Chen wrote: > Thanks Guozhang, > > I think most of the complexity comes from our intention to benefit older > clients. After a second thought, I think the add-on complexity counteracts > the gain here as only 2.5 client is

Build failed in Jenkins: kafka-trunk-jdk8 #4444

2020-04-17 Thread Apache Jenkins Server
See Changes: [github] MINOR: Switch order of sections on tumbling and hopping windows in [github] MINOR: improve test coverage for dynamic LogConfig(s) (#7616) [github] KAFKA-9881: Convert integration test to

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Boyang Chen
Thanks Guozhang, I think most of the complexity comes from our intention to benefit older clients. After a second thought, I think the add-on complexity counteracts the gain here as only 2.5 client is getting a slice of the resilience improvement, not for many older versions. So I decide to drop

Jenkins build is back to normal : kafka-trunk-jdk11 #1363

2020-04-17 Thread Apache Jenkins Server
See

Re: [DISCUSS] KIP-587 Suppress detailed responses for handled exceptions in security-sensitive environments

2020-04-17 Thread Christopher Egerton
Hi Connor, That's great, but I think you may have mistaken Colin for me :) One more thing that should be addressed--the "public interfaces" section isn't just for Java interfaces, it's for any changes to any public part of Kafka that users and external developers interact with. As far as Connect

[jira] [Resolved] (KAFKA-9818) Flaky Test RecordCollectorTest.shouldNotThrowStreamsExceptionOnSubsequentCallIfASendFailsWithContinueExceptionHandler

2020-04-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-9818. Fix Version/s: 2.6.0 Resolution: Fixed > Flaky Test >

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-17 Thread Boyang Chen
Thanks Colin for the suggestions! I have added metrics to track the number of total messages being forwarded as a more generic monitoring. Right now the list of metrics are: - num-client-forwarding-to-controller-rate - num-client-fowarding-to-controller-count -

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-17 Thread Colin McCabe
On Thu, Apr 16, 2020, at 12:33, Jose Garcia Sancio wrote: > Hi Boyang, > > Thanks for the KIP. The KIP looks good. I have a few questions and comments. > > > As part of the KIP-500 >

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-17 Thread Ismael Juma
Hi Colin, The read/modify/write is protected by the zk version, right? Ismael On Fri, Apr 17, 2020 at 12:53 PM Colin McCabe wrote: > On Thu, Apr 16, 2020, at 08:51, Ismael Juma wrote: > > I don't think these requests are necessarily infrequent under multi > tenant > > environments though.

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-17 Thread Colin McCabe
On Thu, Apr 16, 2020, at 08:51, Ismael Juma wrote: > I don't think these requests are necessarily infrequent under multi tenant > environments though. I've seen Controller availability being an issue for > describe topics for example (before it was changed to go to any broker). Hi Ismael, I

Re: [DISCUSS] KIP-590: Redirect Zookeeper Mutation Protocols to The Controller

2020-04-17 Thread Colin McCabe
Hi David & Boyang, I thought the intention of the "old-client-connections-count" metric was to give some information about how many redirections were going on in the cluster. This is different from the "unknown" software version metric. After all, many versions that support KIP-511 will

Kafka Meetup hosted by Confluent Online, Tuesday 4:00pm, April 21st, 2020

2020-04-17 Thread Guozhang Wang
Hello folks, The Bay Area Kafka meetup will continue to be hosted online this month, next Tuesday (April 21st) 4:00pm. We will be presenting the current on going work for a Kafkaesque Raft Protocol:

Re: [VOTE] KIP-577: Allow HTTP Response Headers Configured for Kafka Connect

2020-04-17 Thread Aneel Nazareth
Thanks Jeff, this seems like it addresses a user need. +1 (non-binding) On Fri, Apr 17, 2020 at 1:28 PM Zhiguo Huang wrote: > > Thanks to everyone for their input. I've incorporated the changes, and I > think this is ready for voting. > > To summarize, the KIP simply proposes to add a feature

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Guozhang Wang
Hi Boyang, Your reply to 3) seems conflicting with your other answers which is a bit confusing to me. Following your other answers, it seems you suggest returning UNKNOWN_PRODUCER_ID so that 2.5 clients can trigger retry logic as well? To complete my reasoning here as a complete picture: a)

Re: [VOTE] KIP-584: Versioning scheme for features

2020-04-17 Thread Guozhang Wang
Thanks for the great KIP Kowshik, +1 (binding). On Fri, Apr 17, 2020 at 11:22 AM Jun Rao wrote: > Hi, Kowshik, > > Thanks for the KIP. +1 > > Jun > > On Thu, Apr 16, 2020 at 11:14 AM Kowshik Prakasam > wrote: > > > Hi all, > > > > I'd like to start a vote for KIP-584. The link to the KIP can

[VOTE] KIP-577: Allow HTTP Response Headers Configured for Kafka Connect

2020-04-17 Thread Zhiguo Huang
Thanks to everyone for their input. I've incorporated the changes, and I think this is ready for voting. To summarize, the KIP simply proposes to add a feature which allows HTTP response headers configured for Kafka Connect.The KIP can be found here:

Re: [VOTE] KIP-584: Versioning scheme for features

2020-04-17 Thread Jun Rao
Hi, Kowshik, Thanks for the KIP. +1 Jun On Thu, Apr 16, 2020 at 11:14 AM Kowshik Prakasam wrote: > Hi all, > > I'd like to start a vote for KIP-584. The link to the KIP can be found > here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-584%3A+Versioning+scheme+for+features > . > >

[jira] [Created] (KAFKA-9887) failed-task-count JMX metric not updated if task fails during startup

2020-04-17 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-9887: Summary: failed-task-count JMX metric not updated if task fails during startup Key: KAFKA-9887 URL: https://issues.apache.org/jira/browse/KAFKA-9887 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk11 #1362

2020-04-17 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9819: Fix flaky test in StoreChangelogReaderTest (#8488) [github] MINOR: Switch order of sections on tumbling and hopping windows in --

[jira] [Created] (KAFKA-9886) Validate segment range before reading in `Log.read`

2020-04-17 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-9886: -- Summary: Validate segment range before reading in `Log.read` Key: KAFKA-9886 URL: https://issues.apache.org/jira/browse/KAFKA-9886 Project: Kafka Issue

[jira] [Resolved] (KAFKA-9819) Flaky Test StoreChangelogReaderTest#shouldNotThrowOnUnknownRevokedPartition[0]

2020-04-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-9819. Fix Version/s: 2.6.0 Resolution: Fixed > Flaky Test

[jira] [Created] (KAFKA-9885) Evict last members of a group when the maximum allowed is reached

2020-04-17 Thread David Jacot (Jira)
David Jacot created KAFKA-9885: -- Summary: Evict last members of a group when the maximum allowed is reached Key: KAFKA-9885 URL: https://issues.apache.org/jira/browse/KAFKA-9885 Project: Kafka

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-04-17 Thread Alexandre Dupriez
Hello all, Apologies for the excavation. +1 (non-binding). One of the advantage I see having the metrics exposed by Kafka, as opposed to a system monitoring tool, is that it can be refined to provide a segregated view on the consumption of disk resources, for instance factored by topic or

[jira] [Created] (KAFKA-9884) Unable to override some client properties in Mirror maker 2.0

2020-04-17 Thread Mithun Kumar (Jira)
Mithun Kumar created KAFKA-9884: --- Summary: Unable to override some client properties in Mirror maker 2.0 Key: KAFKA-9884 URL: https://issues.apache.org/jira/browse/KAFKA-9884 Project: Kafka