Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-05 Thread Cyrus Vafadari
This looks like a useful feature, the strategy makes sense, and the KIP is thorough and nicely written. Thanks! Cyrus On Thu, Aug 1, 2019, 12:40 PM Chris Egerton wrote: > Thanks Arjun! Looks good to me. > > On Thu, Aug 1, 2019 at 12:33 PM Arjun Satish > wrote: > > > Thanks for the feedback, Ch

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-05 Thread Ismael Juma
Harsha, Rogue clients can use the admin client to create topics and partitions. ACLs and policies can help in that case as well as this one. Ismael On Mon, Aug 5, 2019, 7:48 PM Harsha Chintalapani wrote: > Hi Justine, > Thanks for the KIP. > "When server-side auto-creation is disa

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-05 Thread Harsha Chintalapani
Hi Justine, Thanks for the KIP. "When server-side auto-creation is disabled, client-side auto-creation will try to use client-side configurations" If I understand correctly, this KIP is removing any server-side blocking client auto creation of topic? if so this will present potential i

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

2019-08-05 Thread Boyang Chen
Yep, Guozhang I think that would be best as passing in an entire consumer instance is indeed cumbersome. Just saw you updated KIP-429, I will follow-up to change 447 as well. Best, Boyang On Mon, Aug 5, 2019 at 3:18 PM Guozhang Wang wrote: > okay I think I understand your concerns about Consum

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2019-08-05 Thread Justine Olshan
Hi all, I made some changes to the KIP. Hopefully this configuration change will make things a little clearer. https://cwiki.apache.org/confluence/display/KAFKA/KIP-487%3A+Client-side+Automatic+Topic+Creation+on+Producer Please let me know if you have any feedback or questions! Thank you, Justin

[jira] [Created] (KAFKA-8757) Flaky Test SaslSslAdminClientIntegrationTest#testIncrementalAlterConfigsForLog4jLogLevelsCanResetLoggerToCurrentRoot

2019-08-05 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-8757: -- Summary: Flaky Test SaslSslAdminClientIntegrationTest#testIncrementalAlterConfigsForLog4jLogLevelsCanResetLoggerToCurrentRoot Key: KAFKA-8757 URL: https://issues.apache.org/ji

[jira] [Created] (KAFKA-8756) Flaky Test SaslSslAdminClientIntegrationTest#testIncrementalAlterConfigsForLog4jLogLevels

2019-08-05 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-8756: -- Summary: Flaky Test SaslSslAdminClientIntegrationTest#testIncrementalAlterConfigsForLog4jLogLevels Key: KAFKA-8756 URL: https://issues.apache.org/jira/browse/KAFKA-8756

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

2019-08-05 Thread Guozhang Wang
okay I think I understand your concerns about ConsumerGroupMetadata now: if we still want to only call initTxns once, then we should allow the whatever passed-in parameter to reflect the latest value of generation id whenever sending the offset fetch request. Whereas the current ConsumerGroupMetad

Re: [VOTE] KIP-440: Extend Connect Converter to support headers

2019-08-05 Thread Ryanne Dolan
+1, non-binding Ryanne On Mon, Aug 5, 2019 at 3:38 PM Randall Hauch wrote: > If my math is right, we have 3 non-binding +1 votes and 2 binding +1 votes. > > This is a simple but really beneficial KIP for Connect. Can we get another > review and vote by a committer? Thanks! > > Randall > > On Fr

[jira] [Created] (KAFKA-8755) Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store

2019-08-05 Thread Bruno Cadonna (JIRA)
Bruno Cadonna created KAFKA-8755: Summary: Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store Key: KAFKA-8755 URL: https://issues.apache.org/jira/browse/KAFKA-8755 P

Re: [VOTE] KIP-429: Kafka Consumer Incremental Rebalance Protocol

2019-08-05 Thread Guozhang Wang
Hello folks, I've also updated the wiki page by making the augmented `ConsumerPartitionAssignor` out as a public API in `o.a.k.clients.consumer` and deprecate the old `PartitionAssignor` as in `o.a.k.clients.consumer.internals`. Guozhang On Fri, Jun 28, 2019 at 11:30 AM Sophie Blee-Goldman wro

Re: [VOTE] KIP-440: Extend Connect Converter to support headers

2019-08-05 Thread Randall Hauch
If my math is right, we have 3 non-binding +1 votes and 2 binding +1 votes. This is a simple but really beneficial KIP for Connect. Can we get another review and vote by a committer? Thanks! Randall On Fri, May 31, 2019 at 3:37 PM sapie...@gmail.com wrote: > Hey everyone, just bumping this thr

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-05 Thread Colin McCabe
I think it would be useful to have this in AdminClient. Especially if we implement KIP-496: Administrative API to delete consumer offsets. It would be odd to have a way to delete consumer offsets in AdminClient, but not to create them. What do you think? best, Colin On Sun, Aug 4, 2019, at

Re: [DISCUSS] KIP-503: deleted topics metric

2019-08-05 Thread Harsha Chintalapani
Thanks for the KIP. Its useful metric to have. LGTM. -Harsha On Mon, Aug 05, 2019 at 11:24 AM, David Arthur wrote: > Hello all, I'd like to start a discussion for > https://cwiki.apache.org/confluence/display/KAFKA/ > KIP-503%3A+Add+metric+for+number+of+topics+marked+for+deletion > > Thanks!

[jira] [Created] (KAFKA-8754) Connect API: Expose the list of available transformations

2019-08-05 Thread JIRA
Stéphane Derosiaux created KAFKA-8754: - Summary: Connect API: Expose the list of available transformations Key: KAFKA-8754 URL: https://issues.apache.org/jira/browse/KAFKA-8754 Project: Kafka

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-05 Thread Colin McCabe
On Mon, Aug 5, 2019, at 10:02, Tom Bentley wrote: > Hi Colin, > > Thanks for the KIP. > > Currently ZooKeeper provides a convenient notification mechanism for > knowing that broker and topic configuration has changed. While KIP-500 does > suggest that incremental metadata update is expected to co

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-05 Thread Colin McCabe
On Mon, Aug 5, 2019, at 07:49, Viktor Somogyi-Vass wrote: > Hey Colin, > > I think this is a long-awaited KIP, thanks for driving it. I'm excited to > see this in Kafka once. I collected my questions (and I accept the "TBD" > answer as they might be a bit deep for this high level :) ). > 1.) Are t

[DISCUSS] KIP-503: deleted topics metric

2019-08-05 Thread David Arthur
Hello all, I'd like to start a discussion for https://cwiki.apache.org/confluence/display/KAFKA/KIP-503%3A+Add+metric+for+number+of+topics+marked+for+deletion Thanks! David

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-05 Thread Tom Bentley
Hi Colin, Thanks for the KIP. Currently ZooKeeper provides a convenient notification mechanism for knowing that broker and topic configuration has changed. While KIP-500 does suggest that incremental metadata update is expected to come to clients eventually, that would seem to imply that for some

Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

2019-08-05 Thread Boyang Chen
Thank you Guozhang for the reply! I'm curious whether KIP-429 has reflected the latest change on ConsumerGroupMetadata? Also regarding question one, the group metadata needs to be accessed via callback, does that mean we need a separate producer API such like "producer.refreshMetadata(groupMetadata

[jira] [Created] (KAFKA-8753) Add JMX for number of topics marked for deletion

2019-08-05 Thread David Arthur (JIRA)
David Arthur created KAFKA-8753: --- Summary: Add JMX for number of topics marked for deletion Key: KAFKA-8753 URL: https://issues.apache.org/jira/browse/KAFKA-8753 Project: Kafka Issue Type: Impr

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-05 Thread Viktor Somogyi-Vass
Hey Colin, I think this is a long-awaited KIP, thanks for driving it. I'm excited to see this in Kafka once. I collected my questions (and I accept the "TBD" answer as they might be a bit deep for this high level :) ). 1.) Are there any specific reasons for the Controller just periodically persist

Re: [VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

2019-08-05 Thread Andrew Schofield
Hi, I'd like to have a final try at getting some voting on this KIP. I'd like to get it into Kafka 2.4 so get your votes in please. Currently only +2 non-binding votes. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-419%3A+Safely+notify+Kafka+Connect+SourceTask+is+stopped PR: ht

[jira] [Created] (KAFKA-8752) Ensure plugin classes are instantiable when discovering plugins

2019-08-05 Thread Alexandre Dupriez (JIRA)
Alexandre Dupriez created KAFKA-8752: Summary: Ensure plugin classes are instantiable when discovering plugins Key: KAFKA-8752 URL: https://issues.apache.org/jira/browse/KAFKA-8752 Project: Kafka

[jira] [Created] (KAFKA-8751) Trogdor - Add endpoint that deletes all tasks

2019-08-05 Thread Stanislav Kozlovski (JIRA)
Stanislav Kozlovski created KAFKA-8751: -- Summary: Trogdor - Add endpoint that deletes all tasks Key: KAFKA-8751 URL: https://issues.apache.org/jira/browse/KAFKA-8751 Project: Kafka Issue

[jira] [Created] (KAFKA-8750) Utils#murmur2() method switch case block don't have break

2019-08-05 Thread yangkun (JIRA)
yangkun created KAFKA-8750: -- Summary: Utils#murmur2() method switch case block don't have break Key: KAFKA-8750 URL: https://issues.apache.org/jira/browse/KAFKA-8750 Project: Kafka Issue Type: Bug