[jira] [Created] (KAFKA-8822) Remove CompletedFetch type from Fetcher

2019-08-20 Thread Sean Glover (Jira)
Sean Glover created KAFKA-8822: -- Summary: Remove CompletedFetch type from Fetcher Key: KAFKA-8822 URL: https://issues.apache.org/jira/browse/KAFKA-8822 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-360: Improve handling of unknown producer

2019-08-20 Thread Matthias J. Sax
Thanks for the KIP. I just have some clarification questions to make sure I understand the proposal correctly: 1) "Safe Epoch Incrementing" > When the coordinator receives a new InitProducerId request, we will use the > following logic to update the epoch: > > 1. No epoch is provided: the

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Harsha Chintalapani
Maulin, The code parts you are pointing are specific for Spiffe and if you are talking about validate method which uses PKIX check like any other provider does. If you want to default to SunJSSE everywhere you can do so by delegating the calls in these methods to SunJSSE provider.

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Maulin Vasavada
Okay, so I take that you guys agree that I have to write a 'custom' algorithm and a provider to make it work , correct? Now, for Harsha's comment "Here the 'Custom' Algorithm is not an implementation per say , ..." , I diagree. You can refer to

Re: [VOTE] KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting

2019-08-20 Thread Bill Bejeck
Thanks for the KIP. +1 (binding) On Tue, Aug 20, 2019 at 6:28 PM Matthias J. Sax wrote: > +1 (binding) > > > On 6/17/19 2:32 PM, John Roesler wrote: > > I'm +1 (nonbinding) on the current iteration of the proposal. > > > > On Mon, May 27, 2019 at 1:58 PM Paul Whalen wrote: > >> > >> I spoke

Re: [VOTE] KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting

2019-08-20 Thread Matthias J. Sax
+1 (binding) On 6/17/19 2:32 PM, John Roesler wrote: > I'm +1 (nonbinding) on the current iteration of the proposal. > > On Mon, May 27, 2019 at 1:58 PM Paul Whalen wrote: >> >> I spoke too early a month ago, but I believe the proposal is finalized now >> and ready for voting. >> >> KIP: >>

Re: [DISCUSS] KIP-401 TransformerSupplier/ProcessorSupplier enhancements

2019-08-20 Thread Matthias J. Sax
Thanks Paul! On 8/15/19 7:28 PM, Paul Whalen wrote: > I updated the KIP (and PR) to relax the restriction on connecting state > stores via either means; it definitely makes sense to me at this point. > I'd love to hear if there are any other concerns or broad objections to the > KIP. > > Paul >

Re: [DISCUSS] KIP-441: Smooth Scaling Out for Kafka Streams

2019-08-20 Thread John Roesler
In response to Bruno's concern #2, I've also added that section to the "Rejected Alternatives" section. Additionally, after reviewing some other assignment papers, I've developed the concern that specifying which "phases" the assignment algorithm should have, or indeed the logic of it at all,

Re: [DISCUSS] KIP-441: Smooth Scaling Out for Kafka Streams

2019-08-20 Thread John Roesler
Hi All, Thanks for the discussion. I've been considering the idea of giving the "catching up" tasks a different name/role. I was in favor initially, but after working though some details, I think it causes some problems, which I've written up in the "rejected alternatives" part of the KIP:

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-20 Thread Konstantine Karantasis
Thanks for the updates on the KIP Arjun! +1 by me (non-binding) Konstantine On Wed, Aug 14, 2019 at 6:46 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Thanks for the KIP Arjun. > > FYI, I left a few comments on the discussion thread, but mentioning here > too since I

[jira] [Reopened] (KAFKA-8812) Rebalance Producers - yes, I mean it ;-)

2019-08-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-8812: No need to close the ticket, as long as the KIP was not declined :) > Rebalance Producers -

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

2019-08-20 Thread Jason Gustafson
Thanks Jun. Yes, that is probably the simplest way to go. -Jason On Tue, Aug 20, 2019 at 2:14 PM Jun Rao wrote: > Hi, Jason, > > Thanks for the KIP. Sorry for the late reply. +1 from me too. > > Just a comment on the implementation. Perhaps we should only allow one > pending AlterIsrRequest

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

2019-08-20 Thread Jun Rao
Hi, Jason, Thanks for the KIP. Sorry for the late reply. +1 from me too. Just a comment on the implementation. Perhaps we should only allow one pending AlterIsrRequest from a broker to the controller (at least for a given partition) in order to set CurrentZkVersion properly? Jun On Tue, Aug

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Sandeep Mopuri
Hi Maulin, thanks for the discussion. As Harsha pointed out, to use the KIP492, you need to create a new provider, register a *new* custom algorithm for your keymanager and trustmanager factory implementations. After this, the kafka server configuration can be done as given below # Register the

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

2019-08-20 Thread Mitchell
Closing the Vote. 4 Binding, 9 non-binding. No against. On Tue, Aug 20, 2019 at 1:35 PM Jason Gustafson wrote: > > +1 Thanks for the KIP! > > -Jason > > On Thu, Aug 15, 2019 at 4:01 AM Jakub Scholz wrote: > > > +1 (non-binding) > > > > Jakub > > > > On Sat, Aug 10, 2019 at 8:34 PM Stanislav

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

2019-08-20 Thread Jason Gustafson
+1 Thanks for the KIP! -Jason On Thu, Aug 15, 2019 at 4:01 AM Jakub Scholz wrote: > +1 (non-binding) > > Jakub > > On Sat, Aug 10, 2019 at 8:34 PM Stanislav Kozlovski < > stanis...@confluent.io> > wrote: > > > Awesome KIP, +1 (non-binding) > > > > Thanks, > > Stanislav > > > > On Fri, Aug 9,

[VOTE] KIP-444: Refactor and Augment Metrics for Kafka Streams

2019-08-20 Thread Guozhang Wang
Hello folks, I'd like to start a voting thread the following KIP to improve the Kafka Streams metrics mechanism to users. This includes 1) renaming changes in the public StreamsMetrics utils API, and 2) a major cleanup on the Streams' own built-in metrics hierarchy. Details can be found here:

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

2019-08-20 Thread Guozhang Wang
Hello Bruno, I've updated the wiki page again per your comments, here's a brief summary: 1. added the list of removed metrics. 2. added a task-level INFO metric "dropped-records" that covers all scenarios and merges in the existing "late-records-drop", "skipped-records", and

[jira] [Created] (KAFKA-8821) Avoid pattern subscription to allow for stricter ACL settings

2019-08-20 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-8821: -- Summary: Avoid pattern subscription to allow for stricter ACL settings Key: KAFKA-8821 URL: https://issues.apache.org/jira/browse/KAFKA-8821 Project: Kafka

[VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-20 Thread Jason Gustafson
Hi All, I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to fix a long-known shortcoming of URP reporting and to improve reassignment monitoring: https://cwiki.apache.org/confluence/display/KAFKA/KIP-352%3A+Distinguish+URPs+caused+by+reassignment . Note that I have added one

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

2019-08-20 Thread Jason Gustafson
I'm going to close this vote. The final result is +5 with 3 binding votes. -Jason On Wed, Aug 14, 2019 at 10:05 AM Satish Duggana wrote: > +1 (non-binding), Thanks Jason for nice improvements on ISR > propagation protocol! > > On Wed, Aug 14, 2019 at 8:29 PM David Arthur wrote: > > > > +1

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

2019-08-20 Thread Guozhang Wang
Hi Bruno, No it was not intentional, and we can definitely add the total amount sensor as well -- they are just util functions to save users some lines of code anyways, and should be straightforward. Guozhang On Tue, Aug 20, 2019 at 1:05 AM Bruno Cadonna wrote: > Hi Guozhang, > > I totally

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

2019-08-20 Thread Harsha Chintalapani
Answers are in-line On Mon, Aug 19, 2019 at 10:45 PM, Maulin Vasavada wrote: > Hi Colin > > > When I refer to "standard" or "custom" algorithms I am following Java > security Provider Terminology. You can refer to > https://docs.oracle.com/javase/7/docs/technotes/guides/security/ >

Re: Do not have permission to create KIP

2019-08-20 Thread Bill Bejeck
You're all set now. -Bill On Tue, Aug 20, 2019 at 11:28 AM Михаил Ерёменко wrote: > Ah, I see. > > Here is it: yeryomenkom > > вт, 20 авг. 2019 г. в 17:11, Bill Bejeck : > > > Sorry I typed too fast before. > > > > You'll need to create the account and share your username here. > > > > Thanks,

Re: Do not have permission to create KIP

2019-08-20 Thread Михаил Ерёменко
Ah, I see. Here is it: yeryomenkom вт, 20 авг. 2019 г. в 17:11, Bill Bejeck : > Sorry I typed too fast before. > > You'll need to create the account and share your username here. > > Thanks, > Bill > > On Tue, Aug 20, 2019 at 10:26 AM Михаил Ерёменко > wrote: > > > Hi! > > > > So, you should

Re: Permission to create KIP

2019-08-20 Thread David Jacot
Thank you, Bill! Le mar. 20 août 2019 à 16:45, Bill Bejeck a écrit : > Hi David, > > You're all set now. > > -Bill > > On Tue, Aug 20, 2019 at 10:04 AM David Jacot wrote: > > > Hi Bill, > > > > Sure. It is dajac. > > > > Thanks, > > David > > > > On Tue, Aug 20, 2019 at 3:55 PM Bill Bejeck

Re: Do not have permission to create KIP

2019-08-20 Thread Bill Bejeck
Sorry I typed too fast before. You'll need to create the account and share your username here. Thanks, Bill On Tue, Aug 20, 2019 at 10:26 AM Михаил Ерёменко wrote: > Hi! > > So, you should create that account for me? > > вт, 20 авг. 2019 г. в 16:02, Bill Bejeck : > > > Hi Mykhailo, > > > > We

Re: Permission to create KIP

2019-08-20 Thread Bill Bejeck
Hi David, You're all set now. -Bill On Tue, Aug 20, 2019 at 10:04 AM David Jacot wrote: > Hi Bill, > > Sure. It is dajac. > > Thanks, > David > > On Tue, Aug 20, 2019 at 3:55 PM Bill Bejeck wrote: > > > Hi David, > > > > Sure thing, can you share your wiki username? > > > > Thanks, > > Bill

Re: Do not have permission to create KIP

2019-08-20 Thread Михаил Ерёменко
Hi! So, you should create that account for me? вт, 20 авг. 2019 г. в 16:02, Bill Bejeck : > Hi Mykhailo, > > We just need to create a wiki account and share the username here. > > Thanks, > Bill > > On Tue, Aug 20, 2019 at 5:06 AM Михаил Ерёменко > wrote: > > > Hi! > > > > I would like to

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-20 Thread Ron Dagostino
Hi Colin. The diagram up at the top confused me -- specifically, the lines connecting the controller/active-controller to the brokers. I had assumed the arrows on those lines represented the direction of data flow, but that is not the case; the arrows actually identify the target of the action,

Re: Permission to create KIP

2019-08-20 Thread David Jacot
Hi Bill, Sure. It is dajac. Thanks, David On Tue, Aug 20, 2019 at 3:55 PM Bill Bejeck wrote: > Hi David, > > Sure thing, can you share your wiki username? > > Thanks, > Bill > > On Tue, Aug 20, 2019 at 9:53 AM David Jacot wrote: > > > Hey, > > > > I would like to create a KIP but I don't

Re: Do not have permission to create KIP

2019-08-20 Thread Bill Bejeck
Hi Mykhailo, We just need to create a wiki account and share the username here. Thanks, Bill On Tue, Aug 20, 2019 at 5:06 AM Михаил Ерёменко wrote: > Hi! > > I would like to create KIP, but it looks like I do not have permission to > do that. > Can you help me with this? > > Regards, >

Re: Permission to create KIP

2019-08-20 Thread Bill Bejeck
Hi David, Sure thing, can you share your wiki username? Thanks, Bill On Tue, Aug 20, 2019 at 9:53 AM David Jacot wrote: > Hey, > > I would like to create a KIP but I don't have the permission yet. Could > someone grant me the relevant permission to the wiki? > > Thanks, > David >

Permission to create KIP

2019-08-20 Thread David Jacot
Hey, I would like to create a KIP but I don't have the permission yet. Could someone grant me the relevant permission to the wiki? Thanks, David

Build failed in Jenkins: kafka-trunk-jdk8 #3862

2019-08-20 Thread Apache Jenkins Server
See Changes: [me] MINOR: Upgrade ducktape to 0.7.6 -- [...truncated 8.44 MB...] kafka.api.PlaintextConsumerTest > testPatternSubscription PASSED

Do not have permission to create KIP

2019-08-20 Thread Михаил Ерёменко
Hi! I would like to create KIP, but it looks like I do not have permission to do that. Can you help me with this? Regards, Mykhailo

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

2019-08-20 Thread Bruno Cadonna
Hi Guozhang, I totally missed the total invocation count metric in the javadoc. Which brings me to a follow-up question. Should the names of the methods reflect the included total invocation count? We have to rename them anyways. One option would be to simply add `Total` to the method names,

Re: [DISCUSS] KIP-509: Rebalance and restart Producers

2019-08-20 Thread W. D.
The KIP can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-509%3A+Rebalance+and+restart+Producers On Mon, Aug 19, 2019 at 10:00 PM W. D. wrote: > At the moment producers do not get any support when restarting: > where should it start producing data from? > what data had

Re: [DISCUSS] KIP-509: Rebalance and restart Producers

2019-08-20 Thread Werner Daehn
https://cwiki.apache.org/confluence/display/KAFKA/KIP-509%3A+Rebalance+and+restart+Producers On 2019/08/19 20:00:02, "W. D." wrote: > At the moment producers do not get any support when restarting:> > where should it start producing data from?> > what data had been committed successfully in