[jira] [Created] (KAFKA-9434) Replace AlterReplicaLogDirs request/response with automated protocol

2020-01-15 Thread Tom Bentley (Jira)
Tom Bentley created KAFKA-9434: -- Summary: Replace AlterReplicaLogDirs request/response with automated protocol Key: KAFKA-9434 URL: https://issues.apache.org/jira/browse/KAFKA-9434 Project: Kafka

[jira] [Created] (KAFKA-9436) New Kafka Connect SMT for plainText => Struct(or Map)

2020-01-15 Thread whsoul (Jira)
whsoul created KAFKA-9436: - Summary: New Kafka Connect SMT for plainText => Struct(or Map) Key: KAFKA-9436 URL: https://issues.apache.org/jira/browse/KAFKA-9436 Project: Kafka Issue Type:

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread David Jacot
Congrats! On Wed, Jan 15, 2020 at 12:00 AM James Cheng wrote: > Congrats Colin, Vahid, and Manikumar! > > -James > > > On Jan 14, 2020, at 10:59 AM, Tom Bentley wrote: > > > > Congratulations! > > > > On Tue, Jan 14, 2020 at 6:57 PM Rajini Sivaram > > wrote: > > > >> Congratulations Colin,

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-15 Thread Mickael Maison
+1 (binding) Thanks for the KIP On Tue, Jan 14, 2020 at 6:50 PM David Arthur wrote: > > +1 binding > > This will be very nice to have. Thanks for the KIP, Colin. > > -David > > On Tue, Jan 14, 2020 at 11:39 AM Sönke Liebau > wrote: > > > +1 (non-binding) > > > > Thanks for creating this! > > >

Re: [VOTE] KIP-518: Allow listing consumer groups per state

2020-01-15 Thread Manikumar
Hi Mickael, Thanks for the KIP. Can you respond to the comments from David on discuss thread? Thanks,

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

2020-01-15 Thread Viktor Somogyi-Vass
Hey folks, bumping this again as KIP freeze is nearing and I hope to get this into the next release. We need only one binding vote. Thanks, Viktor On Thu, Jan 9, 2020 at 1:56 PM Viktor Somogyi-Vass wrote: > Bumping this in the hope of a vote or additional feedback. > > Viktor > > On Tue, Dec

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread Viktor Somogyi-Vass
Congrats to you guys, it's a great accomplishment! :) On Wed, Jan 15, 2020 at 10:20 AM David Jacot wrote: > Congrats! > > On Wed, Jan 15, 2020 at 12:00 AM James Cheng wrote: > > > Congrats Colin, Vahid, and Manikumar! > > > > -James > > > > > On Jan 14, 2020, at 10:59 AM, Tom Bentley wrote: >

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-15 Thread Mickael Maison
Thanks Colin +1 (binding) On Wed, Jan 15, 2020 at 9:27 AM Manikumar wrote: > > +1 (binding) > > Thanks for the KIP. > > On Wed, Jan 15, 2020 at 2:48 AM Gwen Shapira wrote: > > > +1 (binding, re-vote) > > > > On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe wrote: > > > > > > Hi all, > > > > > >

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-15 Thread Viktor Somogyi-Vass
+1 (non-binding) Viktor On Wed, Jan 15, 2020 at 10:27 AM Manikumar wrote: > +1 (binding) > > Thanks for the KIP. > > On Wed, Jan 15, 2020 at 2:48 AM Gwen Shapira wrote: > > > +1 (binding, re-vote) > > > > On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe > wrote: > > > > > > Hi all, > > > > > >

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-15 Thread Manikumar
+1 (binding) Thanks for the KIP. On Wed, Jan 15, 2020 at 2:48 AM Gwen Shapira wrote: > +1 (binding, re-vote) > > On Tue, Jan 14, 2020 at 11:23 AM Colin McCabe wrote: > > > > Hi all, > > > > I'm reposting this since I've been informed that gmail mashed the > original VOTE thread into a

[jira] [Created] (KAFKA-9435) Replace DescribeLogDirs request/response with automated protocol

2020-01-15 Thread Tom Bentley (Jira)
Tom Bentley created KAFKA-9435: -- Summary: Replace DescribeLogDirs request/response with automated protocol Key: KAFKA-9435 URL: https://issues.apache.org/jira/browse/KAFKA-9435 Project: Kafka

[jira] [Created] (KAFKA-9432) Replace DescribeConfigs request/response with automated protocol

2020-01-15 Thread Tom Bentley (Jira)
Tom Bentley created KAFKA-9432: -- Summary: Replace DescribeConfigs request/response with automated protocol Key: KAFKA-9432 URL: https://issues.apache.org/jira/browse/KAFKA-9432 Project: Kafka

[jira] [Created] (KAFKA-9433) Replace AlterConfigs request/response with automated protocol

2020-01-15 Thread Tom Bentley (Jira)
Tom Bentley created KAFKA-9433: -- Summary: Replace AlterConfigs request/response with automated protocol Key: KAFKA-9433 URL: https://issues.apache.org/jira/browse/KAFKA-9433 Project: Kafka

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Tom Bentley
Hi Konstantine, Thanks for the KIP, I can see how it could be useful. a) Did you consider using a metric for this? I don't think it would satisfy all the use cases you have in mind, but you could mention it in the rejected alternatives. b) If the topic name contains the string "-connector" then

[DISCUSS] KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies

2020-01-15 Thread David Jacot
Hi all, I just posted KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies: https://cwiki.apache.org/confluence/display/KAFKA/KIP-559%3A+Make+the+Kafka+Protocol+Friendlier+with+L7+Proxies Have a look and let me know what you think. Best, David

[jira] [Created] (KAFKA-9437) KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies

2020-01-15 Thread David Jacot (Jira)
David Jacot created KAFKA-9437: -- Summary: KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies Key: KAFKA-9437 URL: https://issues.apache.org/jira/browse/KAFKA-9437 Project: Kafka Issue

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread Ron Dagostino
Congratulations! > On Jan 15, 2020, at 5:04 AM, Viktor Somogyi-Vass > wrote: > > Congrats to you guys, it's a great accomplishment! :) > >> On Wed, Jan 15, 2020 at 10:20 AM David Jacot wrote: >> >> Congrats! >> >>> On Wed, Jan 15, 2020 at 12:00 AM James Cheng wrote: >>> >>> Congrats

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Manikumar
Hi Ron, Thanks for the KIP. KIP looks good to me. Am I correct in understanding that, when we run ZkSecurityMigrator with SASL + SSL, multiple identities will be added to the ACLs? Thanks, On Wed, Jan 15, 2020 at 1:19 AM Rajini Sivaram wrote: > Hi Ron, > > Thanks for the detailed

[jira] [Resolved] (KAFKA-9414) sink-task-metrics.sink-record-lag-max metric is not exposed

2020-01-15 Thread Aidar Makhmutov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aidar Makhmutov resolved KAFKA-9414. Resolution: Not A Bug It's not been implemented yet. >

Re: CompletableFuture?

2020-01-15 Thread Ismael Juma
Good question. I have a draft KIP for the producer change: https://cwiki.apache.org/confluence/display/KAFKA/KIP-XXX%3A+Return+CompletableFuture+from+KafkaProducer.send I was still debating which was the best path forward (i.e. what should be in rejected alternatives versus the actual proposal).

Re: CompletableFuture?

2020-01-15 Thread Vamsi Subahsh
Hi, I'm interested in picking this up as I have already worked on internal code bases to make a wrapper on current Api to make it expose CompletableFuture (using callbacks in the current api). Could you give me comment/edit access to the confluence doc, I can write up the new api and the logic

Re: [DISCUSS] KIP-550: Mechanism to Delete Stray Partitions on Broker

2020-01-15 Thread Dhruvil Shah
Hi Colin, We could add a configuration to disable stray partition deletion if needed, but I wasn't sure if an operator would really want to disable it. Perhaps if the implementation were buggy, the configuration could be used to disable the feature until a bug fix is made. Is that the kind of use

Re: [DISCUSS] KIP-489 Kafka Consumer Record Latency Metric

2020-01-15 Thread Habib Nahas
Hi Sean, Thats great, look forward to it. Thanks, Habib On Tue, Jan 14, 2020, at 2:55 PM, Sean Glover wrote: > Hi Habib, > > Thank you for the reminder. I'll update the KIP this week and address the > feedback from you and Gokul. > > Regards, > Sean > > On Tue, Jan 14, 2020 at 9:06 AM Habib

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Ron Dagostino
Thanks Colin and Rajini. I've updated the KIP to say that KIP-421 functionality is available to encrypt sensitive configs like the ZK key store and trust store passwords. (I've also made it clear that the configs are not dynamically reconfigurable since dynamic values are stored in ZK and the

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-15 Thread John Roesler
Hello again all, I had a bit of inspiration last night and realized that it's not necessary (and maybe even inappropriate) for StreamThreadStateStoreProvider and WrappingStoreProvider to implement the public StateStoreProvider interface. By breaking this dependency, I was able to implement the

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread Sean Glover
Congratulations Colin, Vahid and Manikumar and thank you for all your excellent work on Apache Kafka! On Wed, Jan 15, 2020 at 8:42 AM Ron Dagostino wrote: > Congratulations! > > > On Jan 15, 2020, at 5:04 AM, Viktor Somogyi-Vass < > viktorsomo...@gmail.com> wrote: > > > > Congrats to you guys,

Re: [VOTE] KIP-409: Allow creating under-replicated topics and partitions

2020-01-15 Thread Kamal Chandraprakash
+1 (non-binding). Thanks for the KIP! On Mon, Jan 13, 2020 at 1:58 PM M. Manna wrote: > Hi Mikael, > > Apologies for last minute question, as I just caught up with it. Thanks for > your work on the KIP. > > Just trying to get your thoughts on one thing (I might have misunderstood > it) -

[jira] [Created] (KAFKA-9438) Issue with mm2 active/active replication

2020-01-15 Thread Roman (Jira)
Roman created KAFKA-9438: Summary: Issue with mm2 active/active replication Key: KAFKA-9438 URL: https://issues.apache.org/jira/browse/KAFKA-9438 Project: Kafka Issue Type: Bug Components:

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread am
I agree WARN seems like it would be fine given we already log at that level for the handler. It also seems reasonable to exclude the ClassCastException as that can indicate something other then a simple serialization exception and would keep the current behavior. anna On Tue, Jan 14, 2020 at

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread Colin McCabe
Thanks, everyone! best, Colin On Wed, Jan 15, 2020, at 07:50, Sean Glover wrote: > Congratulations Colin, Vahid and Manikumar and thank you for all your > excellent work on Apache Kafka! > > On Wed, Jan 15, 2020 at 8:42 AM Ron Dagostino wrote: > > > Congratulations! > > > > > On Jan 15, 2020,

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Almog Gavra
Hi Konstantine, Thanks for the KIP! This is going to make automatic integration with Connect much more powerful. My thoughts are mostly around freshness of the data and being able to expose that to users. Riffing on Randall's timestamp question - have we considered adding some interval at which

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Colin McCabe
On Tue, Jan 14, 2020, at 11:49, Rajini Sivaram wrote: > Hi Ron, > > Thanks for the detailed explanation, sounds good to me. > > A few more questions: > > 1) At the moment, all sensitive broker configs including > keystore/truststore passwords can be stored encrypted in ZooKeeper prior to >

Re: [VOTE] KIP-551: Expose disk read and write metrics

2020-01-15 Thread Colin McCabe
Thanks, all. I will close the vote later today. best, Colin On Wed, Jan 15, 2020, at 01:48, Mickael Maison wrote: > +1 (binding) > Thanks for the KIP > > On Tue, Jan 14, 2020 at 6:50 PM David Arthur wrote: > > > > +1 binding > > > > This will be very nice to have. Thanks for the KIP, Colin.

Build failed in Jenkins: kafka-trunk-jdk8 #4161

2020-01-15 Thread Apache Jenkins Server
See Changes: [vvcephei] KAFKA-6144: Add KeyQueryMetadata APIs to KafkaStreams (#7960) -- [...truncated 2.82 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-01-15 Thread John Roesler
Hello all, After a long hiatus, I've just realized that I'm now able to upgrade my non-binding support to a binding +1 for KIP-441. This brings the vote tally to: 3 binding +1s: Guozhang, Bill, and myself 3 non-binding +1s: Bruno, Vinoth, and Sophie Since the vote has been open for at least 72

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Bill Bejeck
Thanks for the KIP. +1 (binding) -Bill On Wed, Jan 15, 2020 at 3:45 PM M. Manna wrote: > +1 (non-binding) > > Thanks for this KIP > > Regards, > > On Wed, 15 Jan 2020 at 20:35, Mitchell wrote: > > > +1(non-binding) > > > > Very useful > > -mitch > > > > On Wed, Jan 15, 2020, 3:29 PM Anna

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-15 Thread Bill Bejeck
Thanks for KIP Vito. Overall the KIP LGTM, but I'd have to agree with others on merging the `StreamsNotRunningException` and `StateStoreNotAvailableException` classes. Since in both cases, the thread state is in `PENDING_SHUTDOWN || NOT_RUNNING || ERROR` I'm not even sure how we could

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-15 Thread John Roesler
Hey Vito, Yes, thanks for the KIP. Sorry the discussion has been so long. Hopefully, we can close it out soon. I agree we can drop StreamsNotRunningException in favor of just StateStoreNotAvailableException. Unfortunately, I have some higher-level concerns. The value of these exceptions is

[jira] [Created] (KAFKA-9440) Add ConsumerGroupCommand to delete static members

2020-01-15 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-9440: -- Summary: Add ConsumerGroupCommand to delete static members Key: KAFKA-9440 URL: https://issues.apache.org/jira/browse/KAFKA-9440 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-3596) Kafka Streams: Window expiration needs to consider more than event time

2020-01-15 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-3596. Resolution: Not A Problem > Kafka Streams: Window expiration needs to consider

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Konstantine Karantasis
Hi Randall, Tom and Almog. I'm excited to read your comments. I'll reply in separate emails, in order. First, to Randall's comments, I'm replying below with a reference to the comment number: 1. Although I can imagine we'd be interested in adding additional metadata in the record value, I didn't

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread M. Manna
+1 (non-binding) Thanks for this KIP Regards, On Wed, 15 Jan 2020 at 20:35, Mitchell wrote: > +1(non-binding) > > Very useful > -mitch > > On Wed, Jan 15, 2020, 3:29 PM Anna McDonald > wrote: > > > Greetings, > > I would like to propose a vote on KIP-399, extending the > >

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Matthias J. Sax
Thanks for pushing this KIP over the finish line! +1 (binding) -Matthias On 1/15/20 12:57 PM, Bill Bejeck wrote: > Thanks for the KIP. > > +1 (binding) > > -Bill > > On Wed, Jan 15, 2020 at 3:45 PM M. Manna wrote: > >> +1 (non-binding) >> >> Thanks for this KIP >> >> Regards, >> >> On

Re: [DISCUSS] KIP-550: Mechanism to Delete Stray Partitions on Broker

2020-01-15 Thread Colin McCabe
On Wed, Jan 15, 2020, at 03:54, Dhruvil Shah wrote: > Hi Colin, > > We could add a configuration to disable stray partition deletion if needed, > but I wasn't sure if an operator would really want to disable it. Perhaps > if the implementation were buggy, the configuration could be used to >

[jira] [Created] (KAFKA-9439) Add more public API tests for KafkaProducer

2020-01-15 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-9439: -- Summary: Add more public API tests for KafkaProducer Key: KAFKA-9439 URL: https://issues.apache.org/jira/browse/KAFKA-9439 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Bill Bejeck
Great all of that sounds good to me. Since we are all in agreement, we can move to a vote. -Bill On Wed, Jan 15, 2020 at 11:51 AM am wrote: > I agree WARN seems like it would be fine given we already log at that > level for the handler. It also seems reasonable to exclude the >

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-15 Thread Guozhang Wang
Thanks for the update of the PR John! I have taken a look at 7962 and it looks good to me overall. Guozhang On Wed, Jan 15, 2020 at 9:35 AM John Roesler wrote: > Hello again all, > > I had a bit of inspiration last night and realized that it's not necessary > (and maybe even inappropriate)

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Colin McCabe
On Wed, Jan 15, 2020, at 10:53, Ron Dagostino wrote: > Thanks Colin and Rajini. > > I've updated the KIP to say that KIP-421 functionality is available to > encrypt sensitive configs like the ZK key store and trust store > passwords. (I've also made it clear that the configs are not >

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Colin McCabe
On Wed, Jan 15, 2020, at 13:41, Ron Dagostino wrote: > Hi Colin. Two things come to mind with respect to ZooKeeper camelCase > style vs Kafka-style config names for ZooKeeper. First, I think it > would be desirable for the client configs and broker configs to be > interoperable. For example, it

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Randall Hauch
Almog, You raise some interesting questions. Comments inline below. On Wed, Jan 15, 2020 at 11:19 AM Almog Gavra wrote: > Hi Konstantine, > > Thanks for the KIP! This is going to make automatic integration with > Connect much more powerful. > > My thoughts are mostly around freshness of the

Build failed in Jenkins: kafka-trunk-jdk11 #1083

2020-01-15 Thread Apache Jenkins Server
See Changes: [vvcephei] KAFKA-6144: Add KeyQueryMetadata APIs to KafkaStreams (#7960) -- [...truncated 5.75 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

[VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Anna McDonald
Greetings, I would like to propose a vote on KIP-399, extending the ProductionExceptionHandler to cover serialization exceptions. This KIP is aimed at improving the error-handling semantics in Kafka Streams when Kafka Streams fails to serialize a message to the downstream sink. KIP details

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Mitchell
+1(non-binding) Very useful -mitch On Wed, Jan 15, 2020, 3:29 PM Anna McDonald wrote: > Greetings, > I would like to propose a vote on KIP-399, extending the > ProductionExceptionHandler to cover serialization exceptions. This KIP > is aimed at improving the error-handling semantics in Kafka

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Konstantine Karantasis
Hi Tom, here are my replies to your comments: a) Interesting point. It's worth indeed adding it in the "Rejected Alternatives" sections. I did not consider it as an implementation option. Connect has already a REST API that, as you note, seems a more natural choice, especially since its already

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Ron Dagostino
Hi Colin. Two things come to mind with respect to ZooKeeper camelCase style vs Kafka-style config names for ZooKeeper. First, I think it would be desirable for the client configs and broker configs to be interoperable. For example, it feels like it would be convenient to be able to pass the

Re: [ANNOUNCE] New Kafka PMC Members: Colin, Vahid and Manikumar

2020-01-15 Thread Vahid Hashemian
Thank you all. Regards, --Vahid On Wed, Jan 15, 2020 at 9:15 AM Colin McCabe wrote: > Thanks, everyone! > > best, > Colin > > On Wed, Jan 15, 2020, at 07:50, Sean Glover wrote: > > Congratulations Colin, Vahid and Manikumar and thank you for all your > > excellent work on Apache Kafka! > > > >

Re: [DISCUSS] KIP-515: Enable ZK client to use the new TLS supported authentication

2020-01-15 Thread Ron Dagostino
Hi Manikumar. Yes, you are correct: if the ZK Security Migrator session authenticates to ZooKeeper with multiple identities — SASL and certificate — then ACLs are applied authorizing both the SASL principal and the certificate DN. Ron > On Jan 15, 2020, at 6:33 AM, Manikumar wrote: > > Hi

Re: [VOTE] On the new KIP-158: Kafka Connect allows source connectors to set topic settings when creating new topics

2020-01-15 Thread Gwen Shapira
+1 (binding) Looks super useful. Thank you. On Mon, Jan 13, 2020 at 8:16 AM Konstantine Karantasis wrote: > > Hi everyone. > > I hope y'all had a nice break. The discussion on KIP-158 seems to have > wrapped up since last year, so I'd like to open the vote on this KIP. > > A reminder that

Re: [VOTE] KIP-555: An admin tools proposal to accomplish the deprecation of zookeeper access that is direct

2020-01-15 Thread M. Manna
+1 (Binding) - a long awaited KIP to have a simpler partition reassignment script (without ZK)> Kudos to you Colin :) On Wed, 15 Jan 2020 at 10:10, Viktor Somogyi-Vass wrote: > +1 (non-binding) > > Viktor > > On Wed, Jan 15, 2020 at 10:27 AM Manikumar > wrote: > > > +1 (binding) > > > >

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Randall Hauch
On Wed, Jan 15, 2020 at 4:36 PM Randall Hauch wrote: > On Wed, Jan 15, 2020 at 2:05 PM Konstantine Karantasis < > konstant...@confluent.io> wrote: > >> >> 9. I assumed that partitioning is implied by default, because there's no >> requirement for complete ordering of topic status records. But

Re: Streams, Kafka windows

2020-01-15 Thread John Roesler
Hi Viktor, I’m not sure why you get two identical outputs in response to a single record. Regardless, since you say that you want to get a single, final result for the window and you expect multiple inputs to the windows, you need Suppression. My guess is that you just sent one record to try

Re: [VOTE] KIP-535: Allow state stores to serve stale reads during rebalance

2020-01-15 Thread Navinder Brar
Thanks, Vinoth and John for making the last minute improvements. I have gone through the PR and looks good to me. On Thursday, 16 January, 2020, 12:42:09 am IST, Guozhang Wang wrote: Thanks for the update of the PR John! I have taken a look at 7962 and it looks good to me overall.

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-15 Thread John Roesler
Hi Vito, Haha, your archive game is on point! What Matthias said in that email is essentially what I figured was the rationale. It makes sense, but the point I was making is that this really doesn’t seem like a good way to structure a production app. On the other hand, considering the

Role of CompositeReadOnlyKeyValueStore for point queries

2020-01-15 Thread Navinder Brar
Hi all, Can someone explain to me the thoughts behind having CompositeReadOnlyKeyValueStore. java class while serving data via APIs in Kafka Streams. It fetches the list of stores for all the running tasks on the machine and then looks for a key one by one in each of the stores. When we already

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Anna McDonald
Done. Thanks for pointing that out. anna On Wed, Jan 15, 2020 at 8:52 PM Guozhang Wang wrote: > > Hi Anna, > > Just a minor comment on the wiki page itself: > > ``` > > The new method, handleSerializationException, in ProductionExceptionHandler > will be invoked when > >1.

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-15 Thread Vito Jeng
Hi John, About `StreamsNotStartedException is strange` -- The original idea came from Matthias, two years ago. :) You can reference here: https://mail-archives.apache.org/mod_mbox/kafka-dev/201806.mbox/%3c6c32083e-b63c-435b-521d-032d45cc5...@confluent.io%3e About omitting the categorization --

[jira] [Resolved] (KAFKA-9410) Make groupId Optional in KafkaConsumer

2020-01-15 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9410. Resolution: Fixed > Make groupId Optional in KafkaConsumer >

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

2020-01-15 Thread Vito Jeng
Hi, folks, Thank you suggestion, really appreciate it. :) I understand your concern. I'll merge StreamsNotRunningException and StateStoreNotAvailableException. --- Vito On Thu, Jan 16, 2020 at 6:22 AM John Roesler wrote: > Hey Vito, > > Yes, thanks for the KIP. Sorry the discussion has been

[jira] [Created] (KAFKA-9441) Refactor commit logic

2020-01-15 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-9441: -- Summary: Refactor commit logic Key: KAFKA-9441 URL: https://issues.apache.org/jira/browse/KAFKA-9441 Project: Kafka Issue Type: Sub-task

Build failed in Jenkins: kafka-trunk-jdk11 #1084

2020-01-15 Thread Apache Jenkins Server
See Changes: [vvcephei] KAFKA-6144: IQ option to query standbys (#7962) [vahid.hashemian] MINOR: Removed accidental double negation in error message. (#7834) --

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Randall Hauch
My responses are inline: On Wed, Jan 15, 2020 at 2:05 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Hi Randall, Tom and Almog. I'm excited to read your comments. I'll reply in > separate emails, in order. > > First, to Randall's comments, I'm replying below with a reference to

Re: [DISCUSS] KIP-558: Track the set of actively used topics by connectors in Kafka Connect

2020-01-15 Thread Konstantine Karantasis
Hey Almog, thanks for the comments! Here's my take: 1) I think that an approximate grouping of topics to highly-active/active/inactive (because a precise one would be too expensive) seems like something we could leave out of this first version of topic tracking. Interestingly, as you point out,

[jira] [Created] (KAFKA-9442) Kafka connect REST API times out when trying to create a connector

2020-01-15 Thread Kedar Shenoy (Jira)
Kedar Shenoy created KAFKA-9442: --- Summary: Kafka connect REST API times out when trying to create a connector Key: KAFKA-9442 URL: https://issues.apache.org/jira/browse/KAFKA-9442 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk8 #4162

2020-01-15 Thread Apache Jenkins Server
See Changes: [vvcephei] KAFKA-6144: IQ option to query standbys (#7962) [vahid.hashemian] MINOR: Removed accidental double negation in error message. (#7834) --

Re: [VOTE] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

2020-01-15 Thread Guozhang Wang
Hi Anna, Just a minor comment on the wiki page itself: ``` The new method, handleSerializationException, in ProductionExceptionHandler will be invoked when 1. ClassCastException is thrown while serializing record key / value. We will continue to throw this exception and not invoke the