[jira] [Created] (KAFKA-9123) Add system test with large number of partitions

David Arthur created KAFKA-9123: --- Summary: Add system test with large number of partitions Key: KAFKA-9123 URL: https://issues.apache.org/jira/browse/KAFKA-9123 Project: Kafka Issue Type: Test

[jira] [Created] (KAFKA-9125) GroupMetadataManager and TransactionStateManager should use metadata cache instead of zkClient

Viktor Somogyi-Vass created KAFKA-9125: -- Summary: GroupMetadataManager and TransactionStateManager should use metadata cache instead of zkClient Key: KAFKA-9125 URL:

[jira] [Reopened] (KAFKA-9124) Topic creation should be done by the controller

[ https://issues.apache.org/jira/browse/KAFKA-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass reopened KAFKA-9124: > Topic creation should be done by the controller >

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

Hi Brian, I was just asking it because of curiosity although I do think it would be useful but it should be done on all tools to keep them consistent. Perhaps it could be a future consideration. I'm +1 on the KIP (non-binding). Viktor On Wed, Oct 30, 2019 at 5:50 PM Brian Byrne wrote: > Hi

[jira] [Resolved] (KAFKA-9124) Topic creation should be done by the controller

[ https://issues.apache.org/jira/browse/KAFKA-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-9124. Resolution: Invalid Opened by mistake. > Topic creation should be done by the

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Wondering if anyone has thoughts on these changes? I liked that the new metadata fetch APIs provide all the information at once with consistent naming.. Any guidance on what you would like to be discussed or fleshed out more before we call a VOTE? On Wed, Oct 30, 2019 at 10:31 AM Navinder Brar

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hello Walker / Matthias, Thanks for your explanation. I can tell you've put a lot of thoughts into this already and it seems we cannot avoid adding new interfaces in any ways, so I will rest my arguments trying to reduce the number of first-class citizens in the Streams DSL :) Guozhang On

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Hey Vinoth, I started going over the KIP again yesterday. There are a lot of updates, and I didn't finish my feedback in one day. I'm working on it now. Thanks, John On Thu, Oct 31, 2019 at 11:42 AM Vinoth Chandar wrote: > > Wondering if anyone has thoughts on these changes? I liked that the

[jira] [Created] (KAFKA-9126) Extend `StreamJoined` to allow more store configs

Matthias J. Sax created KAFKA-9126: -- Summary: Extend `StreamJoined` to allow more store configs Key: KAFKA-9126 URL: https://issues.apache.org/jira/browse/KAFKA-9126 Project: Kafka Issue

[VOTE] KIP-150 - Kafka-Streams Cogroup

Hello all, I'd like to call a vote on the updated KIP-150: Kafka-Streams Cogroup found here Thanks, Walker

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

I just sent out a call for a vote. I think everyone has had a good discussion :). If there are any more thoughts I would love to hear them. Walker On Thu, Oct 31, 2019 at 10:36 AM Guozhang Wang wrote: > Hello Walker / Matthias, > > Thanks for your explanation. I can tell you've put a lot of

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Hey Navinder, Thanks for updating the KIP, it's a lot easier to see the current state of the proposal now. A few remarks: 1. I'm sure it was just an artifact of revisions, but you have two separate sections where you list additions to the KafkaStreams interface. Can you consolidate those so we

[jira] [Created] (KAFKA-9124) Topic creation should be done by the controller

Viktor Somogyi-Vass created KAFKA-9124: -- Summary: Topic creation should be done by the controller Key: KAFKA-9124 URL: https://issues.apache.org/jira/browse/KAFKA-9124 Project: Kafka

[jira] [Resolved] (KAFKA-9114) Kafka broker fails to establish secure zookeeper connection via SSL.

[ https://issues.apache.org/jira/browse/KAFKA-9114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gangadhar Balikai resolved KAFKA-9114. -- Resolution: Invalid > Kafka broker fails to establish secure zookeeper connection via

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Thanks John for going through this. - +1, makes sense - +1, no issues there - Yeah the initial patch I had submitted for K-7149(https://github.com/apache/kafka/pull/6935) to reduce assignmentInfo object had taskIds but the merged PR had similar size according to Vinoth and it was

Deferred processing

Hi We have a use case where we have to defer processing the messages by one hour from the time it got published. Please suggest if there is an elegant solution to handle this. Thanks Naveen kumar M. -- Naveen kumar M.

Build failed in Jenkins: kafka-2.4-jdk8 #46

See Changes: [wangguoz] KAFKA-8972 (2.4 blocker): bug fix for restoring task (#7617) -- [...truncated 2.69 MB...] org.apache.kafka.streams.internals.KeyValueStoreFacadeTest

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi, Harsha, I am still looking at the KIP and the PR. A couple of quick comments/questions. 20. It's fine to keep the HDFS binding temporarily in the PR. We just need to remove it before it's merged to trunk. As Victor mentioned, we can provide a reference implementation based on a mocked

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

Hi Vikto, That's an interesting idea. However, I think it's better to consider it outside the context of this KIP. I think one-letter abbreviations would be controversial, and aren't really related to fixing the ZK dependency here. +1 (binding). Thanks, Brian. best, Colin On Thu, Oct 31,

Re: [VOTE] KIP-543: Expand ConfigCommand's non-ZK functionality

Sorry, that should read "Hi Viktor",-C. On Thu, Oct 31, 2019, at 14:08, Colin McCabe wrote: > Hi Vikto, > > That's an interesting idea. However, I think it's better to consider > it outside the context of this KIP. I think one-letter abbreviations > would be controversial, and aren't

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

1. Was trying to spell them out separately. but makes sense for readability. done 2. No I immediately agree :) .. makes sense. @navinder? 3. I actually attempted only sending taskIds while working on KAFKA-7149. Its non-trivial to handle edges cases resulting from newly added topic partitions

[jira] [Created] (KAFKA-9127) Needless group coordination overhead for GlobalKTables

Chris Toomey created KAFKA-9127: --- Summary: Needless group coordination overhead for GlobalKTables Key: KAFKA-9127 URL: https://issues.apache.org/jira/browse/KAFKA-9127 Project: Kafka Issue

[jira] [Resolved] (KAFKA-9073) Kafka Streams State stuck in rebalancing after one of the StreamThread encounters java.lang.IllegalStateException: No current assignment for partition

[ https://issues.apache.org/jira/browse/KAFKA-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-9073. -- Resolution: Fixed > Kafka Streams State stuck in rebalancing after one of the StreamThread >

[jira] [Resolved] (KAFKA-9093) NullPointerException in KafkaConsumer with group.instance.id

[ https://issues.apache.org/jira/browse/KAFKA-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxihx resolved KAFKA-9093. --- Resolution: Fixed > NullPointerException in KafkaConsumer with group.instance.id >

[jira] [Created] (KAFKA-9122) Externalizing DB password is not working

Dwijadas created KAFKA-9122: --- Summary: Externalizing DB password is not working Key: KAFKA-9122 URL: https://issues.apache.org/jira/browse/KAFKA-9122 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-280: Enhanced log compaction

Thanks for picking up this KIP, Senthil. (1) As far as I remember, the main issue of the original proposal was a missing topic level configuration for the compaction strategy. With this being addressed, I am in favor of this KIP. (2) With regard to (1), it seems we would need a new topic level

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Interesting discussion. My personal take is as follows: (1) Co-group is not a special case of a multi-way KTable join, because multiple record wit the same key from a single input stream should be aggregated together and there are not update semantics. A co-group is rather a muti-stream

RE: [DISCUSS] KIP-280: Enhanced log compaction

Hi Matthias Thanks for the response. (1) Yes (2) Yes, and the config name will be the same (i.e. `log.cleaner.compaction.strategy` & `log.cleaner.compaction.strategy.header`) at broker level and topic level (to override broker level default compact strategy). Please let me know if we need to