Re: [ANNOUNCE] New committer: Boyang Chen

2020-06-24 Thread Rajini Sivaram
Congratulations, Boyang! Regards, Rajini On Wed, Jun 24, 2020 at 2:26 AM Adam Bellemare wrote: > Just adding my congratulations, Boyang! Thank you for all your > contributions and effort! > > On Tue, Jun 23, 2020 at 9:14 PM Kowshik Prakasam > wrote: > > > Congrats, Boyang! :) > > > > > >

Build failed in Jenkins: kafka-2.4-jdk8 #228

2020-06-24 Thread Apache Jenkins Server
See Changes: [matthias] HOTFIX: use equals() to compare HostInfo in StreamsMetadataState (#8919) -- [...truncated 7.19 MB...]

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-06-24 Thread Gokul Ramanan Subramanian
Ismael, I am open to using any error code and am not attached to one TBH. Colin had suggested creating a new resource code called RESOURCE_LIMIT_EXCEEDED. I am happy to reuse the error code corresponding to PolicyViolation. Is it safe to rename errors and corresponding exception names? If so, I'd

Re: [ANNOUNCE] New committer: Boyang Chen

2020-06-24 Thread Ismael Juma
Congratulations Boyang! Ismael On Mon, Jun 22, 2020 at 4:26 PM Guozhang Wang wrote: > The PMC for Apache Kafka has invited Boyang Chen as a committer and we are > pleased to announce that he has accepted! > > Boyang has been active in the Kafka community more than two years ago. > Since then

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2020-06-24 Thread Ismael Juma
Hi Boyang, "One thing I'm not fully convinced of is why we need to deprecate the server side auto topic creation logic," The point is to eventually remove auto topic creation logic from MetadataRequest. It makes no sense for MetadataRequest to cause topics to be created. It's an unintuitive

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-06-24 Thread Ismael Juma
Error code names are safe to rename at the moment as they are in an internal package. The exception class is in a public package though. I was thinking that PolicyViolationException could be a subclass of the more generic exception. This approach would mean that older clients would understand the

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Ismael Juma
Hi Sankalp, Is there a reason why you cannot upgrade to Apache Kafka 2.5.0 instead? We are working on the 2.5.1 release, which would be the recommended release. Ismael On Wed, Jun 24, 2020 at 6:18 AM Sankalp Bhatia wrote: > Hi All, > > I would like to know if there are any plans to release a

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Sankalp Bhatia
Thanks Ismael for the response. For our clusters running 2.3.1 and 2.4.1, we saw some issues which had 2.3.2 and 2.4.2 as the fix versions. I looked at 2.5.0 but since it introduces some major changes like support for ZK encryption and a few others, I was wondering if we should choose a smaller

Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Sankalp Bhatia
Hi All, I would like to know if there are any plans to release a 2.3.2 and 2.4.2 versions for Apache Kafka in the near future. I see there are some issues marked as fixed in these two versions (https://tinyurl.com/ycdpz5cb). However, I could not find a branch/tag corresponding to these versions

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-06-24 Thread Gokul Ramanan Subramanian
Alex, 904. For multi-tenancy, yes, the limits in this KIP will apply on a first-come-first-serve basis. If and when multi-tenancy based limiting rolls out, it would probably introduce a different way to specific per-client limits on partitions. I suppose those will apply in addition to and not

Jenkins build is back to normal : kafka-trunk-jdk8 #4665

2020-06-24 Thread Apache Jenkins Server
See

contributor request

2020-06-24 Thread Dániel Urbán
Hi, I'd like to work on some KIPs. Can you please add me to the contributors? My username is durban on both JIRA and Confluence. Thanks in advance, Daniel

Re: [DISCUSS] KIP-578: Add configuration to limit number of partitions

2020-06-24 Thread Gokul Ramanan Subramanian
Boyang, Tom and I discussed earlier in this thread whether we could use the topic creation policy for limiting the number of partitions. Basically, we realized that while we can use it for limiting the number of partitions, we cannot use it for assigning partitions in a way that respects the

回复: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

2020-06-24 Thread Jiamei Xie
Hi Boyang, Justin, " 1. Could we include a link to KIP-464 and explain its relation to KIP-487? It's very hard to read through the proposal when readers only have a reference number to some KIP that is not briefed. " For KIP-464

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Tom Bentley
Congratulations Xi! On Wed, Jun 24, 2020 at 5:34 PM Guozhang Wang wrote: > The PMC for Apache Kafka has invited Xi Hu as a committer and we are > pleased to announce that he has accepted! > > Xi Hu has been actively contributing to Kafka since 2016, and is well > recognized especially for his

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Bill Bejeck
Congratulations Xi! -Bill On Wed, Jun 24, 2020 at 12:45 PM Tom Bentley wrote: > Congratulations Xi! > > On Wed, Jun 24, 2020 at 5:34 PM Guozhang Wang wrote: > > > The PMC for Apache Kafka has invited Xi Hu as a committer and we are > > pleased to announce that he has accepted! > > > > Xi Hu

[ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Guozhang Wang
The PMC for Apache Kafka has invited Xi Hu as a committer and we are pleased to announce that he has accepted! Xi Hu has been actively contributing to Kafka since 2016, and is well recognized especially for his non-code contributions: he maintains a tech blog post evangelizing Kafka in the

Re: [ANNOUNCE] New committer: Boyang Chen

2020-06-24 Thread Raymond Ng
Congrats Boyang! Look forward to more awesome contributions from you in the future. Regards, Ray On Wed, Jun 24, 2020 at 6:07 AM Ismael Juma wrote: > Congratulations Boyang! > > Ismael > > On Mon, Jun 22, 2020 at 4:26 PM Guozhang Wang wrote: > > > The PMC for Apache Kafka has invited Boyang

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Gwen Shapira
Kafka follows a "fully compatible model" and new features are always optional. We have extensive compatibility testing to make sure rolling upgrades are safe and painless. This investment in compatibility allows us to reduce the effort needed in maintaining old versions (Security CVE is the

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Ismael Juma
I think there's some confusion here. You can upgrade to AK 2.5.0 and completely ignore ZK and TLS. It's completely optional. Ismael On Wed, Jun 24, 2020 at 10:20 AM Gokul Ramanan Subramanian < gokul24...@gmail.com> wrote: > Any updates on Kafka 2.3.2 and 2.4.2? Given the complexity of migrating

Re: [DISCUSS] KIP-308: GetOffsetShell: new KafkaConsumer API, support for multiple topics, minimize the number of requests to server

2020-06-24 Thread Dániel Urbán
Hi, I see that this KIP turned somewhat inactive - I'd like to pick it up and work on it if it is okay. Part of the work is done, as switching to the Consumer API is already in trunk, but some functionality is still missing. I've seen the current PR and the discussion so far, only have a few

[jira] [Created] (KAFKA-10197) Elect preferred leader when completing a partition reassignment

2020-06-24 Thread Bob Barrett (Jira)
Bob Barrett created KAFKA-10197: --- Summary: Elect preferred leader when completing a partition reassignment Key: KAFKA-10197 URL: https://issues.apache.org/jira/browse/KAFKA-10197 Project: Kafka

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Gokul Ramanan Subramanian
Any updates on Kafka 2.3.2 and 2.4.2? Given the complexity of migrating from non-encrypted TLS to encrypted TLS connection for ZooKeeper, it would be nice to have a bug-free version of 2.3 and 2.4. Is there a technical reason why we hesitate to get these versions out? Or is it that no one has got

Re: contributor request

2020-06-24 Thread Bill Bejeck
Hi Daniel, You're all set as a contributor in Jira and can create/edit KIPs as well. Thanks for your interest in Apache Kafka. -Bill On Wed, Jun 24, 2020 at 6:16 AM Dániel Urbán wrote: > Hi, > I'd like to work on some KIPs. Can you please add me to the contributors? > My username is durban

Re: [DISCUSS] KIP-595: A Raft Protocol for the Metadata Quorum

2020-06-24 Thread Guozhang Wang
@Jun Rao Regarding your comment about log compaction. After some deep-diving into this we've decided to propose a new snapshot-based log cleaning mechanism which would be used to replace the current compaction mechanism for this meta log. A new KIP will be proposed specifically for this idea.

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Gokul Ramanan Subramanian
I agree that is an option, but is there any reason to not have a 2.3.2 and 2.4.2 released? If so, it would be nice to know about these reasons. Appreciate your time on this. On Wed, Jun 24, 2020 at 6:35 PM Ismael Juma wrote: > I think there's some confusion here. You can upgrade to AK 2.5.0 and

[DISCUSS] KIP-623: Add "internal-topics" option to streams application reset tool

2020-06-24 Thread Joel Wee
Hi all I would like to start a vote for KIP-623, which adds the option --internal-topics to the streams-application-reset-tool: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158862177. Please let me know what you think. Best Joel

[VOTE] KIP-623: Add "internal-topics" option to streams application reset tool

2020-06-24 Thread Joel Wee
Apologies. Changing the subject. On 24 Jun 2020, at 9:14 PM, Joel Wee mailto:joel@outlook.com>> wrote: Hi all I would like to start a vote for KIP-623, which adds the option --internal-topics to the streams-application-reset-tool:

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Mickael Maison
Congratulations Xi! On Wed, Jun 24, 2020 at 7:25 PM Matthias J. Sax wrote: > > Congrats Xi! > > On 6/24/20 9:45 AM, Tom Bentley wrote: > > Congratulations Xi! > > > > On Wed, Jun 24, 2020 at 5:34 PM Guozhang Wang wrote: > > > >> The PMC for Apache Kafka has invited Xi Hu as a committer and we

Re: [DISCUSS] KIP-629: Use racially neutral terms in our codebase

2020-06-24 Thread Jorge Esteban Quilcate Otoya
Hi Xavier, Thank you for this proposal! It's awesome to see the community taking action on this. `include` and `exclude` make sense to me. Probably obvious but is documentation and website considered as well as part of the KIP? This would be interesting because it could be also important to

[jira] [Resolved] (KAFKA-10135) Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager

2020-06-24 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-10135. - Resolution: Fixed > Extract Task#executeAndMaybeSwallow to be a general utility function into

Re: Question around release plans for Apache Kafka 2.3.2 and 2.4.2

2020-06-24 Thread Gokul Ramanan Subramanian
Thanks Gwen and Ismael. On Wed, Jun 24, 2020 at 6:52 PM Gwen Shapira wrote: > Kafka follows a "fully compatible model" and new features are always > optional. We have extensive compatibility testing to make sure rolling > upgrades are safe and painless. > This investment in compatibility allows

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Matthias J. Sax
Congrats Xi! On 6/24/20 9:45 AM, Tom Bentley wrote: > Congratulations Xi! > > On Wed, Jun 24, 2020 at 5:34 PM Guozhang Wang wrote: > >> The PMC for Apache Kafka has invited Xi Hu as a committer and we are >> pleased to announce that he has accepted! >> >> Xi Hu has been actively contributing

Re: [ANNOUNCE] New committer: Boyang Chen

2020-06-24 Thread Liquan Pei
Congrats! On Wed, Jun 24, 2020 at 9:42 AM Raymond Ng wrote: > Congrats Boyang! Look forward to more awesome contributions from you in the > future. > > Regards, > Ray > > On Wed, Jun 24, 2020 at 6:07 AM Ismael Juma wrote: > > > Congratulations Boyang! > > > > Ismael > > > > On Mon, Jun 22,

[jira] [Created] (KAFKA-10198) Dirty tasks may be recycled instead of closed

2020-06-24 Thread Sophie Blee-Goldman (Jira)
Sophie Blee-Goldman created KAFKA-10198: --- Summary: Dirty tasks may be recycled instead of closed Key: KAFKA-10198 URL: https://issues.apache.org/jira/browse/KAFKA-10198 Project: Kafka

[jira] [Reopened] (KAFKA-8197) Flaky Test kafka.server.DynamicBrokerConfigTest > testPasswordConfigEncoderSecretChange

2020-06-24 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman reopened KAFKA-8197: Saw this fail again:  kafka.server.DynamicBrokerConfigTest >

Re: [DISCUSS] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-06-24 Thread John Roesler
Hi Will, This proposal looks good to me overall. Thanks for the contribution! Just a couple of minor notes: The system time method would return a cached timestamp that Streams looks up once when it starts processing a record. This may be confusing, so it might be good to state it in the

Re: [DISCUSS] Apache Kafka 2.6.0 release

2020-06-24 Thread Sophie Blee-Goldman
Hey all, Just a heads up that we discovered a new blocker. The fix is pretty straightforward and there's already a PR for it so it should be resolved quickly. Here's the ticket: https://issues.apache.org/jira/browse/KAFKA-10198 On Sat, May 30, 2020 at 12:52 PM Randall Hauch wrote: > Hi,

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-24 Thread Vahid Hashemian
Congrats Xi! --Vahid On Wed, Jun 24, 2020 at 2:56 PM Mickael Maison wrote: > Congratulations Xi! > > On Wed, Jun 24, 2020 at 7:25 PM Matthias J. Sax wrote: > > > > Congrats Xi! > > > > On 6/24/20 9:45 AM, Tom Bentley wrote: > > > Congratulations Xi! > > > > > > On Wed, Jun 24, 2020 at 5:34 PM

Re: [DISCUSS] Apache Kafka 2.6.0 release

2020-06-24 Thread Boyang Chen
Hey Randal, There was another spotted blocker: https://issues.apache.org/jira/browse/KAFKA-10173 As of current, John is working on a fix. Boyang On Wed, Jun 24, 2020 at 4:08 PM Sophie Blee-Goldman wrote: > Hey all, > > Just a heads up that we discovered a new blocker. The fix is pretty >

Jenkins build is back to normal : kafka-2.3-jdk8 #217

2020-06-24 Thread Apache Jenkins Server
See