[jira] [Created] (KAFKA-15046) Produce performance issue under high disk load

Haruki Okada created KAFKA-15046: Summary: Produce performance issue under high disk load Key: KAFKA-15046 URL: https://issues.apache.org/jira/browse/KAFKA-15046 Project: Kafka Issue Type:

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1882

See Changes: -- [...truncated 571631 lines...] [2023-05-31T22:48:57.111Z] [INFO] Scanning for projects... [2023-05-31T22:48:57.111Z] [INFO]

Re: [DISCUSS] Adding non-committers as Github collaborators

Hi Greg and Andrew, Sorry, I don't know how to re-sent the invitation. It looks like it is auto sent after the .asf.yaml updated. Since updating collaborator list is part of release process based on the doc , I just created a new list and opened a PR:

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.5 #11

See

[jira] [Resolved] (KAFKA-15044) Snappy v.1.1.9.1 NoClassDefFound on ARM machines

[ https://issues.apache.org/jira/browse/KAFKA-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15044. Resolution: Fixed > Snappy v.1.1.9.1 NoClassDefFound on ARM machines >

Re: [VOTE] 3.5.0 RC0

Hey Mickael -- This is done. Thanks! On Wed, May 31, 2023 at 11:24 AM Mickael Maison wrote: > Hi Justine, > > Yes you can merge that into 3.5. > > Thanks, > Mickael > > On Wed, May 31, 2023 at 7:56 PM Justine Olshan > wrote: > > > > FYI -- I just saw this PR regarding a dependency for ARM. We

[jira] [Created] (KAFKA-15045) Move Streams task assignor to public configs

A. Sophie Blee-Goldman created KAFKA-15045: -- Summary: Move Streams task assignor to public configs Key: KAFKA-15045 URL: https://issues.apache.org/jira/browse/KAFKA-15045 Project: Kafka

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Thanks Sophie! I've just made the updates :) On Wed, May 31, 2023 at 2:29 PM Sophie Blee-Goldman wrote: > Thanks Hao -- I really like Walker's idea to separate the assignment > strategy from the rack awareness option. > Having two different configs for these makes a lot of sense to me and seems

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Hi all, I've updated the KIP based on the feedback. Major changes I made: 1. Add rack aware assignment to `StickyTaskAssignor` 2. Reject `Prefer reliability and then find optimal cost` option in standby task assignment. On Wed, May 31, 2023 at 12:09 PM Hao Li wrote: > Hi all, > > Thanks for

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Thanks Hao -- I really like Walker's idea to separate the assignment strategy from the rack awareness option. Having two different configs for these makes a lot of sense to me and seems like it will benefit the most users. Given your KIP already has a lot going on with it, I would be happy to

Re: [DISCUSS] KIP-921 OpenJDK CRaC support

Hmm. This isn't something I say very often, but it might be a bit too early for a KIP about this. Support hasn't even landed in any version of Java yet. How do we know that it will look simlar to the draft proposal when and if it does land? Also, what is the advantage to the user? Maybe I

Re: [DISCUSS] Adding non-committers as Github collaborators

Hi all, Like Greg I also received an invitation to collaborate but was too slow to accept the invite :( I'm also wondering if there's a way to resend the invite? I'm andymg3 on GitHub. Thanks, Andrew On Tue, May 30, 2023 at 12:12 PM Greg Harris wrote: > Hey all, > > I received an invitation to

Re: [DISCUSS] Re-visit end of life policy

Ultimately, maintaining both the ZK-based controller and the KRaft-based controller takes a substantial resource toll on the community. That's the reason why we agreed to drop the ZK based controller in Apache Kafka 4.0, as part of KIP-833. If there is a serious bug in the last bridge release,

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #10

See Changes: -- [...truncated 563061 lines...] [2023-05-31T20:00:14.769Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 177 > KStreamKStreamIntegrationTest

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1881

See Changes: -- [...truncated 477324 lines...] [2023-05-31T19:36:45.380Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 178 >

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Hi all, Thanks for the feedback! I will update the KIP accordingly. *For Sophie's comments:* 1 and 2. Good catch. Fixed these. 3 and 4 Yes. We can make this public config and call out the clientConsumer config users need to set. 5. It's ideal to take the previous assignment in HAAssignor into

Re: [VOTE] 3.5.0 RC0

Hi Justine, Yes you can merge that into 3.5. Thanks, Mickael On Wed, May 31, 2023 at 7:56 PM Justine Olshan wrote: > > FYI -- I just saw this PR regarding a dependency for ARM. We may want to > get this in for 3.5 as well. It should be quick. > >

Re: [VOTE] 3.5.0 RC0

FYI -- I just saw this PR regarding a dependency for ARM. We may want to get this in for 3.5 as well. It should be quick. https://issues.apache.org/jira/browse/KAFKA-15044 https://github.com/apache/kafka/pull/13786 Justine On Wed, May 31, 2023 at 9:28 AM David Arthur wrote: > Mickael, > >

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hey Omnia, I was doing a bit of snooping (I actually just get updates for the KIP page) and I saw this draft was in progress. I shared it with some of my colleagues as well who I previously discussed the issue with. The initial look was pretty promising to me. I appreciate the detailing of the

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Hi Hao, Most of the comments I had on this kip are already mentioned, but I did want to share my two major concerns. 1. Stability. I worry about stability. If we only have the HA assignor work with rack awareness we will have a lot of state movement in many cases. Sophie and Bruno have this

Re: [DISCUSS] solutions for broker OOM caused by many producer IDs

Hi Justine and Luke, I started a KIP draft here https://cwiki.apache.org/confluence/display/KAFKA/KIP-936%3A+Throttle+number+of+active+PIDs for a proposal would appreciate it if you could provide any initial feedback before opening a broader discussion. Thanks On Wed, Feb 22, 2023 at 4:35 PM

[jira] [Created] (KAFKA-15044) Snappy v.1.1.9.1 NoClassDefFound on ARM machines

David Mao created KAFKA-15044: - Summary: Snappy v.1.1.9.1 NoClassDefFound on ARM machines Key: KAFKA-15044 URL: https://issues.apache.org/jira/browse/KAFKA-15044 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-15043) Create a kcontroller metric for expired broker heartbeats

Colin McCabe created KAFKA-15043: Summary: Create a kcontroller metric for expired broker heartbeats Key: KAFKA-15043 URL: https://issues.apache.org/jira/browse/KAFKA-15043 Project: Kafka

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #9

See Changes: -- [...truncated 561044 lines...] [2023-05-31T16:42:52.981Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 177 >

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1880

See

Re: [VOTE] 3.5.0 RC0

Mickael, Colin has approved my patch for KAFKA-15010, I'm just waiting on a build before merging. I'll go ahead and backport the other fixes that need to precede this one into 3.5. -David On Wed, May 31, 2023 at 11:52 AM Mickael Maison wrote: > Hi, > > The issue mentioned by Greg has been

Re: [VOTE] 3.5.0 RC0

Hi, The issue mentioned by Greg has been fixed. As soon as the fix for KAFKA-15010 is merged I'll build another RC. Thanks, Mickael On Tue, May 30, 2023 at 10:33 AM Mickael Maison wrote: > > Hi David, > > Feel free to backport the necessary fixes to 3.5. > > Thanks, > Mickael > > On Tue, May

[GitHub] [kafka-site] divijvaidya commented on a diff in pull request #516: MINOR: Add blog for 3.5.0 release

divijvaidya commented on code in PR #516: URL: https://github.com/apache/kafka-site/pull/516#discussion_r1211826401 ## blog.html: ## @@ -0,0 +1,70 @@ + + + + + + + + +Blog Review Comment: should we call this "news" or "announcements"

[jira] [Created] (KAFKA-15042) Clarify documentation on Tagged fields

Nicolas created KAFKA-15042: --- Summary: Clarify documentation on Tagged fields Key: KAFKA-15042 URL: https://issues.apache.org/jira/browse/KAFKA-15042 Project: Kafka Issue Type: Wish

[GitHub] [kafka-site] mimaison commented on pull request #516: MINOR: Add blog for 3.5.0 release

mimaison commented on PR #516: URL: https://github.com/apache/kafka-site/pull/516#issuecomment-1570318966 @divijvaidya That's a good idea, I've added a sentence. This also made me notice that we're missing the upgrade steps in the 3.5 documentation. I'll open another PR to address it. --

[GitHub] [kafka-site] mimaison commented on a diff in pull request #516: MINOR: Add blog for 3.5.0 release

mimaison commented on code in PR #516: URL: https://github.com/apache/kafka-site/pull/516#discussion_r1211785265 ## blog.html: ## @@ -0,0 +1,69 @@ + + + + + + + + +Blog + + + +

[jira] [Resolved] (KAFKA-8713) [Connect] JsonConverter NULL Values are replaced by default values even in NULLABLE fields

[ https://issues.apache.org/jira/browse/KAFKA-8713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-8713. --- Resolution: Fixed > [Connect] JsonConverter NULL Values are replaced by default values even

Re: [DISCUSS] KIP-858: Handle JBOD broker disk failure in KRaft

Hi all, I have created a TLA+ specification for this KIP, available here: https://github.com/soarez/kafka/blob/kip-858-tla-plus/tla/Kip858.tla If there are no further comments I'll start a voting thread next week. -- Igor

Re: [DISCUSS] KIP-925: rack aware task assignment in Kafka Streams

Hi Hao, Thank you for the KIP! Really interesting! In general, I think the KIP is a bit too vague. You explain the main algorithm and different options. It is not clear to me on what option we will start voting. One way out of this situation would be to cut the KIP down to the simplest

[jira] [Created] (KAFKA-15041) Source Connector auto topic creation fails when topic is deleted and brokers don't support auto topic creation

Sagar Rao created KAFKA-15041: - Summary: Source Connector auto topic creation fails when topic is deleted and brokers don't support auto topic creation Key: KAFKA-15041 URL:

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1879

See Changes: -- [...truncated 574001 lines...] [2023-05-31T10:04:14.111Z] [WARNING] [2023-05-31T10:04:14.111Z] [WARNING] For more or less details, use

[jira] [Created] (KAFKA-15040) segment copy to remote storage won't work in KRaft mode

Luke Chen created KAFKA-15040: - Summary: segment copy to remote storage won't work in KRaft mode Key: KAFKA-15040 URL: https://issues.apache.org/jira/browse/KAFKA-15040 Project: Kafka Issue