Re: KRaft Observer node unable to recover after (re-)bootstrapping to Follower node

It's awesome you already reproduced the issue, Alyssa! But @Justin, if possible, could you still share the logs and the quorum state store file on the observer like Alyssa requested? Thank you. Luke On Fri, May 30, 2025 at 9:40 AM Alyssa Huang wrote: > Wanted to correct my wording and resend t

[jira] [Created] (KAFKA-19336) Upgrade jackson libs since v2.16 is not maintained anymore

Luke Chen created KAFKA-19336: - Summary: Upgrade jackson libs since v2.16 is not maintained anymore Key: KAFKA-19336 URL: https://issues.apache.org/jira/browse/KAFKA-19336 Project: Kafka Issue

[jira] [Created] (KAFKA-19331) No error handling for leader unregistered in applyLocalFollowersDelta

Luke Chen created KAFKA-19331: - Summary: No error handling for leader unregistered in applyLocalFollowersDelta Key: KAFKA-19331 URL: https://issues.apache.org/jira/browse/KAFKA-19331 Project: Kafka

Re: [VOTE] KIP-1179: Introduce remote.log.manager.follower.thread.pool.size config

+1 binding from me. Thanks. Luke On Thu, May 22, 2025 at 12:54 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (binding). Thanks for the KIP! > > On Wed, May 21, 2025 at 9:04 PM Chia-Ping Tsai wrote: > > > +1 (binding) > > > > Kuan-Po Tseng 於 2025年5月21日 週三 下午8:10寫道： > > >

Re: [DISCUSS] KIP-1179: Introduce remote.log.manager.follower.thread.pool.size config

ally configures remote.log.manager.follower.thread.pool.size=8, the > follower thread pool size will be 8. > > I’ll also update this example in the "Compatibility, Deprecation, and > Migration Plan" section. > > Best, > Kuan-Po > > &

Re: [DISCUSS] KIP-1179: Introduce remote.log.manager.follower.thread.pool.size config

Hi Kuan-Po, Thanks for the KIP! Questions: 1. If `remote.log.manager.thread.pool.size` is set and `remote.log.manager.follower.thread.pool.size` is not set, we will still honor the setting in `remote.log.manager.thread.pool.size`, right? We should add in KIP to make it clear. 2. Suppose `remote.

Re: [VOTE] KIP-1135: Deprecated "org.apache.kafka.disallowed.login.modules"

+1 (binding) from me. Thanks for the KIP. Luke On Mon, May 19, 2025 at 11:12 PM Chia-Ping Tsai wrote: > +1 (binding) > > 龚宣璋 於 2025年5月14日 週三 下午7:29寫道： > > > Hi Chia-Ping > > You are right. > > I don't think this should be optional. We should enforce the use of the > > allowlist > > > > Chia-Pi

Re: [DISCUSS] KIP-1159: Large message reference based Serializer

ome smart work for example > to avoid s3 throttling the recommended way on their doc it to create sub > paths under the original bucket, to decide this we might hash the data to > find a suitable sub-path. > > Here is an example of how I would generate an path for s3 file > > `

Re: [VOTE] KIP-1123: Rack-aware partitioning for Kafka Producer

+1 binding from me. Thanks. Luke On Wed, May 14, 2025 at 7:14 PM Satish Duggana wrote: > Thanks Ivan for the KIP, +1 (binding) > > ~Satish. > > On Wed, 14 May 2025 at 16:09, Christo Lolov > wrote: > > > > Hello! > > > > Apologies for coming late to the party. I think this is a great idea and >

Re: [VOTE] 3.9.1 RC2

Hi all, Thanks for the vote so far! Update for the system test: 1. The system tests have some failed tests: http://jenkins.opensource4you.tw/job/kafka-e2e/317 2. The failed benchmark_test and OffsetValidationTest can pass in another run: https://drive.google.com/drive/folders/1lf7EH-m8PNPGfv7Fj3-

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

high watermark between leader and follower.", but what if we > add each partition's metadata in the actual partition - i.e an event with a > pointer to the S3 object and the {byte-range,offset-range} of said > partitions' data in said S3 object > > On 2025/05/08 07:57:

Re: [DISCUSS] KIP-1183: Unified Shared Storage

Hi Xinyu, Thanks for the proposal! I agree abstracting the log and logSegment layers can allow users to implement any kinds of storage mediums. Some high-level comments: 1. I expect to see the future plan of this KIP, but can't find it. What do you expect to achieve the "shared storage" goal afte

Re: [DISCUSS] Apache Kafka 3.9.1 release

Hi Jose, Please hold on cherry-picking the commit. I'm afraid the RC tag will be wrong if there's new commits. Please let us finish the build first. Thank you. Luke On Tue, May 13, 2025 at 11:15 PM José Armando García Sancio wrote: > HI TengYao and Luke, > > I would like to cherry-pick the com

[jira] [Created] (KAFKA-19275) client-state and thread-state metrics are always ""Unavailable"

Luke Chen created KAFKA-19275: - Summary: client-state and thread-state metrics are always ""Unavailable" Key: KAFKA-19275 URL: https://issues.apache.org/jira/browse/KAFKA-19275

Re: [DISCUSS] KIP-1161: cleanup.policy shouldn't be empty

ne for users wanting infinite > retention > > to opt in explicitly. > > > > We could also consider supporting an empty cleanup.policy by fixing the > > issue in remote storage. But then the user may never realize this if > it's a > > mistake. > > > &

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

ere working on a proposal to persist a metadata topic snapshot to avoid > the reading from scratch on broker restart, not sure whether they have > submitted the proposal or not. > > On Friday, May 9, 2025 at 03:19:51 AM PDT, Luke Chen < > show...@gmail.com> wrote: >

Re: [DISCUSS] KIP-1161: cleanup.policy shouldn't be empty

Hi Jiunn-Yang, Thanks for the KIP. Comments: 1. "it is acceptable because an empty cleanup.policy is considered invalid in Kafka. Therefore, this trade-off is justified." Could you explain more about this? Why is this trade-off justified? If we think the empty cleanup.policy is invalid in kafka,

Re: [VOTE] 3.9.1 RC1

run all tests > > >`OffsetsApiIntegrationTest` failed, but re-run passed > > > - run quick start (core/connect/streams) using Scala 2.12 binaries > > > (KRaft mode) > > > - downloaded both docker images; started them up, and re-run parts of > > > q

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

ometime in the future. Thank you. Luke On Fri, May 9, 2025 at 5:52 AM Henry Haiying Cai wrote: > Luke, > Thanks for your continued suggestions on this KIP, see an my answer below > inline with HC> indentation. > On Thursday, May 8, 2025 at 02:10:07 AM PDT, Luke Chen < &

[jira] [Resolved] (KAFKA-19209) "index.interval.bytes" config doc should also mention time index

[ https://issues.apache.org/jira/browse/KAFKA-19209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-19209. --- Fix Version/s: 4.1.0 Resolution: Fixed > "index.interval.bytes" config do

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

. Luke Thanks. Luke On Thu, May 8, 2025 at 3:57 PM Luke Chen wrote: > Hi Xinyu and Henry, > > I think the WAL metadata in KIP1176 is not for log recover, the log > recovery still loads log segments locally. > The WAL metadata is for leader <-> follower information sharing

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

age threshold) and later the current > > tiered > > > storage code will upload this closed log segment onto S3 when this > > segment > > > file is more than 1 hour old. These 2 activities (uploading to S3E1Z > and > > > uploading to S3) are independently run,

Re: [VOTE] 3.9.1 RC1

hat, +1 (binding) from me. > > Thanks, > Justine > > > On Fri, May 2, 2025 at 9:27 AM Justine Olshan > wrote: > > > I will try to take a look today. Thanks folks > > > > Justine > > > > On Thu, May 1, 2025 at 7:42 PM Luke Chen wrote: > >

Re: [VOTE] 3.9.1 RC1

, > > > > > > > > I've completed the following tests on my local machine: > > > > > > > > - Built from source using JDK 21 & JDK 17 (Zulu) > > > > - Executed all unit and integration tests > > > > - Ran the quickstart s

Re: [DISCUSS] KIP-1176: Tiered Storage for Active Log Segment

Hi Henry, This is a very interesting proposal! I love the idea to minimize the code change to be able to quickly get delivered. Thanks for proposing this! Some questions: 1. In this KIP, we add one more tier of storage. That is: local disk -> fast object store -> slow object store. Why can't we a

[jira] [Created] (KAFKA-19219) improve KRaft migration log

Luke Chen created KAFKA-19219: - Summary: improve KRaft migration log Key: KAFKA-19219 URL: https://issues.apache.org/jira/browse/KAFKA-19219 Project: Kafka Issue Type: Improvement Affects

[jira] [Created] (KAFKA-19209) "index.interval.bytes" config doc should also mention time index

Luke Chen created KAFKA-19209: - Summary: "index.interval.bytes" config doc should also mention time index Key: KAFKA-19209 URL: https://issues.apache.org/jira/browse/KAFKA-19209 Proj

Re: [DISCUSS] KIP-1159: Large message reference based Serializer

Hi Omnia, Thanks for proposing this feature that many users expected. Some comments: 1. It's quite interesting to see the idea of chained serializer/deserializer used here. I like it. 2. It's not clear how the "retry count" comes into play in the KIP. It needs more explanation. 3. What does "La

Re: [DISCUSS] KIP-1150 Diskless Topics

Hi Xiaorui, It's great to hear that the AutoMQ team is also interested in contributing to Apache Kafka. Of course the AutoMQ's solution is also an option for the community to consider. It would be good if you can write a KIP for your solution, and we can discuss it in the community. Let me know i

Re: [VOTE] 3.9.1 RC1

Hi all, Update the system tests result link: https://drive.google.com/drive/folders/1lf7EH-m8PNPGfv7Fj3-FLnz_UH3un41n Summary: All the 1st run failed tests can pass after re-run. So they are just flaky. No issues found. Thanks. Luke On Wed, Apr 23, 2025 at 11:57 PM Kuan Po Tseng wrote: > Hi

Re: [VOTE] 3.9.1 RC0

Hi David, Thanks for the fix! I've tested it and it fixed the orphan commit issue. We'll close this VOTE thread and rebuild RC1 later. Thanks. Luke On Sun, Apr 20, 2025 at 12:28 PM TengYao Chi wrote: > Hi David > Thanks for the quick fix! > I will test this patch. > > Sincerely, > TengYao > >

[jira] [Resolved] (KAFKA-19166) Release tags are orphaned

[ https://issues.apache.org/jira/browse/KAFKA-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-19166. --- Fix Version/s: 3.9.1 Resolution: Fixed > Release tags are orpha

Re: [DISCUSS] KIP-1150 Diskless Topics

Hi Josep, Thanks for the KIP! Quite exciting to see this feature brought into Apache Kafka Comments: 1. "Permit multi-region active-active topics with automatic failover" I didn't see any future work mentioning this. Does it mean, with diskless topic MVP, this will work by default? 2. "Write

Re: Kafka tags on repo

Thank you, David! Luke On Thu, Apr 17, 2025 at 9:26 AM David Arthur wrote: > Matthias pointed out to me that there are indeed some orphaned tags for > recent releases. This appears to be a bug in our release script or release > process. I've filed https://issues.apache.org/jira/browse/KAFKA-191

[jira] [Resolved] (KAFKA-19071) Incorrect documentation for remote.storage.enable

[ https://issues.apache.org/jira/browse/KAFKA-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-19071. --- Fix Version/s: 4.1.0 3.9.1 4.0.1 Resolution: Fixed

Re: Requesting permission to contribute to Apache Kafka

Hi Divya, Your account should be all set. Thanks. Luke On Sat, Apr 12, 2025 at 7:25 PM Divya Guduru wrote: > I have access to the JIRA > board but cannot assign issues to myself. Could you give contributor > permissions to Jira ID: divya2890 > > > Regards > Divya > > > On Sat, Apr 12, 2025 at

Re: [DISCUSS] Apache Kafka 3.9.1 release

Hi all, To help the new committer TengYao to get familiar with the release process, I've talked with him and he is willing to be the co-release manager for v3.9.1 release. Thank you TengYao! Luke On Thu, Apr 10, 2025 at 11:25 PM José Armando García Sancio wrote: > Thanks Luke. > > I cherry pi

Re: [ANNOUNCE] New committer: Apoorv Mittal

Congrats Apoorv! Well deserved! Luke On Fri, Apr 11, 2025 at 9:25 AM PoAn Yang wrote: > Congrats, Apoorv! > > > On Apr 11, 2025, at 2:15 AM, Randall Hauch wrote: > > > > Congrats, Apoorv! > > > > On Thu, Apr 10, 2025 at 11:30 AM Lianet M. wrote: > > > >> Congrats Apoorv! Well deserved! > >>

Re: [DISCUSS] Apache Kafka 3.9.1 release

> https://github.com/apache/kafka/pull/19263 > any review to push it through would be welcome - thanks > > > On Fri, 28 Mar 2025 at 02:45, Luke Chen wrote: > > > > Hi Stig, > > > > Let's make sure we have consensus from the community for this backporting > &

[jira] [Resolved] (KAFKA-18984) failed to reset interval.ms by using kafka-client-metrics.sh

[ https://issues.apache.org/jira/browse/KAFKA-18984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18984. --- Resolution: Fixed > failed to reset interval.ms by using kafka-client-metrics

Re: KRaft controller disaster recovery

t; docker images and so on to do this. > > best, > Colin > > > On Mon, Apr 7, 2025, at 14:26, José Armando García Sancio wrote: > > Thanks Luke. > > > > On Thu, Apr 3, 2025 at 7:14 AM Luke Chen wrote: > >> In addition to the approaches you provided, m

Re: KRaft controller disaster recovery

Hi Juha, Thanks for bringing this. I agree having a way to recover from this "majority of controller nodes down" issue is valuable, even though this is rare. In addition to the approaches you provided, maybe we can have a way to "force" KRaft to honor "controller.quorum.voters" config, instead of

Re: Kafka support for Java 24+

> > >> is right, because I don't know which JDKs are available on Jenkins, and > >> without admin access, I don't believe I can get a list anywhere. > >> > >> If someone has full access to Jenkins, maybe they can take a look? > >> > &g

[jira] [Resolved] (KAFKA-18923) resource leak in RSM fetchIndex inputStream

[ https://issues.apache.org/jira/browse/KAFKA-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18923. --- Resolution: Fixed > resource leak in RSM fetchIndex inputStr

Re: Kafka support for Java 24+

gt; >>> > > > > >>> Thank you Stig, Anton, Danish, Mateusz, Manfred, Monica, Istvan, > > > > Vincent, > > > > >>> Clement, Anshu, Foivos, Drakgoku, Severin, Ozan, Georgios, and > > > > Guillaume, > > > > >>> members

[jira] [Resolved] (KAFKA-15667) preCheck the invalid configuration for tiered storage replication factor

[ https://issues.apache.org/jira/browse/KAFKA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15667. --- Resolution: Invalid > preCheck the invalid configuration for tiered storage replication fac

Re: [DISCUSS] Apache Kafka 3.9.1 release

pen a new ticket for the backport, separate > > from the original for 4.0.0 at > > https://issues.apache.org/jira/browse/KAFKA-17078? > > > > Den tors. 27. mar. 2025 kl. 03.03 skrev Luke Chen : > > > >> Hi Matthias, > >> > >> Thanks for your sugge

Re: Kafka support for Java 24+

Hi Stig, Thanks for bringing this to us. I'm +1 for backporting to 3.9 branch since there's no workaround for Java 24. Thanks. Luke On Tue, Mar 18, 2025 at 1:14 AM Stig Rohde Døssing wrote: > Thanks Ismail, > > I've opened https://github.com/apache/kafka/pull/19221 just to get any > test > f

Re: Fetch Request performing remote KIP-405 reads gotcha

Hi Stanislav, Thanks for raising this! I totally forgot about it! For the documentation improvement, I think more is better. All you listed above can be done together. Also "remote.fetch.max.wait.ms" config is also a good place to add this missing info. Thanks. Luke On Thu, Mar 27, 2025 at 4:27

Re: [DISCUSS] Apache Kafka 3.9.1 release

; mid of April" might be a little bit too fuzzy. > > > -Matthias > > On 3/24/25 3:50 PM, Luke Chen wrote: > > Hi all, > > > > I'll be the release manager for a bug fix release of the 3.9.1 release. > > > > Here you have the release plan for 3.9.1:

Re: [DISCUSS] KIP-1110: Metadata of sensitive configuration

Bump this thread to seek for feedback or suggestions. Thanks. Luke On Tue, Nov 26, 2024 at 4:30 PM Luke Chen wrote: > Hi Federico, > > Thanks for the comment. > I've updated the field name to LastUpdateTimestampMs. > > Luke > > On Tue, Nov 19, 2024 at 7:21 PM Fede

[DISCUSS] Apache Kafka 3.9.1 release

Hi all, I'll be the release manager for a bug fix release of the 3.9.1 release. Here you have the release plan for 3.9.1: https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.9.1 I'm aiming to have the first release candidate *by the mid of April* since there are still some opening i

Re: [ANNOUNCE] Apache Kafka 4.0.0

Thanks David for running the release, and all the contributors! Luke On Wed, Mar 19, 2025 at 4:03 AM Greg Harris wrote: > This is fantastic news! > > Thank you to everyone who contributed to 3.x to get ready for this > milestone, and everyone who worked on 4.0 to get it over the line. > > Thank

Re: 3.9.1 bug-fix release?

Hi Matthias, I can work on the 3.9.1 release if there are no objections. Thanks. Luke On Thu, Mar 13, 2025 at 3:06 AM Matthias J. Sax wrote: > Hello, > > with 4.0.0 slowly winding down, it seems it's time to get the ball > rolling for 3.9.1 bug-fix release? > > It would be great if somebody wo

Re: [VOTE] 4.0.0 RC4

a 3.9 broker > > +1 (non-binding) > > Best Regards > Jiunn-Yang > > > David Jacot 於 2025年3月15日 下午5:53 寫道： > > > > Hi Luke, > > > > The link works for me. Have you tried from a different browser or a > > different device? > > > > Bes

[jira] [Created] (KAFKA-18979) kraft.version wrongly upgraded to 1 after upgrading to v4.0

Luke Chen created KAFKA-18979: - Summary: kraft.version wrongly upgraded to 1 after upgrading to v4.0 Key: KAFKA-18979 URL: https://issues.apache.org/jira/browse/KAFKA-18979 Project: Kafka Issue

Re: [VOTE] 4.0.0 RC4

hat only me seeing this? Thanks. Luke On Sat, Mar 15, 2025 at 4:09 PM Luke Chen wrote: > Hi David, > > From @Jusine > > I was able to run the tests I wanted and I found when I use the upgrade > --release version flag, we are still saying we are upgrading kraft.version > to 1

Re: [VOTE] 4.0.0 RC4

Hi David, >From @Jusine > I was able to run the tests I wanted and I found when I use the upgrade --release version flag, we are still saying we are upgrading kraft.version to 1 even though we do not. I don't think this is necessarily a blocker, but wanted to call it out. I've opened KAFKA-18992

[jira] [Created] (KAFKA-18993) 2 "Notable changes in 4.0.0" section for v4.0

Luke Chen created KAFKA-18993: - Summary: 2 "Notable changes in 4.0.0" section for v4.0 Key: KAFKA-18993 URL: https://issues.apache.org/jira/browse/KAFKA-18993 Project: Kafka

[jira] [Created] (KAFKA-18992) kafka-features.sh wronly output `upgraded kraft.version to 1`

Luke Chen created KAFKA-18992: - Summary: kafka-features.sh wronly output `upgraded kraft.version to 1` Key: KAFKA-18992 URL: https://issues.apache.org/jira/browse/KAFKA-18992 Project: Kafka

Re: [VOTE] 4.0.0 RC3

Hi José, Thanks for the fix. The change makes sense to me. Thanks. Luke On Fri, Mar 14, 2025 at 8:03 AM José Armando García Sancio < jsan...@confluent.io> wrote: > Hi Luke and David, > > I merged my fix to the trunk and cherry-picked it to the 4.0 branch. > It is not a full fix but it improves

Re: [VOTE] 4.0.0 RC3

tgz, I got some failures that recovered > as > > > > TengYao > > > > > >> > > >>> mentioned, but I noticed a persistent failure locally, > in > > > > > >> > > >>> MetadataSchemaCheckerToolTest.testVerifyEvolutionGit. > This

[jira] [Created] (KAFKA-18964) Allow to set weights for controller nodes for leader election

Luke Chen created KAFKA-18964: - Summary: Allow to set weights for controller nodes for leader election Key: KAFKA-18964 URL: https://issues.apache.org/jira/browse/KAFKA-18964 Project: Kafka

Re: [VOTE] 4.0.0 RC3

as a blocker. However, there is already a PR inflight to improve it. > > Overall, system tests look pretty good to me. Do you agree? > > Best, > David > > On Tue, Mar 11, 2025 at 8:53 AM Luke Chen wrote: > > > Hi David, > > > > I saw there are some failed

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

review: > > > - Adding the APIs and support for producer and consumer plugins: > > > https://github.com/apache/kafka/pull/17511 > > > - Adding support for Connect plugins: > https://github.com/apache/kafka/pull/17804 > > > > > > Thanks, > > &g

Re: [VOTE] 4.0.0 RC3

Hi David, I saw there are some failed tests in system test , what does that mean for this R

[jira] [Created] (KAFKA-18930) KRaft MigrationEvent won't retry when failing to write data to ZK

Luke Chen created KAFKA-18930: - Summary: KRaft MigrationEvent won't retry when failing to write data to ZK Key: KAFKA-18930 URL: https://issues.apache.org/jira/browse/KAFKA-18930 Project:

[jira] [Created] (KAFKA-18923) resource leak in RSM fetchIndex inputStream

Luke Chen created KAFKA-18923: - Summary: resource leak in RSM fetchIndex inputStream Key: KAFKA-18923 URL: https://issues.apache.org/jira/browse/KAFKA-18923 Project: Kafka Issue Type: Bug

[jira] [Resolved] (KAFKA-18911) alterPartition gets stuck when getting out-of-date errors

[ https://issues.apache.org/jira/browse/KAFKA-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18911. --- Resolution: Invalid > alterPartition gets stuck when getting out-of-date err

[jira] [Created] (KAFKA-18911) alterPartition gets stuck when getting out-of-date errors

Luke Chen created KAFKA-18911: - Summary: alterPartition gets stuck when getting out-of-date errors Key: KAFKA-18911 URL: https://issues.apache.org/jira/browse/KAFKA-18911 Project: Kafka Issue

[jira] [Created] (KAFKA-18860) Clarify the KRaft missing feature

Luke Chen created KAFKA-18860: - Summary: Clarify the KRaft missing feature Key: KAFKA-18860 URL: https://issues.apache.org/jira/browse/KAFKA-18860 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-1134: Virtual Clusters in Kafka

; > > > really depends on the previous points. Then there is an argument > > > > whether > > > > > > ACLs are transparent or not. In the current design, they're > > > > transparent, > > > > > > which means that being part of

Re: [DISCUSS] KIP-1134: Virtual Clusters in Kafka

Hi Viktor, This is a very interesting KIP! It keeps most of the existing concept and design to create a new virtual cluster concept. Nice! Some questions: 1. ACL after migrating to VC: Suppose we have an ACL entry to user Bob, after Bob joining VC1, how does the ACL migration work? Will Bob still

[jira] [Resolved] (KAFKA-18805) Consumer Heartbeat closed change should be locked

[ https://issues.apache.org/jira/browse/KAFKA-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18805. --- Fix Version/s: 4.1.0 Resolution: Fixed > Consumer Heartbeat closed change should be loc

[jira] [Created] (KAFKA-18806) Resource leak in MemoryRecordsBuilder in RecordsUtil

Luke Chen created KAFKA-18806: - Summary: Resource leak in MemoryRecordsBuilder in RecordsUtil Key: KAFKA-18806 URL: https://issues.apache.org/jira/browse/KAFKA-18806 Project: Kafka Issue Type

[jira] [Created] (KAFKA-18805) Consumer Heartbeat closed change should be locked

Luke Chen created KAFKA-18805: - Summary: Consumer Heartbeat closed change should be locked Key: KAFKA-18805 URL: https://issues.apache.org/jira/browse/KAFKA-18805 Project: Kafka Issue Type: Bug

[jira] [Resolved] (KAFKA-18771) Flaky test KRaftClusterTest .testDescribeQuorumRequestToControllers

[ https://issues.apache.org/jira/browse/KAFKA-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18771. --- Fix Version/s: 4.0.0 Resolution: Fixed > Flaky test KRaftClusterT

[jira] [Resolved] (KAFKA-16372) max.block.ms behavior inconsistency with javadoc and the config description

[ https://issues.apache.org/jira/browse/KAFKA-16372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16372. --- Fix Version/s: 4.0.0 Resolution: Fixed > max.block.ms behavior inconsistency with java

Re: [DISCUSS] Apache Kafka 4.0.0 release

> >>>> back > > > > >>>>>> in > > > > >>>>>>> case we cannot make it or if we discover issues? > > > > >>>>>>> > > > > >>>>>>> Andrew/Divij: > > > > >>>>>>> Thanks! > > > &g

[jira] [Resolved] (KAFKA-18451) Flaky RemoteLogManagerTest#testRLMOpsWhenMetadataIsNotReady

[ https://issues.apache.org/jira/browse/KAFKA-18451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-18451. --- Fix Version/s: 4.0.0 Resolution: Fixed > Flaky RemoteLogManagerT

Re: Proposal for converting Apache Kafka Documentation and Website source to Markdown

Hi Harish, Thanks for the proposal. Sounds good to me. But please write a formal KIP for this idea. Thanks. Luke On Tue, Jan 14, 2025 at 10:22 PM David Arthur wrote: > Harish, thanks for putting this together! I ha

[jira] [Created] (KAFKA-18455) Add logs or metrics when connection count exceeds

Luke Chen created KAFKA-18455: - Summary: Add logs or metrics when connection count exceeds Key: KAFKA-18455 URL: https://issues.apache.org/jira/browse/KAFKA-18455 Project: Kafka Issue Type

Re: [DISCUSS] KIP-1123: Rack-aware partitioning for Kafka Producer

Hi Ivan, Thanks for the KIP! This is a great improvement from the cost and latency perspective! Some comments: 1. In the description of `partitioner.rack.aware` config, it'd be better to make it clear that this setting has no effect if a custom partitioner is used. 2. "Select the next partition

Re: [DISCUSS] Java 23 Support for 3.9.x

Hi Greg, Thanks for raising this. My main question is: Is Kafka v3.9 a LTS release? It seems many people think it is, but I never see this kind of discussion anywhere. In Kafka, IIRC, we have never had this kind of LTS release before. We need to clearly define the meaning of LTS release for v3.9.x

Re: [ANNOUNCE] Apache Kafka 3.7.2

in use at large and small companies worldwide, > including > > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, > Rabobank, > > > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > > > > > A big thank you for the following

[jira] [Created] (KAFKA-18230) admin client will fail immediately when the active controller changed

Luke Chen created KAFKA-18230: - Summary: admin client will fail immediately when the active controller changed Key: KAFKA-18230 URL: https://issues.apache.org/jira/browse/KAFKA-18230 Project: Kafka

[jira] [Resolved] (KAFKA-17094) Make it possible to list registered KRaft nodes in order to know which nodes should be unregistered

[ https://issues.apache.org/jira/browse/KAFKA-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-17094. --- Resolution: Fixed > Make it possible to list registered KRaft nodes in order to know which no

Re: [DISCUSS] Apache Kafka 4.0.0 release

t; > > > > > > > On Tue, Nov 26, 2024 at 9:37 AM TengYao Chi wrote: > > > > > Hello David, > > > > > > I’ve just added KIP-653, which aims to upgrade Log4j to Log4j2. > > > This KIP was accepted a long time ago, and the PR is now almost

Re: [VOTE] 3.7.2 RC1

Hi Matthias, Thanks for running the release. I did the following steps: - Validated the checksums and keys - Browsed the javadocs - Browsed the documentation - Ran all the quickstart using KRaft with kafka_2.13-3.7.2.tgz - Ran basic quickstart using 3.7.2-rc1 docker image +1 bind

Re: [ANNOUNCE] New committer: Andrew Schofield

Congratulations! Well deserved! Luke On Tue, Dec 3, 2024 at 5:55 AM Matthias J. Sax wrote: > Congrats! > > On 12/2/24 12:02 PM, Kirk True wrote: > > Congratulations Andrew Well-deserved. > > > > On Mon, Dec 2, 2024, at 6:13 AM, David Arthur wrote: > >> The Project Management Committee (PMC)

[jira] [Created] (KAFKA-18131) Improve logs for voters

Luke Chen created KAFKA-18131: - Summary: Improve logs for voters Key: KAFKA-18131 URL: https://issues.apache.org/jira/browse/KAFKA-18131 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-1030: Change constraints and default values for various configurations

Hi Divij, +1 (binding) from me. Thanks for the KIP! Luke On Thu, Nov 28, 2024 at 12:21 AM Divij Vaidya wrote: > Corresponding discussion thread: > https://lists.apache.org/thread/kxw9kxn49kt30kwckzwp8vq3k7kmrz9m > > Please vote if you agree with proceeding ahead with the proposal. Votes > from

Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hi Divij and Jun, Thanks for your comments. I'm good we put the default value of num.recovery.threads.per.data.dir to 2 since there are many factors that need to be considered. And James, good point of min.insync.replicas validation. If it's complicated or will confuse users, I'd propose we leave

Re: [DISCUSS] KIP-1110: Metadata of sensitive configuration

r only > needs to know if the value changed, not when it changed (in that case > clock drift could be a problem). > > Should we name the new DescribeConfigs field as > "LastUpdateTimestampMs" instead of "LastUpdateTimestamp"? > > On Fri, Nov 15, 2024 at 8:37 A

Re: [DISCUSS] Apache Kafka 4.0.0 release

> > > > David > > > > > > > > On Wed, Nov 20, 2024 at 12:02 PM David Jacot > wrote: > > > > > > > >> Hi Divij, > > > >> > > > >> I think that we can make a small exception for KIP-1030 but we &g

[jira] [Resolved] (KAFKA-17953) Docker image fails when setting env vars

[ https://issues.apache.org/jira/browse/KAFKA-17953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-17953. --- Resolution: Not A Problem Improving doc in this PR: https://github.com/apache/kafka/pull/17894

Re: [DISCUSS] Apache Kafka 4.0.0 release

0th November 2024, > > one week from today. > > > > Best, > > David > > > > On Thu, Nov 7, 2024 at 5:09 AM Luke Chen wrote: > > > > > Hi Kamal, > > > > > > Just added them in to the release plan. > > > > > > Tha

Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hi Divij, One comment: For the default value of "num.recovery.threads.per.data.dir", we change from 1 to 2. Could we change the value based on the CPU core number? Ex: default value = CPU core num / num of log dirs. WDYT? Luke On Wed, Nov 20, 2024 at 5:53 PM Tommi Vainikainen wrote: > Hi, >

Re: [DISCUSS] Apache Kafka 4.0.0 release

review, it would be great to include it in 4.0.0. > > > > [1] > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1073%3A+Return+fenced+brokers+in+DescribeCluster+response > > [2] https://lists.apache.org/thread/zm5kk9st54rj6t5pw9v1dnxhk64mho4m > > [3] https:/

Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hi Divij, Thanks for the KIP! The change makes sense to me. For what Greg pointed out, some configs might be set to a small value for tests on purpose, it is used in Kafka now, for example: [1] [2]. But IMO, after removing ZK and related tests, the test speed should improve a lot. It should be fi

  1   2   3   4   5   6   7   8   9   10   >