Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-05-27 Thread Colin McCabe
On Tue, May 27, 2025, at 09:28, José Armando García Sancio wrote: > Hi Colin, > > On Fri, May 2, 2025 at 6:52 PM Colin McCabe wrote: >> >> On Fri, May 2, 2025, at 09:54, José Armando García Sancio wrote: >> That seems pretty clear? It's also already implemented

Re: [VOTE] KIP-1150: Diskless Topics

2025-05-19 Thread Colin McCabe
:50, Colin McCabe wrote: > Hi Anatolii, > > I think we haven't had time to fully review this KIP yet. I see open > questions on the DISCUSS thread from Jun, Ivan, and a few others. > > There are still a lot of unknowns here. For example, what relationship > does this have

[jira] [Created] (KAFKA-19294) Fix BrokerLifecycleManager RPC timeouts

2025-05-16 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-19294: Summary: Fix BrokerLifecycleManager RPC timeouts Key: KAFKA-19294 URL: https://issues.apache.org/jira/browse/KAFKA-19294 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-15 Thread Colin McCabe
Hi Xinyu Zhou, Thanks for the KIP. It's good to see more people contributing to the community. I think this is your first KIP, so please forgive me for giving some negative feedback. KIPs need to be written in a vendor-neutral manner, for the whole community. So please do not do things like be

Re: [DISCUSS] KIP-1150 Diskless Topics

2025-05-13 Thread Colin McCabe
Hi Josep, Thanks for the KIP. I think there's a bit of confusion in the motivation and naming here. As Jun said, what's being proposed here is not truly "diskless" -- we're still storing a fair amount of metadata on local disks. The proposal talks about "Unification/Relationship with Tiered St

Re: [VOTE] KIP-1150: Diskless Topics

2025-05-13 Thread Colin McCabe
Hi Anatolii, I think we haven't had time to fully review this KIP yet. I see open questions on the DISCUSS thread from Jun, Ivan, and a few others. There are still a lot of unknowns here. For example, what relationship does this have with the KIP-1176 proposal (Tiered Storage for Active Log Seg

Re: [VOTE] KIP-1166: Improve high-watermark replication

2025-05-07 Thread Colin McCabe
Hi Jose, Thanks for the KIP. +1 (binding) best, Colin On Wed, May 7, 2025, at 11:02, David Arthur wrote: > Thanks José, > > +1 binding > > -David A > > On Mon, May 5, 2025 at 4:47 PM Alyssa Huang > wrote: > >> Hey Jose, >> >> KIP looks good to me as well! >> >> On Mon, May 5, 2025 at 10:08 AM

[VOTE] KIP-1155: Metadata Version Downgrades

2025-05-07 Thread Colin McCabe
Hi all, I'd like to start a vote on KIP-1155: Metadata Version Downgrades. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1155%3A+Metadata+Version+Downgrades The discussion thread is here: https://lists.apache.org/thread/cwhkf26np3kjt33yhlsvp3k3jf7ofgf3 best, Colin

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-05-02 Thread Colin McCabe
On Fri, May 2, 2025, at 09:54, José Armando García Sancio wrote: > Hi Colin, > > On Fri, Apr 25, 2025 at 5:42 PM Colin McCabe wrote: >> > The "initiating the downgrade" section states "It will also check that >> > the requested metadata version is

Re: [DISCUSS] KIP-1162: Redesign ClientQuotaCallback#updateClusterMetadata

2025-04-30 Thread Colin McCabe
ically create the object >> > > based on the new interface first, and then try the old interface for >> > > users >> > > who are still using it. Additionally, a callback implementing the old >> > > interface can be wrapped to conform to the new interf

Re: [DISCUSS] KIP-1166: Improve high-watermark replication

2025-04-30 Thread Colin McCabe
García Sancio wrote: > Hi Colin, > > On Mon, Apr 28, 2025 at 8:43 PM Colin McCabe wrote: >> Maybe I missed something, but even after reading the discussion below, I >> still don't understand the rationale for separating the "RPC version too >> old" and &qu

Re: [DISCUSS] KIP-1119: Add support for SSL hot reload

2025-04-29 Thread Colin McCabe
Gaurav brings up a good point. However, we can work around that bug by doing polling every 5 or 10 minutes in addition to using WatchService. best, Colin On Wed, Apr 23, 2025, at 00:39, Gaurav Narula wrote: > Hi Moncef, > > Have you tried experimenting the behaviour of JDK's WatchService with >

Re: [VOTE] KIP-1141: Simplifying MetadataQuorumCommand by Leveraging Admin API for Controller Management

2025-04-28 Thread Colin McCabe
directory. This should be documented a bit more prominently if we want people to use this mechanism. best, Colin On Mon, Apr 28, 2025, at 13:21, Colin McCabe wrote: > Hi Kuan Po Tseng, > > It is not possible to add a new controller purely by ID. You need > controller ID, directory ID, an

Re: [DISCUSS] KIP-1166: Improve high-watermark replication

2025-04-28 Thread Colin McCabe
Hi José, Thank you for this much-needed improvement to metadata propagation. Maybe I missed something, but even after reading the discussion below, I still don't understand the rationale for separating the "RPC version too old" and "high watermark not known" cases. Is the idea that separating t

Re: [DISCUSS] - KIP-1064: Upgrade slf4j to 2.x

2025-04-28 Thread Colin McCabe
Hi, Can you clarify the discussion about compatibility? I wasn't able to figure out whether the change is compatible or not based on reading through the KIP. If it is not compatible, what specific actions are needed from users? best, Colin On Tue, Mar 25, 2025, at 05:10, TengYao Chi wrote: > H

Re: [VOTE] KIP-1148: Remove log.cleaner.enable configuration

2025-04-28 Thread Colin McCabe
+1 (binding) thanks, Colin On Thu, Apr 10, 2025, at 05:22, Jhen-Yung Hsu wrote: > Thanks for the KIP! > > +1 (non-binding) > > On 2025/04/10 07:51:53 TengYao Chi wrote: >> Hello everyone, >> >> I would like to call a vote on KIP-1148 >> . >> >> Here

Re: [ANNOUNCE] New committer: Apoorv Mittal

2025-04-28 Thread Colin McCabe
Congrats, Apoorv! best, Colin On Sat, Apr 12, 2025, at 06:58, Mingdao wrote: > Congratulations, Apoorv! > > On Sat, Apr 12, 2025 at 9:39 PM Kirk True wrote: > >> Congratulations Apoorv. Justly deserved! >> >> On Sat, Apr 12, 2025, at 12:22 AM, Kamal Chandraprakash wrote: >> > Congratulations, Ap

Re: [ANNOUNCE] New committer: TengYao Chi

2025-04-28 Thread Colin McCabe
Congratulations! best, Colin On Sat, Apr 12, 2025, at 06:39, Kirk True wrote: > Congratulations TengYao! > > On Sat, Apr 12, 2025, at 12:21 AM, Kamal Chandraprakash wrote: >> Congrats, TengYao! Well deserved!! >> >> On Sat, Apr 12, 2025 at 7:19 AM Guozhang Wang >> wrote: >> >> > Congratulation

Re: [DISCUSS] KIP-1162: Redesign ClientQuotaCallback#updateClusterMetadata

2025-04-28 Thread Colin McCabe
Hi Kuan Po Tseng, Thank you for starting this discussion. This would be a very nice improvement. It seems that the current scheme requires everyone writing an implementation of ClientQuotaCallackHandler to implement the old updateClusterMetadata function that takes a Cluster. That seems less th

Re: [VOTE] KIP-1141: Simplifying MetadataQuorumCommand by Leveraging Admin API for Controller Management

2025-04-28 Thread Colin McCabe
Hi Kuan Po Tseng, It is not possible to add a new controller purely by ID. You need controller ID, directory ID, and controller endpoints. These cannot be obtained through describeMetadataQuorum since that only describes the existing quorum, not what you would like the new one to be. Also, obv

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-26 Thread Colin McCabe
a Kafka Streams application has to be restarted, that is regrettable, but it's definitely something a user would be willing to do if the alternative is keeping the cluster down for everyone. I've helped many users downgrade their clusters when they experienced a bug. The usefulness o

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-25 Thread Colin McCabe
On Fri, Apr 25, 2025, at 14:26, José Armando García Sancio wrote: > Hi Colin, > > Thanks for the KIP. I have a few questions and comments. > > The "initiating the downgrade" section states "It will also check that > the requested metadata version is compatible with all the other > KIP-584 features

[jira] [Created] (KAFKA-19192) Old bootstrap.metadata files cause problems with post-KAFKA-18601 servers

2025-04-23 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-19192: Summary: Old bootstrap.metadata files cause problems with post-KAFKA-18601 servers Key: KAFKA-19192 URL: https://issues.apache.org/jira/browse/KAFKA-19192 Project

Re: [VOTE] KIP-1131: Improved controller-side monitoring of broker states

2025-04-22 Thread Colin McCabe
+1 (binding) thanks, Colin On Tue, Apr 15, 2025, at 02:25, Federico Valeri wrote: > +1 non binding > > Thanks > > On Tue, Mar 25, 2025 at 11:29 PM Kevin Wu wrote: >> >> Hello all, >> >> I am manually bumping this thread. >> Any feedback or votes would be appreciated. >> >> Best regards, >> Kevin

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-22 Thread Colin McCabe
Jun > > On Mon, Apr 7, 2025 at 3:05 PM Colin McCabe wrote: > >> Hi all, >> >> I posted a new KIP about support metadata.version downgrades. Take a look >> here: >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1155%3A+Metadata+Version+Downgrades >> >> thanks, >> Colin >>

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-04-21 Thread Colin McCabe
García Sancio wrote: > Hi Colin and Kevin, > > On Mon, Apr 14, 2025 at 5:25 PM Colin McCabe wrote: >> How about something like this? >> 10 = fenced >> 20 = controlled shutdown >> 30 = active > > Very few users are going to know what these values mean. For ex

Re: Kafka upgrade guide from 2.8.2 to 3.5.1

2025-04-16 Thread Colin McCabe
On Wed, Apr 16, 2025, at 02:23, Kartik Dev wrote: > Hi team, > > I have two queries regarding the upgrade path mentioned in the link: > https://kafka.apache.org/35/documentation.html#upgrade > > 1. I want to upgrade my kafka cluster from 2.8.2 to 3.5.1. My 2.8.2 cluster > is running on JDK 8 > So

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-04-14 Thread Colin McCabe
Hi Kevin, The values for kafka.controller:type=KafkaController,name=BrokerRegistrationState seem a bit unintuitive. Using 0 for active might be confusing to systems that treat metrics that aren't present as 0. Or to people just scanning the graph visually. How about something like this? 10

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-14 Thread Colin McCabe
ic metadata version. However, for things like the group coordinator, it wouldn't really make sense to tie to a metadata.version, since it's so separate. Hope this helps. best, Colin > > Thanks, > Andrew > > > From: Colin McC

Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer

2025-04-14 Thread Colin McCabe
I would suggest adding a configuration key which controls whether the null values are added. That configuration key can default to true in 4.x and false in 5.x. This will give people a chance to test the new behavior before 5.x. best, Colin On Fri, Mar 14, 2025, at 04:30, 黃竣陽 wrote: > Hello eve

[jira] [Created] (KAFKA-19130) Do not add fenced brokers to BrokerRegistrationTracker on startup

2025-04-11 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-19130: Summary: Do not add fenced brokers to BrokerRegistrationTracker on startup Key: KAFKA-19130 URL: https://issues.apache.org/jira/browse/KAFKA-19130 Project: Kafka

KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-10 Thread Colin McCabe
Hi all, I posted a new KIP about support metadata.version downgrades. Take a look here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1155%3A+Metadata+Version+Downgrades thanks, Colin

Re: [DISCUSS] Apache Kafka 4.1.0 release

2025-04-10 Thread Colin McCabe
+1. Thanks, Mickael! Colin On Wed, Mar 19, 2025, at 04:58, Mickael Maison wrote: > Hi, > > I'd like to volunteer to be the release manager for the Apache Kafka > 4.1.0 release. > > If there are no objections, I'll start building the release plan in > the next few days. > > Thanks, > Mickael

Re: kafka 4.0.0 running on java 11

2025-04-10 Thread Colin McCabe
No. Kafka 4.0 only supports Java 17 and above. Note that if you're still using JDK11, you probably are no longer receiving security updates. All of the major vendors have declared it EOL. So you are probably out of compliance with security guidelines. [Some vendors offer extended support for a

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-08 Thread Colin McCabe
data.version to 3.9-IV0, will the controller check TransactionVersion >> and reject >> the request? >> >> PY2: After users downgrade metadata.version, can they also restart server >> with old binary >> version? If they can, we need to add related end-to-

Re: KRaft controller disaster recovery

2025-04-08 Thread Colin McCabe
ole or something), since I DON'T want the folks making > > docker images and so on to do this. > > Sounds like a good idea! > It would be good to understand the scenario we're trying to solve a bit more. I'm thinking it's something like "no controller node

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

2025-04-08 Thread Colin McCabe
. best, Colin > > Thanks, > PoAn > >> On Apr 8, 2025, at 6:04 AM, Colin McCabe wrote: >> >> Hi all, >> >> I posted a new KIP about support metadata.version downgrades. Take a look >> here: >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1155%3A+Metadata+Version+Downgrades >> >> thanks, >> Colin

Re: KRaft controller disaster recovery

2025-04-07 Thread Colin McCabe
Hi José, I think you make a valid point that our guarantees here are not actually different from zookeeper. In both systems, if you lose quorum, you will probably lose some data. Of course, how much data you lose depends on luck. If the last node standing was the active controller / zookeeper,

Re: Unexpected UNCLEAN leader election behaviour in KRaft vs. Zookeeper

2025-04-07 Thread Colin McCabe
Did you have unclean leader election enabled here? best, Colin On Mon, Apr 7, 2025, at 11:49, Julian Bergner wrote: > Hi, > We primarily operate a Kafka cluster consisting of 3 brokers (IDs: 4, > 5, 6) running Kraft in version 3.9.0. However, we tested the same > scenario in a Zookeeper-managed

[jira] [Created] (KAFKA-19104) Support metadata version downgrade in Kafka

2025-04-07 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-19104: Summary: Support metadata version downgrade in Kafka Key: KAFKA-19104 URL: https://issues.apache.org/jira/browse/KAFKA-19104 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-03-05 Thread Colin McCabe
On Thu, Feb 27, 2025, at 12:19, Kevin Wu wrote: >> >> I guess my concern is that the time-based metrics would reset to 0 on >> every failover (if I understand the proposed implementation correctly). >> That seems likely to create confusion. > > Yeah that makes sense to me. I'm fine with moving towa

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-02-25 Thread Colin McCabe
On Tue, Feb 25, 2025, at 14:40, Colin McCabe wrote: > On Tue, Feb 25, 2025, at 14:12, Kevin Wu wrote: >> Hey Colin, >> >> Thanks for the review. >> >> Regarding the metrics that reflect times: my initial thinking was to indeed >> have these be "soft st

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-02-25 Thread Colin McCabe
On Tue, Feb 25, 2025, at 14:12, Kevin Wu wrote: > Hey Colin, > > Thanks for the review. > > Regarding the metrics that reflect times: my initial thinking was to indeed > have these be "soft state", which would be reset when a controller failover > happens. I'm not sure if it's a big issue if these

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

2025-02-19 Thread Colin McCabe
Hi Kevin, Thanks for the KIP. I notice that you have some metrics that reflect times here, such as LongestPendingStartupTimeMs, LongestPendingControlledShudownTimeMs, etc. I think this may be difficult to do with complete accuracy because we don't include times in the metadata log events for r

[jira] [Resolved] (KAFKA-18442) Downgrades to 3.3.2 will fail from versions 3.6.0 and above when JBOD is enabled

2025-01-29 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-18442. -- Resolution: Won't Fix So firstly, this is a real bug, which we cannot fix since 3.3

Re: [DISCUSS] Apache Kafka 4.0.0 release

2025-01-27 Thread Colin McCabe
Thu, Jan 23, 2025 at 1:07 PM Justine Olshan >> > >> > wrote: >> > >> > > Hey Colin -- yes, we have a blocker to bump the stable metadata >> versions >> > > https://issues.apache.org/jira/browse/KAFKA-18611. I can open that >> today

Re: [DISCUSS] Apache Kafka 4.0.0 release

2025-01-23 Thread Colin McCabe
the feature freeze but before the >> > >>> code >> > >>>>> freeze. >> > >>>>> >> > >>>>> Hi Chia-Ping, >> > >>>>> Let me take a look at KAFKA-18225. >> > >>>>> >>

[jira] [Created] (KAFKA-18635) Re-enable the unclean shutdown detection when in ELR mode

2025-01-23 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18635: Summary: Re-enable the unclean shutdown detection when in ELR mode Key: KAFKA-18635 URL: https://issues.apache.org/jira/browse/KAFKA-18635 Project: Kafka

[jira] [Created] (KAFKA-18634) Fix ELR metadata version issues

2025-01-23 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18634: Summary: Fix ELR metadata version issues Key: KAFKA-18634 URL: https://issues.apache.org/jira/browse/KAFKA-18634 Project: Kafka Issue Type: Bug

[jira] [Resolved] (KAFKA-17972) Default all JUnit tests to KRaft

2025-01-15 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-17972. -- Resolution: Fixed All tests are now KRaft > Default all JUnit tests to KR

[jira] [Created] (KAFKA-18459) Use ZooKeeper 3.4 in ducktape in order to support AK 2.1

2025-01-09 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18459: Summary: Use ZooKeeper 3.4 in ducktape in order to support AK 2.1 Key: KAFKA-18459 URL: https://issues.apache.org/jira/browse/KAFKA-18459 Project: Kafka

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-12-10 Thread Colin McCabe
t; > > >>>>>> Hi Divij, >> > > > > > > >>>>>> >> > > > > > > >>>>>> I agree we _should_ include it into v4.0.0 to make the >> > kafka >> > > > > default >> > > > > > > >

[jira] [Resolved] (KAFKA-18106) Brokers should be removed from the ISR if possible on unclean restart

2024-12-09 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-18106. -- Resolution: Fixed > Brokers should be removed from the ISR if possible on unclean rest

[jira] [Resolved] (KAFKA-18001) KafkaNetworkChannel missing UpdateRaftVoterRequestData logic

2024-12-03 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-18001. -- Assignee: Alyssa Huang Resolution: Fixed > KafkaNetworkChannel miss

[jira] [Created] (KAFKA-18147) kraft version of standalone controller should be 1 not 0

2024-12-03 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18147: Summary: kraft version of standalone controller should be 1 not 0 Key: KAFKA-18147 URL: https://issues.apache.org/jira/browse/KAFKA-18147 Project: Kafka

[jira] [Created] (KAFKA-18146) tests/kafkatest/tests/core/upgrade_test.py needs to be re-added as KRaft

2024-12-03 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18146: Summary: tests/kafkatest/tests/core/upgrade_test.py needs to be re-added as KRaft Key: KAFKA-18146 URL: https://issues.apache.org/jira/browse/KAFKA-18146 Project

[jira] [Created] (KAFKA-18138) The controller must add all extant brokers to BrokerHeartbeatTracker when activating

2024-12-02 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18138: Summary: The controller must add all extant brokers to BrokerHeartbeatTracker when activating Key: KAFKA-18138 URL: https://issues.apache.org/jira/browse/KAFKA-18138

[jira] [Created] (KAFKA-18108) De-flake DynamicBrokerReconfigurationTest.testUncleanLeaderElectionEnable

2024-11-27 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18108: Summary: De-flake DynamicBrokerReconfigurationTest.testUncleanLeaderElectionEnable Key: KAFKA-18108 URL: https://issues.apache.org/jira/browse/KAFKA-18108 Project

[jira] [Created] (KAFKA-18106) Brokers should be removed from the ISR if possible on unclean restart

2024-11-27 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18106: Summary: Brokers should be removed from the ISR if possible on unclean restart Key: KAFKA-18106 URL: https://issues.apache.org/jira/browse/KAFKA-18106 Project: Kafka

[jira] [Resolved] (KAFKA-18051) Disallow creating ACLs with principals that do not contain a colon

2024-11-22 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-18051. -- Resolution: Fixed > Disallow creating ACLs with principals that do not contain a co

[jira] [Created] (KAFKA-18070) Update kafka-metadata-quorum.sh output in docs to match post-KIP-853 appearance

2024-11-22 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18070: Summary: Update kafka-metadata-quorum.sh output in docs to match post-KIP-853 appearance Key: KAFKA-18070 URL: https://issues.apache.org/jira/browse/KAFKA-18070

[jira] [Resolved] (KAFKA-18063) SnapshotRegistry should not leak memory

2024-11-21 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-18063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-18063. -- Resolution: Fixed > SnapshotRegistry should not leak mem

[jira] [Created] (KAFKA-18063) SnapshotRegistry should not leak memory

2024-11-21 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18063: Summary: SnapshotRegistry should not leak memory Key: KAFKA-18063 URL: https://issues.apache.org/jira/browse/KAFKA-18063 Project: Kafka Issue Type: Bug

Re: [ANNOUNCE] Apache Kafka 3.9.0

2024-11-20 Thread Colin McCabe
ka-packages. >> > > >> > > Best, >> > > Bruno >> > > >> > > On 12.11.24 12:16, Chia-Ping Tsai wrote: >> > > > hi Colin >> > > > >> > > > 3.9.1 is nonexistent in the >> &

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

2024-11-20 Thread Colin McCabe
Hi Josep, If the level numbers are not to be used, then let's remove them from the KIP. (Also, I guess this is water under the bridge now, but I disagree that they were ever useful in the discussion; to me, they just added confusion) I also agree with Ismael that API stability should be up to

[jira] [Created] (KAFKA-18051) Disallow creating ACLs with principals that do not contain a colon

2024-11-20 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-18051: Summary: Disallow creating ACLs with principals that do not contain a colon Key: KAFKA-18051 URL: https://issues.apache.org/jira/browse/KAFKA-18051 Project: Kafka

[jira] [Resolved] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16657. -- Resolution: Won't Fix ZK mode is deprecated and will be removed in the upcoming 4.0 re

[jira] [Resolved] (KAFKA-14327) Unify KRaft snapshot generation between broker and controller

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14327. -- Fix Version/s: 3.4.0 Resolution: Fixed > Unify KRaft snapshot generation betw

[jira] [Created] (KAFKA-17987) Remove assorted ZK-related files

2024-11-11 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-17987: Summary: Remove assorted ZK-related files Key: KAFKA-17987 URL: https://issues.apache.org/jira/browse/KAFKA-17987 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-12733) KRaft: always bump leader epoch when changing the ISR for a controlled shutdown

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12733. -- Resolution: Won't Fix > KRaft: always bump leader epoch when changing the I

[jira] [Resolved] (KAFKA-13083) In KRaft mode, ISR is set incorrectly when creating a manual partition assignment with fenced broker(s)

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13083. -- Fix Version/s: 3.0.0 Resolution: Fixed > In KRaft mode, ISR is set incorrectly w

[jira] [Resolved] (KAFKA-12886) Enable request forwarding by default

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12886. -- Resolution: Won't Fix At this point we won't be changing the behavior of ZK mode.

[jira] [Resolved] (KAFKA-9837) New RPC for notifying controller of failed replica

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-9837. - Fix Version/s: 3.7.0 Resolution: Fixed > New RPC for notifying controller of fai

[jira] [Resolved] (KAFKA-15430) Kafla create replca partition on controller node

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15430. -- Resolution: Invalid Replicas aren't created "on the controller." However, the

[jira] [Resolved] (KAFKA-13502) Support configuring BROKER_LOGGER on controller-only KRaft nodes

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13502. -- Fix Version/s: 3.7.0 Assignee: Colin McCabe Resolution: Fixed KIP-919 made it

[jira] [Resolved] (KAFKA-14290) Fix bugs that could block KRaft controlled shutdown indefinitely

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14290. -- Resolution: Fixed > Fix bugs that could block KRaft controlled shutdown indefinit

[jira] [Resolved] (KAFKA-13841) Fix a case where we were unable to place on fenced brokers In KRaft mode

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13841. -- Fix Version/s: 3.3.0 Assignee: Colin McCabe Resolution: Fixed > Fix a c

[jira] [Resolved] (KAFKA-13515) Fix KRaft config validation issues

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13515. -- Fix Version/s: 3.2.0 Assignee: Colin McCabe Resolution: Fixed > Fix KR

[jira] [Resolved] (KAFKA-13245) KIP 771: Change behavior of how controller metrics are exposed when using KRaft.

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13245. -- Resolution: Won't Fix We decided to have standby KRaft controllers expose their me

Re: [VOTE] KIP-866 ZooKeeper to KRaft Migration

2024-11-11 Thread Colin McCabe
ote: > Hey everyone, I'm going to close out the vote. There were three +1 > votes from PMC members and no committer or community votes. > > +1 PMC votes: > * Colin McCabe > * Jason Gustafson > * Jun Rao > > With that, the vote passes. Thanks to everyone who reviewed this

[jira] [Resolved] (KAFKA-15509) KRaft Controller doesn't consider brokers that are shutting down during topic creating

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15509. -- Resolution: Duplicate > KRaft Controller doesn't consider brokers that are shutt

[jira] [Resolved] (KAFKA-13142) KRaft brokers do not validate dynamic configs before forwarding them to controller

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13142. -- Fix Version/s: 3.2.0 Resolution: Fixed This was fixed in Kafka 3.2 > KRaft brokers

[jira] [Resolved] (KAFKA-15861) In Kraft mode, "ssl.keystore.key" private keys are accesible to all the controllers and brokers

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15861. -- Resolution: Won't Fix If you want to distributed keys separately, consider using a key

[jira] [Resolved] (KAFKA-12806) KRaft: Confusing leadership status exposed in metrics for controller without quorum

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12806. -- Fix Version/s: 3.0.0 Resolution: Fixed I think this was one of many metrics bugs we

[jira] [Resolved] (KAFKA-13835) Fix two bugs related to dynamic broker configs in KRaft

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13835. -- Fix Version/s: 3.2.0 Resolution: Fixed This was merged a long time ago, in Kafka 3.2

[jira] [Resolved] (KAFKA-14313) Kraft: immediately producing to new topics occasionally returns NOT_LEADER_FOR_PARTITION

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14313. -- Resolution: Won't Fix This isn't a bug. It results from metadata propagation d

[jira] [Resolved] (KAFKA-14312) Kraft + ProducerStateManager: produce requests to new partitions with a non-zero sequence number should be rejected

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14312. -- Resolution: Won't Fix Based on the discussion, this behavior isn't unique to KRaft,

[jira] [Resolved] (KAFKA-14129) KRaft must check manual assignments for createTopics are contiguous

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14129. -- Fix Version/s: 3.3.0 Resolution: Fixed This was implemented in Kafka 3.3 > KRaft m

[jira] [Resolved] (KAFKA-16370) offline rollback procedure from kraft mode to zookeeper mode.

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16370. -- Resolution: Won't Fix I don't think there are any plans to implement offlin

[jira] [Resolved] (KAFKA-14057) Support dynamic reconfiguration in KRaft remote controllers

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14057. -- Fix Version/s: 3.5.0 Resolution: Fixed This was fixed a long time ago, starting in 3.5

[jira] [Resolved] (KAFKA-14182) KRaft and ACL + GSSAPI

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14182. -- Resolution: Fixed This was fixed a long time ago, probably in 3.2 > KRaft and ACL + GSS

[jira] [Resolved] (KAFKA-14538) Implement KRaft metadata transactions

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14538. -- Fix Version/s: 3.7.0 Resolution: Fixed > Implement KRaft metadata transacti

[jira] [Resolved] (KAFKA-13827) Data Loss on moving to KRAFT mode from v3.1

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-13827. -- Fix Version/s: 3.0.0 Resolution: Won't Fix This related to the kraft support i

[jira] [Resolved] (KAFKA-15749) KRaft support in KafkaMetricReporterClusterIdTest

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15749. -- Resolution: Not A Problem > KRaft support in KafkaMetricReporterClusterIdT

[jira] [Resolved] (KAFKA-14077) KRaft should support recovery from failed disk

2024-11-11 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14077. -- Fix Version/s: 3.9.0 Resolution: Fixed KIP-853 shipped in Apache Kafka 3.9 > KR

[jira] [Created] (KAFKA-17986) Fix ConsumerRebootstrapTest and ProducerRebootstrapTest

2024-11-11 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-17986: Summary: Fix ConsumerRebootstrapTest and ProducerRebootstrapTest Key: KAFKA-17986 URL: https://issues.apache.org/jira/browse/KAFKA-17986 Project: Kafka

[jira] [Created] (KAFKA-17972) Annotate all tests that still require ZK

2024-11-08 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-17972: Summary: Annotate all tests that still require ZK Key: KAFKA-17972 URL: https://issues.apache.org/jira/browse/KAFKA-17972 Project: Kafka Issue Type: Sub

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

2024-11-07 Thread Colin McCabe
Hi Josep, Continuing the discussion from the other thread: I'm still not 100% convinced that we need a distinction between "in development" and "experimental." In both cases, end-users can't make use of the feature. So why not condense them into one step? With regard to "preview", I think "use

Re: [DISCUSS] Require KIPs to include "How to teach this section"

2024-11-07 Thread Colin McCabe
ll changes. >>> >>> I also support moving away from our current practice of pointing users to >>> KIPs for first- or second-level documentation, because they are written for >>> contributors & committers, not end users. >>> As part of enforcing th

[ANNOUNCE] Apache Kafka 3.9.0

2024-11-07 Thread Colin McCabe
.com, Wang Xiaoqing, Xiduo You, xijiu, Xuan-Zhang Gong, Yash Mayya, Zhengke Zhou We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/ Thank you! Regards, Colin McCabe Release Manager for Ap

  1   2   3   4   5   6   7   8   9   10   >