Re: [VOTE] KIP-979 Allow independently stop KRaft processes
Hi everyone, With +3 binding votes (and +2 non-binding), the vote passes. *KIP-979* Allow independently stop KRaft processes is Adopted! Thank you all for your reviews and votes. Regards, Hailey On Fri, Nov 17, 2023 at 12:43 PM Hailey Ni wrote: > Thanks Justine! > > On Fri, Nov 17, 2023 at 10:40 AM Justine Olshan > wrote: > >> Thanks Hailey for the update. +1 (binding) from me :) >> >> Justine >> >> On Thu, Nov 16, 2023 at 11:32 AM Hailey Ni >> wrote: >> >> > Hey Justine, >> > >> > Thank you very much for the review. >> > I've updated the KIP to add a log line stating that when both flags are >> > given, node-id will take precedence. >> > >> > Thanks, >> > Hailey >> > >> > On Wed, Nov 15, 2023 at 3:37 PM Justine Olshan >> > >> > wrote: >> > >> > > Hey Hailey, >> > > >> > > Thanks for the KIP. >> > > I wonder if it would be better to either not allow both flags or if we >> > > choose to have node take precedence, at least have a log line stating >> > such. >> > > >> > > Otherwise the KIP makes sense to me. >> > > >> > > Justine >> > > >> > > On Tue, Nov 14, 2023 at 10:17 AM Colin McCabe >> > wrote: >> > > >> > > > Thanks, Hailey. >> > > > >> > > > +1 (binding) >> > > > >> > > > Colin >> > > > >> > > > On Mon, Nov 13, 2023, at 15:13, Hailey Ni wrote: >> > > > > Hi Colin, >> > > > > >> > > > > Thank you for your review. I removed the "absolute path need to be >> > > > > provided" line from the KIP, and will modify the code to get the >> > > absolute >> > > > > path to the config files using some bash in the kafka-server-start >> > > file. >> > > > > For your second question, I've added a line in the KIP: "If both >> > > > parameters >> > > > > are provided, the value for node-id parameter will take >> precedence, >> > > i.e, >> > > > > the process with node id specified will be killed, no matter >> what's >> > the >> > > > > process role provided." >> > > > > >> > > > > What do you think? >> > > > > >> > > > > Thanks, >> > > > > Hailey >> > > > > >> > > > > On Thu, Nov 9, 2023 at 4:03 PM Colin McCabe >> > > wrote: >> > > > > >> > > > >> Hi Hailey, >> > > > >> >> > > > >> Thanks for the KIP. >> > > > >> >> > > > >> It feels clunky to have to pass an absolute path to the >> > configuration >> > > > file >> > > > >> when starting the broker or controller. I think we should >> consider >> > one >> > > > of >> > > > >> two alternate options: >> > > > >> >> > > > >> 1. Use JMXtool to examine the running kafka.Kafka processes. >> > > > >> I believe ID is available from kafka.server, type=app-info,id=1 >> > > > (replace 1 >> > > > >> with the actual ID) >> > > > >> >> > > > >> Role can be deduced by the presence or absence of >> > > > >> kafka.server,type=KafkaServer,name=BrokerState for brokers, or >> > > > >> kafka.server,type=ControllerServer,name=ClusterId for >> controllers. >> > > > >> >> > > > >> 2. Alternately, we could inject the ID and role into the command >> > line >> > > in >> > > > >> kafka-server-start.sh. Basically add -Dkafka.node.id=1, >> > > > >> -Dkafka.node.roles=broker. This would be helpful to people just >> > > > examining >> > > > >> the output of ps. >> > > > >> >> > > > >> Finally, you state that either command-line option can be given. >> > What >> > > > >> happens if both are given? >> > > > >> >> > > > >> best, >> > > > >> Colin >> > > > >> >> > > > >> >> > > > >> On Mon, Oct 23, 2023, at 12:20, Hailey Ni wrote: >> &g
Re: [VOTE] KIP-979 Allow independently stop KRaft processes
Thanks Justine! On Fri, Nov 17, 2023 at 10:40 AM Justine Olshan wrote: > Thanks Hailey for the update. +1 (binding) from me :) > > Justine > > On Thu, Nov 16, 2023 at 11:32 AM Hailey Ni > wrote: > > > Hey Justine, > > > > Thank you very much for the review. > > I've updated the KIP to add a log line stating that when both flags are > > given, node-id will take precedence. > > > > Thanks, > > Hailey > > > > On Wed, Nov 15, 2023 at 3:37 PM Justine Olshan > > > > wrote: > > > > > Hey Hailey, > > > > > > Thanks for the KIP. > > > I wonder if it would be better to either not allow both flags or if we > > > choose to have node take precedence, at least have a log line stating > > such. > > > > > > Otherwise the KIP makes sense to me. > > > > > > Justine > > > > > > On Tue, Nov 14, 2023 at 10:17 AM Colin McCabe > > wrote: > > > > > > > Thanks, Hailey. > > > > > > > > +1 (binding) > > > > > > > > Colin > > > > > > > > On Mon, Nov 13, 2023, at 15:13, Hailey Ni wrote: > > > > > Hi Colin, > > > > > > > > > > Thank you for your review. I removed the "absolute path need to be > > > > > provided" line from the KIP, and will modify the code to get the > > > absolute > > > > > path to the config files using some bash in the kafka-server-start > > > file. > > > > > For your second question, I've added a line in the KIP: "If both > > > > parameters > > > > > are provided, the value for node-id parameter will take precedence, > > > i.e, > > > > > the process with node id specified will be killed, no matter what's > > the > > > > > process role provided." > > > > > > > > > > What do you think? > > > > > > > > > > Thanks, > > > > > Hailey > > > > > > > > > > On Thu, Nov 9, 2023 at 4:03 PM Colin McCabe > > > wrote: > > > > > > > > > >> Hi Hailey, > > > > >> > > > > >> Thanks for the KIP. > > > > >> > > > > >> It feels clunky to have to pass an absolute path to the > > configuration > > > > file > > > > >> when starting the broker or controller. I think we should consider > > one > > > > of > > > > >> two alternate options: > > > > >> > > > > >> 1. Use JMXtool to examine the running kafka.Kafka processes. > > > > >> I believe ID is available from kafka.server, type=app-info,id=1 > > > > (replace 1 > > > > >> with the actual ID) > > > > >> > > > > >> Role can be deduced by the presence or absence of > > > > >> kafka.server,type=KafkaServer,name=BrokerState for brokers, or > > > > >> kafka.server,type=ControllerServer,name=ClusterId for controllers. > > > > >> > > > > >> 2. Alternately, we could inject the ID and role into the command > > line > > > in > > > > >> kafka-server-start.sh. Basically add -Dkafka.node.id=1, > > > > >> -Dkafka.node.roles=broker. This would be helpful to people just > > > > examining > > > > >> the output of ps. > > > > >> > > > > >> Finally, you state that either command-line option can be given. > > What > > > > >> happens if both are given? > > > > >> > > > > >> best, > > > > >> Colin > > > > >> > > > > >> > > > > >> On Mon, Oct 23, 2023, at 12:20, Hailey Ni wrote: > > > > >> > Hi Ron, > > > > >> > > > > > >> > I've added the "Rejected Alternatives" section in the KIP. > Thanks > > > for > > > > the > > > > >> > comments and +1 vote! > > > > >> > > > > > >> > Thanks, > > > > >> > Hailey > > > > >> > > > > > >> > On Mon, Oct 23, 2023 at 6:33 AM Ron Dagostino < > rndg...@gmail.com> > > > > wrote: > > > > >> > > > >
Re: [VOTE] KIP-979 Allow independently stop KRaft processes
Hey Justine, Thank you very much for the review. I've updated the KIP to add a log line stating that when both flags are given, node-id will take precedence. Thanks, Hailey On Wed, Nov 15, 2023 at 3:37 PM Justine Olshan wrote: > Hey Hailey, > > Thanks for the KIP. > I wonder if it would be better to either not allow both flags or if we > choose to have node take precedence, at least have a log line stating such. > > Otherwise the KIP makes sense to me. > > Justine > > On Tue, Nov 14, 2023 at 10:17 AM Colin McCabe wrote: > > > Thanks, Hailey. > > > > +1 (binding) > > > > Colin > > > > On Mon, Nov 13, 2023, at 15:13, Hailey Ni wrote: > > > Hi Colin, > > > > > > Thank you for your review. I removed the "absolute path need to be > > > provided" line from the KIP, and will modify the code to get the > absolute > > > path to the config files using some bash in the kafka-server-start > file. > > > For your second question, I've added a line in the KIP: "If both > > parameters > > > are provided, the value for node-id parameter will take precedence, > i.e, > > > the process with node id specified will be killed, no matter what's the > > > process role provided." > > > > > > What do you think? > > > > > > Thanks, > > > Hailey > > > > > > On Thu, Nov 9, 2023 at 4:03 PM Colin McCabe > wrote: > > > > > >> Hi Hailey, > > >> > > >> Thanks for the KIP. > > >> > > >> It feels clunky to have to pass an absolute path to the configuration > > file > > >> when starting the broker or controller. I think we should consider one > > of > > >> two alternate options: > > >> > > >> 1. Use JMXtool to examine the running kafka.Kafka processes. > > >> I believe ID is available from kafka.server, type=app-info,id=1 > > (replace 1 > > >> with the actual ID) > > >> > > >> Role can be deduced by the presence or absence of > > >> kafka.server,type=KafkaServer,name=BrokerState for brokers, or > > >> kafka.server,type=ControllerServer,name=ClusterId for controllers. > > >> > > >> 2. Alternately, we could inject the ID and role into the command line > in > > >> kafka-server-start.sh. Basically add -Dkafka.node.id=1, > > >> -Dkafka.node.roles=broker. This would be helpful to people just > > examining > > >> the output of ps. > > >> > > >> Finally, you state that either command-line option can be given. What > > >> happens if both are given? > > >> > > >> best, > > >> Colin > > >> > > >> > > >> On Mon, Oct 23, 2023, at 12:20, Hailey Ni wrote: > > >> > Hi Ron, > > >> > > > >> > I've added the "Rejected Alternatives" section in the KIP. Thanks > for > > the > > >> > comments and +1 vote! > > >> > > > >> > Thanks, > > >> > Hailey > > >> > > > >> > On Mon, Oct 23, 2023 at 6:33 AM Ron Dagostino > > wrote: > > >> > > > >> >> Hi Hailey. I'm +1 (binding), but could you add a "Rejected > > >> >> Alternatives" section to the KIP and mention the > "--required-config " > > >> >> option that we decided against and the reason why we made the > > decision > > >> >> to reject it? There were some other small things (dash instead of > > dot > > >> >> in the parameter names, --node-id instead of --broker-id), but > > >> >> cosmetic things like this don't warrant a mention, so I think > there's > > >> >> just the one thing to document. > > >> >> > > >> >> Thanks for the KIP, and thanks for adjusting it along the way as > the > > >> >> discussion moved forward. > > >> >> > > >> >> Ron > > >> >> > > >> >> > > >> >> Ron > > >> >> > > >> >> On Mon, Oct 23, 2023 at 4:00 AM Federico Valeri < > > fedeval...@gmail.com> > > >> >> wrote: > > >> >> > > > >> >> > +1 (non binding) > > >> >> > > > >> >> > Thanks. > > >> >> > > > >> >> > On Mon, Oct 23, 2023 at 9:48 AM Kamal Chandraprakash > > >> >> > wrote: > > >> >> > > > > >> >> > > +1 (non-binding). Thanks for the KIP! > > >> >> > > > > >> >> > > On Mon, Oct 23, 2023, 12:55 Hailey Ni > > > >> >> wrote: > > >> >> > > > > >> >> > > > Hi all, > > >> >> > > > > > >> >> > > > I'd like to call a vote on KIP-979 that will allow users to > > >> >> independently > > >> >> > > > stop KRaft processes. > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+processes > > >> >> > > > > > >> >> > > > Thanks, > > >> >> > > > Hailey > > >> >> > > > > > >> >> > > >> > > >
Re: [VOTE] KIP-979 Allow independently stop KRaft processes
Hi Colin, Thank you for your review. I removed the "absolute path need to be provided" line from the KIP, and will modify the code to get the absolute path to the config files using some bash in the kafka-server-start file. For your second question, I've added a line in the KIP: "If both parameters are provided, the value for node-id parameter will take precedence, i.e, the process with node id specified will be killed, no matter what's the process role provided." What do you think? Thanks, Hailey On Thu, Nov 9, 2023 at 4:03 PM Colin McCabe wrote: > Hi Hailey, > > Thanks for the KIP. > > It feels clunky to have to pass an absolute path to the configuration file > when starting the broker or controller. I think we should consider one of > two alternate options: > > 1. Use JMXtool to examine the running kafka.Kafka processes. > I believe ID is available from kafka.server, type=app-info,id=1 (replace 1 > with the actual ID) > > Role can be deduced by the presence or absence of > kafka.server,type=KafkaServer,name=BrokerState for brokers, or > kafka.server,type=ControllerServer,name=ClusterId for controllers. > > 2. Alternately, we could inject the ID and role into the command line in > kafka-server-start.sh. Basically add -Dkafka.node.id=1, > -Dkafka.node.roles=broker. This would be helpful to people just examining > the output of ps. > > Finally, you state that either command-line option can be given. What > happens if both are given? > > best, > Colin > > > On Mon, Oct 23, 2023, at 12:20, Hailey Ni wrote: > > Hi Ron, > > > > I've added the "Rejected Alternatives" section in the KIP. Thanks for the > > comments and +1 vote! > > > > Thanks, > > Hailey > > > > On Mon, Oct 23, 2023 at 6:33 AM Ron Dagostino wrote: > > > >> Hi Hailey. I'm +1 (binding), but could you add a "Rejected > >> Alternatives" section to the KIP and mention the "--required-config " > >> option that we decided against and the reason why we made the decision > >> to reject it? There were some other small things (dash instead of dot > >> in the parameter names, --node-id instead of --broker-id), but > >> cosmetic things like this don't warrant a mention, so I think there's > >> just the one thing to document. > >> > >> Thanks for the KIP, and thanks for adjusting it along the way as the > >> discussion moved forward. > >> > >> Ron > >> > >> > >> Ron > >> > >> On Mon, Oct 23, 2023 at 4:00 AM Federico Valeri > >> wrote: > >> > > >> > +1 (non binding) > >> > > >> > Thanks. > >> > > >> > On Mon, Oct 23, 2023 at 9:48 AM Kamal Chandraprakash > >> > wrote: > >> > > > >> > > +1 (non-binding). Thanks for the KIP! > >> > > > >> > > On Mon, Oct 23, 2023, 12:55 Hailey Ni > >> wrote: > >> > > > >> > > > Hi all, > >> > > > > >> > > > I'd like to call a vote on KIP-979 that will allow users to > >> independently > >> > > > stop KRaft processes. > >> > > > > >> > > > > >> > > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+processes > >> > > > > >> > > > Thanks, > >> > > > Hailey > >> > > > > >> >
Re: [VOTE] KIP-979 Allow independently stop KRaft processes
Hi Ron, I've added the "Rejected Alternatives" section in the KIP. Thanks for the comments and +1 vote! Thanks, Hailey On Mon, Oct 23, 2023 at 6:33 AM Ron Dagostino wrote: > Hi Hailey. I'm +1 (binding), but could you add a "Rejected > Alternatives" section to the KIP and mention the "--required-config " > option that we decided against and the reason why we made the decision > to reject it? There were some other small things (dash instead of dot > in the parameter names, --node-id instead of --broker-id), but > cosmetic things like this don't warrant a mention, so I think there's > just the one thing to document. > > Thanks for the KIP, and thanks for adjusting it along the way as the > discussion moved forward. > > Ron > > > Ron > > On Mon, Oct 23, 2023 at 4:00 AM Federico Valeri > wrote: > > > > +1 (non binding) > > > > Thanks. > > > > On Mon, Oct 23, 2023 at 9:48 AM Kamal Chandraprakash > > wrote: > > > > > > +1 (non-binding). Thanks for the KIP! > > > > > > On Mon, Oct 23, 2023, 12:55 Hailey Ni > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to call a vote on KIP-979 that will allow users to > independently > > > > stop KRaft processes. > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+processes > > > > > > > > Thanks, > > > > Hailey > > > > >
[VOTE] KIP-979 Allow independently stop KRaft processes
Hi all, I'd like to call a vote on KIP-979 that will allow users to independently stop KRaft processes. https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+processes Thanks, Hailey
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Federico, Thanks for your comments. SureI will update the KIP accordingly. Thanks, Hailey On Tue, Oct 3, 2023 at 1:29 AM Federico Valeri wrote: > Hi Hailey, thanks for the KIP. > > I also agree that the two mutually exclusive args are better. In order > to be consistent with the other tools, I would suggest to use > --process-role and --node-id (hyphen instead of dot). Can you also > update the KIP? > > On Mon, Oct 2, 2023 at 10:18 PM Hailey Ni > wrote: > > > > Hi Kamal, > > > > I think the broker.id property has been replaced with the `node.id` > property > > in KRaft. The documentation for `node.id` says it is required ( > > > https://github.com/apache/kafka/blob/72e275f6ea867747e6b4e524c80d5ebd726ac25b/core/src/main/scala/kafka/server/KafkaConfig.scala#L741 > ), > > and the QuickStart files all use it ( > > > https://github.com/apache/kafka/tree/72e275f6ea867747e6b4e524c80d5ebd726ac25b/config/kraft > ). > > It is technically true that these two configs are treated as synonyms of > > one another ( > > > https://github.com/apache/kafka/blob/72e275f6ea867747e6b4e524c80d5ebd726ac25b/core/src/main/scala/kafka/server/KafkaConfig.scala#L1587-L1597 > ), > > so if you specify either one the process will still recognize it and > > start. But it makes sense to exclusively use `node.id` in KRaft > because a > > node isn't necessarily a broker anymore; it could be a controller (or > even > > a combined broker+controller). > > > > Thanks, > > Hailey > > > > On Mon, Oct 2, 2023 at 1:17 PM Hailey Ni wrote: > > > > > Hi Ismeal, > > > > > > Thanks for the comments. I'll change the implementation to use a pair > of > > > mutually exclusive args --process.roles and --node.id. > > > > > > Thanks, > > > Hailey > > > > > > On Mon, Oct 2, 2023 at 6:34 AM Ismael Juma wrote: > > > > > >> Hi Ron, > > >> > > >> Yes, that's what I am proposing, yes. > > >> > > >> Ismael > > >> > > >> On Sat, Sep 30, 2023 at 2:30 PM Ron Dagostino > wrote: > > >> > > >> > Thanks, Ismael. I think you are proposing a pair of mutually > exclusive > > >> > args --process.roles and --node.id, right? I agree that is more > > >> > user-friendly than the --required-config arg, and it comes at the > > >> possible > > >> > expense of generality. So that’s the tradeoff between the two, I > think. > > >> > No other config comes to mind now that we’ve identified these two. > I > > >> think > > >> > the two specific and mutually exclusive parameters would be the way > to > > >> go > > >> > unless someone else identifies still more options that people might > > >> want. > > >> > > > >> > Did I get that right, or were you proposing something different? > > >> > > > >> > Ron > > >> > > > >> > > On Sep 30, 2023, at 10:42 AM, Ismael Juma > wrote: > > >> > > > > >> > > Hi, > > >> > > > > >> > > Thanks for the KIP. I think this approach based on configs is a > bit > > >> too > > >> > > open ended and not very user friendly. Why don't we simply provide > > >> flags > > >> > > for the things a user may care about? So far, it seems like we > have > > >> two > > >> > > good candidates (node id and process role). Are there any others? > > >> > > > > >> > > Ismael > > >> > > > > >> > >> On Fri, Sep 29, 2023 at 6:19 PM Hailey Ni > > > >> > wrote: > > >> > >> > > >> > >> Hi Ron, > > >> > >> > > >> > >> I think you made a great point, making the "name" arbitrary > instead > > >> of > > >> > >> hard-coding it will make the functionality much more flexible. > I've > > >> > updated > > >> > >> the KIP and the code accordingly. Thanks for the great idea! > > >> > >> > > >> > >> Thanks, > > >> > >> Hailey > > >> > >> > > >> > >> > > >> > >>> On Fri, Sep 29, 2023 at 2:34 PM Ron Dagostino < > rndg...@gmail.com> > > >> > wrote: > > >> > >>&
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Kamal, I think the broker.id property has been replaced with the `node.id` property in KRaft. The documentation for `node.id` says it is required ( https://github.com/apache/kafka/blob/72e275f6ea867747e6b4e524c80d5ebd726ac25b/core/src/main/scala/kafka/server/KafkaConfig.scala#L741), and the QuickStart files all use it ( https://github.com/apache/kafka/tree/72e275f6ea867747e6b4e524c80d5ebd726ac25b/config/kraft). It is technically true that these two configs are treated as synonyms of one another ( https://github.com/apache/kafka/blob/72e275f6ea867747e6b4e524c80d5ebd726ac25b/core/src/main/scala/kafka/server/KafkaConfig.scala#L1587-L1597), so if you specify either one the process will still recognize it and start. But it makes sense to exclusively use `node.id` in KRaft because a node isn't necessarily a broker anymore; it could be a controller (or even a combined broker+controller). Thanks, Hailey On Mon, Oct 2, 2023 at 1:17 PM Hailey Ni wrote: > Hi Ismeal, > > Thanks for the comments. I'll change the implementation to use a pair of > mutually exclusive args --process.roles and --node.id. > > Thanks, > Hailey > > On Mon, Oct 2, 2023 at 6:34 AM Ismael Juma wrote: > >> Hi Ron, >> >> Yes, that's what I am proposing, yes. >> >> Ismael >> >> On Sat, Sep 30, 2023 at 2:30 PM Ron Dagostino wrote: >> >> > Thanks, Ismael. I think you are proposing a pair of mutually exclusive >> > args --process.roles and --node.id, right? I agree that is more >> > user-friendly than the --required-config arg, and it comes at the >> possible >> > expense of generality. So that’s the tradeoff between the two, I think. >> > No other config comes to mind now that we’ve identified these two. I >> think >> > the two specific and mutually exclusive parameters would be the way to >> go >> > unless someone else identifies still more options that people might >> want. >> > >> > Did I get that right, or were you proposing something different? >> > >> > Ron >> > >> > > On Sep 30, 2023, at 10:42 AM, Ismael Juma wrote: >> > > >> > > Hi, >> > > >> > > Thanks for the KIP. I think this approach based on configs is a bit >> too >> > > open ended and not very user friendly. Why don't we simply provide >> flags >> > > for the things a user may care about? So far, it seems like we have >> two >> > > good candidates (node id and process role). Are there any others? >> > > >> > > Ismael >> > > >> > >> On Fri, Sep 29, 2023 at 6:19 PM Hailey Ni >> > wrote: >> > >> >> > >> Hi Ron, >> > >> >> > >> I think you made a great point, making the "name" arbitrary instead >> of >> > >> hard-coding it will make the functionality much more flexible. I've >> > updated >> > >> the KIP and the code accordingly. Thanks for the great idea! >> > >> >> > >> Thanks, >> > >> Hailey >> > >> >> > >> >> > >>> On Fri, Sep 29, 2023 at 2:34 PM Ron Dagostino >> > wrote: >> > >>> >> > >>> Thanks, Hailey. Is there a reason to restrict it to just >> > >>> process.roles and node.id? Someone might want to do >> > >>> "--required-config any.name=whatever.value", for example, and at >> first >> > >>> glance I don't see a reason why the implementation should be any >> > >>> different -- it seems it would probably be easier to not have to >> worry >> > >>> about restricting to specific cases, actually. WDYT? >> > >>> >> > >>> Ron >> > >>> >> > >>> On Fri, Sep 29, 2023 at 5:12 PM Hailey Ni > > >> > >>> wrote: >> > >>>> >> > >>>> Updated. Please let me know if you have any additional comments. >> Thank >> > >>> you! >> > >>>> >> > >>>> On Thu, Sep 21, 2023 at 3:02 PM Hailey Ni >> wrote: >> > >>>> >> > >>>>> Hi Ron. Thanks for the response. I agree with your point. I'll >> make >> > >> the >> > >>>>> corresponding changes in the KIP and KAFKA-15471 >> > >>>>> <https://issues.apache.org/jira/browse/KAFKA-15471>. >> > >>>>> >> > >>>>> On Thu, Sep 21, 2023 at 1:40
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Ismeal, Thanks for the comments. I'll change the implementation to use a pair of mutually exclusive args --process.roles and --node.id. Thanks, Hailey On Mon, Oct 2, 2023 at 6:34 AM Ismael Juma wrote: > Hi Ron, > > Yes, that's what I am proposing, yes. > > Ismael > > On Sat, Sep 30, 2023 at 2:30 PM Ron Dagostino wrote: > > > Thanks, Ismael. I think you are proposing a pair of mutually exclusive > > args --process.roles and --node.id, right? I agree that is more > > user-friendly than the --required-config arg, and it comes at the > possible > > expense of generality. So that’s the tradeoff between the two, I think. > > No other config comes to mind now that we’ve identified these two. I > think > > the two specific and mutually exclusive parameters would be the way to go > > unless someone else identifies still more options that people might want. > > > > Did I get that right, or were you proposing something different? > > > > Ron > > > > > On Sep 30, 2023, at 10:42 AM, Ismael Juma wrote: > > > > > > Hi, > > > > > > Thanks for the KIP. I think this approach based on configs is a bit too > > > open ended and not very user friendly. Why don't we simply provide > flags > > > for the things a user may care about? So far, it seems like we have two > > > good candidates (node id and process role). Are there any others? > > > > > > Ismael > > > > > >> On Fri, Sep 29, 2023 at 6:19 PM Hailey Ni > > wrote: > > >> > > >> Hi Ron, > > >> > > >> I think you made a great point, making the "name" arbitrary instead of > > >> hard-coding it will make the functionality much more flexible. I've > > updated > > >> the KIP and the code accordingly. Thanks for the great idea! > > >> > > >> Thanks, > > >> Hailey > > >> > > >> > > >>> On Fri, Sep 29, 2023 at 2:34 PM Ron Dagostino > > wrote: > > >>> > > >>> Thanks, Hailey. Is there a reason to restrict it to just > > >>> process.roles and node.id? Someone might want to do > > >>> "--required-config any.name=whatever.value", for example, and at > first > > >>> glance I don't see a reason why the implementation should be any > > >>> different -- it seems it would probably be easier to not have to > worry > > >>> about restricting to specific cases, actually. WDYT? > > >>> > > >>> Ron > > >>> > > >>> On Fri, Sep 29, 2023 at 5:12 PM Hailey Ni > > >>> wrote: > > >>>> > > >>>> Updated. Please let me know if you have any additional comments. > Thank > > >>> you! > > >>>> > > >>>> On Thu, Sep 21, 2023 at 3:02 PM Hailey Ni wrote: > > >>>> > > >>>>> Hi Ron. Thanks for the response. I agree with your point. I'll make > > >> the > > >>>>> corresponding changes in the KIP and KAFKA-15471 > > >>>>> <https://issues.apache.org/jira/browse/KAFKA-15471>. > > >>>>> > > >>>>> On Thu, Sep 21, 2023 at 1:40 PM Ron Dagostino > > >>> wrote: > > >>>>> > > >>>>>> Hi Hailey. No, I just looked, and zookeeper-server-stop does not > > >> have > > >>>>>> any facility to be specific about which ZK nodes to signal. So > > >>>>>> providing the ability in kafka-server-stop to be more specific > than > > >>>>>> just "signal all controllers" or "signal all brokers" would be a > > >> bonus > > >>>>>> and therefore not necessarily required. But if it is easy to > > >> achieve > > >>>>>> and doesn't add any additional cognitive load -- and at first > glance > > >>>>>> it does seem so -- we should probably just support it. > > >>>>>> > > >>>>>> Ron > > >>>>>> > > >>>>>> On Wed, Sep 20, 2023 at 6:15 PM Hailey Ni > > >>> > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> Hi Ron, > > >>>>>>> > > >>>>>>> Thank you very much for the comment. I think it makes sense to me > > >>> that
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Ron, I think you made a great point, making the "name" arbitrary instead of hard-coding it will make the functionality much more flexible. I've updated the KIP and the code accordingly. Thanks for the great idea! Thanks, Hailey On Fri, Sep 29, 2023 at 2:34 PM Ron Dagostino wrote: > Thanks, Hailey. Is there a reason to restrict it to just > process.roles and node.id? Someone might want to do > "--required-config any.name=whatever.value", for example, and at first > glance I don't see a reason why the implementation should be any > different -- it seems it would probably be easier to not have to worry > about restricting to specific cases, actually. WDYT? > > Ron > > On Fri, Sep 29, 2023 at 5:12 PM Hailey Ni > wrote: > > > > Updated. Please let me know if you have any additional comments. Thank > you! > > > > On Thu, Sep 21, 2023 at 3:02 PM Hailey Ni wrote: > > > > > Hi Ron. Thanks for the response. I agree with your point. I'll make the > > > corresponding changes in the KIP and KAFKA-15471 > > > <https://issues.apache.org/jira/browse/KAFKA-15471>. > > > > > > On Thu, Sep 21, 2023 at 1:40 PM Ron Dagostino > wrote: > > > > > >> Hi Hailey. No, I just looked, and zookeeper-server-stop does not have > > >> any facility to be specific about which ZK nodes to signal. So > > >> providing the ability in kafka-server-stop to be more specific than > > >> just "signal all controllers" or "signal all brokers" would be a bonus > > >> and therefore not necessarily required. But if it is easy to achieve > > >> and doesn't add any additional cognitive load -- and at first glance > > >> it does seem so -- we should probably just support it. > > >> > > >> Ron > > >> > > >> On Wed, Sep 20, 2023 at 6:15 PM Hailey Ni > > >> wrote: > > >> > > > >> > Hi Ron, > > >> > > > >> > Thank you very much for the comment. I think it makes sense to me > that > > >> we > > >> > provide an even more specific way to kill individual > > >> controllers/brokers. > > >> > I have one question: does the command line for ZooKeeper cluster > provide > > >> > such a way to kill individual controllers/brokers? > > >> > > > >> > Thanks, > > >> > Hailey > > >> > > > >> > On Tue, Sep 19, 2023 at 11:01 AM Ron Dagostino > > >> wrote: > > >> > > > >> > > Thanks for the KIP, Hailey. It will be nice to provide some > > >> > > fine-grained control for when people running the broker and > controller > > >> > > this way want to stop just one of them. > > >> > > > > >> > > One thing that occurs to me is that in a development environment > > >> > > someone might want to run multiple controllers and multiple > brokers > > >> > > all on the same box, and in that case they might want to actually > stop > > >> > > just one controller or just one broker instead of all of them. > So I'm > > >> > > wondering if maybe instead of supporting kafka-server-stop > > >> > > [--process.roles ] we might want to instead support > > >> > > kafka-server-stop [--required-config ]. If someone > wanted > > >> > > to stop any/all controllers and not touch the broker(s) they could > > >> > > still achieve that by invoking kafka-server-stop --required-config > > >> > > process.roles=controller. But if they did want to stop a > particular > > >> > > controller they could then also achieve that via kafka-server-stop > > >> > > --required-config node.id=1 (for example). What do you think? > > >> > > > > >> > > Ron > > >> > > > > >> > > On Thu, Sep 14, 2023 at 5:56 PM Hailey Ni > > > >> > > wrote: > > >> > > > > > >> > > > Hi all, > > >> > > > > > >> > > > I would like to start the discussion about *KIP-979: Allow > > >> independently > > >> > > > stop KRaft controllers or brokers* < > > >> > > > > > >> > > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+controllers+or+brokers > > >> > > > > > > >> > > > It proposes adding an optional field "--process.roles " > in > > >> the > > >> > > > script to allow users to independently stop either KRaft broker > > >> processes > > >> > > > or controller processes. While in the past, all processes were > > >> killed > > >> > > using > > >> > > > a single script. > > >> > > > Please let me know if you have any questions or comments. Much > > >> > > appreciated. > > >> > > > > > >> > > > Thanks & Regards, > > >> > > > Hailey > > >> > > > > >> > > > >
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Updated. Please let me know if you have any additional comments. Thank you! On Thu, Sep 21, 2023 at 3:02 PM Hailey Ni wrote: > Hi Ron. Thanks for the response. I agree with your point. I'll make the > corresponding changes in the KIP and KAFKA-15471 > <https://issues.apache.org/jira/browse/KAFKA-15471>. > > On Thu, Sep 21, 2023 at 1:40 PM Ron Dagostino wrote: > >> Hi Hailey. No, I just looked, and zookeeper-server-stop does not have >> any facility to be specific about which ZK nodes to signal. So >> providing the ability in kafka-server-stop to be more specific than >> just "signal all controllers" or "signal all brokers" would be a bonus >> and therefore not necessarily required. But if it is easy to achieve >> and doesn't add any additional cognitive load -- and at first glance >> it does seem so -- we should probably just support it. >> >> Ron >> >> On Wed, Sep 20, 2023 at 6:15 PM Hailey Ni >> wrote: >> > >> > Hi Ron, >> > >> > Thank you very much for the comment. I think it makes sense to me that >> we >> > provide an even more specific way to kill individual >> controllers/brokers. >> > I have one question: does the command line for ZooKeeper cluster provide >> > such a way to kill individual controllers/brokers? >> > >> > Thanks, >> > Hailey >> > >> > On Tue, Sep 19, 2023 at 11:01 AM Ron Dagostino >> wrote: >> > >> > > Thanks for the KIP, Hailey. It will be nice to provide some >> > > fine-grained control for when people running the broker and controller >> > > this way want to stop just one of them. >> > > >> > > One thing that occurs to me is that in a development environment >> > > someone might want to run multiple controllers and multiple brokers >> > > all on the same box, and in that case they might want to actually stop >> > > just one controller or just one broker instead of all of them. So I'm >> > > wondering if maybe instead of supporting kafka-server-stop >> > > [--process.roles ] we might want to instead support >> > > kafka-server-stop [--required-config ]. If someone wanted >> > > to stop any/all controllers and not touch the broker(s) they could >> > > still achieve that by invoking kafka-server-stop --required-config >> > > process.roles=controller. But if they did want to stop a particular >> > > controller they could then also achieve that via kafka-server-stop >> > > --required-config node.id=1 (for example). What do you think? >> > > >> > > Ron >> > > >> > > On Thu, Sep 14, 2023 at 5:56 PM Hailey Ni >> > > wrote: >> > > > >> > > > Hi all, >> > > > >> > > > I would like to start the discussion about *KIP-979: Allow >> independently >> > > > stop KRaft controllers or brokers* < >> > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+controllers+or+brokers >> > > > > >> > > > It proposes adding an optional field "--process.roles " in >> the >> > > > script to allow users to independently stop either KRaft broker >> processes >> > > > or controller processes. While in the past, all processes were >> killed >> > > using >> > > > a single script. >> > > > Please let me know if you have any questions or comments. Much >> > > appreciated. >> > > > >> > > > Thanks & Regards, >> > > > Hailey >> > > >> >
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Ron. Thanks for the response. I agree with your point. I'll make the corresponding changes in the KIP and KAFKA-15471 <https://issues.apache.org/jira/browse/KAFKA-15471>. On Thu, Sep 21, 2023 at 1:40 PM Ron Dagostino wrote: > Hi Hailey. No, I just looked, and zookeeper-server-stop does not have > any facility to be specific about which ZK nodes to signal. So > providing the ability in kafka-server-stop to be more specific than > just "signal all controllers" or "signal all brokers" would be a bonus > and therefore not necessarily required. But if it is easy to achieve > and doesn't add any additional cognitive load -- and at first glance > it does seem so -- we should probably just support it. > > Ron > > On Wed, Sep 20, 2023 at 6:15 PM Hailey Ni > wrote: > > > > Hi Ron, > > > > Thank you very much for the comment. I think it makes sense to me that we > > provide an even more specific way to kill individual controllers/brokers. > > I have one question: does the command line for ZooKeeper cluster provide > > such a way to kill individual controllers/brokers? > > > > Thanks, > > Hailey > > > > On Tue, Sep 19, 2023 at 11:01 AM Ron Dagostino > wrote: > > > > > Thanks for the KIP, Hailey. It will be nice to provide some > > > fine-grained control for when people running the broker and controller > > > this way want to stop just one of them. > > > > > > One thing that occurs to me is that in a development environment > > > someone might want to run multiple controllers and multiple brokers > > > all on the same box, and in that case they might want to actually stop > > > just one controller or just one broker instead of all of them. So I'm > > > wondering if maybe instead of supporting kafka-server-stop > > > [--process.roles ] we might want to instead support > > > kafka-server-stop [--required-config ]. If someone wanted > > > to stop any/all controllers and not touch the broker(s) they could > > > still achieve that by invoking kafka-server-stop --required-config > > > process.roles=controller. But if they did want to stop a particular > > > controller they could then also achieve that via kafka-server-stop > > > --required-config node.id=1 (for example). What do you think? > > > > > > Ron > > > > > > On Thu, Sep 14, 2023 at 5:56 PM Hailey Ni > > > wrote: > > > > > > > > Hi all, > > > > > > > > I would like to start the discussion about *KIP-979: Allow > independently > > > > stop KRaft controllers or brokers* < > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+controllers+or+brokers > > > > > > > > > It proposes adding an optional field "--process.roles " in the > > > > script to allow users to independently stop either KRaft broker > processes > > > > or controller processes. While in the past, all processes were killed > > > using > > > > a single script. > > > > Please let me know if you have any questions or comments. Much > > > appreciated. > > > > > > > > Thanks & Regards, > > > > Hailey > > > >
Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi Ron, Thank you very much for the comment. I think it makes sense to me that we provide an even more specific way to kill individual controllers/brokers. I have one question: does the command line for ZooKeeper cluster provide such a way to kill individual controllers/brokers? Thanks, Hailey On Tue, Sep 19, 2023 at 11:01 AM Ron Dagostino wrote: > Thanks for the KIP, Hailey. It will be nice to provide some > fine-grained control for when people running the broker and controller > this way want to stop just one of them. > > One thing that occurs to me is that in a development environment > someone might want to run multiple controllers and multiple brokers > all on the same box, and in that case they might want to actually stop > just one controller or just one broker instead of all of them. So I'm > wondering if maybe instead of supporting kafka-server-stop > [--process.roles ] we might want to instead support > kafka-server-stop [--required-config ]. If someone wanted > to stop any/all controllers and not touch the broker(s) they could > still achieve that by invoking kafka-server-stop --required-config > process.roles=controller. But if they did want to stop a particular > controller they could then also achieve that via kafka-server-stop > --required-config node.id=1 (for example). What do you think? > > Ron > > On Thu, Sep 14, 2023 at 5:56 PM Hailey Ni > wrote: > > > > Hi all, > > > > I would like to start the discussion about *KIP-979: Allow independently > > stop KRaft controllers or brokers* < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+controllers+or+brokers > > > > > It proposes adding an optional field "--process.roles " in the > > script to allow users to independently stop either KRaft broker processes > > or controller processes. While in the past, all processes were killed > using > > a single script. > > Please let me know if you have any questions or comments. Much > appreciated. > > > > Thanks & Regards, > > Hailey >
[jira] [Created] (KAFKA-15471) Allow independently stop KRaft controllers or brokers
Hailey Ni created KAFKA-15471: - Summary: Allow independently stop KRaft controllers or brokers Key: KAFKA-15471 URL: https://issues.apache.org/jira/browse/KAFKA-15471 Project: Kafka Issue Type: Improvement Reporter: Hailey Ni Assignee: Hailey Ni Some users run KRaft controllers and brokers on the same machine (not containerized, but through tarballs, etc). Prior to KRaft, when running ZooKeeper and Kafka on the same machine, users could independently stop the ZooKeeper node and Kafka broker since there were specific shell scripts for each (zookeeper-server-stop and kafka-server-stop, respectively). However in KRaft mode, they can't stop the KRaft controllers independently from the Kafka brokers because there is just a single script that doesn't distinguish between the two processes and signals both of them. We need to provide a way for users to kill either controllers or brokers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers
Hi all, I would like to start the discussion about *KIP-979: Allow independently stop KRaft controllers or brokers* < https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+controllers+or+brokers > It proposes adding an optional field "--process.roles " in the script to allow users to independently stop either KRaft broker processes or controller processes. While in the past, all processes were killed using a single script. Please let me know if you have any questions or comments. Much appreciated. Thanks & Regards, Hailey
Requesting permission to contribute to Apache Kafka
Hi, This is Hailey. Wiki ID: hni. May I request edit permission to the Kafka Wiki please? Thanks, Hailey
Request to Get Edit Permission
Hi, Can I get edit access to Apache Kafka's Wiki please? Thanks, Hailey
