Hi everyone, I have updated the proposed solution of the KIP. Instead of extending the DescribeQuorum API, it now suggests updating the DescribeCluster API to optionally return fenced brokers. This should address the comments from Colin and Jose.
Please let me know if you have any questions. Regards, Gantigmaa On Thu, Aug 15, 2024 at 9:00 AM Gantigmaa Selenge <gsele...@redhat.com> wrote: > Hi Colin and Jose, > > Thanks for taking a look at the KIP and providing feedback. > > I overlooked that the memory state containing observers will be lost when > a controller restarts. You also made a good point about other observers > that might be connecting to the quorum to read the current metadata. I > agree that we should implement a way to list the fenced brokers using the > DescribeClusterRequest instead of the DescribeQuorum API. > > I will update the KIP and get back to you all for further feedback. Thanks. > > Regards, > Gantigmaa > > On Fri, Aug 9, 2024 at 11:48 PM José Armando García Sancio > <jsan...@confluent.io.invalid> wrote: > >> Thanks for the KIP Gantigmaa! >> >> I agree with the motivation but it is not clear to me that this should be >> solved in the KRaft layer. The KRaft leader only keeps track, in-memory, >> of >> observers that have fetched. It is possible, after a kraft leader change, >> for this state to get lost. If the shutdown brokers never fetch they will >> not show up in the kraft leader's observer list. >> >> As Colin has pointed out it is better to solve this in the metadata module >> and the KRaft/Quorum controller. The QuorumController knows of all of the >> registered brokers and their state: fence, unfence and even the heartbeat >> state. Take a look at >> org.apache.kafka.controller.BrokerHeartbeatManager.BrokerHeartbeatState >> for >> some of the heartbeat state tracked. >> >> To expose this information we can extend the kafka-cluster CLI, and also >> extend Metadata RPC or introduce a new RPC that returns more broker level >> information. >> >> Thanks, >> >> On Thu, Jul 25, 2024 at 9:21 AM Gantigmaa Selenge <gsele...@redhat.com> >> wrote: >> >> > Hi everyone, >> > >> > I would like to start a discussion on KIP-1073 that includes inactive >> > observer nodes in the response for describeQuorum request. >> > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1073%3A+Return+inactive+observer+nodes+in+DescribeQuorum+response >> > >> > The initial discussion on this issue is here, if you would like to see >> what >> > was previously discussed: >> > https://lists.apache.org/list.html?dev@kafka.apache.org >> > >> > Any feedback and suggestions for the KIP are welcome in this email >> thread. >> > >> > Thank you. >> > Regards, >> > Gantigmaa Selenge >> > >> >> >> -- >> -José >> >
