Hi Colin and Jose, Thanks for taking a look at the KIP and providing feedback.
I overlooked that the memory state containing observers will be lost when a controller restarts. You also made a good point about other observers that might be connecting to the quorum to read the current metadata. I agree that we should implement a way to list the fenced brokers using the DescribeClusterRequest instead of the DescribeQuorum API. I will update the KIP and get back to you all for further feedback. Thanks. Regards, Gantigmaa On Fri, Aug 9, 2024 at 11:48 PM José Armando García Sancio <jsan...@confluent.io.invalid> wrote: > Thanks for the KIP Gantigmaa! > > I agree with the motivation but it is not clear to me that this should be > solved in the KRaft layer. The KRaft leader only keeps track, in-memory, of > observers that have fetched. It is possible, after a kraft leader change, > for this state to get lost. If the shutdown brokers never fetch they will > not show up in the kraft leader's observer list. > > As Colin has pointed out it is better to solve this in the metadata module > and the KRaft/Quorum controller. The QuorumController knows of all of the > registered brokers and their state: fence, unfence and even the heartbeat > state. Take a look at > org.apache.kafka.controller.BrokerHeartbeatManager.BrokerHeartbeatState for > some of the heartbeat state tracked. > > To expose this information we can extend the kafka-cluster CLI, and also > extend Metadata RPC or introduce a new RPC that returns more broker level > information. > > Thanks, > > On Thu, Jul 25, 2024 at 9:21 AM Gantigmaa Selenge <gsele...@redhat.com> > wrote: > > > Hi everyone, > > > > I would like to start a discussion on KIP-1073 that includes inactive > > observer nodes in the response for describeQuorum request. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1073%3A+Return+inactive+observer+nodes+in+DescribeQuorum+response > > > > The initial discussion on this issue is here, if you would like to see > what > > was previously discussed: > > https://lists.apache.org/list.html?dev@kafka.apache.org > > > > Any feedback and suggestions for the KIP are welcome in this email > thread. > > > > Thank you. > > Regards, > > Gantigmaa Selenge > > > > > -- > -José >
