Hi Alexey, Here is the link to the kudu master recovery guide I used. https://kudu.apache.org/releases/1.16.0/docs/administration.html
Additionally, I found some of the warnings in kudu tservers that some of the tablets report uuids that are unknown to the cluster. to be specific here is the cluster configuration 5 kudu masters 6 kudu tservers replication factor -3 affected kudu master - master -1 warning on kudu tserver - 3 - Invalid argument:RequestConsensusVote: Wrong destination UUID requested. local UUID - localuuid. Requested UUID : wronguuid from tserver-1. Note : above warning is there even though master 1 is not in the cluster and cluster reports healthy. Thanks, Panduka. On Thu, Jul 18, 2024 at 11:35 PM Alexey Serbin <aser...@cloudera.com> wrote: > Hi Panduka, > > Most likely, something was missing in the implementation of the documented > procedure. > BTW, what exact one did you use -- could you share a link to the document? > > There was an outdated one with a lot of manual intervention, and there is > a contemporary one > using `kudu master add` and `kudu master remove` CLI utilities. > > To find more insights on what's wrong, you could check out INFO logs of > the kudu-master process > with the LEARNER role and the leader master. > > > Kind regards, > > Alexey > > On Thu, Jul 18, 2024 at 10:43 AM Panduka Senevirathna < > paneng1...@gmail.com> wrote: > >> Hi, >> >> Kudu version - 1.16.0 >> Anyone know about kudu master role - LEARNER? i followed the steps for >> recovering from a dead kudu master as per the steps of kudu administration >> guide.and this kudu master went into a state called LEARNER instead of >> FOLLOWER. >> After removing and reconfiguring the cluster to run with rest of the >> masters kudu ksck responds healthy. There is no error. But once this master >> is added and run ksck it indicates an error saying master consensus errors >> found. And there are 5 masters in our cluster, this newly added master does >> not accept the elected leader as per the ksck output. Plus in the web gui >> this master does not have the cluster id updated. It’s empty. added kudu >> master has this warning in the logs as well. >> >> Thanks, >> Panduka. >> >
