Update: problem solved. Turns out that DHCP was handing out the wrong IP address for the DNS server. We had started the install on on IP, then moved it to another. We thought we had caught all of the changes in IP, but clearly not quite. Once we updated, then reran makedhcp, nodediscovery has started working.
Thank you all for the input. On Mon, 4 Feb 2019 at 19:04, Casandra H Qiu <cxh...@us.ibm.com> wrote: > There should have more message after nodediscoverstart in the cluster.log. > did u boot the node after issue this command? > > Can u check the node status? then run nodedisoverstart command with -V > command, (-V , verbose) will have more log messages. > > > Thanks, > Casandra > > > ................................................................... > Casandra Hong Qiu > Phone: (845) 433-9291, t/l 293-9291 > Office: Building 8, 3-B-04 > cxh...@us.ibm.com > > > > [image: Inactive hide details for Nathan Harper ---02/04/2019 12:34:58 > PM---Hi, As requested, from cluster.log]Nathan Harper ---02/04/2019 > 12:34:58 PM---Hi, As requested, from cluster.log > > From: Nathan Harper <nathan.har...@cfms.org.uk> > To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net> > Date: 02/04/2019 12:34 PM > Subject: Re: [xcat-user] Unrecognized directive (dest=) > ------------------------------ > > > > Hi, > > As requested, from cluster.log > > Feb 4 17:25:35 mgt06 xcat[28865]: INFO xCAT: Allowing > nodediscoverstart noderange=vkosctrl02 for root from localhost > > > And then from computes.log > > Feb 4 17:25:40 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 230 seconds > Feb 4 17:25:50 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 220 seconds > Feb 4 17:26:00 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 210 seconds > Feb 4 17:26:10 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 200 seconds > Feb 4 17:26:20 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 190 seconds > Feb 4 17:26:30 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 180 seconds > Feb 4 17:26:40 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 170 seconds > Feb 4 17:26:50 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 160 seconds > Feb 4 17:27:00 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 150 seconds > Feb 4 17:27:10 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 140 seconds > Feb 4 17:27:20 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 130 seconds > Feb 4 17:27:30 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 120 seconds > Feb 4 17:27:40 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 110 seconds > Feb 4 17:27:50 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 100 seconds > Feb 4 17:28:00 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 90 seconds > Feb 4 17:28:11 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 80 seconds > Feb 4 17:28:21 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 70 seconds > Feb 4 17:28:31 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 60 seconds > Feb 4 17:28:41 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 50 seconds > Feb 4 17:28:51 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 40 seconds > Feb 4 17:29:01 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 30 seconds > Feb 4 17:29:11 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 20 seconds > Feb 4 17:29:21 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 10 seconds > Feb 4 17:29:31 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > Running getdestiny --> *172.19.24.1:3001* <http://172.19.24.1:3001> > Feb 4 17:29:41 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > Received destiny= > Feb 4 17:29:41 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > The destiny=, destiny parameters= > Feb 4 17:29:41 172.19.24.214 [localhost] ERR xcat.genesis.doxcat: > Unrecognized directive (dest=) > Feb 4 17:29:48 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 150 seconds > Feb 4 17:29:58 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 140 seconds > Feb 4 17:30:08 172.19.24.214 [localhost] INFO xcat.genesis.doxcat: > ... Will retry xCAT in 130 seconds > > > On Mon, 4 Feb 2019 at 16:59, Casandra H Qiu <*cxh...@us.ibm.com* > <cxh...@us.ibm.com>> wrote: > > Hi, Nathan: > > Please show us the logs in the xCAT MN /var/log/xcat/ during the > discovery happens for this node. > after nodediscoverstart, then boot the node, you should see following > message in the log: > > > xcat.discovery.seqdiscovery > xcat.genesis.doxcat > xcat.discovery.aaadiscovery > > > Can u also try ip address instead of hostname? nodediscoverstart > noderange=172.19.24.12 -V > > > Thanks, > Casandra > > ................................................................... > Casandra Hong Qiu > Phone: (845) 433-9291, t/l 293-9291 > Office: Building 8, 3-B-04 > *cxh...@us.ibm.com* <cxh...@us.ibm.com> > > > > [image: Inactive hide details for Nathan Harper ---02/04/2019 10:29:50 > AM---Hi, @ Cassandra, as requested:]Nathan Harper ---02/04/2019 > 10:29:50 AM---Hi, @ Cassandra, as requested: > > From: Nathan Harper <*nathan.har...@cfms.org.uk* > <nathan.har...@cfms.org.uk>> > To: xCAT Users Mailing list <*xcat-user@lists.sourceforge.net* > <xcat-user@lists.sourceforge.net>> > Date: 02/04/2019 10:29 AM > Subject: Re: [xcat-user] Unrecognized directive (dest=) > ------------------------------ > > > > Hi, > > @ Cassandra, as requested: > nodediscoverstart noderange=vkosctrl02 > > tabdump discoverydata is empty. > [root@mgt06 ~]# lsdef vkosctrl02 > Object name: vkosctrl02 > bmc=vkosctrl02-bmc > groups=all,openstack,controllers > ip=172.19.24.12 > mgt=ipmi > postbootscripts=otherpkgs > postscripts=syslog,remoteshell,syncfiles > [root@mgt06 ~]# nodediscoverstatus > Sequential discovery is running. > The parameters used for discovery: noderange=vkosctrl02 > > > @ Ryan - that is what I would expect - however, they aren't running > the discovery at all, unless we SSH into them (via their dynamic range IP) > and run dodiscovery by hand. > > > On Mon, 4 Feb 2019 at 15:03, Casandra H Qiu <*cxh...@us.ibm.com* > <cxh...@us.ibm.com>> wrote: > Hi, Nathan: > > you are mentioned you were using sequential-based discovery, > what's the commands did you run ? > > Can u run 'tabdump discoverydata` to check if this node to be > add to discovery? > what's the node definition? make sure mac address is not > defined.. > the command `nodediscoverstatus` show anything? > > > Thanks, > Casandra > > > ................................................................... > Casandra Hong Qiu > Phone: (845) 433-9291, t/l 293-9291 > Office: Building 8, 3-B-04 > *cxh...@us.ibm.com* <cxh...@us.ibm.com> > > > > [image: Inactive hide details for Ryan Bowen ---02/04/2019 > 09:10:16 AM---If you don’t give the nodes something to do after > discovery (]Ryan > Bowen ---02/04/2019 09:10:16 AM---If you don’t give the nodes > something to > do after discovery (nodeset shell, runcmd=, osimage/install > > From: Ryan Bowen <ryan_bo...@shi.com> > To: xCAT Users Mailing list <*xcat-user@lists.sourceforge.net* > <xcat-user@lists.sourceforge.net>> > Date: 02/04/2019 09:10 AM > Subject: Re: [xcat-user] Unrecognized directive (dest=) > ------------------------------ > > > > If you don’t give the nodes something to do after discovery > (nodeset shell, runcmd=, osimage/install) they will wait for you to > give it > something. From my experience, the nodes seem to be discovered and > they’re > waiting for something to do. Can you pping/ssh to the nodes? > > * Ryan Bowen *| Integration Systems Engineer | *SHI International Corp* | > *C:* (609)619-2571 | *E:* *ryan_bo...@shi.com* > <ryan_bo...@shi.com> > > * From:* Nathan Harper <*nathan.har...@cfms.org.uk* > <nathan.har...@cfms.org.uk>> > * Sent:* Monday, February 4, 2019 4:44 AM > * To:* xCAT Users Mailing list <*xcat-user@lists.sourceforge.net* > <xcat-user@lists.sourceforge.net>> > * Subject:* Re: [xcat-user] Unrecognized directive (dest=) > > @ Casandra, > > There is nothing to the chain entry for this node, as it has not > been discovered yet. From the flowchart here: > > > > *https://xcat-docs.readthedocs.io/en/stable/advanced/chain/chain_table.html* > > <https://xcat-docs.readthedocs.io/en/stable/advanced/chain/chain_table.html> > > as the node is unknown, it should run a discovery - however, > this is not running. > > On Fri, 1 Feb 2019 at 19:05, Casandra H Qiu <*cxh...@us.ibm.com* > <cxh...@us.ibm.com>> wrote: > > Can u show us the chain attribute for this node? > > also, can u post some logs from /var/log/xcat/cluster.log during > the sequential discovery for this node? > > > Thanks, > Casandra > > > > ................................................................... > Casandra Hong Qiu > Phone: (845) 433-9291, t/l 293-9291 > Office: Building 8, 3-B-04 > *cxh...@us.ibm.com* <cxh...@us.ibm.com> > > > > [image: Inactive hide details for Nathan Harper ---01/31/2019 > 12:36:00 PM---Hi all, We're in the process of setting up two new > xCAT sys]Nathan > Harper ---01/31/2019 12:36:00 PM---Hi all, We're in the process of > setting > up two new xCAT systems. As it's a small > > From: Nathan Harper <*nathan.har...@cfms.org.uk* > <nathan.har...@cfms.org.uk>> > To: xCAT Users Mailing list <*xcat-user@lists.sourceforge.net* > <xcat-user@lists.sourceforge.net>> > Date: 01/31/2019 12:36 PM > Subject: [xcat-user] Unrecognized directive (dest=) > > ------------------------------ > > > > > Hi all, > > We're in the process of setting up two new xCAT systems. As it's > a small number of nodes we're just doing sequential discovery, and > on the > first system (done 2 weeks ago) the node discovery works as expected. > However, on the next one (today), the systems will boot into the > genesis > image happily enough, but won't run the discovery process. > > It looks like the getdestiny process is working properly, but > it's not being given anything to work with, and we see the error: > > Jan 31 16:49:39 172.19.24.214 [localhost] INFO > xcat.genesis.doxcat: Received destiny= > Jan 31 16:49:39 172.19.24.214 [localhost] INFO > xcat.genesis.doxcat: The destiny=, destiny parameters= > Jan 31 16:49:39 172.19.24.214 [localhost] ERR > xcat.genesis.doxcat: Unrecognized directive (dest=) > > We can't see what we've done differently between the two systems > (both latest xCAT, CentOS). Any idea what we might have missed? > > _______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > _______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > -- > * Nathan* *Harper* // IT Systems Lead > > * e: **nathan.har...@cfms.org.uk* <nathan.har...@cfms.org.uk> *t*: 0117 > 906 1104 *m*: 0787 551 0891 *w: **www.cfms.org.uk* > <http://www.cfms.org.uk/> > CFMS Services Ltd // Bristol & Bath Science Park // Dirac > Crescent // Emersons Green // Bristol // BS16 7FR > > CFMS Services Ltd is registered in England and Wales No 05742022 > - a subsidiary of CFMS Ltd > CFMS Services Ltd registered office // 43 Queens Square // > Bristol // BS1 4QP_______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > > _______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > > -- > * Nathan* *Harper* // IT Systems Lead > > * e: **nathan.har...@cfms.org.uk* <nathan.har...@cfms.org.uk> *t*: 0117 > 906 1104 *m*: 0787 551 0891 *w: **www.cfms.org.uk* > <http://www.cfms.org.uk/> > CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // > Emersons > Green // Bristol // BS16 7FR > > CFMS Services Ltd is registered in England and Wales No 05742022 - a > subsidiary of CFMS Ltd > CFMS Services Ltd registered office // 43 Queens Square // Bristol // > BS1 4QP_______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > > _______________________________________________ > xCAT-user mailing list > *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net> > *https://lists.sourceforge.net/lists/listinfo/xcat-user* > <https://lists.sourceforge.net/lists/listinfo/xcat-user> > > > > -- > *Nathan* *Harper* // IT Systems Lead > > *e: **nathan.har...@cfms.org.uk* <nathan.har...@cfms.org.uk> *t*: 0117 > 906 1104 *m*: 0787 551 0891 *w: **www.cfms.org.uk* > <http://www.cfms.org.uk/> > CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons > Green // Bristol // BS16 7FR > > CFMS Services Ltd is registered in England and Wales No 05742022 - a > subsidiary of CFMS Ltd > CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 > 4QP_______________________________________________ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > > > > _______________________________________________ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > -- *Nathan Harper* // IT Systems Lead *e: *nathan.har...@cfms.org.uk *t*: 0117 906 1104 *m*: 0787 551 0891 *w: *www.cfms.org.uk CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP
_______________________________________________ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
