Thank you so much, Victor and Casandra! It's making sense now, and I to it
at least partially to work.
So to summarize:
- I predefine a node with the final name and attributes I want.
- bmcdiscover uses nmap to scan for new systems with IPMI, but otherwise
powered off, and generates temporary nodes for those.
- I copy a few of the attributes from the temporary node to the
corresponding predefined one.
The attributes to be copied are: serial, mtm, postbootscript,
postscripts, mgt, cons.
- when the system is powered on, xCAT will use PXE to boot the genesis
kernel, which will identify more details about the hardware and send those
attributes back. xCAT will match up those two based on the serial number
and the mtm.
xCAT will then reconfigure the BMC for the IP address, user name and
password in the predefined node.
xCAT will store those remaining attributes in its table, which then
allows the discovered node to boot normally.
It's making sense to me now, and seems to work. I'm still having a problem
with the PXE boot, but that appears to be unrelated.
On Tue, Mar 6, 2018 at 1:13 PM, Victor Hu <v...@us.ibm.com> wrote:
> Hi Kevin
>
> Discovery is tricky and it looks like the doc does not clear up a lot of
> this ... let me see if i can give some more details...
>
>
> bmcdiscover generates a temporary node definition for the BMC, this allows
> admins to power control the unknown server through xCAT, We need a new
> server to boot the host and xCAT will send the genesis kernel to servers
> that it does not recognize MAC addresses of. So the temporary node
> definition is this guy: node-dell-poweredge-r230-7q3z7m2, When xCAT
> makes a match between the temporary node definition and the pre-defined
> node definition (through MTMS, switch based discovery), it will
> automatically remove the "node-<MT-SN>" definition from the xCAT database.
>
> Typically, the node IPs and BMC IPs would not be part of the dynamic
> range, so when the BMC comes up, it will get a IP address from the dynamic
> range, let's just say your range is 192.168.101.200-192.168.101.250 (50
> IPs)
>
> You can set the static information in the pre-defined node definition,
> cn-002.....
>
> For example, here's a simplified discovery process
>
> 1. The server BMC comes up and gets a DHCP IP address from the dyn range,
> lets say it gets the first one, 192.168.101.200.
>
> 2. Then you run bmcdiscover --range 192.168.101.200 and write this
> temporary node definition to the BMC....
>
> 3. Then create a pre-defined node definition cn-002....
>
> cn-002:
> bmc=192.168.101.202
> ip=192.168.101.102
> chain=runcmd=bmcsetup,standby
> ...
> mtm=DELL:PowerEdge R230
> serial=7Q3Z7M2
>
> 4. Then set the server to network boot and power it on.... That's the
> commands Casandra gave:
>
> rsetboot node-dell-poweredge-r230-7q3z7m2 net
> rpower node-dell-poweredge-r230-7q3z7m2 boot
>
> 5. This powers on the host side and on any MAC address that xCAT does not
> know about it, it will send over the genesis kernel (a diskless linux image
> that does discovery and sends back the info to xCATd)
>
>
> 6. In that cn-002, I set some properties of the chain table... chain
> attribute will run bmcsetup script inside discovery that if it makes a
> match (in this case using MTMS discovery) it will send over the properties
> of the node and will configure the BMC IP address to be 192.168.101.202
>
> At this point, you should be able to power control using `rpower cn-002
> state` without any need to merge information between the two definitions.
>
>
> Regards,
> Victor
>
>
>
>
>
> ----- Original message -----
> From: Kevin Keane <kke...@sandiego.edu>
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Cc:
> Subject: Re: [xcat-user] How to configure IPMI?
> Date: Tue, Mar 6, 2018 12:49 PM
>
> Thank you, Casandra.
>
> So in effect I have to manually merge the BMC information into my
> predefined node? I think I can do that.
>
> One more question. I'm still confused about how xCAT gets the correct IP
> address into the correct system. This is what may new merged node looks
> like.
>
> Object name: cn-002
> arch=x86_64
> bmc=192.168.101.201
> bmcpassword=calvin
> bmcusername=root
> chain=runcmd=bmcsetup,shell
> cons=ipmi
> groups=ipmi,dellr230,all,compute
> hwtype=bmc
> installnic=mac
> ip=192.168.101.4
> mgt=ipmi
> mtm=DELL:PowerEdge R230
> netboot=xnba
> nichostnamesuffixes.em1=-comp
> nichostnamesuffixes.em2=-mgt
> nicips.em1=192.168.100.4
> nicips.em2=192.168.101.4
> nodetype=mp
> os=rhels7.4
> postbootscripts=otherpkgs,confignics
> postscripts=syslog,remoteshell,syncfiles,setupntp
> primarynic=mac
> profile=hpccn
> provmethod=rhels7.4-x86_64-netboot-hpccn
> serial=7Q3Z7M2
>
>
> On Mon, Mar 5, 2018 at 2:02 PM, Casandra H Qiu <cxh...@us.ibm.com> wrote:
>
> bmcdiscover command will created this node definition to xcatdb:
>
> node-dell-poweredge-r230-7q3z7m2:
> objtype=node
> groups=all
> bmc=192.168.101.201
> cons=ipmi
> mgt=ipmi
> mtm=DELL:PowerEdge R230
> serial=7Q3Z7M2
> bmcusername=root
> bmcpassword=calvin
>
> For your predefine the compute node, you need to update bmc info, mtm,
> serial.....
> chdef cn-001 bmc=192.168.101.201 bmcusername=root bmcpassword=calvin
> mtm=DELL:PowerEdge R230 serial=7Q3Z7M2
>
> then, reboot the bmc node:
> rsetboot node-dell-poweredge-r230-7q3z7m2 net
> rpower node-dell-poweredge-r230-7q3z7m2 boot
>
> bmc node definition will be removed from xcat database once compute node
> discovered via mtms.
>
> refer this doc for more information:
> http://xcat-docs.readthedocs.io/en/stable/guides/admin-guide
> s/manage_clusters/ppc64le/discovery/mtms/discovery_using_defined.html
>
>
>
>
> Thanks,
> Casandra
>
> ...................................................................
> Casandra Hong Qiu
> Phone: (845) 433-9291, t/l 293-9291
> Office: Building 8, 3-B-04
> cxh...@us.ibm.com
>
>
>
> [image: Inactive hide details for Kevin Keane ---03/05/2018 03:47:15
> PM---Thank you, that filled in a big part of the puzzle! I am stil]Kevin
> Keane ---03/05/2018 03:47:15 PM---Thank you, that filled in a big part of
> the puzzle! I am still confused how to get bmcdiscover to ad
>
> From: Kevin Keane <kke...@sandiego.edu>
> To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>
> Date: 03/05/2018 03:47 PM
> Subject: Re: [xcat-user] How to configure IPMI?
>
> ------------------------------
>
>
>
> Thank you, that filled in a big part of the puzzle!
>
> I am still confused how to get bmcdiscover to add the information to the
> existing node, instead of creating a new one.
>
> bmcdiscover seems to generate a new second node, and I'm not sure how to
> correlate that to the existing one. The page you pointed me to seems to
> suggest this is a manual process?
>
>
> node-dell-poweredge-r230-7q3z7m2:
> objtype=node
> groups=all
> bmc=192.168.101.201
> cons=ipmi
> mgt=ipmi
> mtm=DELL:PowerEdge R230
> serial=7Q3Z7M2
> bmcusername=root
> bmcpassword=calvin
>
> And this is the one I already have predefined.
>
> Object name: cn-001
> arch=x86_64
> cons=ipmi
> groups=ipmi,all,all,compute
> installnic=mac
> ip=192.168.101.3
> mac=50:9A:4C:74:58:0F
> mgt=ipmi
> netboot=xnba
> nichostnamesuffixes.em1=-comp
> nichostnamesuffixes.em2=-mgt
> nicips.em1=192.168.100.3
> nicips.em2=192.168.101.3
> os=rhels7.4
> postbootscripts=otherpkgs,confignics
> postscripts=syslog,remoteshell,syncfiles,setupntp
> primarynic=mac
> profile=hpccn
> provmethod=rhels7.4-x86_64-netboot-hpccn
> updatestatus=failed
> updatestatustime=02-20-2018 16:26:47
>
>
>
> On Mon, Mar 5, 2018 at 11:33 AM, Casandra H Qiu <*cxh...@us.ibm.com*
> <cxh...@us.ibm.com>> wrote:
>
> - predefined compute node definition, and use one of discovery method
> to find BMC . xCAT will match BMC and compute nodes during the discover
> phase and setting all the required fields (mac, ip address, hostname and
> etc..)
>
> Refer to this doc:
> *http://xcat-docs.readthedocs
>
> <http://xcat-docs.readthedocs>.io/en/stable/guides/admin-guides/manage_clusters/ppc64le/discovery/mtms/discovery.html?highlight=bmcdiscover*
>
>
>
> Thanks,
> Casandra
>
>
> ...................................................................
> Casandra Hong Qiu
> Phone: *(845) 433-9291* <(845)%20433-9291>, t/l 293-9291
> Office: Building 8, 3-B-04
> *cxh...@us.ibm.com* <cxh...@us.ibm.com>
>
>
>
> [image: Inactive hide details for Kevin Keane ---03/05/2018 02:08:11
> PM---I am trying to understand how the IP addresses for IPMI on my]Kevin
> Keane ---03/05/2018 02:08:11 PM---I am trying to understand how the IP
> addresses for IPMI on my nodes is actually assigned. I create m
>
> From: Kevin Keane <*kke...@sandiego.edu* <kke...@sandiego.edu>>
> To: xCAT Users Mailing list <*xcat-user@lists.sourceforge.net*
> <xcat-user@lists.sourceforge.net>>
> Date: 03/05/2018 02:08 PM
> Subject: [xcat-user] How to configure IPMI?
> ------------------------------
>
>
>
>
> I am trying to understand how the IP addresses for IPMI on my nodes is
> actually assigned. I create my nodes manually (not by any discovery
> method).
>
> What I am not understanding right now is how to correlate the BMC MAC
> address (which is only in the dhcp leases file) to the correct node, and
> how to set a specific IP address, host name etc. for the BMC so the rpower,
> rcons etc. commands work correctly.
>
> Here is a sample of one of my nodes. My nodes are Dell servers with
> iDRAC 9 IPMI is configured to share the first NIC, and use DHCP.
>
> Object name: cn-001
> arch=x86_64
> cons=ipmi
> groups=ipmi,all,all,compute
> installnic=mac
> ip=192.168.101.3
> mac=50:9A:4C:74:58:0F
> mgt=ipmi
> netboot=xnba
> nichostnamesuffixes.em1=-comp
> nichostnamesuffixes.em2=-mgt
> nicips.em1=192.168.100.3
> nicips.em2=192.168.101.3
> os=rhels7.4
> postbootscripts=otherpkgs,confignics
> postscripts=syslog,remoteshell,syncfiles,setupntp
> primarynic=mac
> profile=hpccn
> provmethod=rhels7.4-x86_64-netboot-hpccn
> updatestatus=failed
> updatestatustime=02-20-2018 16:26:47
>
> Currently, the iDRAC adapter obtains an IP address via DHCP from the
> dynamic range, without a good way for me to identify which lease belongs to
> which node.
>
> Thanks!
>
> --
>
> _______________________________________________________________________
> Kevin Keane | Systems Architect | University of San Diego ITS |
> *kke...@sandiego.edu* <kke...@sandiego.edu>
> Maher Hall, 192 |5998 Alcalá Park | San Diego, CA 92110-2492 |
> *619.260.6859* <%28619%29%20260-2298>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org!
>
> *https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=pxebkHxVnnl4SYlApInHfzX0kgfydADs-2hbz1qNERA&s=Sb092eM7ZXsFzqlTvwz2zYc8nz1sgu4Ek0G03ASa3aM&e=*
>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=pxebkHxVnnl4SYlApInHfzX0kgfydADs-2hbz1qNERA&s=Sb092eM7ZXsFzqlTvwz2zYc8nz1sgu4Ek0G03ASa3aM&e=>
> _______________________________________________
> xCAT-user mailing list
> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
>
>
> *https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=pxebkHxVnnl4SYlApInHfzX0kgfydADs-2hbz1qNERA&s=We1fPhISvNCdgvy89b1Y2XhHJWz4x1MF3J7qJ-kUcu8&e=*
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=pxebkHxVnnl4SYlApInHfzX0kgfydADs-2hbz1qNERA&s=We1fPhISvNCdgvy89b1Y2XhHJWz4x1MF3J7qJ-kUcu8&e=>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! *http://sdm.link/slashdot
> <http://sdm.link/slashdot>*
> _______________________________________________
> xCAT-user mailing list
> *xCAT-user@lists.sourceforge.net* <xCAT-user@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/xcat-user*
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&m=DedaiUMRN7B7Ii624N18mXpzoCUcSlWzFWc4UzW4rT8&s=apeJ9e0b545GFf9xu9LHerHUZBIhVfDItqHhwGWo_LM&e=>
>
>
>
>
> --
>
>
> _______________________________________________________________________
> Kevin Keane | Systems Architect | University of San Diego ITS |
> *kke...@sandiego.edu* <kke...@sandiego.edu>
> Maher Hall, 192 |5998 Alcalá Park | San Diego, CA 92110-2492 |
> *619.260.6859* <%28619%29%20260-2298>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! https://urldefense.proofpoint.
> com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_iaSHvJ
> ObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS0nCzyNXY&
> m=DedaiUMRN7B7Ii624N18mXpzoCUcSlWzFWc4UzW4rT8&s=FP4VOnsbrR_M
> p7yvAuYRkgmE6RaNjOzh9ArI9R5tROQ&e=
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.s
> ourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_
> iaSHvJObTbx-siA1ZOg&r=n1LR_Py9TQX0dVqfGTbLHUMGx25-C8VtBDS
> 0nCzyNXY&m=DedaiUMRN7B7Ii624N18mXpzoCUcSlWzFWc4UzW4rT8&s=ape
> J9e0b545GFf9xu9LHerHUZBIhVfDItqHhwGWo_LM&e=
>
>
>
>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=87c6pZdGQvqXbdlQTeOmYw&m=5Wpd6clQ4Pcjrqp88602dj6j8YfwTZ33UIybRbSr1vs&s=D3fSjikTQBHfglcFbGlZaJBkPj5Lxoc53pu6XEjQbwc&e=>
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=87c6pZdGQvqXbdlQTeOmYw&m=5Wpd6clQ4Pcjrqp88602dj6j8YfwTZ33UIybRbSr1vs&s=EKoae2ipUCmfckBrBdikPYts6qUeWMMEQFbxZiswSeE&e=>
>
>
>
>
>
> --
>
> _______________________________________________________________________
> Kevin Keane | Systems Architect | University of San Diego ITS |
> kke...@sandiego.edu
> Maher Hall, 192 |5998 Alcalá Park | San Diego, CA 92110-2492 |
> 619.260.6859 <%28619%29%20260-2298>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! https://urldefense.proofpoint.
> com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_
> iaSHvJObTbx-siA1ZOg&r=87c6pZdGQvqXbdlQTeOmYw&m=
> 5Wpd6clQ4Pcjrqp88602dj6j8YfwTZ33UIybRbSr1vs&s=
> D3fSjikTQBHfglcFbGlZaJBkPj5Lxoc53pu6XEjQbwc&e=
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.
> sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&
> c=jf_iaSHvJObTbx-siA1ZOg&r=87c6pZdGQvqXbdlQTeOmYw&m=
> 5Wpd6clQ4Pcjrqp88602dj6j8YfwTZ33UIybRbSr1vs&s=
> EKoae2ipUCmfckBrBdikPYts6qUeWMMEQFbxZiswSeE&e=
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> xCAT-user mailing list
> xCAT-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xcat-user
>
>
--
_______________________________________________________________________
Kevin Keane | Systems Architect | University of San Diego ITS |
kke...@sandiego.edu
Maher Hall, 192 |5998 Alcalá Park | San Diego, CA 92110-2492 | 619.260.6859
<%28619%29%20260-2298>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
