Thanks for the response. The actual deployment we're looking at right now has some stumbling blocks, although I did get a dynamic range added and the IPMIs get an IP so that's solved.
The nodes we are deploying for testing come with UEFI turned on and set only boot from port one on a Mellanox ConnectX 3 card. The customer wants port 1 to be IB and port to be 10gb so at present there are a couple of ways this could be handled. Booting diskless over IB would be the most efficient method but that's not currently supported under xCAT so that's out. What we are doing with our current setup is to assign the IPMI a dynamic address via DHCP (easily solved) and then running a script from the MN using IPMI lanplus to reconfigure the nodes to Legacy boot and enable booting over port 2 on the ConnectX card, then booting a diskless image that's used for testing. My vision would be to use xCAT to boot with ib0 being the installnic, using switch+port discovery, and doing all the config using bmcsetup and post/postboot scripts. Eventually we'll automate the testing but all the scripts we use for that were organically developed with no standards over the last five years and that's a totally different ball of wax. I still have to work out switch+port discovery but that doesn't seem like it'll be an issue based on my reading so far and I'm waiting for access to the switches we'll be using so I can set that up. My only real hold-up at this point is booting diskless over IB. Hell, I wouldn't mind being a guinea pig for it if there's anyone working on it or if there's a hack out there. The end result of this project will be a system that will deploy and test a cluster from any manufacturer with very little configuration work at time of deployment. The initial setup will be complex but in the end I think it's really the only way to go. Regards, Mark L. Potter ________________________________ From: Jarrod Johnson [[email protected]] Sent: Tuesday, November 10, 2015 9:07 AM To: xCAT Users Mailing list Subject: Re: [xcat-user] xCAT 2.10 non-node DHCP assigenment and more questions Like Arif said, the most mindless, cross-vendor approach that should generally work is to pxe boot the node and leave it to 'bmcsetup' to handle the IPMI configuration. For switches, depending on your vendor strategy, there's a few things, some proprietary, some standard, but depends heavily upon the 'default' behavior of your switch vendors as to whether it's actually viable. Now for going a bit off the beaten path for potentially useful stuff: Incidentally for non-node things via DHCP without knowing the macs, strictly speaking that will happen naturally with a dynamic range (we usually describe that as being relevant to 'discovery', but the dhcp range will answer things like switches, service processors, and laptops plugged in for a bit). Depending on the DHCP request and if makedns is right, then their requested hostname will be put into DNS. However, failing that harvesting pertinent details from /var/lib/dhcp/dhcpd.leases may be interesting. I have done this upon occasion, but I don't feel like it ever shows promise as a robust method to base an automation strategy around. Note that the automatic discovery I plan to take to new levels in confluent. At least for Lenovo equipment, I'm comfortable that we can get to the point where DHCP or no DHCP, we can automate the configuration of systems as long as we are on the same vlan as them. I'll of course be doing my best to follow strategies that make most use out of standards, but like above some of these standards are not ubiquitous or on by default. I hope to bake this design more in the coming months to get a more concrete wish list for things that would be good defaults to request of vendors. From: Arif Ali [mailto:[email protected]] Sent: Monday, November 09, 2015 4:19 PM To: [email protected] Subject: Re: [xcat-user] xCAT 2.10 non-node DHCP assigenment and more questions Hi Mark, Good to see you on-board All the latest docs are located on the new documentation site readthedocs [1], but you may want to start on sourceforge [2], but note that a lot of the ducmentation currently is being migrated from sourceforge to readthedocs. To answer your questions, see below on my take on it, others may have more and different answers That said I am looking for information on how to do the following thing. I have extensively Googled and haven't found answers for these as of yet, even links would be greatly appreciated! Deploy diskless nodes over IB - I saw this as a priority three on the 2.10 roadmap but haven't see much of anything about it. If anyone has this working I don't mind doing the legwork. I have managed to get somewhere on this but end up with an error: "dracut warning: No root device "1" found" and haven't managed to get past that. This is the most important piece to be honest. I have also seen this in the roadmap as well, but actually seen it in action, I can't see it in the release notes [3], maybe something I am missing. I would love to be able to do this myself Configure non-node entities via DHCP - A good for instance would be setting up IPMI or switches via DHCP without knowing the mac. We have quite a bit of volume and automatic setup of these items would greatly speed up our process. On this note, the key word is discovery for search criteria in the readthedocs. Depending on the HW you have, and the switches you have, you can discover in ~3 methods 1. Automagic through switch discovery, This is detailed in [4], [5] and [6] 2. Sequential Discovery -- resources include [7], [8] and [9] 3. Flex or bladecenter node discovery -- resource include [10] Hopefully, this will give you a lot of reading material to go through the paces. I know the documentation is in quite a few places, but that is due to things getting migrated to readthedocs. If you have any issues, give us a shout on IRC, some of us hang out on the #xcat channel [1] http://xcat-docs.readthedocs.org/en/latest/ [2] http://sourceforge.net/p/xcat/wiki/Main_Page/ [3] https://github.com/xcat2/xcat-core/wiki/XCAT_2.10_Release_Notes [4] http://sourceforge.net/p/xcat/wiki/XCAT_NeXtScale_Clusters/#node-definition-and-discovery [5] http://sourceforge.net/p/xcat/wiki/XCAT_iDataPlex_Cluster_Quick_Start/#option-2-switch-discovery [6] http://xcat-docs.readthedocs.org/en/latest/guides/admin-guides/manage_clusters/ppc64le/discovery/switch_discovery.html [7] http://sourceforge.net/p/xcat/wiki/Sequential_Discovery/ [8] http://sourceforge.net/p/xcat/wiki/XCAT_iDataPlex_Cluster_Quick_Start/#option-1-sequential-discovery [9] http://xcat-docs.readthedocs.org/en/latest/guides/admin-guides/manage_clusters/ppc64le/discovery/seq_discovery.html [10] http://sourceforge.net/p/xcat/wiki/XCAT_system_x_support_for_IBM_Flex/#cmm-discovery-and-configuration regards, Arif
------------------------------------------------------------------------------
_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
