Re: [xcat-user] [External] search in resolv.conf
This can be achieved by adding each search domain as a network table entry. See my post from April 3, 2022. > On 12/20/2022 7:09 AM Christian Caruthers wrote: > > > > The only way I know of is through a custom postscript. > > > NOTE that with RHEL/CentOS/etc. 8.x resolv.conf is managed by > NetworkManager. To avoid it being accidentally overwritten, you can rename it > to resolv.somethingelse and lilnks resolv.conf to that. NetworkManager will > not modify a symlink. > > > Regards, > > Christian Caruthers > > Lenovo Professional Services > > Mobile: 757-289-9872 > > > From: SOPORTE MODEMAT via xCAT-user > Sent: Wednesday, December 14, 2022 6:05 PM > To: xCAT Users Mailing list > Cc: SOPORTE MODEMAT > Subject: [External] [xcat-user] search in resolv.conf > Importance: High > > > Hi. > > > > Please tell me how can I get the “search domain” populated in the network > interfaces or “search” in the /etc/resolv.conf in each compute node, I am > using xcat 2.16.4 on Centos 8.4. All the information about domain, forwarders > ands nameserver are in the networks and site table. > > > > Thank you in advance for your help. > > > > Kind regards. > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Booting from one interface and then switching to another interface
You could set up your node object with the DNS name and IP address of the 100G interface, and just set the MAC to that of the 1Gb boot interface. Create a postscript that updates the interface config files (e.g. "ifcfg-p1p1" and "ifcfg-em1") so that traffic flows over the 100G interface when the node reboots into the OS. Set that postscript as the last one in the postscript list. > On 04/12/2022 11:53 AM Hannum, Keith wrote: > > > > We’ve done this in our environment. Use confignetwork to set up your 100G > NID first then, the key is to use setroute to add a route to continue to talk > to the xcat master over the original boot interface, otherwise, xcat gets > requests from an unknown node. > > > > -Keith > > > > Keith Hannum > > keith.han...@lmco.com mailto:keith.han...@lmco.com > > > > From: Jeffrey R. Lang > Sent: Thursday, March 31, 2022 6:05 PM > To: xcat-user@lists.sourceforge.net > Subject: EXTERNAL: [xcat-user] Booting from one interface and then > switching to another interface > > > So I have a new machine that has 4 x 1g ports and a Mellanox HBA in 100G > Ethernet mode.I need to boot the node from the 1g interface (BIOS doesn’t > support booting from the Mellanox HBA) then switch over to using the 100G > after the node is booted. > > > I believe that I have seen this type of request in the past. Anyone out > their doing this that can provide some knowledge on how you were able to > accomplish this? > > > Thanks, > > Jeff > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] option domain-search in dhcpd.conf
You just need to make additional entries in the networks database, one for each additional search domain. Example entries: "a.domain.com","10.10.1.0","255.255.255.0",,"a.domain.com",,, "b.domain.net","10.10.2.0","255.255.255.0",,"b.domain.net",,, "c.domain.us","10.10.3.0","255.255.255.0",,"c.domain.us",,, The "netname", "net", and "mask" are required but you can put in any values. Set the "domain" field to the value you want to have in the search line. tabch netname="a.domain.com" networks.net="10.10.1.0" networks.mask="255.255.255.0" networks.domain="a.domain.com" After you run "makedhcp -n" the dhcpd.conf file will contain the search line: option domain-search "site.com", "a.domain.com", "b.domain.net", "c.domain.us"; Remember that the first entry is the "domain" from the "site" table. > On 04/01/2022 6:13 AM Roosen, Nicolas wrote: > > > On Sat, 2022-03-12 at 12:01 -0500, Russ Auld wrote: > > This definitely works - I've done it myself. > > > > Can you show us your networks table? > > > > > > Right now it is empty of any domain, because I couldn't find the proper > syntax to add multiple domain names in it. > > Could you share your example which works? > > Thanks. > > > > > > On 03/11/2022 6:41 AM Roosen, Nicolas > > > wrote: > > > > > > > > > On Thu, 2022-03-10 at 18:17 +0100, Thomas HUMMEL wrote: > > > > > > > > > > > > On 10/03/2022 11:26, Roosen, Nicolas wrote: > > > > > Hello, > > > > > > > > > > I'm having a hard time to get our nodes /etc/resolv.conf search > > > > > option > > > > > correctly setup. > > > > > > > > Hello, > > > > > > > > just to be sure : are you sure your NetworkManager settings are > > > > correct > > > > (if the nic is NM managed) ? There is a quite complex > > > > combinatorics > > > > about if, how, and by who resolv.conf is handled which may or may > > > > not > > > > involve what's received via DHCP (see dns and rc-manager for > > > > start in > > > > NetworkManager.conf man) > > > > > > Hello, my main concern here is to have xCAT generate a proper > > > dhcpd.conf file with a custom "option domain-search". > > > > > > If I setup the file by hand, it works on the client. > > > > > > But I couldn't find a xCAT way of doing this. > > > > > > > > > ___ > > > xCAT-user mailing list > > > xCAT-user@lists.sourceforge.net > > > INVALID URI REMOVED > > > tinfo/xcat-user__;!!NpxR!0BMrwk9rq2- > > > tCRhqEKSmNvMt6VChWuNWzLFgFURqC7ffhVXdr4aQLhu7Baj5FAME$ > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] option domain-search in dhcpd.conf
This definitely works - I've done it myself. Can you show us your networks table? > On 03/11/2022 6:41 AM Roosen, Nicolas wrote: > > > On Thu, 2022-03-10 at 18:17 +0100, Thomas HUMMEL wrote: > > > > > > On 10/03/2022 11:26, Roosen, Nicolas wrote: > > > Hello, > > > > > > I'm having a hard time to get our nodes /etc/resolv.conf search > > > option > > > correctly setup. > > > > Hello, > > > > just to be sure : are you sure your NetworkManager settings are > > correct > > (if the nic is NM managed) ? There is a quite complex combinatorics > > about if, how, and by who resolv.conf is handled which may or may not > > involve what's received via DHCP (see dns and rc-manager for start in > > NetworkManager.conf man) > > Hello, my main concern here is to have xCAT generate a proper > dhcpd.conf file with a custom "option domain-search". > > If I setup the file by hand, it works on the client. > > But I couldn't find a xCAT way of doing this. > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Node nic config error during install
If you would like the Ethernet interface configured, add it to the nics table. The IP address shown by lsdef is probably coming from the hosts table.The hosts table is useful if you want to build out /etc/hosts by running makehosts.It?s also the right place to list interfaces that are not configured in-band, such as bmc?s. The nics table is where you want to list all in-band interfaces (to be configured by the os).On Thu, May 13, 2021 at 10:58 AM wrote:Going to respond to both of you in a single email.Russell,Was not aware of confignics going away. Will look into replacing that piece correctly.Nathan,The nics table says this about the node:"c001","bmc!172.16.4.26,ib0!172.16.8.26",,,"ib0!Infiniband,bmc!bmc",,"bmc!dracnet,ib0!ibnet","ib0!c001.ib.cluster,bmc!c001.drac.cluster","ib0!Type=InfiniBand CONNECTED_MODE=yes MTU=65520"Also, one of my team mates pointed out that the network-scripts RPM is not being installed by default.Adding that RPM to the osimage template and addingsystemctl enable network-scripts systemctl start network-scripts To the %post section of the template has corrected the issue.It does seem a bit odd though that something like that wouldn't be included and enabled by default.On Thu, May 13, 2021 at 2:59 AM Russell Auldwrote:First, I think confignics has been superseded by confignetwork; Regardless of that, you should only call one of them.Second, are you using the ?nics? table? If not, please look into it. If yes, please show the settings for the node. From: aaronhc...@gmail.com Sent: Wednesday, May 12, 2021 4:13 PMTo: xCAT Users Mailing list Subject: [xcat-user] Node nic config error during install xCAT 2.16.1CentOS 8.3 When the node finishes installing, the network doesn't come up. The postscript is set to do it.postscripts=syslog,remoteshell,syncfiles,confignics --ibaports=1,confignetwork -sHowever, it doesn't. If I attempt to run it manually, I get this error:[root@c001 xcatpost]# /xcatpost/confignetwork -s[I]: NetworkManager is active[E]:Error: Can not determine proper install nic.[I]: There is no other nic device to configure. There is an ifcfg-eno1 file present, which is wrong:[root@c001 network-scripts]# cat ifcfg-eno1# Generated by parse-kickstartTYPE=EthernetDEVICE=eno1UUID=3f02c4a7-daf5-4404-8fe7-43492454ef92_ONBOOT_=yesBOOTPROTO=dhcpIPV6INIT=yesPROXY_METHOD=noneBROWSER_ONLY=noDEFROUTE=yesIPV4_FAILURE_FATAL=noIPV6_AUTOCONF=yesIPV6_DEFROUTE=yesIPV6_FAILURE_FATAL=noNAME="System eno1" There is also an ifcfg-xcat-eno1 file, which is what should be configured.[root@c001 network-scripts]# cat ifcfg-xcat-eno1DEVICE=eno1IPADDR=172.16.0.26NETMASK=255.255.252.0BOOTPROTO=none_ONBOOT_=yesNAME=xcat-eno1AUTOCONNECT_PRIORITY=9MTU=1500 2: eno1: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 18:03:73:1c:77:b8 lsdef for the node shows these:ip=172.16.0.26mac=18:03:73:1c:77:b8 Any idea why this is failing? This is a first attempt to get this set up on 8.x ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Installing the slightly awkward software
Postscripts are the best solution to this problem. Encapsulating the installation procedure for a piece of software lends itself quite well to moving to a configuration management platform later (i.e. Chef, Puppet, or Ansible). It's also self documenting and atomic. The downside is a lack of versioning in a CN build. In other words, which version of the postscript was used when node42 was imaged? There are ways to solve this problem of course, but they tend to make things more complex. > On 07/02/2020 9:00 PM Lachlan Musicman wrote: > > > Hola, > > We are installing Open On Demand which has a requirement of > websockify>0.8.0 which in turn has a python3 requirement. > > CentOS 7 (for eg) only packages an older version of python-websockify, > so we need to deploy manually. IE python3 /path/to/websockify/setup.py > install > > What's the recommended way to do this - do we create a software kit, > or a postscript, or just push it out with xdsh and add some > documentation to remind ourselves to update it when we update > underlying python? > > Cheers > L. > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] nodeset command not working
Try running "mknb x86_64" Also ensure that your site table is setup correctly. If nodeset continues to hang, try using "strace" in another shell on the process to see what it's doing - it must be waiting on something when it's hanging. Double check that all your networking is configured properly, including DNS (both backwards and forwards). > On 07/06/2020 10:03 AM Huette, Antoine wrote: > > > lsdef works fine > > We ended up reinstalling xcat from scratch because it looked like there was > some kind of problem with the xcat data base. > > However after reinstalling and configuring xcat again we still cannot deploy > a compute node. It goes into PXE and then stops immediately, doing nothing. > The xcatprobe -osdeploy command only says « via tftp download xcat/xnba.kpxe > » and « stop at stage download bootloader ». > > The mentioned kpxe file doesn’t exist for some reason. On another cluster > this file is present, and I don’t think we did something special to generate > it. > > Is there a way to generate properly all files related to pxe boot ? > > Regards, > > Antoine Huette > HPC engineer > Bechtle > > > Le 5 juil. 2020 à 16:23, Russ Auld a écrit : > > > > Please ensure that all of the xCAT services are running on the master > > node. You could just restart them all to be safe. > > Are you able to 'lsdef' the node that you're supplying to the 'nodeset' > > command? > > > > > >>> On 07/03/2020 9:42 AM Huette, Antoine wrote: > >> Hello, > >> On an Centos 7.7 cluster with xcat 2.14.6, we have issues using the > >> nodeset command. It simply does nothing, the command hangs and never > >> stops. Using the -V parameter doesn’t show anything either. > >> The nodeset also seem not to change any file. The files in /tftpboot, > >> /install/autoinst and the mypostscript are all empty. > >> The xcat tables all seem to be filled properly, according to the online > >> documentation. > >> Did anyone else have this issue ? > >> Best regards, > >> Antoine Huette > >> HPC engineer > >> Bechtle > >> ___ > >> xCAT-user mailing list > >> xCAT-user@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/xcat-user > > > > > > ___ > > xCAT-user mailing list > > xCAT-user@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xcat-user > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] What is the right way to add new DHCP client to xCAT that is not a node nor a switch
If you have devices that obtain addresses via DHCP and would like to make those addresses static, one way to do it would be to add your own, manually-created leases to "/var/lib/dhcpd/dhcpd.leases" You could either hand-edit the file (make sure that dhcpd is not running when you edit it), or use "omshell" (see man page) to add DHCP lease objects to the running dhcpd daemon process. Running 'makedhcp -n' will only update the dhcp configuration file in /etc, so the leases file should be safe. I would make a copy of the working configuration after you have it all setup correctly. If you want xCAT to manage the leases you would need to have a separate network object (defines the subnet and dhcp range), then manually add the node objects using their MAC addresses and set the IP address on the node object. You would run 'makedhcp ' to make the leases in the lease file. Under the hood, xCAT just runs "omshell" to add the leases. You could leave the node objects as basic as possible. You could also set the chain value to 'boot' to ensure that nothing funny happens with regard to discovery. You may have to experiment a little to get it just right. Of course, the MN should have an address in each of the networks. Ensure that the 'dhcpinterfaces' field in the 'site' table has a valid interface name for each of the networks. Be sure to run 'makedhcp -n' after adding the network object and updating 'dhcpinterfaces'. > On 07/06/2020 10:10 AM Daniel Hilst via xCAT-user > wrote: > > > Thanks Russ, > > This surely helps to get devices in an isolated network but my question > is, if I want to assign them specific IP addresses I still need to create > node objects, and assign IP and mac addresses to this node objects right? > > Regards > > > - > De: Russ Auld > Enviado: domingo, 5 de julho de 2020 11:42 > Para: xCAT Users Mailing list > Assunto: Re: [xcat-user] What is the right way to add new DHCP client to > xCAT that is not a node nor a switch > > xCAT will serve DHCP leases to the network segment used by your compute > nodes. Check the man page for 'networks'. If you want the extra devices to be > in a separate network segment, you can create another network xCAT object. > You set the address range on the network object. > Then 'makedhcp' will rebuild the dhcp files will the correct address > ranges. > > > > On 06/23/2020 12:42 PM Daniel Hilst via xCAT-user > wrote: > > > > > > Hi, > > > > I need to get some peripherals to get IP from DHCP that runs in the > > headnode. My guess is that I should not edit dhcpd.conf because it will be > > overwritten by `makedhcp -n`. I look at the node manual (man node) and see > > that there is no "other" option in nodetype attribute. > > > > So how to add clients (I want they to have fixed address based on > > MAC) that are not nodes nor switches, to xCAT dhcpd? > > > > Regards, > > ___ > > xCAT-user mailing list > > xCAT-user@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xcat-user > > > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] What is the right way to add new DHCP client to xCAT that is not a node nor a switch
xCAT will serve DHCP leases to the network segment used by your compute nodes. Check the man page for 'networks'. If you want the extra devices to be in a separate network segment, you can create another network xCAT object. You set the address range on the network object. Then 'makedhcp' will rebuild the dhcp files will the correct address ranges. > On 06/23/2020 12:42 PM Daniel Hilst via xCAT-user > wrote: > > > Hi, > > I need to get some peripherals to get IP from DHCP that runs in the > headnode. My guess is that I should not edit dhcpd.conf because it will be > overwritten by `makedhcp -n`. I look at the node manual (man node) and see > that there is no "other" option in nodetype attribute. > > So how to add clients (I want they to have fixed address based on MAC) > that are not nodes nor switches, to xCAT dhcpd? > > Regards, > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] nodeset command not working
Please ensure that all of the xCAT services are running on the master node. You could just restart them all to be safe. Are you able to 'lsdef' the node that you're supplying to the 'nodeset' command? > On 07/03/2020 9:42 AM Huette, Antoine wrote: > > > Hello, > > On an Centos 7.7 cluster with xcat 2.14.6, we have issues using the nodeset > command. It simply does nothing, the command hangs and never stops. Using the > -V parameter doesn’t show anything either. > > The nodeset also seem not to change any file. The files in /tftpboot, > /install/autoinst and the mypostscript are all empty. > > The xcat tables all seem to be filled properly, according to the online > documentation. > > Did anyone else have this issue ? > > > Best regards, > > > Antoine Huette > HPC engineer > Bechtle > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] configeth hostname setting on NetworkManager only systems
You should examine your hostname resolution settings - not only your DNS configuration (/etc/resolv.conf) but also '/etc/hosts' and most importantly '/etc/nsswitch.conf'. On the CN, NetworkManager will perform a reverse lookup of the host's IP address, so you just need to control the answer that it gets. I find this command particularly helpful when debugging these things: 'getent hosts ' This command does exactly what getHostNameByAddr() and getHostAddrByName() would do by consulting 'nsswitch.conf' and the databases that it lists for 'hosts'. Barring some organizational policy, you have full control of how the names are mapped to IP addresses, usually via the 'hosts' file, but sometimes with the DNS server running on the head node if you enable that. In short, focus on the source of answers rather than the processes that set or change the hostname values. -Russ > On April 22, 2020 at 1:49 PM Thomas HUMMEL wrote: > > > Hello, > > I'm successfully using xCAT 2.15.1 to provision CentOS 8.1 stateless nodes. > > For that matter my osimage has got the confignetwork -s postscript > configured. > > By default genimage does not generate a hostname-mode directive in > NetworkManager.conf(5) causing it to have its default value. > > This causes the following behavior : > > - osimage without confignetwork -s : node hostname ends up being a non > fqdn name (ex: maestro-1000) : this is good > > - osimage with confignetwork -s : node hostname ends up being the fqdn > name : (ex : maestro-1000.maestro.pasteur.fr) : this is to me not > recommended and anyway not consistent with the above case > > This is because DHCP sends the non fqdn hostname and dracut/initrd sets > the transient hostname to the fqdn hostname and, according to the > semantic of the default value of NetworkManager(5) hostname-mode, the > original hostname (i.e. initrd/dracut fqdn one in this case) is restored > once the DHCP connection is deactivated, which is the case when the > xcat- static profile is created by confignetwork -s takes over. > > This can be worked around by inserting into the image the > hostname-mode=none NetworkManager.conf(5) setting. > > However I saw in configeth script code (which is called be > confignetwork) that the transient hostname is set using the hostname(1) > command (hostname $NODE) > > I think it may be the problem as NetworkManager has no way to be aware > of it (as it uses systemd-hostnamed service as a proxy). > > Wouldn't configeth rather use dbus-send or hostnamectl(1) --transient to > achieve the same ? > > What do you think ? > > Thanks for your help > > -- > Thomas HUMMEL > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
[xcat-user] Expired CA cert
The xCAT CA certificate "/etc/xcat/certs/ca.pem" has expired on our MN. What's the proper procedure to regenerate the SSL certificates? xcatconfig -c ? I found this discussion https://xcat-docs.readthedocs.io/en/stable/advanced/security/certs.html It looks like I could regenerate the CA cert with /opt/xcat/share/xcat/scripts/setup-xcat-ca.sh ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] how to use the specify partition file for all compute node
Your script has to create the /tmp/partitionfile file. Echo the contents to the file.On Fri, Feb 21, 2020 at 10:55 AM peter CZ1 Peng wrote:Hi ,Dear I try to deploy the raid0 software setup on the compute node ,and it work if I manually update the kickstart file after nodeset command run . but now I don't want to manually update the kickstart file ,is there any way to do so ? I try with this and not work for me https://xcat-docs.readthedocs.io/en/stable/guides/admin-guides/manage_clusters/ppc64le/diskful/customize_image/raid_cfg.html#deploy-diskful-nodes-with-raid1-setup-on-redhat I try to do as below : [root@mgt40 autoinst]# cat /install/custom/partition/SW_RAID_SD530_2SSD part /boot/efi --size 500 --fstype efi --ondisk sda part /boot --size 500 --fstype ext3 --ondisk sda part swap --size 512 --ondisk sdb #part /var --size 1024 --ondisk sdb part swap --size 512 --ondisk sda part raid.01 --size 1 --grow --ondisk sda part raid.11 --size 1 --grow --ondisk sdb raid pv.01 --fstype xfs --device pv.01 --level=RAID0 raid.01 raid.11 volgroup sysvg pv.01 logvol / --vgname=sysvg --size=8 --name=root 858 chdef -t osimage -o rhels8.0.0-x86_64-install-compute partitionfile="s:/install/custom/partition/SW_RAID_SD530_2SSD" 859 lsdef -t osimage rhels8.0.0-x86_64-install-compute 860 nodeset node05 osimage=rhels8.0.0-x86_64-install-compute 861 vim node05 862 rsetboot node05 net -u 863 rpower node05 reset But in rcons ,it report that the file is missing 15:34:39 Running pre-installation scripts The following problem occurred on line 0 of the kickstart file: Unable to open input kickstart file: curl#37 - "Couldn't open file /tmp/partitionfile" Best wishes, Peter CZ Peng ??? Global Engineering - Complex Solutions TE Lenovo China Lenovo R & D Center, Gao Xin Road No.16, Nanshan District science and Technology Park, Shenzhen, China Phone: +86 181 2997 7350 peng...@lenovo.com ___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/xcat-user___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] question: partition definition file and/or script - can we associate them with a 'group'?
The best approach is to use a script to generate the partitionfile that gets included in the kickstart file. If you read "partitionfile" section of the man page for linuximage, you'll see that you can set a prefix on the file name that indicates to xCAT that it is a script. For instance in your osimage definition for a Linux machine: partitionfile=s:/install/custom/install/rh/my-partitions.sh The script runs under the Anaconda environment in RHEL systems, which means you have access to Anaconda functions like "list-harddrives" You can parse the output and dynamically generate your disk layout, PV groups, volume groups, whatever you want. Just output the final configuration to /tmp/partitionfile in your script. To pick it up in the kickstart, the template file must contain this block: #XCAT_PARTITION_START# %include /tmp/partitionfile #XCAT_PARTITION_END# I use this all the time to make different disk layouts depending on the number of drives and their sizes. > On February 2, 2020 at 10:03 PM Imam Toufique wrote: > > I have been attaching a partition def. file with osimage for my stateful > installs. > > so, here is my current one: > > bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda > zerombr > clearpart --all --initlabel > part /boot --size 300 --fstype ext3 > part swap --size 4096 > part / --size 1 --grow --fstype xfs > > Now, if I understand correctly, partition file/script cant be associated > with a 'group', only with an 'osimage' . > > the above is good for a single disk setup. If I have 2 drives in a > system and I want software raid (i.e. RAID1, in this case), then I have to > attach another script to it, for the selected 'osimage'. > > How do I make this a bit more dynamic, so I can select a 'group' ( or > something )that will select a specific partition definition file? I can > write something at a higher level to instruct my wrapper script that machine > X has 2 drives, therefore, select a RAID1 specific partition definition file > -- but, I think that's a bit tacky way to do this. > > How do you guys handle this in general? > thanks > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Delete old osimages
I think I asked a question about this a few years ago, and the dev team said that you have to clean up /tftpboot manually. When you make a new osimage, xCAT creates files in /tftpboot for it. If your root partition is limited in size, you could run out of space in /tftpboot. > On February 15, 2020 at 11:31 AM Vinícius Ferrão via xCAT-user > wrote: > > Hi Cassandra, issuing rmdef will remove all the definitions about the > image on xCAT, right? > > But the files will stay on the disk, and they must be manually removed. > > Is there’s anything more than /install/*images* to be deleted? Or this is > sufficient? > > Thanks, > > > > > > On 14 Feb 2020, at 16:35, Casandra H Qiu < > cxh...@us.ibm.com mailto:cxh...@us.ibm.com > wrote: > > > > > > we don't use `rmosdistro`, > > only need `rmdef` command, you may need to issue multiple time > > because there are multiple osimage name with the same DISTRO > > > > > > ... > > Casandra Hong Qiu > > Phone: (845) 433-9291, t/l 293-9291 > > Office: Building 8, 3-B-04 > > cxh...@us.ibm.com mailto:cxh...@us.ibm.com > > > > > > > > Andrew Loftus ---02/14/2020 02:14:31 PM---> ... it > > seems like rmosdistro with the -f flag did what I expected. I'll have to > > take that back. Af > > > > From: Andrew Loftus > mailto:alof...@illinois.edu > > > To: xCAT Users Mailing list > mailto:xcat-user@lists.sourceforge.net > > > Date: 02/14/2020 02:14 PM > > Subject: [EXTERNAL] Re: [xcat-user] Delete old osimages > > > > > > - > > > > > > > > > ... it seems like rmosdistro with the -f flag did what I > > expected. > > I'll have to take that back. After running rmosdistro, lsdef -t > > osimage still lists the images. I had to also run "rmdef" (as pointed out > > by Aaronhcarr and Casandra H Qiu). > > > > So, in summary, what is the proper way to delete an old osdistro > > that was created from copycds? > > > > I was hoping there was a single command to do this, but it seems it > > requires: > > > > 1. lsdef -t osimage | grep | cut -d' ' -f1 | xargs -n1 > > rmdef -t osimage > > 2. rmosdistro > > > > Please add to this list if it is missing anything. > > > > On Fri, Feb 14, 2020 at 1:06 PM Casandra H Qiu < cxh...@us.ibm.com > > mailto:cxh...@us.ibm.com > wrote: `rmimage` only remove the files under > > `rootimgdir=` attribute for the diskless/netboot image. > > > > `rmdef -t osimage imagename ` should remove the image created by > > the `copycds` > > > > what kind of error message did u see when u run `rmdef` command? > > > > > > Casandra Qiu > > > > > > ... > > Casandra Hong Qiu > > Phone: (845) 433-9291, t/l 293-9291 > > Office: Building 8, 3-B-04 > > cxh...@us.ibm.com mailto:cxh...@us.ibm.com > > > > > > > > Andrew Loftus ---02/14/2020 01:50:59 PM---No, I > > didnt' try the -f flag because it didn't seem like the right approach. If > > copycds always creat > > > > From: Andrew Loftus < alof...@illinois.edu > > mailto:alof...@illinois.edu > > > To: xCAT Users Mailing list < xcat-user@lists.sourceforge.net > > mailto:xcat-user@lists.sourceforge.net > > > Date: 02/14/2020 01:50 PM > > Subject: [EXTERNAL] Re: [xcat-user] Delete old osimages > > > > - > > > > > > > > No, I didnt' try the -f flag because it didn't seem > > like the right approach. > > > > > > If copycds always creates an "install" image and rmdef > > doesn't support removing the "install" image, then "rmosdistro" will never > > run without the -f flag. That seems broken to me. > > > > > > That aside, it seems like rmosdistro with the -f flag > > did what I expected. > > > > > > On Fri, Feb 14, 2020 at 11:07 AM Mark Gurevich < > > gurev...@us.ibm.com mailto:gurev...@us.ibm.com > > > wrote: > > Have you tried "rmosdistro -f" to force remove ? > > > > Mark Gurevich > > Poughkeepsie Development Lab > > HPC Software Development - xCAT > > > > "If we knew what it was we were doing, it would not be called > > research, would it?" > > --Albert Einstein > > > > > > > > Andrew Loftus ---02/14/2020 12:00:33 PM---Is there a > > way to clean up old osimages (created from copycds)? I tried "rmosdistro" > > but that fails > > > > From: Andrew Loftus < alof...@illinois.edu > > mailto:alof...@illinois.edu > > > To: xCAT Users Mailing list < xcat-user@list
Re: [xcat-user] DNS Setup
Echoing what Dave said: Be sure that your "/etc/resolv.conf" file includes a line like "search cluster.com" Also be aware that reverse lookup is important, so ensure that you have an entry for your master node in "/etc/hosts" with the public IP address and the full name of the machine. For instance: 127.0.0.1 localhost 52.16.78.90 master.cluster.com master If your machine is registered in your external DNS source (the ones that you list in resolv.conf) then you don't need this step. > On October 18, 2019 at 3:39 PM david_john...@brown.edu wrote: > > Is your resolv.conf set up with a domain directive or a search directive? > New nodes should get their resolv.conf configured automatically but if you > have only just now made the change on your management node it may not > reconfigure the resolver for you. > > -- ddj > Dave Johnson > > > > > On Oct 18, 2019, at 3:34 PM, Anthony Ruth > wrote: > > > > > > > > > > Hello, > > > > I am trying to set up xcat on Ubuntu server 18.04. I am stuck at > > the point of setting up DNS. > > > > I am mainly following the guide located > > at:http://xcat-docs.readthedocs.io/en/stable/guides/get-started/quick_start.html > > > > I am able to get through the setup of the management node. When I > > reach the makehosts command under stage 1, I receive a message that the > > domain name is not set: > > > > root@clusteradmin:/home/anthony# makehosts steve0 > > Warning: [clusteradmin]: No domain can be determined for node > > 'steve0'. The domain of the xCAT node must be provided in an xCAT network > > definition or the xCAT site definition. > > > > I found an additional guide which appears to fix this issue at: > > https://xcat-docs.readthedocs.io/en/stable/advanced/domain_name_resolution/domain_name_resolution.html > > > > The domain is declared by: > > > > > > chdef -t site domain=cluster.com > > > > I then did the makehosts command and makedns -n which both appear > > successful. I tried the dns test using > > > > root@clusteradmin:/home/anthony# nslookup steve0 192.168.250.3 > > Server: 192.168.250.3 > > Address: 192.168.250.3#53 > > > > ** server can't find steve0: NXDOMAIN > > > > However, I can find steve if I include the domain > > > > root@clusteradmin:/home/anthony# nslookuphttp://steve0.cluster.com > > 192.168.250.3 > > Server: 192.168.250.3 > > Address: 192.168.250.3#53 > > > > Name:http://steve0.cluster.com > > Address: 192.168.250.200 > > > > This seems strange to me. Originally makehost steve0 was > > complaining that it did not have a domain for steve. Then we set a domain > > and makehost became happy, but nslookup is not happy and does not know > > steve0's domain. Trying xcatprobe xcatmn fails due to lack of dns. I do not > > understand the -i warning > > > > root@clusteradmin:/home/anthony# xcatprobe xcatmn > > [mn]: Checking all xCAT daemons are running... > > [ OK ] > > [mn]: Checking xcatd can receive command request... > > [ OK ] > > [mn]: Checking 'site' table is configured... > > [ OK ] > > [mn]: No interface provided by '-i' option, detected site table IP > > attribute 192.168.250.3, c...[WARN] > > [mn]: If this is incorrect, rerun with -i option > > [WARN] > > [mn]: Checking provision network is configured... > > [ OK ] > > [mn]: Checking 'passwd' table is configured... > > [ OK ] > > [mn]: Checking important directories(installdir,tftpdir) are > > configured... [ OK ] > > [mn]: Checking SELinux is disabled... > > [ OK ] > > [mn]: Checking HTTP service is configured... > > [ OK ] > > [mn]: Checking TFTP service is configured... > > [ OK ] > > [mn]: Checking DNS service is configured... > > [FAIL] > > [mn]: DNS service isn't ready on 192.168.250.3 > > [mn]: Checking DHCP service is configured... > > [ OK ] > > [mn]: Checking NTP service is configured... > > [ OK ] > > [mn]: Checking rsyslog service is configured... > > [ OK ] > > [mn]: Checking firewall is disabled...
Re: [xcat-user] Stateless nodes hostkeys
The postscript you want is 'remoteshell'. It will install the _same_ host keys on all nodes. If you bake host keys into the image, the sshd daemon will not create new keys when it starts. Since the host keys are fixed, you can create a 'ssh_known_hosts' file with entries for each node and distribute it to your login/submit/bastion hosts. There was a discussion regarding this behavior on the list recently. I recommend reading the message from Jarrod Johnson. > On October 11, 2019 at 9:15 AM Thomas HUMMEL wrote: > > > Hello, > > For an HPC cluster, using xCAT-server-2.14.6 on CentOS 7.7 x86_64, I'm > booting stateless nodes from a single osimage. > > My question is about how to deal with the fact that their ssh hostkeys > change each time they boot. > > Previously only the HPC "submit" node could ssh to the compute nodes so > we made an ssh_config file on it which would ignore the change of ssh > hostkeys of the computes. > > Now almost anyone will be allowed to ssh to the compute nodes, thus the > need for those to always have the same ssh hostkey across reboots. > > What is the best way to implement this ? > > I'm not sure about xcatconfig (and what are the keys in > /etc/xcat/hostkeys for). > > My idea was to externally generate one host key per node on the > management node just after the node creation (nodeadd) and to sync them > using a postscript (not sure if the postscript would occur soon enough, > i.e. before sshd-keygen.service, though) > > What do you think ? > > Thanks > > -- > Thomas HUMMEL > > > > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Installing node from a different subnet
Can you use curl or wget to download the kickstart file? Does the file look reasonable? Can you use a different node in the other subnet to download the ks file?On May 21, 2018 7:43 AM, Fabio Moreira wrote:Hi,Yes, I've run the nodeset command in order to generate the kickstart file and the file /install/autoinst/node01 exists on the master server.Em dom, 20 de mai de 2018 às 09:28, Russell Auldescreveu: Did you run "nodeset" to generate the kickstart file for node01?In other words, does /install/autoinst/node01 exist on the master server?On May 18, 2018 at 10:14 AM Fabio Moreira wrote: Hi,I'm trying to install a node from a different subnet of my Management xCAT server. We've set up a DHCP server in the node's subnet and change the "next server" to the xCAT server. The node boots from xnba but the installation stops at stage 'download_kickstart'.See the exit from probe the node01:xcatprobe osdeploy -n node01The install NIC in current server is eno1 [INFO]All nodes to be deployed are valid [ OK ]-Start capturing every message during OS provision process-[node01] 09:55:25 Via HTTP get /tftpboot/xcat/xnba/nodes/node01[node01] 09:55:25 Via HTTP get /tftpboot/xcat/osimage/__centos7.4-x86_64-install-compute...[node01] 09:55:25 Via HTTP get /tftpboot/xcat/osimage/__centos7.4-x86_64-install-compute...[node01] 09:55:49 Via HTTP get /install/centos7.4/x86_64/.treeinfo[node01] 09:55:49 Via HTTP get /install/centos7.4/x86_64/LiveOS/squashfs.img[node01] 09:55:49 Via HTTP get /install/centos7.4/x86_64/images/updates.img[node01] 09:55:49 Via HTTP get /install/centos7.4/x86_64/images/product.img[node01] 09:55:57 Via HTTP get /install/autoinst/node0160 minutes have expired, stop monitoring [INFO]== Summary =There is 1 node provision failuresnode01 : stop at stage 'download_kickstart' [FAIL]Any tips?Best Regards,Fábio Souza--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdot___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/xcat-user -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user -- Fábio MS -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Confignetworks and default route
Are you running the latest version of xCAT? The script is named "confignetwork". You could download the latest version of that script from github.On Jan 10, 2018 3:48 AM, Nathan Harper wrote:Hi Yuan,Unfortunately confignetworks doesn't appear to be setting the gateway for me, despite it being set in the networks table. I'll take a look at your other suggestions though, thanks.On 10 January 2018 at 02:45, Yuan Y Bai <bybai@cn.ibm.com> wrote: I missed one line in last mail: confignetwork migrate routes to new interface. you can also use nicextraparams in nics table to customize something for specific interface, the nicextraparams content will be added into ifcfg-xxx file. Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: bybai@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.China 100193IBM环宇大厦北京市海淀区东北旺西路8号,中关村软件园28号楼邮编:100193 - Original message -From: "Yuan Y Bai" <bybai@cn.ibm.com>To: xcat-user@lists.sourceforge.netCc: xcat-user@lists.sourceforge.netSubject: Re: [xcat-user] Confignetworks and default routeDate: Wed, Jan 10, 2018 10:12 AM Hi Nathan, Thanks Russ. We cannot configure gateway in nics table. You can configure gateway in networks table for the specific networks , confignetwork also use the gateway from networks table, but confignetworks If you want to configure the default gateway as the static gateway, after running confignetworks , you can use makeroutes or setroute script to do that , here is my draft doc for these 2 command/script https://github.com/xcat2/xcat-core/pull/4580/commits/e9fd1c9e345997e409b54229be159bafadc3de73 Best Regards--Yuan Bai (白媛)CSTL HPC System Management DevelopmentTel:86-10-82451401E-mail: bybai@cn.ibm.comAddress: IBM ZGC Campus. Ring Building 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District,Beijing P.R.China 100193IBM环宇大厦北京市海淀区东北旺西路8号,中关村软件园28号楼邮编:100193 - Original message -From: Russ Auld <russauld@comcast.net>To: xCAT Users Mailing list <xcat-user@lists.sourceforge.net>Cc:Subject: Re: [xcat-user] Confignetworks and default routeDate: Wed, Jan 10, 2018 8:28 AM The gateway field should be used to set the default route. Make sure there's just one gateway set if you use multiple nics, otherwise the last one will win. On Jan 9, 2018 12:16 PM, Nathan Harper <nathan.harper@cfms.org.uk> wrote: Hi, We've been using confignetworks post OS install to take the installnic and bond it with another interface. As the default gateway is set by DHCP, is there some config I'm missing in the nics table to get it to set the default gateway? -- Nathan Harper // IT Systems Lead e: nathan.harper@cfms.org.uk t: 0117 906 1104 m: 0787 551 0891 w: www.cfms.org.uk CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP --Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=uiTcPxjMR44SPRNNb6l_nA&m=wUdq-uqfTIZWkxxk9T5fq0Ms-QlKJVymzSNR1jIlnGc&s=i8XDzvVqchn8wGhQme7nVtB0-_M9R4j-KdhkqIfJb4U&e= ___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=uiTcPxjMR44SPRNNb6l_nA&m=wUdq-uqfTIZWkxxk9T5fq0Ms-QlKJVymzSNR1jIlnGc&s=pja89jnFc264DoFp_9moXhy7Dek9Iwaa-UjoMZFKOzo&e= --Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! https://urldefense.proofpoint.com/v2/url?u=http-3A__sdm.link_slashdot&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=uiTcPxjMR44SPRNNb6l_nA&m=di4U8_rLCg5toqXle7vmgzpG20nBiVUT4oD9fvMG6D8&s=1AcmAI7smXKekNM6RFIRMi_pF8ZnZbXVbw3aaNPTrn4&e= ___xCAT-user mailing listxCAT-user@lists.sourceforge.nethttps://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_xcat-2Duser&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=uiTcPxjMR44SPRNNb6l_nA&m=di4U8_rLCg5toqXle7vmgzpG20nBiVUT4oD9fvMG6D8&s=82RnRo29CkkG45FNQ0S5-fLghIkcGAuTmagPxZ2-AWs&e= --
Re: [xcat-user] Confignetworks and default route
The gateway field should be used to set the default route. Make sure there's just one gateway set if you use multiple nics, otherwise the last one will win. On Jan 9, 2018 12:16 PM, Nathan Harper wrote:Hi,We've been using confignetworks post OS install to take the installnic and bond it with another interface.As the default gateway is set by DHCP, is there some config I'm missing in the nics table to get it to set the default gateway?-- Nathan Harper // IT Systems Leade: nathan.harper@cfms.org.uk t: 0117 906 1104 m: 0787 551 0891 w: www.cfms.org.uk CFMS Services Ltd // Bristol & Bath Science Park // Dirac Crescent // Emersons Green // Bristol // BS16 7FR CFMS Services Ltd is registered in England and Wales No 05742022 - a subsidiary of CFMS Ltd CFMS Services Ltd registered office // 43 Queens Square // Bristol // BS1 4QP -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Debugmode solves stateful installation hang after postscripts
Ensure that the node can ssh back to the MN in the anaconda environment. The updateflag.awk script can hang trying to update the node's status at the end of postscripts.On Jan 1, 2018 9:32 AM, Daniel Letai wrote: Hello, I have encountered a strange issue where sending any node to rinstall "hangs" after finishing the postscripts - it never reboots, and therefore never continue to the postbootscripts. Trying to diagnose the issue led to the strange bit. Setting xcatdebugmode=1 in site table SOLVED the issue while still not showing any error in any log. We have verified this is indeed the case - setting it to 0 reverts to non functioning rinstall, re-setting to 1 and rinstall works without an issue. We would like to work without debugmode - what might be the issue and how can we solve this? xCAT version - 2.13.8 xCAT node OS - RHEL 7.4 Nodes OS - RHEL 6.5 Thanks, Daniel Letai -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
[xcat-user] Bootparams table is empty
I'm setting up a new xCAT master on a RHEL7 system. I'm using v2.13.8.I ran copycds on the rhel7 iOS.When I run nodeset the bootparams table remains empty and there is no initrd or kcmdline set for the node. Dracut then fails when the node boots. It's set for pxe boot and it does do that. What updates the bootparams table? -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Local scratch for stateless compute nodes
If you're using netboot and local disk, then isn't that "statelite"?Do the satellite instructions not work? On Nov 27, 2017 9:26 AM, Gilad Berman wrote: All, I would like to join this question – Does even localdisk works with stateless? From the docs it seems that should be supported (because it is under stateless), however – - the instructions are taken from statelite and refer to statelite code (litefile) - The rc.localdisk code is under statelite - In the linuximage man – “Partitionfile - Only available for diskful osimages and statelite osimages(localdisk enabled)“ A very quick trial on my statless nodes results in nothing J, it seems there is simply no reference to localdisk with stateless. So, can someone please help clarify it? ** as always, there is a chance I missed something very basic and it should be working J THX in advance! Gilad Berman HPC Architect Lenovo EMEA +972-52-2554262 gberman@lenovo.com Lenovo.com Twitter | Facebook | Instagram | Blogs | Forums From: Vinícius Ferrão [mailto:ferrao@versatushpc.com.br] Sent: Wednesday, November 22, 2017 4:09 AM To: xcat-user@lists.sourceforge.net Subject: [xcat-user] Local scratch for stateless compute nodes Hello, I would like to enable swap and local /tmp on my stateless nodes, but after following the documentation on the following link nothing appears to work: http://xcat-docs.readthedocs.io/en/stable/advanced/hierarchy/provision/diskless_sn.html I’m aware that the documentation is for service nodes and not for compute nodes, but I was thinking the procedure would be similar. At this point I’m with this settings on osimage: [root@headnode xcat]# lsdef -t osimage centos7.4-x86_64-netboot-compute exlist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.exlist imagetype=linux osarch=x86_64 osdistroname=centos7.4-x86_64 osname=Linux osvers=centos7.4 otherpkgdir=/install/post/otherpkgs/centos7.4/x86_64 partitionfile=/install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile permission=755 pkgdir=/install/centos7.4/x86_64 pkglist=/opt/xcat/share/xcat/netboot/centos/compute.centos7.pkglist postinstall=/opt/xcat/share/xcat/netboot/centos/compute.centos7.postinstall profile="" /> provmethod=netboot rootimgdir=/install/netboot/centos7.4/x86_64/compute synclists=/install/custom/netboot/compute.synclist And the content of the partition file is the following: cat /install/custom/netboot/centos7.4-x86_64-netboot-compute/partitionfile enable=yes enablepart=yes [disk] dev=/dev/sda clear=yes parts=10,90 [swapspace] dev=/dev/sda1 [localspace] dev=/dev/sda2 fstype=xfs Finally the following commands were executed: chtab priority=7.1 policy.commands=getpartition policy.rule=allow chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/var/log/ litefile.options=localdisk chtab litefile.image=centos7.4-x86_64-netboot-compute litefile.file=/tmp/ litefile.options=localdisk After a new genimage/packimage the local scratch does not appears to be made nor working. Someone knows what to do next? Thanks, V. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Xcat setup -- stateful provisioning for centOS
Please start here: https://sourceforge.net/p/xcat/wiki/XCAT_iDataPlex_Cluster_Quick_Start/ On Fri, 2017-10-20 at 16:54 -0700, Imam Toufique wrote: > Hello everyone, > > I am very new to xcat, installed it an hour ago ;-) . I need your > help on setting up stateful provisioning for centOS. > > Can you share any documents that are concise enough to get me started > on this? I see many documents in the web, but I have not been able to > figure out what is the right way to set this up. > > Thanks a lot in advance! > > > --- > --- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Problem with syncfiles after install.
Typically this is indicative of a problem with ssh connectivity between the compute node and the master/service node. Ensure that DNS is also working correctly. On Mon, 2017-10-09 at 19:00 +0300, Hakan Bayındır wrote: > Hello All, > > I'm having a problem with the default syncfiles script, which is > fired by default as a "postscript" after an installation. The > installation completes as it should be and the system reboots, > however I get an error in the xcat.log which states that the > syncfiles have returned 1, hence failed. The strange thing is, when I > fire the same syncfile with xdcp, everything works as it should. Did > anyone had this error and nudge me in the right direction? > > Best regards, > > Hakan > > Hakan BAYINDIR > Uzman Araştırmacı > Ağ Teknolojileri Birimi > YÖK Binası B-5 Blok Kat:4 > 06539 Bilkent ANKARA > T +90 312 298 9373 > F +90 312 266 5181 > [ http://www.ulakbim.gov.tr/ | www.ulakbim.gov.tr ] > hakan.bayin...@tubitak.gov.tr > . > ... > > [ http://www.ulakbim.gov.tr/ ] > [ http://www.tubitak.gov.tr/sorumlulukreddi | > Sorumluluk Reddi ] > > --- > --- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Preventing xCAT from updating systems.
I was thinking that this must be CentOS - thanks for confirming that. I was also thinking that a postscript could be used to interfere with the update process. I'm not sure if there is an official/best-practice approach to solving this. I would disable all non-xCAT repos with a post-script, and place that in the list before otherpkgs. In general I don't want my nodes getting updates from non-xCAT sources. If you feel that you want that, you could create another postscript that re-enables the non-xCAT repos and run that after otherpkgs. I would avoid modifying otherpkgs because when you update xCAT later, your changes could get overwritten. Or, if you make your own otherpkgs variant, e.g. "my_otherpkgs", you would have to manually merge changes from the stock otherpkgs into your version. Good luck and let us know what you decide to do. On Wed, 2017-10-04 at 16:18 +0300, Hakan Bayındır wrote: > Hi Russ, > > The default files for internet repositories are present. Base, > Updates > and Extras are enabled. My extra repository files and xCAT's own > repositories are also present. > > The systems get all the updates from the internet it seems. > > How can I disable the default CentOS repositories then? I guess the > answer is a small post-install script, but I'm interested in the best > practice. > > Thanks for all the help, > > Regards, > > Hakan > > On 10/04/2017 03:37 PM, Russ Auld wrote: > > You are correct - my mistake. > > > > Can you look at the repo files that get created in > > /etc/yum.repos.d/ > > (assuming RHEL/Centos)? > > > > In general, the repos will be a result of the directories set for > > "pkgdir" and directories called out in the otherpkgs file in the > > osimage. > > > > Are the machines reaching out to the internet for updates? > > > > > > > > > > On Wed, 2017-10-04 at 15:18 +0300, Hakan Bayındır wrote: > > > Hi Russ, > > > > > > Thanks for your answer. To test, I reinstalled the host with only > > > "otherpkgs" post boot script. After checking the duplicates (with > > > /usr/bin/yum --showduplicates list ), it actually > > > ran > > > "yum > > > -y upgrade" and upgraded to system to latest packages before > > > installing > > > my other packages. > > > > > > Actually in file /install/postscripts/otherpkgs, line 783; the > > > script > > > explicitly calls result=`eval $envlist yum -y upgrade 2>&1` and > > > upgrades > > > the whole system. > > > > > > I can modify the script, but wanted to ask whether there's a > > > better > > > way > > > to modify behavior of the said script. > > > > > > Regards, > > > > > > Hakan > > > > > > On 10/04/2017 02:27 PM, Russ Auld wrote: > > > > I'm pretty sure that it does not update the operating system to > > > > the > > > > latest version automatically. > > > > > > > > The "otherpkgs" postscript creates repos based on the files and > > > > their > > > > directories specified in the "otherpkgs" file set in the > > > > "osimage". > > > > It then uses the platform's package manager to install the > > > > packages > > > > listed in the "otherpkgs" file. > > > > It doesn't run "yum update" (for instance) as you are > > > > suggesting. > > > > > > > > You should take a closer look at the postscripts that are being > > > > run > > > > and > > > > the repositories that are created on the node. > > > > > > > > > > > > On Wed, 2017-10-04 at 13:55 +0300, Hakan Bayındır wrote: > > > > > Hello all, > > > > > > > > > > The xCAT's otherpackages script updates the operating system > > > > > to > > > > > the > > > > > latest version automatically. Is there a way, or best > > > > > practice to > > > > > install "other packages" without updating the operating > > > > > system to > > > > > the > > > > > latest version? > > > > > > > > > > Thanks in advance, > > > > > > > > > > Regards, > > > > > > > > > > Hakan Bayindir > > > > > -
Re: [xcat-user] Preventing xCAT from updating systems.
You are correct - my mistake. Can you look at the repo files that get created in /etc/yum.repos.d/ (assuming RHEL/Centos)? In general, the repos will be a result of the directories set for "pkgdir" and directories called out in the otherpkgs file in the osimage. Are the machines reaching out to the internet for updates? On Wed, 2017-10-04 at 15:18 +0300, Hakan Bayındır wrote: > Hi Russ, > > Thanks for your answer. To test, I reinstalled the host with only > "otherpkgs" post boot script. After checking the duplicates (with > /usr/bin/yum --showduplicates list ), it actually ran > "yum > -y upgrade" and upgraded to system to latest packages before > installing > my other packages. > > Actually in file /install/postscripts/otherpkgs, line 783; the script > explicitly calls result=`eval $envlist yum -y upgrade 2>&1` and > upgrades > the whole system. > > I can modify the script, but wanted to ask whether there's a better > way > to modify behavior of the said script. > > Regards, > > Hakan > > On 10/04/2017 02:27 PM, Russ Auld wrote: > > I'm pretty sure that it does not update the operating system to the > > latest version automatically. > > > > The "otherpkgs" postscript creates repos based on the files and > > their > > directories specified in the "otherpkgs" file set in the "osimage". > > It then uses the platform's package manager to install the packages > > listed in the "otherpkgs" file. > > It doesn't run "yum update" (for instance) as you are suggesting. > > > > You should take a closer look at the postscripts that are being run > > and > > the repositories that are created on the node. > > > > > > On Wed, 2017-10-04 at 13:55 +0300, Hakan Bayındır wrote: > > > Hello all, > > > > > > The xCAT's otherpackages script updates the operating system to > > > the > > > latest version automatically. Is there a way, or best practice to > > > install "other packages" without updating the operating system to > > > the > > > latest version? > > > > > > Thanks in advance, > > > > > > Regards, > > > > > > Hakan Bayindir > > > --- > > > > > > --- > > > Check out the vibrant tech community on one of the world's most > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > ___ > > > xCAT-user mailing list > > > xCAT-user@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/xcat-user > > > > - > > - > > Check out the vibrant tech community on one of the world's most > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > ___ > > xCAT-user mailing list > > xCAT-user@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xcat-user > > > > --- > --- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Preventing xCAT from updating systems.
I'm pretty sure that it does not update the operating system to the latest version automatically. The "otherpkgs" postscript creates repos based on the files and their directories specified in the "otherpkgs" file set in the "osimage". It then uses the platform's package manager to install the packages listed in the "otherpkgs" file. It doesn't run "yum update" (for instance) as you are suggesting. You should take a closer look at the postscripts that are being run and the repositories that are created on the node. On Wed, 2017-10-04 at 13:55 +0300, Hakan Bayındır wrote: > Hello all, > > The xCAT's otherpackages script updates the operating system to the > latest version automatically. Is there a way, or best practice to > install "other packages" without updating the operating system to the > latest version? > > Thanks in advance, > > Regards, > > Hakan Bayindir > --- > --- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Pointers to using confluent without xCAT?
I feel like this has been discussed before on the list. Did you search the list archives? On Wed, 2017-08-23 at 15:26 +1000, Chris Samuel wrote: > Hi folks, > > We have been asked to help a group out with their (non-xCAT) cluster > and one > of the things they are missing is any console logging of their bare > metal > nodes. > > I was looking at using conserver from xCAT as we know how to > configure that, > but then remembered confluent but as we've not had the chance to play > with it > yet (sorry Jarrod) I'm not sure how feasible it would be to run that > outside > of xCAT? > > Are there any pointers on configuring & running it in the absence of > xCAT? > > All the best, > Chris -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
[xcat-user] tftpboot folder size
I noticed that the size of the /tftpboot folder increases over time. Each osimage definition creates a new directory like /tftpboot/xcat/osimage/[image_name] that contains "initrd.img" and "vmlinuz". Do I have to manually clean up this folder? If I "rmdef -t osimage [image_name]" the contents of the /tftpboot/xcat/osimage/[image_name] folder do not get removed. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] Pem files are world readable on imaged node
On further inspection, these PEM files are being copied to the nodes from the MASTER node when they are provisioned. The PEM files aren't owned by any package - I'll assume that they are created during the installation of xCAT. Is it safe to lock these files down to mode 0600? /install/postscripts/_xcat/ca.pem /install/postscripts/ca/ca-cert.pem -Russ On Thu, 2017-04-27 at 10:23 -0400, Russell Auld wrote: > I just noticed that there are two world-readable pem files in > /xcatpost after a diskfull image of a node. > Shouldn't those files be restricted or deleted? > > --- > --- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xcat-user > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] configfirewall postscript
Need to know the version of xcat that you are using. Original message From: Kilian Cavalotti Date: To: xCAT Users Mailing list Subject: [xcat-user] configfirewall postscript Hi all, I'm trying to use the configfirewall from /install/postscripts to configure iptables on a stateful node. I added the script to the node's postscripts.postbootscripts, but it complains about a missing Perl dependency when executing. Is that expected? # nodels login postscripts.postbootscripts sugi-ln01: setroute,configfirewall --private eth1 --public eth0 --nat --ports 22 At installation, I see this: Tue Mar 3 06:53:00 PST 2015 Running postscript: configfirewall --private eth1 --public eth0 --nat --ports 22 Can't locate xCAT/Utils.pm in @INC (@INC contains: /opt/xcat/lib/perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./configfirewall line 12. BEGIN failed--compilation aborted at ./configfirewall line 12. Is that expected? Thanks, -- Kilian -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
Re: [xcat-user] How do provisioned nodes get their hostkeys?
Lisa and Wang, thanks for clarifying!However, on our provisioned nodes, I can confirm that the public hostkeys only live in /xcat/postscripts/hostkeys.There is no code in the remoteshell script that would copy them to /etc/ssh.Also, there is no sudoer postscript.These issues appear to have been addressed in version 2.8.4Another good reason to upgrade!Again, thanks. Original message From: Lissa Valletta Date: To: xCAT Users Mailing list Subject: Re: [xcat-user] How do provisioned nodes get their hostkeys? When you run updatenode -k, it runs the remoteshell script on the node. This will update the ssh hostkeys on the node. Run it in verbose mode and you will see. Updatenode first calls xdsh -K to setup the root/.ssh keys, which is why it prompts for root password; so that we have passwordless ssh setup to the node. Once this is done, it can the call xdsh command below to run the remoteshell script. This is 2.8 but yours will be similar. updatenode compute-01 -k manage-02: Internal call command: xdsh compute-01 --nodestatus -s -v -e /install/postscripts/xcatdsklspost 5 -m x.xx.xx.xx 'remoteshell' --tftp /tftpboot --installdir /install --nfsv4 no -c -F -V The remoteshell script running on the node calls getcredentials.pm on the Management Node ( or Service node if hierarchical) to send it the private hostkeys. These must be downloaded securely. If you notice in /install/postscripts/hostkeys are the matching public hostkeys. These do not need a secure transfer, so they are just copied from the directory, since this directory is on the node. The keys in /install/postscript/hostkeys are are definitely used. remoteshell must be in your postscripts list for this to happen, which is why it is a default. For install, process is similar, after the install, the postscripts will be wget to the node into /xcatpost directory. The /xcatpost/mypostscript file will be sent to the node and run. That file list the postscripts for the node, one of which is remoteshell. Again you are running remoteshell on the node and the processing is as above. Lissa K. Valletta 8-3/B10 Poughkeepsie, NY 12601 (tie 293) 433-3102 Xiao Peng Wang ---10/09/2014 05:46:17 AM---See my comments inside the previous email. Thanks From: Xiao Peng Wang To: xCAT Users Mailing list Date: 10/09/2014 05:46 AM Subject: Re: [xcat-user] How do provisioned nodes get their hostkeys? See my comments inside the previous email. Thanks Best Regards -- Wang Xiaopeng (王晓朋) IBM China System Technology Laboratory Tel: 86-10-82453455 Email: w...@cn.ibm.com Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193 "Russell Auld" wrote on 2014/10/09 09:42:45: > From: "Russell Auld" > To: > Date: 2014/10/09 09:42 > Subject: [xcat-user] How do provisioned nodes get their hostkeys? > > I’m trying to determine how the hostkeys end up on a provisioned > node (stateful). It appears that the keys live in /etc/xcat/hostkeys. > The man page for updatenode indicates that it can manage hostkeys > for the nodes with the -k option. > However, the “remoteshell” postscript also appears to manage hostkeys. > It will use the getcredentials.awk script to pull hostkeys from the > management node and install them in /etc/ssh. > The Perl module /opt/xcat/lib/perl/xCAT_plugin/credentials.pm will > handle the requests and respond by reading from /etc/xcat/hostkeys. > The question is, when a node is provisioned, and assuming that > ‘remoteshell’ is in the postscript list, which of these two scripts > is used to set the hostkeys? During deployment, the remoteshell will be used. updatenode will rerun all the postscripts, so if remoteshell in the postscript attribute, it will be run too. updatenode -k can be used update the /root/.ssh/authorized_keys for the target compute node. This is useful when remoteshell failed during the os deployment. > Do all the relevant host keys live in /etc/xcat/hostkeys? Yes > Additionally, it appears that there are copies of the public host > keys in /xcat/postscripts/hostkeys (or /install/postscripts/ > hostkeys), however they don’t appear to be used by anything in xCAT > (i.e. distributed to the provisioned nodes). Can someone confirm > and/or comment on why they are in this place to begin with? The ones in /install/postscripts/hostkeys are used by postscript sudoer. I think you can just ignore it. They are just a copy. > This is xCAT 2.7.5 on RHEL 6.4 > Thanks! > -- > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/g