Morning all,
This has taken me days to sort out and lots of searching the net but finally
have ubuntu 22.04 deploying on xcat2 for pxe and xnba.
I'm sure there are a load of ways to complete this however this is how I did it
(obviously modify paths to suit your xcat setup):
Tables:
linuximage:
"ubuntu22.04-x86_64-install","/install/vessel/templates/subiquity.tmpl",,,"/install/vessel/templates/ubuntu22.pkglist","/install/global/os/ubuntu22.04/x86_64",,,,,,,,,,,,,,,,,,
osimage:
"ubuntu22.04-x86_64-install",,"v.ubuntu22","linux",,"install",,,,,"Linux","ubuntu22.04","x86_64","/install/vessel/templates/v.synclist",,,,,,,,
noderes: "ubuntu22",,"xnba",,,,,,"enp3s0f0","enp3s0f0",,,,,,,,,,,, #<-- not
sure if you need to specify the nic. Will test without later
nodehm: "ubuntu22","ipmi","ipmi",,,,,,,,,,,,,
nodetype:
"v.ubuntu22","ubuntu22.04","x86_64","v.ubuntu22","ubuntu22.04-x86_64-install",,,,
presecripts: "ubuntu22",,"vessel/v.ubuntu-xcat-installer",,
For our example node of shark099 the nodelist tab is as follows:
nodelist: "shark099","v.ubuntu22,ubuntu22,all",
Iso image:
Put the ubuntu-22.04.5-live-server-amd64.iso file in /install/isos
Extract the iso:
# copycds ubuntu-22.04.5-live-server-amd64.iso
We then have to move the extract directory /install/ubuntu22.04.5 to
/install/global/os/ubuntu22.04.5 and create a symlink to ubuntu22.04.
This is just our site and setup. You dont have to do this but you will need to
make sure the naming is consistent elsewhere.
Important for PXE installs:
/install/vessel/syncfiles/ubuntu/ldlinux.c32 #<-- this file comes from an
existing ubuntu install. You need to make sure you have syslinux-common on the
source system and then copy the file over. The pre script below copies this to
/tftpboot which is required for PXE to work
Scripts and templates
The template file "/install/vessel/templates/subiquity.tmpl" is as follows
(with sensitive info redacted):
#cloud-config
autoinstall:
version: 1
user-data:
hostname: #HOSTNAME#
disable_root: false
package_update: true
package_upgrade: true
timezone: UTC
keyboard:
layout: us
toggle: null
variant: ''
locale: en_US
storage:
layout:
name: direct
network:
version: 2
ethernets:
id0:
match:
macaddress: '#TABLE:mac:$NODE:mac#'
addresses:
- __NODEIP__/24
nameservers:
addresses: [#TABLE:site:key=master:value#, 8.8.8.8]
routes:
- to: default
via: '__GATEWAY__'
ssh:
allow-pw: true
install-server: true
authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu...........
apt:
primary:
- arches: [amd64]
uri: http://us.archive.ubuntu.com/ubuntu/
packages:
- bash
- nfs-common
- openssl
- isc-dhcp-client
- libc-bin
- openssh-server
- openssh-client
- wget
- vim
- rsync
- busybox-static
- gawk
- dnsutils
- chrony
- gpg
- snmp
- apt-transport-https
- ca-certificates
- curl
- gnupg
- lsb-release
- ipmitool
early-commands:
- echo "Hi!"
late-commands:
- echo "PermitRootLogin yes" >> /target/etc/ssh/sshd_config
- echo "StrictHostKeychecking=no" >> /target/etc/ssh/ssh_config
- systemctl daemon-reload
- mkdir -p /target/root/.ssh
- echo "#TABLE:site:key=idrsapub:value#" >>
/target/root/.ssh/authorized_keys
- systemctl restart sshd
- echo "Copy /etc/hosts file"
- wget http://#XCATVAR:XCATMASTER#:80/install/ubuntu/hosts -O
/target/etc/hosts
- mkdir -p /target/etc
- echo "Disabling systemd-resolved..."
- rm -f /target/etc/resolv.conf
- echo "domain #TABLE:site:key=domain:value#" > /target/etc/resolv.conf
- echo "nameserver #XCATVAR:XCATMASTER#" >> /target/etc/resolv.conf
- echo "nameserver 8.8.8.8" >> /target/etc/resolv.conf
- sed -i
's/GRUB_CMDLINE_LINUX_DEFAULT=""/GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1"/g'
/target/etc/default/grub
- curtin in-target --target /target update-grub
- echo 'fs.file-max = 188898' >> /target/etc/sysctl.conf
- echo 'fs.inotify.max_user_instances=1280' >> /target/etc/sysctl.conf
- echo 'fs.inotify.max_user_watches=655360' >> /target/etc/sysctl.conf
- echo 'session required pam_limits.so' >> /target/etc/pam.d/common-session
- echo 'session required pam_limits.so' >>
/target/etc/pam.d/common-session-noninteractive
- echo '* soft nofile 188898' >> /target/etc/security/limits.conf
- echo '* hard nofile 188898' >> /target/etc/security/limits.conf
- echo "ulimit -Sn 188898" >> /target/root/.bashrc
- mkdir -p /target/etc/apt/keyrings
- mkdir -p /target/etc/apt/sources.list.d
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor
-o /target/etc/apt/keyrings/docker.gpg
- echo "deb [arch=$(dpkg --print-architecture)
signed-by=/etc/apt/keyrings/docker.gpg]
https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee
/target/etc/apt/sources.list.d/docker.list
- apt-get update
- echo 'export HISTTIMEFORMAT="%d/%m/%y %H:%M:%S "' >> /target/root/.bashrc
- echo "export HISTSIZE=10000" >> /target/root/.bashrc
- echo "export HISTFILESIZE=10000" >> /target/root/.bashrc
- echo "export PS1='\[\e[31;1m\][\u@#TABLE:site:key=location:value# -
\[\e[34;1m\]\H \W]\# \[\e[0m\]'" >> /target/root/.bashrc
- echo "alias minicom='TERM=linux minicom'" >> /target/root/.bashrc
- sed -i
'/^session[[:space:]]\+optional[[:space:]]\+pam_motd.so[[:space:]]\+motd=\/run\/motd.dynamic/s/^/#/'
/target/etc/pam.d/sshd
- sed -i
'/^session[[:space:]]\+optional[[:space:]]\+pam_motd.so[[:space:]]\+noupdate/s/^/#/'
/target/etc/pam.d/sshd
- systemctl restart ssh
The prescript file "/install/vessel/prescripts/v.ubuntu-xcat-installer" is
follows:
#!/bin/bash
printenv > /tmp/postscript_env_$$.log
TFTPBOOT1="/tftpboot/pxelinux.cfg"
TFTPBOOT2="/tftpboot/xcat/osimage/ubuntu22.04-x86_64-install"
TFTPBOOTXNBA="/tftpboot/xcat/xnba/nodes"
AUTOINSTDIR="/install/autoinst"
SOURCEFILESDIR="/install/global/os/ubuntu22.04/x86_64/casper"
osimageNode(){
# Create the PXE files for the individual nodes
target="/tftpboot/ldlinux.c32"
source="/install/vessel/syncfiles/ubuntu/ldlinux.c32"
if [ ! -f "$target" ]; then
cp -f "$source" "$target"
fi
#
# Copy vmlinuz and initrd to the tftpbootdir location
#
cp -f ${SOURCEFILESDIR}/vmlinuz ${TFTPBOOT2}/vmlinuz
cp -f ${SOURCEFILESDIR}/initrd ${TFTPBOOT2}/initrd.img
for NODE in ${NODES//,/ }; do
IP=$(awk -v host="${NODE}" '$2 == host {print $1}' /etc/hosts)
# Extract the first 3 octets
NFSSERVER="${IP%.*}.10"
GATEWAY="${IP%.*}.1"
# Write the rest of this shite out to the pxe file
echo "#install ubuntu22.04-x86_64-v.ubuntu22" >
/tftpboot/pxelinux.cfg/${NODE}
echo "DEFAULT xCAT" >> /tftpboot/pxelinux.cfg/${NODE}
echo "LABEL xCAT" >> /tftpboot/pxelinux.cfg/${NODE}
echo " KERNEL xcat/osimage/ubuntu22.04-x86_64-install/vmlinuz" >>
/tftpboot/pxelinux.cfg/${NODE}
echo " INITRD xcat/osimage/ubuntu22.04-x86_64-install/initrd.img" >>
/tftpboot/pxelinux.cfg/${NODE}
echo " APPEND root=/dev/ram0 ramdisk_size=1500000 ignore_uuid ip=dhcp
boot=casper
url=http://${NFSSERVER}/install/isos/ubuntu-22.04.5-live-server-amd64.iso
cloud-config-url=http://${NFSSERVER}:80/install/autoinst/${NODE}/user-data
autoinstall" >> /tftpboot/pxelinux.cfg/${NODE}
# Modify the xnba files as well
echo "#!gpxe" > ${TFTPBOOTXNBA}/${NODE}
echo "imgfetch -n kernel
http://\${next-server}/tftpboot/xcat/osimage/ubuntu22.04-x86_64-install/vmlinuz";
>> ${TFTPBOOTXNBA}/${NODE}
echo "imgload kernel" >> ${TFTPBOOTXNBA}/${NODE}
echo "imgargs kernel ignore_uuid ip=dhcp boot=casper
url=http://\${next-server}/install/isos/ubuntu-22.04.5-live-server-amd64.iso
cloud-config-url=http://\${next-server}/install/autoinst/shark099/user-data
autoinstall" >> ${TFTPBOOTXNBA}/${NODE}
echo "imgfetch -n initrd
http://\${next-server}/tftpboot/xcat/osimage/ubuntu22.04-x86_64-install/initrd.img";
>> ${TFTPBOOTXNBA}/${NODE}
echo "imgexec kernel" >> ${TFTPBOOTXNBA}/${NODE}
cp -f ${TFTPBOOTXNBA}/${NODE} ${TFTPBOOTXNBA}/${NODE}.uefi
# Do some substitutions in the user-data file
sed -i "s/__NODEIP__/${IP}/" ${AUTOINSTDIR}/${NODE}/user-data
sed -i "s/__GATEWAY__/${GATEWAY}/" ${AUTOINSTDIR}/${NODE}/user-data
done
}
cleanupPXE(){
# Remove PXE files for the individual nodes
for NODE in ${NODES//,/ }; do
find /install/autoinst/ -name '${NODE}' -exec rm -rf {} +
find /tftpboot/pxelinux.cfg -name '${NODE}' -exec rm -rf {} +
find /tftpboot/pxelinux.cfg -xtype l -delete
find /tftpboot/xcat/xnba/nodes -name '${NODE}' -exec rm -rf {} +
done
}
case ${ACTION} in
osimage ) osimageNode;;
boot ) cleanupPXE;;
esac
Thats pretty much it. When you run "nodeset shark099 osimage" the prescript
will create the working pxe and xnba files so you can use either "pxe" or
"xnba" in noderes depending on your preference.
You can then use nodeset shark099 boot to change the state back to boot and
clean up the pxe and xnba files.
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
