On Mon, 2022-03-07 at 19:55 +0000, Mark Gurevich wrote: > Nicolas, > Would you mind opening an xcat-core issue at > https://github.com/xcat2/xcat-core/issues > And describe the working and not working cases there ? > I think it will be much easier to have a conversation there than this > mailing list thread. >
Sure, I've opened https://github.com/xcat2/xcat-core/issues/7125#issue-1162377447 > -----Original Message----- > From: Roosen, Nicolas <nicolas.roo...@hpe.com> > Sent: Monday, March 7, 2022 3:24 AM > To: xcat-user@lists.sourceforge.net > Subject: [EXTERNAL] Re: [xcat-user] addkcmdline persistent after > installation > > Hello, > > On Fri, 2022-03-04 at 20:35 +0000, Mark Gurevich wrote: > > So with "%include /tmp/partitionfile" commented out, the > > addkcmdline="R::...." entries get correctly appended to grub and > > applied on each reboot ? > > > > no sorry, bad english: > > I meant to say, when I *uncomment* the line "%include > /tmp/partitionfile" *AND* put my partition definition in a partition > file ; then it works. > > If I comment out the line "%include /tmp/partitionfile", the extra > kernel args are not saved into the grub.cfg. > > So to summarize with two templates examples: > > - this works: > > #XCAT_PARTITION_START# > #%include /tmp/partitionfile > #XCAT_PARTITION_END# > > (I wrote the same partition scheme as bellow in a partition file > defined in the osimage: > partitionfile="/install/custom/profiles/custom- > node.part"). > > - this doesn't work: > > #XCAT_PARTITION_START# > #%include /tmp/partitionfile > part biosboot --fstype=biosboot --size=1 part /boot --size 512 -- > fstype ext4 part swap --size 8192 part / --size 40960 --fstype ext4 > part /var --size 20480 --fstype ext4 part /tmp --size 10240 --fstype > ext4 part /opt/tools --size 512000 --fstype ext4 part /scratchl -- > size 1 --grow --fstype ext4 #XCAT_PARTITION_END# > > > > -----Original Message----- > > From: Roosen, Nicolas <nicolas.roo...@hpe.com> > > Sent: Friday, March 4, 2022 10:41 AM > > To: xcat-user@lists.sourceforge.net > > Subject: [EXTERNAL] Re: [xcat-user] addkcmdline persistent after > > installation > > > > Things work so much better when I comment out > > > > %include /tmp/partitionfile > > > > in the node installation template ! > > > > We originally used a partition scheme directly inside the template, > > thus commented the include directive: > > > > #XCAT_PARTITION_START# > > #%include /tmp/partitionfile > > part biosboot --fstype=biosboot --size=1 part /boot --size 512 -- > > fstype ext4 part swap --size 8192 part / --size 40960 --fstype ext4 > > part /var --size 20480 --fstype ext4 part /tmp --size 10240 -- > > fstype > > ext4 part /opt/tools --size 512000 --fstype ext4 part /scratchl -- > > size 1 --grow --fstype ext4 #XCAT_PARTITION_END# > > > > Which apparently doesn't suits the bootlader appending stuff to > > grub ? > > > > > > On Fri, 2022-03-04 at 09:05 +0000, Roosen, Nicolas wrote: > > > Hello, > > > > > > On Thu, 2022-03-03 at 20:20 +0000, Mark Gurevich wrote: > > > > Yes, unfortunately /tmp/partitionfile is not preserved after > > > > the > > > > system boots. > > > > You can try the following: > > > > > > > > 1. Set debug on: chdef -t site clustersite xcatdebugmode=1 2. > > > > Start > > > > installation 3. When node "status" attribute changes to > > > > "installing", on Management node run > > > > runcmdinstaller <node> "cat /tmp/partitionfile". There is > > > > also > > > > a " /tmp/pre-install.log" > > > > which might have something interesting. > > > > > > > > > > thank you for the debug steps, I could see the following: > > > > > > 1. The "addkcmdline" parameters when set into the osimage > > > definition > > > are properly applied into the / install/autoinst/<node> file: > > > > > > grep nouveau /install/autoinst/node01-clu if [ -n > > > "earlyprintk=ttyS0,115200 rd.blacklist=nouveau > > > nouveau.modeset=0 " ];then > > > sed -i -e /bootloader/s#\'#\"#g -e '/bootloader/s/-- > > > append=\([^"]\S*[^"]\)/--append="\1"/g' -e '/bootloader/s/-- > > > append="\(.*\)"/--append="\1 earlyprintk=ttyS0,115200 > > > rd.blacklist=nouveau nouveau.modeset=0 "/g' /tmp/partitionfile > > > sed -i -e '/bootloader/{/append=/!s/$/& -- > > > append="earlyprintk=ttyS0,115200 rd.blacklist=nouveau > > > nouveau.modeset=0 > > > " /}' /tmp/partitionfile > > > grep -q bootloader /tmp/partitionfile >/dev/null 2>&1 || echo > > > - > > > e "bootloader --append=\"earlyprintk=ttyS0,115200 > > > rd.blacklist=nouveau > > > nouveau.modeset=0 \"" >>/tmp/partitionfile > > > > > > > > > 2. The /tmp/partitionfile does have the parameters: > > > > > > runcmdinstaller node01-clu "cat /tmp/partitionfile" > > > > > > [anaconda root@node01-clu ~]# cat /tmp/partitionfile ignoredisk > > > --only-use=/dev/sda part /boot --fstype=xfs --asprimary > > > --ondisk=/dev/sda --size=1024 part pv.000997 --grow --asprimary > > > --ondisk=/dev/sda --size=1024 volgroup xcatvg --pesize=4096 > > > pv.000997 > > > logvol swap --name=swap --vgname=xcatvg --recommended logvol / > > > --fstype=xfs --name=root --vgname=xcatvg --size=1024 --grow > > > bootloader > > > --boot-drive=sda --append="rd.blacklist=nouveau > > > nouveau.modeset=0 " > > > > > > > > > 3. But then the grub menu entry is not properly setup (this is an > > > edit > > > of the grub menu when booting): > > > > > > load_video set gfx_payload=keep insmod gzio linux > > > ($root)/vmlinuz-4.18.0-240.el8.x86_64 root=UUID=c1069add-a9e2- > > > 4764-81c6-\ > > > 7ee28338e3c1 ro crashkernel=auto resume=UUID=2ee99570-dc09-4115- > > > b244- > > > f6ff60de4\ > > > e79 console=ttyS1,115200n8r initrd > > > ($root)/initramfs-4.18.0-240.el8.x86_64.img $tuned_initrd > > > > > > > > > 4. I do see in /var/log/xcat/computes.log that the grub.cfg is > > > generated during the installation, BUT the "boot args" are not > > > correct: > > > > > > Mar 4 09:23:05 node01-clu anaconda[2401]: INFO anaconda: > > > installation: Task started: Configure the bootloader (12/44) Mar > > > 4 > > > 09:23:07 node01-clu org.fedoraproject.Anaconda.Modul WARNING > > > INFO:program:Running in chroot '/mnt/sysroot'... grub2-install -- > > > no- > > > floppy /dev/sda Mar 4 09:23:08 node01-clu > > > org.fedoraproject.Anaconda.Modul WARNING INFO:program:Installing > > > for > > > i386-pc platform. > > > Mar 4 09:23:08 node01-clu org.fedoraproject.Anaconda.Modul > > > WARNING > > > INFO:program:Installation finished. No error reported. > > > Mar 4 09:23:08 node01-clu org.fedoraproject.Anaconda.Modul > > > WARNING > > > DEBUG:program:Return code: 0 Mar 4 09:23:08 node01-clu > > > org.fedoraproject.Anaconda.Modul WARNING > > > INFO:anaconda.modules.storage.bootloader.grub2:bootloader.py: > > > used > > > boot > > > args: crashkernel=auto resume=UUID=2ee99570-dc09-4115-b244- > > > f6ff60de4e79 > > > console=ttyS1,115200n8r > > > Mar 4 09:23:08 node01-clu org.fedoraproject.Anaconda.Modul > > > WARNING > > > INFO:program:Running in chroot '/mnt/sysroot'... grub2-set- > > > default > > > b47a8ffe2045445ba9c2ca3e1e21579f-4.18.0-240.el8.x86_64 > > > Mar 4 09:23:08 node01-clu org.fedoraproject.Anaconda.Modul > > > WARNING > > > DEBUG:program:Return code: 0 Mar 4 09:23:08 node01-clu > > > org.fedoraproject.Anaconda.Modul WARNING INFO:program:Running in > > > chroot '/mnt/sysroot'... grub2-mkconfig -o /boot/grub2/grub.cfg > > > > > > > > > So somehow the grub.cfg is not properly setup during the > > > installation > > > ? > > > > > > > > > > -----Original Message----- > > > > From: Roosen, Nicolas <nicolas.roo...@hpe.com> > > > > Sent: Thursday, March 3, 2022 3:04 PM > > > > To: xcat-user@lists.sourceforge.net > > > > Subject: [EXTERNAL] Re: [xcat-user] addkcmdline persistent > > > > after > > > > installation > > > > > > > > On Thu, 2022-03-03 at 18:55 +0000, Mark Gurevich wrote: > > > > > Nicolas, > > > > > > > > > > Can you check if the file /install/autoinst/<nodename> > > > > > contains > > > > > your kernel parameters after line: > > > > > "#specify the kernel options which will be persistent after > > > > > installation" > > > > > > > > I've checked another node (I'm trying many things), and the > > > > result > > > > are different. > > > > > > > > So, first node: > > > > > > > > grep nouveau /install/autoinst/node-g003-clu > > > > > > > > if [ -n "nouveau.blacklist=yes " ];then > > > > sed -i -e /bootloader/s#\'#\"#g -e '/bootloader/s/-- > > > > append=\([^"]\S*[^"]\)/--append="\1"/g' -e '/bootloader/s/-- > > > > append="\(.*\)"/--append="\1 nouveau.blacklist=yes "/g' > > > > /tmp/partitionfile > > > > sed -i -e '/bootloader/{/append=/!s/$/& -- > > > > append="nouveau.blacklist=yes " /}' /tmp/partitionfile > > > > grep -q bootloader /tmp/partitionfile >/dev/null 2>&1 || > > > > echo > > > > - > > > > e "bootloader --append=\"nouveau.blacklist=yes \"" > > > > > > /tmp/partitionfile > > > > > > > > The node definition: > > > > > > > > lsdef node-g003-clu -i addkcmdline > > > > Object name: node-g003-clu > > > > addkcmdline=R::nouveau.blacklist=yes nouveau.blacklist=yes > > > > > > > > Second node: > > > > > > > > grep nouveau /install/autoinst/spiro-g302-clu > > > > > > > > if [ -n "earlyprintk=ttyS0,115200 rd.blacklist=nouveau > > > > nouveau.modeset=0 " ];then > > > > sed -i -e /bootloader/s#\'#\"#g -e '/bootloader/s/-- > > > > append=\([^"]\S*[^"]\)/--append="\1"/g' -e '/bootloader/s/-- > > > > append="\(.*\)"/--append="\1 earlyprintk=ttyS0,115200 > > > > rd.blacklist=nouveau nouveau.modeset=0 "/g' /tmp/partitionfile > > > > sed -i -e '/bootloader/{/append=/!s/$/& -- > > > > append="earlyprintk=ttyS0,115200 rd.blacklist=nouveau > > > > nouveau.modeset=0 " /}' /tmp/partitionfile > > > > grep -q bootloader /tmp/partitionfile >/dev/null 2>&1 || > > > > echo > > > > - > > > > e "bootloader --append=\"earlyprintk=ttyS0,115200 > > > > rd.blacklist=nouveau > > > > nouveau.modeset=0 \"" >>/tmp/partitionfile The node definition: > > > > > > > > lsdef node-g302-clu -i addkcmdline > > > > Object name: node-g302-clu > > > > addkcmdline=R::earlyprintk=ttyS0,115200 > > > > R::rd.blacklist=nouveau > > > > R::nouveau.modeset=0 > > > > > > > > > > > > So it seems that the parameters are added into the autoinst > > > > file, > > > > but not applied on the node during installation ... > > > > > > > > Some problems with /tmp/partitionfile maybe ? > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Roosen, Nicolas <nicolas.roo...@hpe.com> > > > > > Sent: Thursday, March 3, 2022 12:27 PM > > > > > To: xcat-user@lists.sourceforge.net > > > > > Subject: [EXTERNAL] [xcat-user] addkcmdline persistent after > > > > > installation > > > > > > > > > > Hello, > > > > > > > > > > I'm doing a new install of xCAT 2.16.3, and I'm still > > > > > struggling > > > > > to have the kernel parameters persistent after the initial > > > > > installation of a diskfull compute node (an HPE PRoliant in > > > > > legacy > > > > > bios mode if that matters). > > > > > > > > > > This is what I've tried: > > > > > > > > > > chdef -t osimage rhel8.3-x86_64-node > > > > > addkcmdline="R::earlyprintk=ttyS0,115200 > > > > > R::rd.blacklist=nouveau > > > > > R::nouveau.modeset=0 earlyprintk=ttyS0,115200 > > > > > rd.driver.blacklist=nouveau nouveau.modeset=0" > > > > > > > > > > and also: > > > > > > > > > > chdef -t node <node> addkcmdline="R::earlyprintk=ttyS0,115200 > > > > > R::rd.blacklist=nouveau R::nouveau.modeset=0" > > > > > > > > > > The settings are taken into account for the initial install, > > > > > but > > > > > as soon as the node reboot they are gone. > > > > > > > > > > Any ideas ? > > > > > > > > > > Thanks. > > > > > > > > > > -- > > > > > Nicolas > > > > > > > > > > _______________________________________________ > > > > > xCAT-user mailing list > > > > > xCAT-user@lists.sourceforge.net > > > > > INVALID URI REMOVED > > > > > nfo/xcat- > > > > > user__;!!NpxR!z2z0NRtWUWDzNpoHrqgFjBhHmXlVu4JL8HQweoonBWTeh9O > > > > > SR > > > > > Rx > > > > > 01 > > > > > oW > > > > > G3EiYqpxS$ > > > > > > > > > > > > > > > _______________________________________________ > > > > > xCAT-user mailing list > > > > > xCAT-user@lists.sourceforge.net > > > > > INVALID URI REMOVED > > > > > nfo/xcat- > > > > > user__;!!NpxR!z2z0NRtWUWDzNpoHrqgFjBhHmXlVu4JL8HQweoonBWTeh9O > > > > > SR > > > > > Rx > > > > > 01 > > > > > oW > > > > > G3EiYqpxS$ > > > > > > > > > > > > > > _______________________________________________ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > INVALID URI REMOVED > nfo/xcat- > user__;!!NpxR!x1XqHlvAFK_BvHdDGHacNdMvkyfZfyL5bFlXyRsuHpveY75UYYCMFBF > m-ykz8aT2$ > > _______________________________________________ > xCAT-user mailing list > xCAT-user@lists.sourceforge.net > INVALID URI REMOVED > nfo/xcat- > user__;!!NpxR!x1XqHlvAFK_BvHdDGHacNdMvkyfZfyL5bFlXyRsuHpveY75UYYCMFBF > m-ykz8aT2$ -- Nicolas Roosen Technical Consultant HPC & AI Business Group nicolas.roo...@hpe.com +33 170486483 Office +33 777161256 Mobile _______________________________________________ xCAT-user mailing list xCAT-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xcat-user
