Re: Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?
On Fri, 2023-01-27 at 14:01:08 +0100, Thomas Lange wrote:
> Here are some code snippets I found for dhcpd.conf:
>
>
>if substring(option vendor-class-identifier, 0, 20) =
> "PXEClient:Arch:0" {
> filename "fai/pxelinux.0";
>}
>if substring(option vendor-class-identifier, 0, 20) =
> "PXEClient:Arch:7" {
> filename "fai/syslinux.efi";
>}
> }
>
>
> if option arch = 00:00 {
> filename "fai/pxelinux.0";
> } elsif option arch = 00:09 {
> filename "fai/bootx64.efi";
> } elsif option arch = 00:07 {
> filename "fai/grubnetx64.efi";
> } elsif option arch = 00:06 {
> filename "fai/bootx64.efi";
> }
I've been using those for years now with syslinux, they are x86(-64) only.
For ARM64 (11 decimal) there's *only* grubnetaa64.efi which will then load
grub/grub.cfg - which doesn't seem to be customizable the way we're used to
with MAC or IP addresses. (There are signed and unsigned versions of that
file; I've chosen the signed one for now.) No means to read the individual
pxelinux.cfg/* file. (I'll have to check the syslinux issues whether someone
else has asked for non-x86 support.)
Meanwhile, I was able to boot the single machine into "sysinfo", but only
with a Buster kernel (4.19), a Bullseye one (5.10) apparently runs into
issues with the many network interfaces provided (but not connected) of the
Cavium ThunderX SoC, and eventually freezes. I'll give that one another go
next week.
Thanks so far, S
Re: Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?
Here are some code snippets I found for dhcpd.conf:
if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:0"
{
filename "fai/pxelinux.0";
}
if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:7"
{
filename "fai/syslinux.efi";
}
}
if option arch = 00:00 {
filename "fai/pxelinux.0";
} elsif option arch = 00:09 {
filename "fai/bootx64.efi";
} elsif option arch = 00:07 {
filename "fai/grubnetx64.efi";
} elsif option arch = 00:06 {
filename "fai/bootx64.efi";
}
https://github.com/cobbler/cobbler/discussions/2687
--
regards Thomas
Re: Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?
On Thu, 2023-01-26 at 12:15:07 +0100, Thomas Lange wrote: > > On Thu, 26 Jan 2023 11:06:14 +0100, Steffen Grunewald > > said: > > > Which info do you need? Would the output of `ls -lR /boot` (on a booting > > machine) be sufficient, or even too much? Any selections from dpkg > output? > I'm not sure which infos I need. Maybe the pathes for grub. > I also need to know if some grub commands in fai-cd need to be called > with different parameters. What about --format=x86_64-efi? Do we need > to add some grub modules for --install-modules=...? > What about the xorriso call? There are also intel arch specific options. This seems to be more complicated than I expected :( There's good news though: PXE is not *broken* on the machine, it's just *different*. Different in terms of the "PXEClient:ARCH:x" magic number it sends to the DHCP server. Now knowing that x is 00011 (0x0b), I could assign an initial bootloader for UEFI, like I'm doing with my x86_64 machines. If I only knew which file to use... Jan 27 13:18:39 faiserver dhcpd[125420]: DHCPOFFER on 10.150.100.100 to 1c:1b:0d:ef:76:7a via eth2 Jan 27 13:18:43 faiserver dhcpd[125420]: DHCPREQUEST for 10.150.100.100 (10.150.100.109) from 1c:1b:0d:ef:76:7a via eth2 Jan 27 13:18:43 faiserver dhcpd[125420]: DHCPACK on 10.150.100.100 to 1c:1b:0d:ef:76:7a via eth2 Jan 27 13:18:43 faiserver in.tftpd[76654]: RRQ from 10.150.100.100 filename grubnetaa64.efi Jan 27 13:18:43 faiserver in.tftpd[76654]: tftp: client does not accept options Jan 27 13:18:43 faiserver in.tftpd[76899]: RRQ from 10.150.100.100 filename grubnetaa64.efi Jan 27 13:18:47 faiserver in.tftpd[80100]: RRQ from 10.150.100.100 filename /grub/arm64-efi/command.lst Jan 27 13:18:47 faiserver in.tftpd[80100]: sending NAK (1, File not found) to 10.150.100.100 Jan 27 13:18:47 faiserver in.tftpd[80102]: RRQ from 10.150.100.100 filename /grub/arm64-efi/fs.lst Jan 27 13:18:47 faiserver in.tftpd[80102]: sending NAK (1, File not found) to 10.150.100.100 Jan 27 13:18:47 faiserver in.tftpd[80142]: RRQ from 10.150.100.100 filename /grub/arm64-efi/crypto.lst Jan 27 13:18:47 faiserver in.tftpd[80142]: sending NAK (1, File not found) to 10.150.100.100 Jan 27 13:18:47 faiserver in.tftpd[80192]: RRQ from 10.150.100.100 filename /grub/arm64-efi/terminal.lst Jan 27 13:18:47 faiserver in.tftpd[80192]: sending NAK (1, File not found) to 10.150.100.100 Jan 27 13:18:47 faiserver in.tftpd[80298]: RRQ from 10.150.100.100 filename /grub/grub.cfg Jan 27 13:18:47 faiserver in.tftpd[80298]: sending NAK (1, File not found) to 10.150.100.100 So "grubnetaa64.efi" (as suggested for RasPi PXE boot) doesn't seem to be the right one. The debian-installer-11-netboot-arm64 contains "bootnetaa64.efi" which looks a bit better, but the second-step loader will be "grubaa64.efi", which gets me to the "grub> " prompt again. I feel I'm getting closer but I'm not there yet... back to reading now. - S
Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?
> On Thu, 26 Jan 2023 11:06:14 +0100, Steffen Grunewald > said: > Which info do you need? Would the output of `ls -lR /boot` (on a booting > machine) be sufficient, or even too much? Any selections from dpkg output? I'm not sure which infos I need. Maybe the pathes for grub. I also need to know if some grub commands in fai-cd need to be called with different parameters. What about --format=x86_64-efi? Do we need to add some grub modules for --install-modules=...? What about the xorriso call? There are also intel arch specific options. I also have a small arm64 device here (Hikey960), currently updating it to bullseye and try to install the fai packages on it. > I the meantime: what would I see if I properly assign a nfsroot but > "forget" to set the $FAI_* variables? Typing long command lines into grub > is so failure-prone... Have a look at your pxelinux.cfg. There you see the parameters. These are important ro rootovl ip=dhcp root=1.2.3.4:/srv/fai/nfsroot,nfsvers=3 Others like FAI_CONFIG_SRC may be hardcoded inside the nfsroot in /etc/fai/fai.conf. Not really sure if this still works. -- regards Thomas
Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?
On Wed, 2023-01-25 at 14:41:25 +0100, Steffen Grunewald wrote: > On Wed, 2023-01-25 at 14:12:56 +0100, Thomas Lange wrote: > > > I'm happy for some hints how to support fai-cd on arm64, and maybe > > also cross-arch available. But I'm not sure if grub-mkstandalone can > > do cross-arch. Which info do you need? Would the output of `ls -lR /boot` (on a booting machine) be sufficient, or even too much? Any selections from dpkg output? I the meantime: what would I see if I properly assign a nfsroot but "forget" to set the $FAI_* variables? Typing long command lines into grub is so failure-prone... Thanks, S
Re: Create fai-cd for other architecture?
On Wed, 2023-01-25 at 14:12:56 +0100, Thomas Lange wrote: > > On Wed, 25 Jan 2023 13:43:30 +0100, Steffen Grunewald > > said: > > > Cross-platform doesn't work since "grub-mkstandalone" would throw an > > "exec format error". > > Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 > hardcoded. > What about running a qemu cross-arch VM on amd64? It would be slow, > but should be doable? Fixing the hard-coded arch is easy if you tell > me how this looks for arm64. I could boot the machine and perform some experiments; a RasPi on the other hand isn't that slow (but at home, not at the office now). I already replaced the few amd64 occurrences in fai-cd with arm64, but got stuck when it proceeded to look for grub-i386. This is pure EFI so no special tricks would be required. Do we actually need the whole grub stuff? > I'm happy for some hints how to support fai-cd on arm64, and maybe > also cross-arch available. But I'm not sure if grub-mkstandalone can > do cross-arch. Perhaps I'm thinking too complicated. All I want to do is start a kernel and initrd, assign a (fixed, as I don't trust DHCP for this one) IP address and set the kernel cmdline I'd otherwise pass to PXE. A first attempt to use a Debian metinstall image, and edit the boot config, has failed, I suspect I got the root= parameter wrong. Got to look that up for the case there had been no proper DHCP handshake before. - S
Re: Create fai-cd for other architecture?
> On Wed, 25 Jan 2023 13:43:30 +0100, Steffen Grunewald > said: > Cross-platform doesn't work since "grub-mkstandalone" would throw an > "exec format error". > Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 hardcoded. What about running a qemu cross-arch VM on amd64? It would be slow, but should be doable? Fixing the hard-coded arch is easy if you tell me how this looks for arm64. I'm happy for some hints how to support fai-cd on arm64, and maybe also cross-arch available. But I'm not sure if grub-mkstandalone can do cross-arch. -- reagrds Thomas
Re: Create fai-cd for other architecture?
On Wed, 2023-01-25 at 13:04:44 +0100, Steffen Grunewald wrote: > Hello, > > I've got an ARM64 server that doesn't do PXE properly, and (of course...) > I don't have a second ARM64 machine at hand to prepare a FAI CD. > The NFSROOT already exists. > Is it possible to run fai-cd on AMD64, with some setup/config tricks, to > create an ISO image that can be booted on the ARM machine, or do I have > to find something with the right architecture (a RasPi running Debian > probably would do)? Answering myself: Cross-platform doesn't work since "grub-mkstandalone" would throw an "exec format error". Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 hardcoded. Any alternative to create a bootable stick for this arm64 machine, to mount a remote nfsroot and access a remote config tree? (Just take an installer stick image and replace the kernel and initrd, and provide a kernel commandline?) - S
Create fai-cd for other architecture?
Hello, I've got an ARM64 server that doesn't do PXE properly, and (of course...) I don't have a second ARM64 machine at hand to prepare a FAI CD. The NFSROOT already exists. Is it possible to run fai-cd on AMD64, with some setup/config tricks, to create an ISO image that can be booted on the ARM machine, or do I have to find something with the right architecture (a RasPi running Debian probably would do)? Thanks, S -- Steffen Grunewald, Cluster Administrator Max Planck Institute for Gravitational Physics (Albert Einstein Institute) Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany ~~~ Fon: +49-331-567 7274 Mail: steffen.grunewald(at)aei.mpg.de ~~~
