Re: problem with mdadm FAI 6.2

[Thomas Lange]

First, check if /dev/md1 is already mounted.
Maybe the FAI boot process (using dracut inside the initrd) detect
this raid and activates it.

You may add a hook which wipes add signature of a soft raid using a
simple script. For e.g. under https://fai-project.org/download/misc/
there are partition.WIPEDISKS and wipe-all

These can also remove the disk partition label.
-- 
regards Thomas

Re: UEFI boot dropping to startup.nsh prompt

[Thomas Lange]

Mmmm. Now I think it's not possible that your image craeted by
fai-diskimage can create a valid EFI boot entry on your
computer. Maybe that's why it's not booting.
Do you think the EFI entries match the UUID of the disk image?



> On Wed, 10 Apr 2024 09:01:08 +, Laura Smith via linux-fai 
>  said:

> Diese Nachricht wurde eingewickelt um DMARC-kompatibel zu sein. Die
> eigentliche Nachricht steht dadurch in einem Anhang.

> This message was wrapped to be DMARC compliant. The actual message
> text is therefore in an attachment.
> From: Laura Smith 
> To: fully automatic installation for Linux 
> Subject: UEFI boot dropping to startup.nsh prompt
> Date: Wed, 10 Apr 2024 09:01:08 +

> I'm building a UEFI image for Debian Bookworkm, but when I load the qcow2 
and attempt to boot it, it
> drops into a startup.nsh prompt as below.

> I have tried building with grub-efi-amd64-signed instead of 
grub-efi-amd64, but that does not make
> any difference.

> Any ideas how I can troubleshoot ?  

> Thanks !

> UEFI Interactive Shell v2.2
> EDK II
> UEFI 2.70 (EDK II, 0x0001)
> Mapping table
> FSO: Alias (s) :HDOb: : BLK1:
> PciRoot (0x0) /Pci (0x4,0x0) /HD 
(1,GPT,41F5FCD7-6BFF-49A0-B726-E168F937E46D,0x800,0x10)
> BLKO: Alias (s) :
> PciRoot (0x0) /Pci (0x4,0×0)
> BLK2: Alias (s) :
> PciRoot (0x0) /Pci (0x4,0x0) /HD (2,GPT 
,F6E9D048-1C74-4A2E-A196-6BFE311BEE82,0×100800,0x1C3800)
> BLK3: Alias (s) :
> PciRoot (0x0) /Pci (0x4,0x0) /HD (3,GPT 
,9F400A0A-EE84-4F72-BA36-431EEB828209,0x2C4000,0x113B800)
> Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell> 

Re: Adding options for qcow2 output ?

[Thomas Lange]

Hi Laura,

with fai-diskimage you can't add options for qcow2. But you can create
a raw image and then manually convert it to qcow2 using any options
you like. In the end fai-diskimage is also just creating a raw image
and converts this to qcow2 using

qemu-img convert -f raw $rawname -O -c -o compression_type=zstd qcow2 $qcowname


> On Tue, 09 Apr 2024 15:14:07 +, Laura Smith via linux-fai 
>  said:


> Hi
> How do I set the "preallocation=off" qcow2 option on the output for 
fai-diskimage ?

-- 
viele Grüße Thomas

Re: No physical disk in FAI 6.2.2 plain text

[Thomas Lange]

The bug in the FAIme service is now fixed.

I forgot to change the /etc/fai/NFSROOT when using FAI 6.2.2 and
therefore systemd was missing  inside the initrd in the nfsroot.

The result is that the disk belongs to group root instead of disk.
That's why FAI stops, because is does no detect a disk.


CONCLUSION for users that build their own FAI ISOs:
  If you upgrade to FAI 6.2.X do check your /etc/fai/NFSROOT. It should
  NOT contain this line:

sysvinit-core systemd-sysv-

-- 
regards Thomas

Re: No physical disk in FAI 6.2.2 plain text

[Thomas Lange]

> On Wed, 27 Mar 2024 13:28:19 +, "Paul, Jochen" 
>  said:

> I wonder why the "Gnome desktop" installation works successful and the 
"Plain text console, no X11" aborts. I've tried several times the
> creation of both images but the result does not change.
I cannot confirm that the GNOME desktop installation works.
For me both ISO I've just created are broken. I guess you are using an
older ISO for GNOME an a new one for the plain console installation.

-- 
viele Grüße Thomas

Re: No physical disk in FAI 6.2.2 plain text

[Thomas Lange]

Hi Jochen,

I can confirm this using a KVM virtual machine.
I try to dig deeper into it, because other also had this problem some
times. I think it's a problem that I mix several FAI version on the
server that creates the FAIme ISOs.



P.S.: Your mail got stuck because it had several pics in it, so the
size was over the limit for this mailing list. Please try to no
include pics, that are not needed for debugging and are only advertising.

-- 
viele Grüße Thomas

Re: mistake in subroutines line 1285

[Thomas Lange]

Hi

ssh is correct. ftp will not work here. We need to log into the
server for changing the boot configuration by calling fai-chboot. This
can't be done using ftp. The login has to be done via ssh.

regards Thomas

FAI 6.2.1 released

[Thomas Lange]

Hi all,

the new FAI version 6.2.1 is now available on fai-project.org.
Some minor bugs were fixed and the whole changelog can be read at

https://tracker.debian.org/media/packages/f/fai/changelog-6.2.1

-- 
best regards Thomas

Re: install_packages: check for debian

[Thomas Lange]

Hi Maximilian,

thanks for the hint. I've already applied a patch which will be
available in the next upcoming FAI version (6.2.1).

regards Thomas

> On Tue, 20 Feb 2024 17:31:25 +0100, Maximilian Stein  
> said:

> Hi there,
> to decide if it is running on a Debian system, install_packages(8) 
> switched to checking for existence of 
> $FAI_ROOT/var/cache/apt/pkgcache.bin a while ago [1].

> Unfortunately, this does not always work in my tests with Bookworm, as 
> the file seems to be deleted and recreated during an apt update. 

Re: FAIme service produces currently broken ISOs

[Thomas Lange]

The old configuration was restoredd and now the FAIme service is
running again. I also found the cause of the problem.

dracut does not install a file that is needed is a certain
environment.

Thomas

>>>>> On Tue, 13 Feb 2024 17:49:08 +0100, Thomas Lange  
>>>>> said:

> Hi all,
> currently the FAIme service creates broken ISOs which do not boot or
> do not recognize the local disks. I try to fix this as soon as
> possible, but currently I do not know the root cause of the problem.

FAIme service produces currently broken ISOs

[Thomas Lange]

Hi all,

currently the FAIme service creates broken ISOs which do not boot or
do not recognize the local disks. I try to fix this as soon as
possible, but currently I do not know the root cause of the problem.

-- 
regards Thomas

Subject: Re:Re: Problem with setup-storage

[Thomas Lange]

Hi all,

other people also have this problem (disks under /dev are not
in group disk but root). I wonder if you have checked /etc/fai/NFSROOT
after upgrading to FAI 6.2. Make sure this line is removed and
recreate the nfsroot.

sysvinit-core systemd-sysv-

It was used with FAI 6.0.x because we did not used systemd inside the
nfsroot. FAI 6.2 now ships an /etc/fai/NFSROOT which will install
systemd that's used nowadays. I hope that this is the cause of your
wrong group ownership for /dev.

Concerning setup-storage, I wonder if it is really broken inside the
FAI boot environment. FAI should define the functions needed for
setup-storage before calling setup-storage.

There may be still problems when you log in via ssh into the FAI
environment and call setup-storage. Then the functions needed are not
defined. A patch is already applied and will be available in the next
FAI version. See

https://github.com/faiproject/fai/compare/eb8f779aa4c69157d448..cd33235318279a962

-- 
viele Grüße Thomas

Re: Problem with setup-storage

[Thomas Lange]

Here's the patch

https://github.com/faiproject/fai/commit/cd33235318279a96267d3377661dbeba93eacac3

-- 
viele Grüße Thomas

Re: Problem with setup-storage

[Thomas Lange]

Hi Remy,

it seems you hit bug https://bugs.debian.org/1063146.
I'm preparing a patch and will upload a new FAI version if it's ready.

But this bug should only appear, if you use setup-storage as a
standalone command. Inside the FAI environment everything should work.
It's not clear to me in which kind of environment you have problems.

-- 
regards Thomas

FAI web pages offline

[Thomas Lange]

Tomorrow, saturday from 8:00 localtime (UTC +0100) to around 4pm the
FAI web pages and services will be offline due to a power
maintenance.

-- 
regards Thomas

Re: Define sda as the smallest disk

[Thomas Lange]

Hi all,

that is the example how to change the shell variable $disklist:
https://fai-project.org/download/misc/99-disklist.sh

Create the script class/99-disklist.sh in your config space (/s/rv/fai/config)

These are the imprtant lines:

if [ -n "$newlist" ]; then
echo New disklist: $newlist
echo disklist=\"$newlist\" >> $LOGDIR/additional.var
fi


This script writes the new valuespf disklist to
$LOGDIR/additional.var. Then FAI will parse it and sets the new value
for disklist before calling setup-storage.

regards Thomas

Re: Define sda as the smallest disk

[Thomas Lange]

> On Wed, 31 Jan 2024 11:15:46 +0100, Rémy Dernat  said:

> Hi Thomas,
> That looks fantastic. Is there any examples to use these functions ?
No, the documentation is really bad. In FAI the disk list is created
by these two steps:

set_bootstick
all_disks_and_size | checkdisk $FAI_BOOTSTICK | once_only



>From the sources (lib/subroutines)

matchdisks() {
# matchdisks PATTERN PATTERN 

checkdisk() {
# read lines with device name and size and check if device is a disk
# $1 can be a device name that will be ignored. Used for the device
# name of the USB stick if we boot from it

disks_by_id() {
# list all disks by ID and their link to the device name

all_disks_and_size() {
# print a list of devices and their block size


# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
all_disks_by_size() {
all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK
}
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
largestdisk()  {
all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK | head -1
}
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
smallestdisk() {
all_disks_and_size | sort -n  -k2 | checkdisk $FAI_BOOTSTICK | head -1
}

set_bootstick() {
# determine the device of the USB stick we boot from
# sets the variable FAI_BOOTSTICK



-- 
regards Thomas

Re: Define sda as the smallest disk

[Thomas Lange]

FAI 6.2 now provides some helper functions to create your own disklist:

grepv_disks()
grep_disks()
notmatchdisks()
matchdisks()
smallestdisk()
largestdisk()
all_disks_by_size()
all_disks_and_size()
once_only()
checkdisk()
disks_by_id()


Here's an older script how to change the disklist:
https://fai-project.org/download/misc/99-disklist.sh


> On Mon, 29 Jan 2024 17:13:39 +0100, Rémy Dernat  said:


> So, I would like to define something like disk1 / sda is :
> fdisk -l 2>/dev/null |awk '/Disk / {print $2,$3}' |sort -n -k2 |head -1

> Then, this would be used by setup-storage.

-- 
regards Thomas

FAI 6.2 released

[Thomas Lange]

After more than one a year, a new minor FAI version is available, but
it includes some interesting new features.

Here a the items from the NEWS file:

fai (6.2) unstable; urgency=low

  * fai-cd can now create live images
  * Use systemd during installation
  * New feature: run FAI inside a screen or tmux session
  * fai-diskimage: do not use compression of qemu-img which is slow
instead provide .qcow2.zst, add option -C
  * fai-kvm: add support for booting from USB storage
  * new tool mk-data-partition adds a data partition to an ISO
  * easy installation of packages from /pkgs/ directories
  * new helper functions for creating custom list of disks
  * new method detect:// for FAI_CONFIG_SRC


In the past the command fai-cd was only used for creating installation
ISOs, that could be used from CD or USB stick.
Now it possible to create a live ISO. Therefore you create your live
chroot environment using 'fai dirinstall' and then convert it to a
bootable live ISO using fai-cd. See man fai-cd(8) for an example.

Years ago I had the idea to use the remaining disk space on an USB
stick after copying an ISO onto it. I've blogged about this recently:

  https://blog.fai-project.org/posts/extending-iso-images/

The new FAI version includes the tool mk-data-partition for adding
a data partition to the ISO itself or to an USB stick.

FAI detects this data partition, mounts it to /media/data and can then
use various configurations from it.  You may want to copy your own set
of .deb packages or your whole FAI config space to this partition. FAI
now automatically searches this partition for usable FAI configuration
data and packages. FAI will install all packages from pkgs/
if the equivalent class is defined. Setting FAI_CONFIG_SRC=detect://
now looks into the data partition for the subdirectory 'config' and
uses this as the config space. So it's now possible to modify an
existing ISO (that is read-only) and make changes to the config space.
If there's no config directory in the data partition FAI uses the
default location on the ISO.

The tool fai-kvm, which starts virtual machines can now boot an ISO
not only as CD but also as USB stick.

Sometimes users want to adjust the list of disks before the
partitioning is startet.
Therefore FAI provides several functions like
 smallestdisk()
 largestdisk()
 matchdisks()

You can select individual disks by their model name or even the serial
number.

Two new FAI flags were added (tmux and screen) that make it easy to
run FAI inside a tmux or screen session.

And finally FAI uses systemd. Yeah!
This technical change was waiting since 2015 in a merge request from
Moritz 'Morty' Strübe, that would enable using systemd during the
installation. Before FAI still was using old-style SYSV init
scripts and did not started systemd. I didn't tried to apply the
patch, because I was afraid that it would need much time to make it
work. But then in may 2023 Juri Grabowski just gave it a try at
MiniDebConf Hamburg, and voilà it just works!
Many, many thanks to Moritz and Juri for their bravery.

The whole changelog can be found at
https://tracker.debian.org/media/packages/f/fai/changelog-6.2

New ISOs for FAI will be prepared during the next days. The FAIme
service for creating customized installation ISOs will get its update
later.

The new packages are available for bookworm by adding this line to
your sources.list:

deb https://fai-project.org/download bookworm koeln

--
best regards Thomas

Re: Making sure to partition the right disk(s)

[Thomas Lange]

> On Fri, 19 Jan 2024 15:33:02 +0100, Diego Zuccato 
>  said:

> But it seems it doesn't get mounted (at least a custom script did not 
> find it mounted). I don't know FAI internals enough :(
This mounting of a partition labeled MY-DATA will only work from FAI
6.2, which is not yet released.

-- 
regards Thomas

Re: Making sure to partition the right disk(s)

[Thomas Lange]

> On Fri, 19 Jan 2024 10:03:50 +0100, Diego Zuccato 
>  said:

> PS: looking at the source, I noticed that a partition labeled "MY-DATA" 
> is automatically mounted to /media/data . Does it work only for boots 
> from CD or also from network? It could be useful to store machine's 
> static data (SSH server key, just to say one)...
It only works for boots from an ISO that is uses as an USB stick.
This is for a initial installation, if all disks are empty.

It may also work if you do a reinstallation via network and the kernel
will find a partition with label MY-DATA. I guess this should also
work.

-- 
regards Thomas

Re: Making sure to partition the right disk(s)

[Thomas Lange]

> On Fri, 19 Jan 2024 09:03:57 +0100, Diego Zuccato 
>  said:

> Hello all.
> It's not too unusual that sometimes disks get recognized in a different 
> order across reboots.
> How can I make sure I'm repartitioning the right disk and not another 
> one containing data? I can't find any way to bind some info about HDD to 
> "disk1" instead of "disk2".
I use this script to manipulate the disklist:
http://fai-project.org/download/misc/99-disklist.sh

In the next FAI version 6.2 (the release may come in the next few
days) there are more functions to change the order of the disks, like
smallestdisk or matchdisks which can match to a certain serial number.

-- 
regards Thomas

Re: Single FAI server, multiple Debian versions?

[Thomas Lange]

> On Wed, 17 Jan 2024 12:27:30 +0100, Diego Zuccato 
>  said:

> Yup, sure!
> And in the list printed during boot there are both, with DEBIAN 
> preceding BOOKWORM. I see both var files being read & printed.

> Since the first workaround didn't work (even after changing 'echo' to 
> 'cat'), I resorted to adding files/etc/apt/sources.list/BOOKWORM64 with 
> the needed lines.
> At least now the packages get fetched from the right repository, but "it 
> smells worse than my yesterday diaper" (Baby Herman)...
I you have some problems, just put the fai.log onto paste.debian.net
and tell us on IRC #linux-fai (oftc.net). Then you can get quick help
from ohter FAI users.

regards Thomas

Re: Single FAI server, multiple Debian versions?

[Thomas Lange]

> On Tue, 16 Jan 2024 14:15:04 +0100, Diego Zuccato 
>  said:

> Tks, that's indeed way easier. And more manageable, especially if the 
> files are kept on a dedicated http server => no changes to nfsroot.
> The con is that the file needs to be saved somewhere on the local system 
> before being extracted, and that could be a problem with small disks, 
No. The tar file is downloaded into a directory that is a RAM disk.

-- 
regards Thomas

Re: Attempt to fai-setup for Bookworm fails -?

[Thomas Lange]

> On Fri, 5 Jan 2024 10:55:57 +0100, Steffen Grunewald 
>  said:

> Using the files in /etc/fai that come with fai-server_6.0.5, I end up 
with this:

> ...
> update-initramfs: deferring update (trigger activated)
> Processing triggers for dracut (059-4) ...
> dracut: Generating /boot/initrd.img-6.1.0-17-amd64
> dracut: dracut module 'network' depends on 'systemd-networkd', which 
can't be installed
> dracut: dracut module 'ifcfg' depends on 'network', which can't be 
installed
> dracut: dracut module 'livenet' depends on 'network', which can't be 
installed
> dracut: dracut module 'nfs' depends on 'network', which can't be installed
> dracut: dracut module 'nvmf' depends on 'network', which can't be 
installed
> dracut: dracut module 'ifcfg' cannot be found or installed.
> dpkg: error processing package dracut (--configure):
>  installed dracut package post-installation script subprocess returned 
error exit status 1
> Errors were encountered while processing:
>  dracut
> ...
You need the dracut module network-legay, not network (which needs systemd)
which is hardocded in fai-make-nfsroot.

> Of course, I've got to do this on a Bullseye machine. What am I doing 
wrong?

Which version of FAI do you have on this bullseye machine? 5.10.X?
Or a FAI 6.x version?

I guess you need to use the newer version of fai-make-nfsroot (from
bookworm) on the bullseye machine. Maybe it's just related to
https://bugs.debian.org/1010906 but I did more changes to this script
for newer dracut versions.

I do not remember correctly if it's possible to use the bookworm
packages of FAI on a bullseye host system to create a bookworm
nfsroot. But I would guess it's possible.

-- 
regards Thomas

Re: Installing metapackage with fai.me

[Thomas Lange]

> On Wed, 29 Nov 2023 10:30:33 +0100 (CET), Jean-Mathieu Chantrein 
>  said:


> package-name: install in no-rec mode
> package-name+: include recommended packages by package-name
> package-name- : exclude package-name from the installation (syntax used 
by apt)
We cannot use + and - because this is also used by apt/apt-get. man apt says:
"The requested action can be overridden for specific packages by
appending a plus (+) to the package name to install this package or a minus (-) 
to remove it."


In FAI itself we use different sections in a config file for install
or install-norec. This could be added to the FAIme web page by just
adding another input field. By do not expect that to be implemented as
fast as yesterday ;-)


> Along the same lines, we could also add a wilcard (perhaps this is 
already the case ...):
> package-name* : installs all packages starting with package-name in 
no-rec mode (syntax used by apt)
> package-name*+: installs all packages starting with package-name in 
recommended mode
apt/apt-get already supports * as glob pattern, but I'm not sure if
FAI also allows this.

-- 
 Thomas

Re: Installing metapackage with fai.me

[Thomas Lange]

Hi all,

on https://fai-project.org/FAIme/
you can now enable installing recommended packages for your custom
package list.

I took more time to add the new checkbox on the web page, than to add
thr function to the backend part of FAIme.

Thanks a lot to Jean-Mathieu for suggesting this feature.

-- 
regards Thomas

Re: Installing metapackage with fai.me

[Thomas Lange]

> On Wed, 29 Nov 2023 09:26:56 +0100 (CET), Jean-Mathieu Chantrein 
>  said:

> Yes, it's games-finest. Thank you.
> apt-cache show games-finest
Ah, now it looks better. I wonder why this metapackage does not use
depends but only recommends.

> So, I think that underlying packages are not installed because they are 
in Recommends.
> In fact, in the FAI conf files, the package appears like this:

> PACKAGES install-norec
> games-finest
Yes, you are right -norec does not install recommends. We have
PACKAGES install but in FAI.me it's currently hardcoded to use install-norec.

> Is there a way to change this default behaviour in fai.me?
Currently not, but I will think about implementing it. Seems like a
nice feature to me. Another checkbox should do the job.

I may change the FAI.me behaviour globally for some tests, maybe this
evening or tomorrow.
-- 
viele Grüße Thomas

Re: Installing metapackage with fai.me

[Thomas Lange]

Salut Jean-Mathieu,

no there's not limit on the size, but games-finest is not a package
name:

$ apt show game-finest
N: Unable to locate package game-finest
N: Unable to locate package game-finest

But this exists:

$ apt show junior-tasks
Package: junior-tasks
Version: 1.30
Priority: optional
Section: misc
Source: debian-junior
Maintainer: Debian Junior 
Installed-Size: 25.6 kB
Depends: tasksel

But junior-tasks only installs the package tasksel, nothing more. I
guess the blends names are not visible to apt.
Therefore blends may be not possible in FAI.
Only package names, that apt knows (for e.g. task-kde-desktop,
task-laptop) can be used in FAI.

-- 
viele Grüße Thomas


> On Wed, 29 Nov 2023 08:24:18 +0100 (CET), Jean-Mathieu Chantrein 
>  said:

> Hello,
> When I want to make an iso on fai.me using metapackages like games-finest 
or junior-tasks, the underlying packages are not present in the iso and are 
therefore not installed.
> Are there any limitations to the use of these packages in fai.me (perhaps 
a limitation on the size of the generated iso?) or am I missing something?

Re: problems in fai-make-nfsroot

[Thomas Lange]

> On Mon, 13 Nov 2023 15:17:02 +, SCHEUER Thomas 
>  said:

> Attached are the fai-setup.log and the fai-make-nfsroot.log
> fai-nfsroot : Depends: fai-client (>= 5.2) but it is not going to be 
installed -> so setup breaks
- Which FAI version are you using?
- Try fai-make-nfsroot -v to get more information.
- We do not need fai-setup.log, parts of fai-make-nfsroot.log are fine

In fai-make-nfsroot we call
  $ROOTCMD apt-get -yf --no-install-recommends install fai-nfsroot 
dracut-config-generic dracut-live dracut-network dracut-squash
which should automatically install fai-client, because the package
fai-nfsroot depends on fai-client. I wonder why this does not work in
your environment.

Since I never installed FAI on an Ubuntu system, maybe others can
help.

-- 
regards Thomas

Re: FAI + SaltStack anybody?

[Thomas Lange]

> On Fri, 06 Oct 2023 21:57:28 +1300, Andrew Ruthven  
> said:

> This isn't ideal as the secrets are still present in the NFSROOT for a 
short
> period of time, but does solve the chicken and egg issue others mentioned
This reminds me of a solution I once saw.
Put some info into a fifo (named pipe), so only one receiver can read
it. After that the fifo is empty.

What about having a daemon on the FAI server which serves some secrect
using:
echo secrect | nc -p 12345 -l

So only one FAI client can read the secrect from port 12345 once.
This may help a little bit.
-- 
regards Thomas

Re: ypbind not enabled after installation

[Thomas Lange]

> On Thu, 05 Oct 2023 16:04:58 +0200, Julien Vion via linux-fai 
>  said:

> Somehow, "systemctl enable" in the script does not work.
Currently FAI does not start systemd during the installation.
That's why a systemctl call in a script cannot work.
But systemctl enable only creates a symlink, so ln -s should help.
The next FAI version will use systemd during the installation.

-- 
regards Thomas

Re: Setting variables and notiying the user - How?

[Thomas Lange]

> On Wed, 27 Sep 2023 12:21:15 +0200, Thomas Kloepfer 
>  said:

> Problem 1: How do I display a message to the User?
Have you tried to redirect the output to the tty/console?
in class/41-warning.sh we do this
out=$(tty)
dialog .. > $out


> Problem 2:

> Scripts in [ config/scripts// ] have access to environment 
> variables such as HOST, FAI_ROOT, KEYMAP ... . But how can I set up my 
> own environment variable in one script and later on read this variable 
> in an other script.
All shell variables you define in class/*.var are available in the
customization scripts.


> For my task I need to set up the password for the sudo user. I want to 
> store this generated password in an environment variable. An I want to 
> read this environment variable later in my script.

> The documentation of FAI says, that variables that have been set up in 
> hooks get sourced.

> So I tried to set up my own hook ( [ 
> config/hooks/myvar. ] ) and set a variable. But the variable 
> I set up (and I exported) was not accessible later.
Not sure if this still works, but try to name your hook
config/hooks/taskname.classname.sh

> I found a workaround for this task, but I am not happy with it:
> I tweaked [ config/hooks/setup.DEFAULT.sh ] added my variable this 
> worked. I could access this variable later in my scripts.
Oh, that's what I just suggested.
Maybe just add a hook
hooks/configure.DEFAULT.sh containing your variable definitions or just
source /tmp/myvars.sh in this hook.
OR name the hook hooks/defvar.DEFAULT.sh


-- 
regards Thomas

Re: Installation overwrites USB boot stick

[Thomas Lange]

Hi Michael,

please check if you are using the newest version of this file.
In 2017 I've added some code to ignore the USB stick (git show 0e071f2ec8e6be1)

https://github.com/faiproject/fai/blob/master/lib/fai-disk-info

> As expected, /sys/block/sd*/removable reads '1' for the USB stick and 
> '0' for the SSD.

> What is the advantage of checking the device group instead of the 
> removable flag in /sys here anyway?
Some people may want to install to a removable disk, for e.g. a micro
SD card.


-- 
regards Thomas

Re: Problem adding systemd-resolved to FAI generated image

[Thomas Lange]

I found some hints in the config space for the official Debian cloud
images:

https://salsa.debian.org/cloud-team/debian-cloud-images/-/blob/master/config_space/sid/hooks/updatebase.BASE
https://salsa.debian.org/cloud-team/debian-cloud-images/-/blob/master/config_space/sid/scripts/CLOUD/20-enable-systemd-resolved


> Now, you would think that is easy :
> 1. Add systemd-resolved to package_config
> 2. Add "$ROOTCMD systemctl enable systemd-resolved" to scripts

> However, the instant I make those two changes, my build breaks.

-- 
regards Thomas

Re: fcopy to a symlinked directory

[Thomas Lange]

> On Tue, 19 Sep 2023 19:07:21 +0200, Christopher Huhn  said:

> Hi y'all
> I try to `fcopy -r /dir_a` where dir_a is a symlink to dir_b – on some 
> systems.

> Looking at the code it looks like this case is simply unhandled by 
> fcopy. Is that correct?
I think you hit
https://bugs.debian.org/985303

-- 

FAI wiki online again

[Thomas Lange]

Hi folks,

the FAI wiki is back online.

-- 
regards Thomas

Re: Installation of package_config/CLASS.gpg

[Thomas Lange]

I would suggest you are using a hook with an fcopy command to put
those files to some other locations.

> On Tue, 18 Jul 2023 21:36:04 +1200, Andrew Ruthven  
> said:

> Hey,
> I see that FAI since 5.8.7 will install package_config/CLASS.gpg
> into /etc/apt/trusted.gpg.d/ . Apt will then trust all the keyrings in
> /etc/apt/trusted.gpg.d . This isn't really ideal, and I'd prefer to use
> Signed-By to specify which GPG keyring to trust for our various additional
> repositories.

> How about having task_repository check for another file, say
> package_config/CLASS.gpg_dest that'd allow us to specify where to copy
> package_config/CLASS.gpg to?

-- 
regards Thomas

Re: Anyone here on the list having an fedora Server (newer versions) for serving diskless-clients (running fedora)?

[Thomas Lange]

> On Wed, 16 Aug 2023 14:16:35 +0200, Sven Schumacher 
>  said:

> Hello,
> in older documentation I can find the info, that there was an option 
> "readonly=yes" to use on fedora for creating an nfsroot to be served for 
> diskless clients but in newer versions this file isn't available 
> anymore. When I boot my client everything is fine, as long as I have a 
> writeable nfs-mount. Otherwise the system hangs on booting. Using more 
> than ONE client isn't possible this way. I'll have to separate /etc and 
> /var. something like overlayfs, like fai is using, would be the 
> solution, perhaps.

> Any other suggestions/hints?
I do not remember where was this option readonly=yes? Inside FAI?
Something fedora specific?
Using the nfsroot for diskless clients always will use overlayfs for
making the whole filesystem writable, even if it's only eported
read-only.

There may be problems, if you export it via NFS v4, but somewhere in
the mailing list archive there's a mail how to fix that.
Read:
https://lists.uni-koeln.de/pipermail/linux-fai/2019-September/012351.html

I have written an article in the FAI wiki (which is currently down)
about using the nfsroot as a live system for diskless clients.
Here's the link to archive.org:
https://web.archive.org/web/20230517095137/https://wiki.fai-project.org/index.php/Use_nfsroot_for_diskless_clients

In the next FAI version, building live systems will be much
easier, you can then also create live ISO images using "fai dirinstall"
and then fai-cd.

-- 
regards Thomas

Re: FAI wiki looks empty

[Thomas Lange]

Sorry, the wiki is down after an upgrade to the new mediawiki version.
I may take 2-3 weeks till I have time to fix this.
-- 
regards Thomas

Re: Next make-nfsroot roadblock

[Thomas Lange]

> On Mon, 31 Jul 2023 18:47:02 +, Lonlone Lee  
> said:

> FAI Server Operating System: bullseye
> FAI Package Version: 6.0.5 (Upgraded with bookworm fai package)

Are you upgrading the nfsroot?
I always create the nfsroot from scratch.

Please prove more info which commands you are calling.
Try fai-make-nfsroot -fv and provide the whole log file. You may put
the log file onto paste.debian.net

-- 
regards Thomas

Re: make-nfsroot error

[Thomas Lange]

> On Fri, 28 Jul 2023 20:15:25 +, Lonlone Lee  
> said:

> FAI Package Version: 6.0.5 (Upgraded with bookworm fai package)

> dpkg: warning: downgrading fai-setup-storage from 6.0.3+deb12u1 to 5.8.4
> Unpacking fai-setup-storage (5.8.4) over (6.0.3+deb12u1) ...

You are trying to downgrade to a very old version.

Please try do recreate the nfsroot from scratch. It seems to
upgrade/downgrade an existing nfsroot.

-- 
regards Thomas

Re: install_packages fails at "aptitude --allow-change-held-packages

[Thomas Lange]

Thanks ti Henning for the patch. It will be included in the next release.

> On Fri, 28 Jul 2023 08:32:53 +0200, Henning Glawe 
>  said:

> Moin Adam,
> On Thu, Jul 27, 2023 at 06:56:27PM -0700, Adam Lewenberg wrote:
>> When running the fai-make-nfsroot program I am getting an
>> install_packages error:
>> 
>> aptitude: unrecognized option '--allow-change-held-packages'
>> aptitude 0.8.13
>> 
>> The call to install_packages looks like this:

> I also ran into this issue, a proposed fix is in:
> https://github.com/faiproject/fai/pull/119

-- 
regards Thomas

New FAI ISO images, Live ISO

[Thomas Lange]

Hi all,

I've created new FAI ISO images, using Debian 12 and the newest FAI
version 6.0.3 They are available at
https://fai-project.org/fai-cd/

There's also a new Live ISO which was created by a not-yet-released
FAI version (the currenty version in the git repository). The next FAI
version will support building live ISO images by combining the two
commands:

# fai dirinstall
# fai-cd

-- 
regards Thomas

Re: LVM mirror

[Thomas Lange]

> On Tue, 21 Mar 2023 17:12:15 +0100, Henning Fehrmann 
>  said:

> Hello,
> we intend to install the OS on a LVM which is put on top of two mirrored
> physical devices without using mdadm. We use FAI v 5.10.3.

> Should I create a hook to achieve this or should I better opt for a more 
recent
> FAI version?
I think you have to create a hook.
I don't remember a change in setup-storage that would support your
setting.

-- 
viele Grüße Thomas

Re: fai-diskimage "ERROR: No classes are defined"

[Thomas Lange]

> On Tue, 23 May 2023 07:48:08 +, "Schulz, Reiner"  
> said:

> Hi Thomas,
> Now with fai 6.0.2 i use
> fai-diskimage --cspace file:///adm/rschulz/fai/config -v -N --size 25G -u 
x-ra5285
> -ra5285.raw
> and got:
> ERROR: No classes are defined

> looking on the fai-diskimage  code at lines
> 182 if [ -z "$classes" ]; then
> 183 die 7 "No classes are defined"
> 184 fi
My code looks different:

if [ -z "$classes" -a -z "$renew" ]; then
die 7 "No classes are defined. Use -c or -N."
fi

Are you reall using the 6.0.2 version?


This change was done in 116a8709bc698344eaf5d00c109a8737d5273de6
-- 
regards thomas

Re: fai-diskimage and /dev/loop0

[Thomas Lange]

> On Tue, 16 May 2023 11:43:45 +, "Schulz, Reiner"  
> said:

> Hi there
> I am on my first steps with fai-diskimage from fai 5.8.4

> but grub ist installed on /dev/loop0 and in fstab „/“ is set to 
/dev/loop0p1
> so systemd-mount tries to mount /dev/loop0p1
> what do i wrong? J
Maybe your FAI version is too old?
Or do your disk_config uses the device name instead of an UUID for the
fstab entries? Please try to use the option fstabkey:uuid in your
disk_config.

-- 
ragrds Thomas

Re: Ubuntu 22.04: problem installing nvidia-driver-525

[Thomas Lange]

Here's the log of an Ubuntu installation using FAI 6 and Ubuntu 22.04
in kvm (no real nvidia HW available)

http://public.cs.uni-koeln.de/lange/ubuntu-fai.log

There's no problem by using the wrong kernel. Have a look at line
9236ff

The installaion ISO was created using https://fai-project.org/FAIme,
and I've selected Ubuntu version and added the two packages

  linux-headers-generic nvidia-driver-525

Maybe this helps
-- 
regards Thomas

Re: Ubuntu 22.04: problem installing nvidia-driver-525

[Thomas Lange]

> On Thu, 20 Apr 2023 13:14:04 +0200, Andreas Sindermann 
>  said:


> It should create the initramfs in the Ubuntu environment, not in the 
> Debian environment...
update-initramfs should recreate the initrd.

-- 
viele Grüße Thomas

Re: Ubuntu 22.04: problem installing nvidia-driver-525

[Thomas Lange]

> On Thu, 20 Apr 2023 10:45:52 +0200, Andreas Sindermann 
>  said:

> Hi all,
> there seems to be a problem installing a Ubuntu 22.04 package that is 
> trying to call update-initramfs:

> update-initramfs: Generating /boot/initrd.img-5.10.0-18-amd64
> W: missing /lib/modules/5.10.0-18-amd64
> W: Ensure all necessary drivers are built into the linux image!
> depmod: ERROR: could not open directory /lib/modules/5.10.0-18-amd64: No 
> such file or directory
Is 5.10.0-18-amd64 the kernel version, that is running during the
installation? Or is it the ubuntu kernel version inside /target?

My guess is that update-initramfs tries to build an initrd for the
wrong kernel version.

Can you just ignore this error, or does the installation aborts?


regards Thomas

Re: FAI 6.0.1 on Debian 12/testing

[Thomas Lange]

Hi Bob,

thanks for your feeback. I found a minor bug in lib/subroutines,
which I could fix with this commit

https://github.com/faiproject/fai/commit/80e5d44bc8aada63febbe9a85aa9fe7c3bc61ab6

best regards thomas

Re: FAI 6.0.1 on Debian 12/testing

[Thomas Lange]

> On Tue, 11 Apr 2023 12:37:45 -0700, Bob A  said:

> Thank you, Thomas.
> I noticed all my package_config/* files had aptitude, so I changed them 
all to "install" or
> "install-norec", but then I had the same problem with aptitude -- which 
confused me.

If you changed to "install" or "install-norec", then there should be no
errors any more. I wonder what you mean by "the same problem with
aptitude" if no aptitude is used.

Oh. Maybe this option was added to apt/apt-get in a version that is
newer than the one you are using (inside the nfsroot).

-- 
best regards thomas

Re: FAI 6.0.1 on Debian 12/testing

[Thomas Lange]

> On Tue, 11 Apr 2023 11:11:53 -0700, Bob A  said:

> Long story short, many packages are not getting installed due to an error 
with some options being sent to aptitude. The installation does not fail, just 
many packages are not installed.
> Error:

> aptitude: unrecognized option '--allow-change-held-packages'
Yes, this is a minor bug in FAI 6. We've added the option
--allow-change-held-packages which aptitude does not recognize. You
should move from aptitude to install (which means apt-get) in the
package_config/* files.

Or you must edit the file bin/install_packages and remove this option
from the perl code.


best regards thomas

Create your own Ubuntu installation ISO

[Thomas Lange]

Hi all,

a new feature is available on https://fai-project.org/FAIme
Build your own customized installation ISO for Ubuntu.

You can select if you want to install an Ubuntu LTS 22.04 server or
desktop and enable Ubuntu LTS packages which are also called Hardware
Enablement (HWE). Different partitioning schemes are available and you
can select a language and keyboard layout and add packages which are
then automatically installed from the customized installation media.
It's possible to upload your public ssh key or specify your
github account name for easy access to the root account.

Advanced users can upload a postinst script, which can be executed
during the first boot of the computer.

-- 
best regards Thomas

RE: RAID 0 Disk config.

[Thomas Lange]

> On Wed, 5 Apr 2023 07:34:11 +, Marc Hoppins via linux-fai 
>  said:

> Would the wipe-all script sit in class, like

> 20-hwdetect.sh
> 21-wipe-all.sh
No, put those script into hooks/
and name them for e.g. partition.WIPEDISKS

Then it get executed before the partitioning task starts if the class
WIPEDISKS is defined.

-- 
viele Grüße Thomas

RE: RAID 0 Disk config.

[Thomas Lange]

Here you can find two small bash scripts, one called wipe-all
which removes all partitions without caring about raid or lvm.
The other script is called partition.WIPEDISKS is a FAI hooks and
first stops lvm and raid then does the wipefs call.

https://fai-project.org/download/misc/

RE: RAID 0 Disk config.

[Thomas Lange]

> On Tue, 4 Apr 2023 12:11:23 +, Marc Hoppins via linux-fai 
>  said:

> Another disk question if I may, does FAI test for existing partitions on 
the target disk?  I think the reason I was getting failures is that the target 
probably had partitions created by a previous, failed FAI.

Yes it tests for existing partition, because you may want to preserve
the partition and/or the data in it.

regards Thomas

Re: RAID 0 Disk config.

[Thomas Lange]

> On Thu, 23 Mar 2023 14:56:35 +, Marc Hoppins via linux-fai 
>  said:


> For a RAID config, the documentation only specifies pre-sized partitions 
which I do not want. I currently have:

Hi Marc,

without checking I guess our tool does not support raw disks for this
setup. But you  can use a size of 1G- which means create a partition
with a size of at least 1Gbytes but maximize the partition with no
limit. That means the partition will get the whole size of the disk.
This may be similar to what you want. It looks likes this:

disk_config /dev/sda fstabkey:uuid align-at:1M
primary - 1G-  -  -  -


Then you can use the partitions to create your raid0.


-- 
regards Thomas

RE: Ubuntu 22

[Thomas Lange]

   > The basefile on YOUR site? It is fine?
It should be fine, but if you say it's not fine, please provide some
more infos.

regards Thomas

Re: Ubuntu 22

[Thomas Lange]

> I see a JAMMY64 basefile on the fai-project site. Our FAI guy here tells 
me that it is not functional, is this the case?
> Marc
Any error messages? Some more info would be good.

-- 
regards Thomas

Launch of new FAI project web site

[Thomas Lange]

Hi,

after more than 13 years, I've launched a new design for the FAI
project web site. It now uses Materialize CSS and will work much
better on mobile devices. Thanks to Thorsten Bülo who did the first
part of converting the web pages to the new design.

I hope you all enjoy the new layout.


best regards Thomas

FAIme service now support Debian 12 aka bookworm

[Thomas Lange]

Hi all,

the FAI web service for building installation and cloud images now
supports Debian 12 aka bookworm. You can give it a try at

  https://fai-project.org/FAIme

It uses FAI 6.0 during installation and the cloud images are also
created by the newest FAI version.

-- 
regards Thomas

Re: dracut fails for bookworm fai-make-nfsroot call

[Thomas Lange]

Yes, you are right. FAI is not yet using systemd, so it needs
network-legacy. A change in the dependencies of dracut 059 now needs a
simple patch for fai-make-nfsroot to make sure dracut-network gets installed.
This is the change
https://github.com/faiproject/fai/commit/9e640bea6848bedc24fcdec32ce19d2605565d32

which is already included in FAI 6.0. I recommend to use FAI 6.0 also
on a bullseye FAI server, to create an bookworm nfsroot.

-- 
best regards Thomas

Re: Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?

[Thomas Lange]

Here are some code snippets I found for dhcpd.conf:


   if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:0" 
{
 filename "fai/pxelinux.0";
   }
   if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:7" 
{
 filename "fai/syslinux.efi";
   }
}


 if option arch = 00:00 {
 filename "fai/pxelinux.0";
 } elsif option arch = 00:09 {
 filename "fai/bootx64.efi";
 } elsif option arch = 00:07 {
 filename "fai/grubnetx64.efi";
 } elsif option arch = 00:06 {
 filename "fai/bootx64.efi";
 }


https://github.com/cobbler/cobbler/discussions/2687

-- 
regards Thomas

Net-boot arm64 with nfsroot, Re: Create fai-cd for other architecture?

[Thomas Lange]

> On Thu, 26 Jan 2023 11:06:14 +0100, Steffen Grunewald 
>  said:

> Which info do you need? Would the output of `ls -lR /boot` (on a booting
> machine) be sufficient, or even too much? Any selections from dpkg output?
I'm not sure which infos I need. Maybe the pathes for grub.
I also need to know if some grub commands in fai-cd need to be called
with different parameters. What about --format=x86_64-efi? Do we need
to add some grub modules for --install-modules=...?
What about the xorriso call? There are also intel arch specific options.


I also have a small arm64 device here (Hikey960), currently updating it to
bullseye and try to install the fai packages on it.


> I the meantime: what would I see if I properly assign a nfsroot but
> "forget" to set the $FAI_* variables? Typing long command lines into grub
> is so failure-prone...
Have a look at your pxelinux.cfg. There you see the parameters.

These are important
ro rootovl ip=dhcp root=1.2.3.4:/srv/fai/nfsroot,nfsvers=3

Others like FAI_CONFIG_SRC may be hardcoded inside the nfsroot in
/etc/fai/fai.conf. Not really sure if this still works.

-- 
regards Thomas

Re: Create fai-cd for other architecture?

[Thomas Lange]

> On Wed, 25 Jan 2023 13:43:30 +0100, Steffen Grunewald 
>  said:

> Cross-platform doesn't work since "grub-mkstandalone" would throw an
> "exec format error".
> Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 
hardcoded.
What about running a qemu cross-arch VM on amd64? It would be slow,
but should be doable? Fixing the hard-coded arch is easy if you tell
me how this looks for arm64.

I'm happy for some hints how to support fai-cd on arm64, and maybe
also cross-arch available. But I'm not sure if grub-mkstandalone can
do cross-arch.

-- 
reagrds Thomas

Re: UEFI boot order, Re: Tip: Remote FAI install

[Thomas Lange]

Hi,

I found this code that move the first boot entry (which is expected to
be the new entry after an installation) to the end of the boot list.

https://community.theforeman.org/t/efi-boot-order-with-centos-7-network-boot-vs-local-boot/10529/2

# the EFI boot manager is only installed on UEFI hosts by Anaconda
if [[ -f /sbin/efibootmgr ]]; then
echo "- Changing EFI boot order to preserve network boot ..."
created_entry=$(efibootmgr | grep "BootOrder" | cut -d " " -f 2 | cut -d 
"," -f 1)
others=$(efibootmgr | grep "BootOrder" | cut -d " " -f 2 | cut -d "," -f 2-)
new_order="${others},${created_entry}"
efibootmgr -q -o ${new_order}
fi

-- 
regards Thomas

Re: UEFI boot order, Re: Tip: Remote FAI install

[Thomas Lange]

Oh, you can set an efi boot entry inactive using -A
Maybe this helps changing the boot device, without changing the
overall boot order.

-- 
regards Thomas

Re: UEFI boot order, Re: Tip: Remote FAI install

[Thomas Lange]

Hi

for one-time changes in the boot order, you can use
efibootmgr --bootnext
and set this to the network device. From man efibootmgr:

BootNext - the boot entry which is scheduled to be run on next
   boot. This supercedes BootOrder for one boot only, and is
   deleted by the boot manager after first use.  This allows
   you to change the next boot behavior without changing
   BootOrder.

But in the end, we want to have a script that changes the boot
order. And we must check if an upgrade of the grub package calls
grub-install again because that may change the order.

First I like to collect the output of efibootmgr. I like to know how
different the network boot options are stored in UEFI BIOS.

Here are two examples:

BootCurrent: 
BootOrder: ,0001
Boot* debian
Boot0001* UEFI: IP4 Intel(R) 82579LM Gigabit Network Connection



BootCurrent: 0001
BootOrder: 0001,0002,001D,0017,0018,0019,001A,001C,,001E,001F,0024,001B
Boot* Windows Boot Manager
Boot0001* debian
Boot0002* Linux-Firmware-Updater
.
.
Boot001D* PCI LAN
Boot001E  Other CD
Boot001F  Other HDD
Boot0020* USBR BOOT CDROM
Boot0021* USBR BOOT Floppy
Boot0022* ATA HDD
Boot0023* ATAPI CD
Boot0024* PCI LAN


I wonder why the second machine has two LAN entries, because it's a
Thinkpad laptop with only one ethernet device.

-- 
regards Thomas

Re: Progressbar for extracting Basefiles

[Thomas Lange]

> On Wed, 18 Jan 2023 12:28:26 +0100, Karsten Nordsiek 
>  said:

>     -1   Use only first tar file matching class name.
>     -c class[class]  Define classes (space separated).
Hmmm, it's hard to see what you have changed. A diff or git diff would
be much better to understand.

My first guess is to add -f to the pv call.

  -f, --force  output even if standard error is not a terminal

Inside FAI we redirect the output to a pipe which also does tee into a
file. Or you may try to force the output (does pv use stderr?) to
/dev/ttyN.

-- 
viele Grüße Thomas

FAI 6.0 released and new ISO images using Debian 12 bookworm/testing

[Thomas Lange]

Hi all,

after more than a year, a new major FAI release is ready to download.

Following new features are included:

  * add support for release specification in package_config via release=
  * the partitioning tool now supports partition labels with GPT
  * support partition labels and partition uuids in fstab
  * support for Alpine Linux and Arch Linux package managers in install_packages
  * Ubuntu 22.04 support added
  * Rocky Linux 9 support added
  * use zstd instead of gzip
  * fai-chboot: variable substitution for $IP and $HOST when copying a
template
  * all customization scripts now write to scripts.log. The old
behaviour can be enable by setting $FAI_USE_OLD_SCRIPT_LOGS
  * add support for NVme devices in fai-kvm
  * add ssh key for root remote access using classes
  * drop support of .asc signatures of your repository, use .gpg
instead

Sure, we have a lot of bug fixes included.

Even FAI 6.0 will only be included into Debian bookworm, you can
install FAI 6.0 on a bullseye FAI server and create a nfsroot
using bookworm without any problems. The combination of a bullseye FAI
server with FAI 6.0 and a bullseye nfsroot should also work.

New ISO images are available at https://fai-project.org/fai-cd/

The FAI.me build service is not yet using FAI 6.0, but support will be
added in the future.

-- 
regards Thomas

Re: Unable to install etckeeper with FAI - triggers break

[Thomas Lange]

Hi Andrew,

unshare --pid is also called in fai-make-nfsroot. Does it also
need to be removed in there or only in bin/fai?

> On Thu, 05 Jan 2023 20:54:28 +1300, Andrew Ruthven  
> said:

> Hey,
> I've run into an interesting issue now that I've updated an nfsroot to 
Bullseye. The change to using unshare in $ROOTCMD in v5.10 is
> breaking the dpkg trigger for etckeeper.

> After editing /usr/sbin/fai and removing ' --pid' I get a successful 
installation, complete with etckeeper.

-- 
viele Grüße Thomas

Re: Identify disks for setup-storage

[Thomas Lange]

Hi Thomas,

I use a script to reorder the variable $disklist, so setup-storage can
mostly use disk1, but I can define the order of the disk depending on
disk types or even the serial number of a disk. The script is 
class/99-disklist.sh:


#! /bin/bash

mydisks() {

find $* -type l -printf "%f %l\n" | egrep -v '^md|-part|wwn-' | egrep ^ata 
| sed -e 's#.*/##g'| tr '\n' ' '
}

# This is really important, because we use shell globbing for creating the list 
of disks
cd /dev/disk/by-id || echo Cannot get disk information


case $HOSTNAME in
server1) newlist=$(mydisks *WD3000FYYZ* *0F400HGN* ) ;;
server2) newlist=$(mydisks *WDC_WD320* *ST31000* *SAMSUNG_HD502* ) ;;
www5)newlist=$(mydisks *_SSD_* *TOSHIBA* ) ;;
esac

if [ -n "$newlist" ]; then
echo New disklist: $newlist
echo disklist=\"$newlist\" >> $LOGDIR/additional.var
fi

-- 
viele Grüße Thomas

Re: Unable to install etckeeper with FAI - triggers break

[Thomas Lange]

I see no reason against removing the --pid option from unshare.
Any objections from others?

-- 
regards Thomas

Re: btrfs with multiple subvolumes in one partition

[Thomas Lange]

> On Tue, 06 Dec 2022 14:50:18 +0100, Christoph Pleger 
>  said:

> Hello,
> when setting up btrfs with setup-storage, is it possible to configure
> multiple subvolumes in one partition, like I tried to configure with

> disk_config btrfs fstabkey:uuid
> btrfs single / disk1.3 subvol=@root
> btrfs single /var disk1.3 subvol=@root/var
> btrfs single /usr disk1.3 subvol=@root/usr
I think this is related to https://bugs.debian.org/964761
The output of format.log would help for the bug report.

I do not know if the btrfs implementation in setup-storage is intended
to support this.
-- 
regards Thomas

Re: pvscan in udev rules - buster vs bullseye

[Thomas Lange]

> On Wed, 7 Sep 2022 09:21:46 +, Markus Rexhepi-Lindberg 
>  said:

> Hi,
> I was affected by the same issue and the suggested patch fixed it.
> Thank you Andrew.

> Can we add this to FAI? I assume it can be added to fai-make-nfsroot.
The next FAI version (6.0) will include this.
See this commit. The syntax error in it was already fixed.
https://github.com/faiproject/fai/commit/64f85a332864b64b449043199c499f6e43cf5f42


My plan is to release FAI 6.0 before the end of this year.
-- 
regards Thomas

Re: ubuntu basefiles and machine uuid

[Thomas Lange]

See https://github.com/faiproject/fai-config/pull/14 for my solution
of the problem.

-- 
regards Thomas

wrong BOOT_DEVICE when using nvme

[Thomas Lange]

In 2019 I got this bug report on IRC. I now can answer it. sorry for
the delay ;-)



07/24/19 [14:02:13] < lindbergio> Hi! The BOOT_DEVICE variable gets the same 
value as the ROOT_DEVICE in the disk_var.sh file (/dev/vg0/root) while it 
should be /dev/nvme0n1p1 or the UUID 
  ("5C48-D830"). This results in GRUB not 
getting installed as it tries to install to $BOOT_DEVICE. Can anyone help me 
understand why the BOOT_DEVICE 
  variable gets the wrong value? Here is a 
paste of the disk_config I use
07/24/19 [14:02:15] < lindbergio> https://paste.debian.net/plain/1092876.
07/24/19 [14:35:22] < lindbergio> It works if I change the mountpoint for the 
boot partition from "/boot/efi" to "/boot". Then the BOOT_DEVICE=/dev/nvme0n1p1.
07/24/19 [14:36:42] < lindbergio> According to the EFI/ESP example in 
setup-storage(8) one should use /boot/efi and not /boot.
07/24/19 [16:10:48] < lindbergio> Looks like find_boot_mnt_point() in Fstab.pm 
only looks for /boot and thus /boot/efi fails. Is that correct?


disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
primary /boot/efi   500 vfatrw
primary -   0-  -   -

disk_config lvm always_format:all
vg  vg0 disk1.2
vg0-swapswap8192swaprw
vg0-var /var10240   ext4rw
vg0-home/home   50% ext4rw
vg0-root/   34816   ext4rw



You are right, the BOOT_DEVICE is set to the same value as
ROOT_PARTITION (btw, ROOT_DEVICE does not exists in FAI).

Here's the disk_var.sh using your disk_config:

BOOT_DEVICE=${BOOT_DEVICE:-"/dev/vg0/root"}
SWAPLIST=${SWAPLIST:-"/dev/vg0/swap"}
ESP_DEVICE=${ESP_DEVICE:-/dev/nvme0n1p1}
ROOT_PARTITION=${ROOT_PARTITION:-/dev/vg0/root}

BUT! In the script GRUB_EFI/10-setup BOOT_DEVICE will be redefined if
BOOT_DEVICE is a link to a /dev/md device. Here's the output with some
added debug output:

=   shell: GRUB_EFI/10-setup   =
ainsl: appending to /target/etc/default/grub: GRUB_DISABLE_OS_PROBER=true

BOOT_DEVICE: /dev/vg0/root
++ readlink -f /dev/vg0/root
+ _bdev=/dev/dm-3
+ '[' /dev/dm = /dev/dm ']'
++ lvs --noheadings -o devices /dev/vg0/root
++ sed -e 's/^*\([^(]*\)(.*$/\1/'
+ BOOT_DEVICE='  /dev/nvme0n1p2(2983)'
BOOT_DEVICE: /dev/nvme0n1p2(2983)


So it's correct the the value is wrong in disk_var.sh but FAI will
correctly install grub into the disks.

-- 
regards Thomas

Re: CD, NFSROOT and expiring DHCP lease

[Thomas Lange]

Hmm, my last patch does not work, becaus building the initrd by dracut
fails if the kill-dhclient.sh script does not exist.
I've tried it another way by adding this to fai-make-nfsroot:

sed -i -e '/kill-dhclient.sh/d' 
$NFSROOT/usr/lib/dracut/modules.d/35network-legacy/module-setup.sh

It worked, the dhclient was started inside the initrd but not killed.
So the dhclient is still running.
But this also had problems when using an nfsroot mount, when I forced
the lease-time to be short. I guess the dhclient cannot renew the IP
because it accesses files from inside the initrd which are not
available any more when FAI is running.

So, I'm not sure if it works from fai-cd. How should we fix #987779?
Any ideas?

-- 
regards Thomas

Re: CD, NFSROOT and expiring DHCP lease

[Thomas Lange]

Hi Andrew,

yes this makes it more worse, I can confirm it.
The new approach will be to stop the initrd from killing dhclient.
Here's the patch, but it's not yet tested:


diff --git a/bin/fai-make-nfsroot b/bin/fai-make-nfsroot
index 3b9fbb6e..02b332a4 100755
--- a/bin/fai-make-nfsroot
+++ b/bin/fai-make-nfsroot
@@ -458,6 +458,7 @@ EOF
 # remove file diversion, then create initrd
 rm $NFSROOT/etc/kernel/postinst.d/dracut
 $ROOTCMD dpkg-divert --rename --remove /etc/kernel/postinst.d/dracut
+rm -f $NFSROOT/usr/lib/dracut/modules.d/35network-legacy/kill-dhclient.sh
 $ROOTCMD dpkg-reconfigure dracut
 
 set_root_pw



> On Fri, 20 Aug 2021 20:20:08 +1200, Andrew Ruthven  
> said:

> On Thu, 2021-04-29 at 22:48 +1200, Andrew Ruthven wrote:
>> Is there an approach so that ISO images created with fai-cd
> perform
>> DHCP lease renewals?
> As a workaround, you could write a hook, that starts dhclient.
> Since the hooks are on the ISO and will be executed, this should 
work
> until I release a new FAI version with a fix.

> Yup, sure thing. I'll see about doing this tomorrow.

> I can now confirm that this didn't work, in fact it made things worse!

> Running dhclient caused a second IP to be allocated to the machine being 
built which then
> caused the nfsroot to become unreachable.

Re: fai-make-nfsroot docs and missing reboot

[Thomas Lange]

Hi Diego,

I remember I had this issue years ago, and I may not had fixed it
correctly. Using the FAI flag reboot we cannot cover three different
cases. In my own config space I've added a line into hooks/savelog.LAST.sh

savelog.LAST.sh:

.
.
.
if [ -s $errfile ]; then
   echo "ERRORS found in log files. See $errfile" >&2
else
   echo "Congratulations! No errors found in log files."
#   export flag_reboot=1
fi


If you uncomment the line "export flag_reboot=1"
the machine will reboot automatically if no errors are found.
Hope this helps.

> On Tue, 31 May 2022 10:56:50 +0200, Diego Zuccato 
>  said:

> Moreover, at the end of the install, after saying there were no errors, 
> FAI asked to press ENTER to reboot, but IIUC that should me automatic, 
> w/o manual confirmation: docs at 
> https://fai-project.org/fai-guide/#_a_id_faiflags_a_fai_flags says that 
> "If no errors occurred, the client will always reboot automatically." 
> and that seems not to be the case (but it's exactly what I wanted: no 
> error = boot into newly installed SO w/o any interaction, while 
> specifying 'reboot' seems to suggest that it reboots also in case of 
> errors).


-- 
regards Thomas

Re: fai-make-nfsroot docs and missing reboot

[Thomas Lange]

> On Tue, 31 May 2022 10:56:50 +0200, Diego Zuccato 
>  said:

> Reading man page for fai-make-nfsroot it seems 'FULL' install is the 
> default. But after having troubles adding Salt repository (could not 
> verify server certificate), I noticed that ca-certificates (installed 
> only in FULL nfsroot) was not present. After using
> fai-make-nfsroot -k -c FULL
> it works. That makes me suspect that the actual default is -s .
> Could the man page be improved by specifying which is the case?
I've checked this. fai-make-nfsroot -fv creates an nfsroot including
ca-certificates and other packages defined in the FULL section.
So I cannot reproduce your bug.

-- 
regards Thomas

Re: FAI from CD with static IP address; $NIC1 is not defined

[Thomas Lange]

Hi Jan,

get-boot-info inside the nfsroot is the right place. I wonder why the
change does not appear on the CD.

It's important that NIC1 is also set in boot.log. If not, that's why
the error occurs.

-- 
viele Grüße Thomas

Re: fai-make-nfsroot fails to download certain packages

[Thomas Lange]

> On Tue, 30 Aug 2022 18:43:11 +0100, Adam Weremczuk 
>  said:


> wget -O fai-project.gpg 
> https://fai-project.org/download/2BF8D9FE074BCDE4.gpg; sudo cp 
> ERROR: The certificate of ‘fai-project.org’ is not trusted.
> ERROR: The certificate of ‘fai-project.org’ has expired.
I have no problem using wget and downloading this file on a bullseye
machine. Maybe your wget version is too old?


> WARNING: These unknown packages are removed from the installation list: 
> module-init-tools console-tools grub
module-init-tools was removed from /etc/fai/NFSROOT in FAI 5.1.
console-tools was removed in 2014 from /etc/fai/NFSROOT.
It seems, you have an old config.

> ***

> Lots of:
> insserv: warning: script 'rcS.distrib' missing LSB tags and overrides
Just ignore these.

-- 
regards Thomas

Re: fai-make-nfsroot fails to download certain packages

[Thomas Lange]

Hi Adam,

please try to write ASCII mails, yours is a HTML only mail.

Concering your problem, you should use a official mirror and not a
local apt-cacher and check if the problem disappears.

-- 
regards Thomas

Re: root file system not found on LVM with Ubuntu "jammy" build

[Thomas Lange]

>>>>> On Fri, 19 Aug 2022 06:13:09 -0700, fai524-f...@lewenberg.com said:

> On 8/19/2022 12:31 AM, Thomas Lange wrote:
>>>>>>> On Fri, 19 Aug 2022 00:09:52 -0700, fai524-f...@lewenberg.com said:
>> 
>> > The server builds but the first boot after imaging drops the system to
>> > the initramfs prompt due to a missing /dev/mapper/vg0-root file system.
>> Maybe you forgot to install the package lvm2.
>> 

> That was the issue.
Fince. You should have this error message in fai.log and error.log,
since FAI checks in scripts/LAST/50-misc if you use a lvm device and
have added the lvm2 package:

  ERROR: Found lvm devices, but the lvm2 package was not installed


P.S.: I'm always very interested in user experiences with FAI. Can you
please fill out the FAI questionnaire
https://fai-project.org/questionnaire and send it back to me when you
finished a project with FAI.

-- 
regards Thomas

Re: root file system not found on LVM with Ubuntu "jammy" build

[Thomas Lange]

> On Fri, 19 Aug 2022 00:09:52 -0700, fai524-f...@lewenberg.com said:

> The server builds but the first boot after imaging drops the system to 
> the initramfs prompt due to a missing /dev/mapper/vg0-root file system.
Maybe you forgot to install the package lvm2.

-- 
regards Thomas

Re: Dracut : Could not boot

[Thomas Lange]

> On Mon, 18 Jul 2022 09:42:04 +0200 (CEST), bruno pinto 
>  said:

> Hello Thomas,
> How you put in the initrd the new firmwar?
What do you mean by new firmware? A new version because of an update,
or was it missing?

You need the package name of the firmware added to /etc/fai/NFSROOT,
then rebuild the nfsroot. Then you have the new initrd, which must be
copied to /srv/tftp/fai/ (should be done automatically), then do a
network boot with the new initrd. 
-- 
regards Thomas

Re: Dracut : Could not boot

[Thomas Lange]

> On Tue, 12 Jul 2022 13:47:29 +0200 (CEST), bruno pinto 
>  said:

> I have inside the NFSROOT : firmware-linux-nonfree
> I know what firmware I need: 
https://fwupd.org/lvfs/devices/com.dell.uefi398c4c37.firmware
Using this command you can see which files are included in the initrd
that FAI is using:

/srv/fai/nfsroot/usr/bin/lsinitrd /srv/fai/nfsroot/boot/initrd.img* | less

-- 
regards Thomas

Re: Dracut : Could not boot

[Thomas Lange]

> On Mon, 11 Jul 2022 16:08:57 +0200 (CEST), bruno pinto 
>  said:

> Well I do the installation by network.
> How I proceed to install the non free firmware in the initrd of FAI?
First check if you have the nonfree firmware inside the nfsroot. Do
you know which firmware you need?

-- 
regards Thomas

Re: Dracut : Could not boot

[Thomas Lange]

> On Mon, 11 Jul 2022 15:02:26 +0200 (CEST), bruno pinto 
>  said:

> Last friday, I receive a new laptop to install by FAI (Dell Precision 
5570) , and when I boot by PXE with FAI, I have a message lie this:
Do you do a network installation with FAI? Or do you boot a FAI ISO
from USB/CD,?

> "dracut: Warning: could not boot"
This may be because of an unknown hardware device. If you do the
network installation, the ethernet device must be activated. Otherwise
the USB device containing the ISO will be used.
Maybe your ethernet device needs some nonfree firmware, which is not
included into the initrd that dracut is using.

Do you have any more detailed warning/error message?
-- 
regards Thomas

Re: Sysemd in the nfsroot

[Thomas Lange]

> On Wed, 15 Jun 2022 12:13:23 +0100, Justin Cattle  said:

> Hi Thomas,
> Did you ever have any time to take a look at this, or have any thoughts 
about it ?
No, I did not had time for this yet.

-- 
viele Grüße Thomas

Re: dhcp issue...

[Thomas Lange]

Hi Andreas,

you should check if the dhcp request from the client (which are
technical bootpc packets are received on the dhcp server.
Does the dhcp server have more than one network card? Then check
/etc/default/isc-dhcp-server if the dhcp server is listening on all
interfaces.

# tcpdump -pnlqte udp and src port bootpc and broadcast

You should also see some messages in the log of the dhcp server, for
e.g. if the server receives the request but has no free IP addresses.

In addition to tcpdump there's also dhcpdump for debugging.

-- 
best regards Thomas

Re: Ubuntu-22.04

[Thomas Lange]

> On Wed, 20 Apr 2022 09:40:17 +0200, Stefan Möding  said:


> Ubuntu-22.04 now uses the zstd(1) compression program for packages.  FAI
> can't install these packages out of the box because debootstrap on
> Debian-11 doesn't know about this compression format.  Here are the things
> I changed.
I would just install debootstrap >= 1.0124 on my machine, because this
version added zstd support. Normally it's easy to install a newer version
of debootstrap because it has no special dependencies.
But I didn't tested this yet.
-- 
regards Thomas

Re: Squashfs Image No space left on device

[Thomas Lange]

> On Thu, 19 May 2022 15:11:24 +0200, Karsten Nordsiek 
>  said:

> Hello,
> on one FAI-Server (we are using multiple FAI-Server) I got an error 
> message no space left on device.
fai-cd uses /tmp for creating temporary files which may big a huge
amount. Check if this partition has enough free space.

-- 
viele Grüße Thomas

Re: Sysemd in the nfsroot

[Thomas Lange]

Hi Justin,

I didn't had time to work on this yet. I do not know how to call the
main fai script from systemd, so its output is shown on the console
and the input is also read from the console. It must also work that
the admin can interrupt the fai run via ctrl-C and get an interactive shell.
Does anyone know how to do this?

Currently (using sysv init) the script is started as /etc/init.d/rcS
and starts some other daemons on demand (for e.g. sshd is only started
if FAI_FLAGS includes sshd). How do we do this using systemd?


> On Fri, 6 May 2022 15:02:30 +0100, Justin Cattle  said:

> Hi,
> I'm just wondering what the current blockers to using systemd in the 
nfsroot are.  Are there some things that simply won't work, or is it more about 
reducing complexity by only supporting
> a single init system, or something else ?


-- 
best regards Thomas

Re: booting FAI without PXE, tftp using kexec

[Thomas Lange]

> On Wed, 6 Apr 2022 09:37:08 +0100, Justin Cattle  said:

> Hi Thomas,
> That's very cool.  Thanks for sharing.
> I guess you are copying the kernel and initrd over manually and putting 
them in your current working
> dir ?
Yes, excatly.

-- 
viele Grüße Thomas

booting FAI without PXE, tftp using kexec

[Thomas Lange]

Hi all,

I just installed a machine in a different subnet where I could no use
network booting using PXE.

But on this machine linux was already running. This is how I managed
to boot the FAI system and to start the installation.

Load the FAI kernel and initrd from the nfsroot and add FAI specific boot 
parameters:

# kexec -l vmlinuz-5.10.0-13-amd64 --initrd=initrd.img-5.10.0-13-amd64 
--append="FAI_FLAGS=sshd,verbose,createvt FAI_ACTION=install rootovl rd.neednet 
ip=dhcp root=1.2.3.4:/srv/fai/nfsroot,nfsvers=3 
FAI_CONFIG_SRC=nfs://faiserver/srv/fai/config"

Then reboot the computer using kexec. This will shutdown the OS and
boot the FAI kernel and initrd without going through the BIOS.

# systemctl kexec

-- 
best regards Thomas

Re: no operating system found

[Thomas Lange]

Hi Matthias,

auf jeden Fall mal die Log Dateien der FAI Installion ansehen.
Irgendetwas ist das nicht richtig gelaufen.
Ist das ein UEFI System?


> On Tue, 15 Mar 2022 09:05:21 +, "Weber, Matthias - LASUB" 
>  said:

> Hallo,
> ich habe FAI Ubuntu 20.04 LTS auf einem Lenovo ThinkCentre M72e 
installiert.
> Beim 1. Neustart nach der Installation erscheint "error: file boot grub 
fonts unicode.pf2 not found", was aber nicht schlimm scheint.
> Beim 2. Neustart aber kommt "error 1962: no operating system found" und 
Ubuntu wird nicht gebootet.
> Was kann ich tun?

> -- 
> Vielen Dank und beste Grüße
> Matthias

-- 
viele Grüße Thomas

Re: LUKS, "no device found"

[Thomas Lange]

> On Tue, 25 Jan 2022 16:06:52 -0800, Ian Eure  said:

> Is it trying to tear down the existent (broken) disk structure 
> instead of overwriting it?  Otherwise, I don’t understand what 
If you like to overwrite all old lvm softraid stsructures, you can
use a hook for that.
I use this script  https://fai-project.org/download/misc/wipe-all

> Sorry for the dumb question, but where would I add the "-d" flag? 
> I’m creating a CD image and booting it on the target machine using 
> Ventoy on a USB stick.
You have to edit the fai source code (lib/subroutines) which calls 
setup-storage.
-- 
viele Grüße Thomas