Re: RAID + UEFI

[Justin Cattle]

Hi,

I think we do have some customisation in the various scripts, including
GRUB_EFI/10-setup, that may account for it.
As our script for grub is quite custom to our use case, I'm not sure I can
share it I'm afraid.  But, that is where you need to start investigating.

This is from vague recollection now, but I think essentially you have to
make grug install directly to the individual disk devices, separately, and
not try and use the raid device.
Hopefully someone else has some more concise advice!


Cheers,
Just


On Wed, 20 Sept 2023 at 10:44, Thomas Kloepfer <
thomas.kloep...@iwr.uni-heidelberg.de> wrote:

> Hello Justin,
>
> thanks for the quick answer.
>
> I tried to use your disk_config. And I get a a raid as expected.
> But the script for settign up grup [ config/scipts/GRUP_EFI/10-setup ]
> fails.
> Hence I end up with a system tha can not boot.
>
> What I find in [ scripts.log ]:
> ---
> =   shell: GRUB_EFI/10-setup   =
> ...
> + echo Install grub on /dev/sda
> + unshare --pid --fork --kill-child
>--mount-proc chroot /target grub-install
>--no-floppy --force-extra-removable /dev/sda
> Installing for x86_64-efi platform.
> grub-install: error: disk `md0' not found.
> ++ error=1
> ...
> Install grub on /dev/sdb
> + unshare --pid --fork --kill-child
>   --mount-proc chroot /target grub-install
>   --no-floppy --force-extra-removable /dev/sdb
> Installing for x86_64-efi platform.
> grub-install: error: disk `md0' not found.
> ++ error=1
> ...
> done
> + exit 1
> GRUB_EFI/10-setupFAILED with exit code 1.
> ---
>
> So my question is: Have you twaked [ GRUP_EFI/10-setup ] ?
> How can I fix this error?
> Why is the script telling me it uses [sda] for installation bun then
> trying [md0]?
>
> Perhaps some of the envirenment-variables might help, so I list a few:
> FAI_VERSION=6.0.5
> disklist='sda sdb '
>
> Regards
> Thomas Kloepfer
>
> On 20.09.23 09:21, Justin Cattle wrote:
>
> Hi,
>
>
> There's probably a few different ways to do this.  Here is a fairly
> generic disk_config example I created last time I was working on FAI.
> I haven't used the config n a while, but it's should be some inspiration
> even if it doesn't work for you as it is:
>
> $ cat disk_config/FAIBASE_EFI_RAID
> # default EFI raid disk config
> #
> disk_config disk1 disklabel:gpt fstabkey:uuid align-at:1M
> primary - 256M-   -
> primary - 256M-   -   -
> primary - RAM:20% -   -
>
> disk_config disk2 sameas:disk1
>
> disk_config raid fstabkey:uuid
> raid1   /boot/efi disk1.1,disk2.1 vfatrw
> mdcreateopts="--metadata=1.0"
> raid1   / disk1.2,disk2.2 ext4defaults,noatime,discard
> raid1   swap  disk1.3,disk2.3 swapsw
>
>
> You can also use LVM as well on top if you like.  Here is my example for
> htat [ again, from last time I looked at this stuff ]:
>
> $ cat LVM_EFI_RAID
> # default LVM EFI raid disk config
> #
> disk_config disk1 disklabel:gpt fstabkey:uuid align-at:1M
> primary - 256M-   -
> primary - 1G  -   -
> primary - 4G- -   -
>
> disk_config disk2  sameas:disk1
>
> disk_config raid fstabkey:uuid
> raid1   /boot/efi disk1.1,disk2.1 vfatrw
> mdcreateopts="--metadata=1.0"
> raid1   /boot disk1.2,disk2.2 ext4defaults,noatime,discard
> raid1   - disk1.3,disk2.3 -   -
>
> disk_config lvm fstabkey:uuid
>
> vg vg md2
> vg-root / 256M-   ext4defaults,noatime,discard
> vg-swap swap  RAM:20% swapsw
>
>
> Hope that helps.
>
> Cheers,
> Just
>
>
> On Wed, 20 Sept 2023 at 08:14, Thomas Kloepfer <
> thomas.kloep...@iwr.uni-heidelberg.de> wrote:
>
>> Hello,
>>
>> I am rather new to FAI. But I am impresst by this great tool.
>> Especially by the documentation that commes with it.
>>
>> But I am suprised I do not find any working example on how to set up a
>> simple RAID 1 with UEFI boot.
>>
>> I read all the discussions in this mailinglist (found two threads
>> dealing with the problem).
>>
>> I am aware that RAID and UEFI do not get together well due to the fact
>> that the RAID information is not available at boot-time and [grup] has
>> to deal with a device [sda] rather than with a partition [md0]. Futher
>> more [grup] needs a partition with partition type [vfat].
>>
>> I

Re: RAID + UEFI

[Justin Cattle]

Hi,


There's probably a few different ways to do this.  Here is a fairly generic
disk_config example I created last time I was working on FAI.
I haven't used the config n a while, but it's should be some inspiration
even if it doesn't work for you as it is:

$ cat disk_config/FAIBASE_EFI_RAID
# default EFI raid disk config
#
disk_config disk1 disklabel:gpt fstabkey:uuid align-at:1M
primary - 256M-   -
primary - 256M-   -   -
primary - RAM:20% -   -

disk_config disk2 sameas:disk1

disk_config raid fstabkey:uuid
raid1   /boot/efi disk1.1,disk2.1 vfatrw
mdcreateopts="--metadata=1.0"
raid1   / disk1.2,disk2.2 ext4defaults,noatime,discard
raid1   swap  disk1.3,disk2.3 swapsw


You can also use LVM as well on top if you like.  Here is my example for
htat [ again, from last time I looked at this stuff ]:

$ cat LVM_EFI_RAID
# default LVM EFI raid disk config
#
disk_config disk1 disklabel:gpt fstabkey:uuid align-at:1M
primary - 256M-   -
primary - 1G  -   -
primary - 4G- -   -

disk_config disk2  sameas:disk1

disk_config raid fstabkey:uuid
raid1   /boot/efi disk1.1,disk2.1 vfatrw
mdcreateopts="--metadata=1.0"
raid1   /boot disk1.2,disk2.2 ext4defaults,noatime,discard
raid1   - disk1.3,disk2.3 -   -

disk_config lvm fstabkey:uuid

vg vg md2
vg-root / 256M-   ext4defaults,noatime,discard
vg-swap swap  RAM:20% swapsw


Hope that helps.

Cheers,
Just


On Wed, 20 Sept 2023 at 08:14, Thomas Kloepfer <
thomas.kloep...@iwr.uni-heidelberg.de> wrote:

> Hello,
>
> I am rather new to FAI. But I am impresst by this great tool.
> Especially by the documentation that commes with it.
>
> But I am suprised I do not find any working example on how to set up a
> simple RAID 1 with UEFI boot.
>
> I read all the discussions in this mailinglist (found two threads
> dealing with the problem).
>
> I am aware that RAID and UEFI do not get together well due to the fact
> that the RAID information is not available at boot-time and [grup] has
> to deal with a device [sda] rather than with a partition [md0]. Futher
> more [grup] needs a partition with partition type [vfat].
>
> I can imaging a solution with FAI that needs special one-time treatment
> done manualy after FAI hast terminated, but this kills the aim of FAI :
> A=fully, A=automated .
>
> so my question is:
> - Can any one supply a solution that manages RAID 1 with UEFI at the
> same time?
>(I would be sadisfied with a solution where the Partition used by
> UEFI-boot is *not* on a RAID.)
>
>or
>
> - Is it realy just not so easy to get RAID and UEFI to work together
> because FAI was established at a time where there was no UEFI boot.
> Hence the stucture of FAI is not intended to solve this in the first
> pace. Attemts to solve this will result in rather ugly workarounds?
>
> I'd appreciate any helpfull reply.
> And thanks again for this great tool.
>
> Thomas Kloepfer
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Launch of new FAI project web site

[Justin Cattle]

Nice - It looks good :)

Thanks again Thomas for your continued support and stewardship of FAI !


Cheers,
Just


On Wed, 8 Mar 2023 at 11:22, Thomas Lange  wrote:

> Hi,
>
> after more than 13 years, I've launched a new design for the FAI
> project web site. It now uses Materialize CSS and will work much
> better on mobile devices. Thanks to Thorsten Bülo who did the first
> part of converting the web pages to the new design.
>
> I hope you all enjoy the new layout.
>
>
> best regards Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Sysemd in the nfsroot

[Justin Cattle]

Hi Thomas,


Did you ever have any time to take a look at this, or have any thoughts
about it ?


Cheers,
Just


On Tue, 10 May 2022 at 20:45, Justin Cattle  wrote:

> Hi Thomas,
>
>
> I found a bit of time to take a look.
>
> I have forked and created a branch with a demo of how it could work.  It's
> not meant to be the finished article, but hopefully it shows how some of it
> could be achieved.
>
> Things to note:
>
>- Relevant part of NFSROOT config needs to be something like this:
>   - systemd systemd-sysv libpam-systemd sysvinit-core-
>- Because of potential race conditions with starting usual tty
>services, I select unused tty for this
>   - fai on tty11
>   - fai-maint on tty12
>- fai-maint is a systemd service which creates a tty with a root bash
>session already logged in.  It's just an example of how that kind of thing
>could be achieved with systemd.
>- Although I tried to make FAI itself still support both SysV and
>Systemd, it's hard to do that in the packaging I think, so for this I
>dropped SysV support in the packaging.
>   - You can use the systemd-sysv compatibility layer to create
>   service units, but that doesn't give as much control over the units [ 
> they
>   are much more basic ], so native systemd is better.
>
>
> Please also note I haven't tested this exhaustively, so far I've only used
> it with a squashfs image, not an nfs root.
>
> I'd be happy to work with you to create a PR and review it together to try
> and integrate it, but I appreciate you may not want to look at this yet.
> It seems to be that multi init system packaging could be the trickiest
> part, and you may not want to drop SysV support any time soon.
>
> Please take a look and let me know what you think:
> https://github.com/ocadotechnology/fai/tree/systemd-support
>
>
> Cheers,
> Just
>
>
> On Fri, 6 May 2022 at 20:40, Thomas Lange  wrote:
>
>> Hi Justin,
>>
>> I didn't had time to work on this yet. I do not know how to call the
>> main fai script from systemd, so its output is shown on the console
>> and the input is also read from the console. It must also work that
>> the admin can interrupt the fai run via ctrl-C and get an interactive
>> shell.
>> Does anyone know how to do this?
>>
>> Currently (using sysv init) the script is started as /etc/init.d/rcS
>> and starts some other daemons on demand (for e.g. sshd is only started
>> if FAI_FLAGS includes sshd). How do we do this using systemd?
>>
>>
>> >>>>> On Fri, 6 May 2022 15:02:30 +0100, Justin Cattle 
>> said:
>>
>> > Hi,
>> > I'm just wondering what the current blockers to using systemd in
>> the nfsroot are.  Are there some things that simply won't work, or is it
>> more about reducing complexity by only supporting
>> > a single init system, or something else ?
>>
>>
>> --
>> best regards Thomas
>>
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Sysemd in the nfsroot

[Justin Cattle]

Hi Thomas,


I found a bit of time to take a look.

I have forked and created a branch with a demo of how it could work.  It's
not meant to be the finished article, but hopefully it shows how some of it
could be achieved.

Things to note:

   - Relevant part of NFSROOT config needs to be something like this:
  - systemd systemd-sysv libpam-systemd sysvinit-core-
   - Because of potential race conditions with starting usual tty services,
   I select unused tty for this
  - fai on tty11
  - fai-maint on tty12
   - fai-maint is a systemd service which creates a tty with a root bash
   session already logged in.  It's just an example of how that kind of thing
   could be achieved with systemd.
   - Although I tried to make FAI itself still support both SysV and
   Systemd, it's hard to do that in the packaging I think, so for this I
   dropped SysV support in the packaging.
  - You can use the systemd-sysv compatibility layer to create service
  units, but that doesn't give as much control over the units [
they are much
  more basic ], so native systemd is better.


Please also note I haven't tested this exhaustively, so far I've only used
it with a squashfs image, not an nfs root.

I'd be happy to work with you to create a PR and review it together to try
and integrate it, but I appreciate you may not want to look at this yet.
It seems to be that multi init system packaging could be the trickiest
part, and you may not want to drop SysV support any time soon.

Please take a look and let me know what you think:
https://github.com/ocadotechnology/fai/tree/systemd-support


Cheers,
Just


On Fri, 6 May 2022 at 20:40, Thomas Lange  wrote:

> Hi Justin,
>
> I didn't had time to work on this yet. I do not know how to call the
> main fai script from systemd, so its output is shown on the console
> and the input is also read from the console. It must also work that
> the admin can interrupt the fai run via ctrl-C and get an interactive
> shell.
> Does anyone know how to do this?
>
> Currently (using sysv init) the script is started as /etc/init.d/rcS
> and starts some other daemons on demand (for e.g. sshd is only started
> if FAI_FLAGS includes sshd). How do we do this using systemd?
>
>
> >>>>> On Fri, 6 May 2022 15:02:30 +0100, Justin Cattle  said:
>
> > Hi,
> > I'm just wondering what the current blockers to using systemd in the
> nfsroot are.  Are there some things that simply won't work, or is it more
> about reducing complexity by only supporting
> > a single init system, or something else ?
>
>
> --
> best regards Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Sysemd in the nfsroot

[Justin Cattle]

Hi Thomas,

Thanks for those initial pointers.
If I get some time I may try and take a look at this and share findings.


Cheers,
Just


On Fri, 6 May 2022 at 20:40, Thomas Lange  wrote:

> Hi Justin,
>
> I didn't had time to work on this yet. I do not know how to call the
> main fai script from systemd, so its output is shown on the console
> and the input is also read from the console. It must also work that
> the admin can interrupt the fai run via ctrl-C and get an interactive
> shell.
> Does anyone know how to do this?
>
> Currently (using sysv init) the script is started as /etc/init.d/rcS
> and starts some other daemons on demand (for e.g. sshd is only started
> if FAI_FLAGS includes sshd). How do we do this using systemd?
>
>
> >>>>> On Fri, 6 May 2022 15:02:30 +0100, Justin Cattle  said:
>
> > Hi,
> > I'm just wondering what the current blockers to using systemd in the
> nfsroot are.  Are there some things that simply won't work, or is it more
> about reducing complexity by only supporting
> > a single init system, or something else ?
>
>
> --
> best regards Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Sysemd in the nfsroot

[Justin Cattle]

Hi,


I'm just wondering what the current blockers to using systemd in the
nfsroot are.  Are there some things that simply won't work, or is it more
about reducing complexity by only supporting a single init system, or
something else ?

We work mostly with Ubunto for various reasons, and would be nice to be
able to consolidate and use that for the nfsroot in FAI as well.

If there are no big technical barriers, would patches for systemd support
be considered ?


Cheers,
Just

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: booting FAI without PXE, tftp using kexec

[Justin Cattle]

Hi Thomas,

That's very cool.  Thanks for sharing.
I guess you are copying the kernel and initrd over manually and putting
them in your current working dir ?


Cheers,
Just


On Tue, 5 Apr 2022 at 14:44, Thomas Lange  wrote:

> Hi all,
>
> I just installed a machine in a different subnet where I could no use
> network booting using PXE.
>
> But on this machine linux was already running. This is how I managed
> to boot the FAI system and to start the installation.
>
> Load the FAI kernel and initrd from the nfsroot and add FAI specific boot
> parameters:
>
> # kexec -l vmlinuz-5.10.0-13-amd64 --initrd=initrd.img-5.10.0-13-amd64
> --append="FAI_FLAGS=sshd,verbose,createvt FAI_ACTION=install rootovl
> rd.neednet ip=dhcp root=1.2.3.4:/srv/fai/nfsroot,nfsvers=3
> FAI_CONFIG_SRC=nfs://faiserver/srv/fai/config"
>
> Then reboot the computer using kexec. This will shutdown the OS and
> boot the FAI kernel and initrd without going through the BIOS.
>
> # systemctl kexec
>
> --
> best regards Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Python Lib for fai-do-scripts(1)

[Justin Cattle]

That's awesome work, I look forward to trying it out.


Cheers,
Just


On Fri, 17 Dec 2021 at 22:33, Maximilian Stein  wrote:

> Hi all,
>
> Today I am very pleased to announce the publication of pyfai [1], a
> Python package providing Pythonic wrappers for the most important
> commands in FAI customization scripts such as fcopy(8). More high-level
> abstractions, e.g. to install systemd units, are planned and will follow
> within the next months. The source code is available on Github [2] and
> extensive documentation is provided on Read the Docs [3].
>
> There are no official Debian packages yet (but planned). Until they
> become available, I would recommend stdeb [4] (Debian package
> python3-stdeb) to build a .deb package: `py2dsc-deb fai-0.1.0.tar.gz`
>
> My current integration in FAI installation and softupdate is based on
> the Debian package that I simply let install by install_packages(8).
> During softupdate this is already enough to actually `import fai` in
> Python customization scripts. During installation, however, Python needs
> to load the package from the target system (unless pyfai is installed in
> the installer system itself, too). Therefore, I set
> `PYTHONPATH="$target/usr/lib/python3/dist-packages"` in
> `class/INSTALL.var`.
>
> I am looking forward to your feedback/ideas/pull requests!
>
> Best,
> Maximilian
>
>
> [1]: https://pypi.org/project/fai/
>
> [2]: https://github.com/st31ny/pyfai
>
> [3]: https://fai.readthedocs.io/
>
> [4]: https://pypi.org/project/stdeb/
>
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Tips for easily creating nfsroot?

[Justin Cattle]

We tend to use either custom packages from our internal repos specified in
the NFSROOT config, or puppet [ as we leverage that a lot anyway ].

I think packages are probably cleaner for your use case, as it can handle
removal of files too, and it's quite simple.
Puppet [ or any other infra orchestration ] can do that too, but can be
more complex to support.


Cheers,
Just


On Wed, 11 Aug 2021 at 15:21, John G Heim via linux-fai <
linux-fai@uni-koeln.de> wrote:

> Diese Nachricht wurde eingewickelt um DMARC-kompatibel zu sein. Die
> eigentliche Nachricht steht dadurch in einem Anhang.
>
> This message was wrapped to be DMARC compliant. The actual message
> text is therefore in an attachment.
>
>
> -- Forwarded message --
> From: John G Heim 
> To: fully automatic installation for Linux 
> Cc:
> Bcc:
> Date: Wed, 11 Aug 2021 09:21:17 -0500
> Subject: Tips for easily creating nfsroot?
> When I recreate the nfsroot for my FAI installation, I have to do a lot
> of extra stuff. I know about adding packages to /etc/fai/NFSROOT and
> writing hooks. Problem is that most of the stuff i have to do is copying
> files. I was thinking about creating a directory within /etc/fai and
> then simply write a hook that does an rsync from /etc/fai/files/ to
> /srv/fai/nfsroot/. Anybody have any ideas for a cleaner way to handle this?
>
>
> --
> John G. Heim; jh...@math.wisc.edu; 608-263-4189
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: unshare causes breakage in docker

[Justin Cattle]

Great - thanks Thomas as always.


Cheers,
Just


On Mon, 21 Jun 2021 at 15:14, Thomas Lange  wrote:

> >>>>> On Tue, 27 Apr 2021 21:05:37 +0100, Justin Cattle 
> said:
>
> > We build FAI images in docker.
> Have a look at
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989547
>
> There's also a workaround how to disable the use of unshare.
>
> --
> viele Grüße Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: unshare causes breakage in docker

[Justin Cattle]

Googling around, there are a few ways.

It looks like this could be the most consistent:

grep -E ":/docker/.+$" /proc/1/cgroup

..which is looking for cgroups under a docker cgroup

It may be refined to:

grep -E ":/docker/[0-9a-f]+$" /proc/1/cgroup

..but I can't say that for sure.

What do you think, does that sound reasonable ?


Cheers,
Just


On Wed, 28 Apr 2021 at 07:58, Thomas Lange  wrote:

> >>>>> On Tue, 27 Apr 2021 21:05:37 +0100, Justin Cattle 
> said:
>
>
> > Can we somehow make unshare vs chroot an option ?  Or, even better
> perhaps, detect docker
> > and don't use unshare in that case ?
>
> Do you know how to detect that the process is running inside docker?
>
> --
> viele Grüße Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

unshare causes breakage in docker

[Justin Cattle]

Hi,


We build FAI images in docker.

It seems like this change has broken that workflow:

commit 5bc6385471b2b8b625c3bbaba76488b3127aebf1
Author: Thomas Lange 
Date:   Thu Sep 24 20:29:08 2020 +0200

Use unshare when calling chroot

Bug report : Daemonized processes inside the chroot cause image
building to hang indefinitely
See https://salsa.debian.org/cloud-team/debian-cloud-images/-/issues/9

diff --git a/bin/fai b/bin/fai
index 433e7eb5..13cddec8 100755
--- a/bin/fai
+++ b/bin/fai
@@ -87,7 +87,7 @@ fai_init() {
   [ $do_init_tasks -eq 1 ] && FAI_ROOT=/target || FAI_ROOT=/
 fi
 # executed command in the environment of the new system
-ROOTCMD="chroot $FAI_ROOT"
+ROOTCMD="unshare --pid --fork --kill-child --mount-proc chroot
$FAI_ROOT"
 # no chroot needed
 [ "$FAI_ROOT" = '/' ] && ROOTCMD=
 target=$FAI_ROOT
diff --git a/bin/fai-make-nfsroot b/bin/fai-make-nfsroot
index 9ac4c367..588a62c0 100755
--- a/bin/fai-make-nfsroot
+++ b/bin/fai-make-nfsroot
@@ -152,7 +152,7 @@ cfdir=$(readlink -f $cfdir) # canonicalize path
 oldnfsroot=$NFSROOT
 deldir=$NFSROOT

-ROOTCMD="chroot $NFSROOT"
+ROOTCMD="unshare --pid --fork --kill-child --mount-proc chroot $NFSROOT"
 export DEBIAN_FRONTEND=noninteractive

 # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I haven't dug into it in great detail yet, but this produces the following
error during fai-make-nfsroot, even running as root user in a container
in privileged mode:


I: Base system installed successfully.
1227
unshare:
umount /proc failed: Invalid argument
1228
mount:
failed to read mtab: No such file or directory
1229
mount:
failed to read mtab: No such file or directory
1230
Log
file written to /var/log/fai/fai-make-nfsroot.log and
/srv/fai/nfsroot/filesystem.dir/var/tmp
1231
ERROR
when calling fai-make-nfsroot.
1232
Log
file written to /var/log/fai/fai-setup.log


If I revert that change, it all works again.

Can we somehow make unshare vs chroot an option ?  Or, even better perhaps,
detect docker and don't use unshare in that case ?




Cheers,
Just

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: FAI confused by DHCP on multiple network interfaces

[Justin Cattle]

BTW - I would also add that I had to achieve a similar config for grub
recently, not syslinux, as I was looking at Secure Boot.

An equivalent for that is adding this to the linux command line:

BOOTIF=$net_default_mac


Cheers,
Just


On Thu, 18 Mar 2021 at 17:13, Thomas Lange  wrote:

> FAI 5.10.1 includes some change in get-boot-info
> which may fix your problems. Instead of using all interfaces that are
> up, boot.log should only contain the parameters of your first NIC.
>
> > Is this a known problem? How can I force the setup back to using the
> initial
> > IPADDR, obtained from the DHCP/TFTP server that is used for the
> install?
> > (I vaguely remember there's a way to feed this into the pxelinux.cfg
> file
> > but ideally I should not need to know whether there are responses
> from other
> > DHCP servers, and if there are any, they should not overwrite IPADDR
> - with
> > a value identical to IP1ADDR.)
> I use IPAPPEND 6 in my pxelinux.cfg files.
> --
> viele Grüße Thomas
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: FAI confused by DHCP on multiple network interfaces

[Justin Cattle]

Hi,


I don't think you mentioned if you are using syslinux or not, but if you
are, you can put something like this in your pxe config file:

ipappend 2

For an entire config might look like this:

default buster

label buster
kernel vmlinuz
ipappend 2
append initrd=initrd.img console=ttyS0,115200n8 console=tty0 ip=dhcp
root=1.2.3.4:/srv/fai/nfsroot/filesystem.dir rootovl
ixgbe.allow_unsupported_sfp=1 FAI_FLAGS=verbose,sshd,createvt,reboot
FAI_ACTION=install rd.net.timeout.carrier=20 rd.net.timeout.ifup=30



Cheers,
Just


On Thu, 18 Mar 2021 at 16:11, Steffen Grunewald <
steffen.grunew...@aei.mpg.de> wrote:

> Hi all,
>
> I'm trying to install an old machine in the basement - from home office,
> not having a simple means to pull network plugs, just a static BMC
> address...
>
> The machine is booted via PXE on *-network:0 (in "lshw"'s terms), and gets
> started properly. But then things go downhill a bit, although the install
> is successful - it's IPADDR that gets mangled, resulting in a chain of
> wrong network settings (and the machine won't boot properly into a state
> I could connect to).
>
> Here's the boot.log returned to the server after installation:
>
> # cat boot.log
> netdevices_all="eth0 eth1 "
> netdevices_up="eth0 eth1 "
> netdevices="eth0 eth1 "
> BROADCAST='10.150.255.255'
> DOMAIN='hypatia.local'
> DNSSRVS='...'
> DNSSRVS_1='...'
> IPADDR='10.150.100.210'
> NETWORK='10.150.0.0'
> NTPSRVS='10.150.100.190'
> NTPSRVS_1='10.150.100.190'
> SERVER='faiserver'
> NETMASK='255.255.0.0'
> IPADDR=172.16.22.77
> SERVER=
> NETMASK=255.255.0.0
> GATEWAYS=172.16.1.1
> BROADCAST=172.16.255.255
> CIDR=172.16.22.77/16
> NIC1=eth1
>
> All is fine (the DNS server is set by DHCP) - until the addition of the
> second IPADDR entry (which indeed belongs to the second NIC).
> Everything that sources boot.log afterwards gets affected by this mistake,
> and variables.log reads a bit strange:
>
> # cat variables.log | grep '^[A-Z].*\..*\..*\..*'
> BROADCAST=172.16.255.255
> CIDR=172.16.22.77/16
> DNSSRVS=...
> DNSSRVS_1=...
> FAI_CONFIG_SRC=nfs://10.150.100.100/srv/fai/config/cluster-buster
> GATEWAYS=172.16.1.1
> 
> IP1ADDR=172.16.22.77
> IPADDR=172.16.22.77
> NETMASK=255.255.0.0
> NETWORK=10.150.0.0
> NTPSRVS=10.150.100.190
> NTPSRVS_1=10.150.100.190
> SERVER=10.150.100.100
>
> The FAI version in the Debian Buster NFSROOT is 5.9.4, the NFSROOT was
> created (and copied to the non-X86_64 server) on Feb 11.
>
> Is this a known problem? How can I force the setup back to using the
> initial
> IPADDR, obtained from the DHCP/TFTP server that is used for the install?
> (I vaguely remember there's a way to feed this into the pxelinux.cfg file
> but ideally I should not need to know whether there are responses from
> other
> DHCP servers, and if there are any, they should not overwrite IPADDR - with
> a value identical to IP1ADDR.)
>
> Patches (to be applied directly to the NFSROOT) are welcome!
>
> Thanks,
>  Steffen
>
>
> --
> Steffen Grunewald, Cluster Administrator
> Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
> Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
> ~~~
> Fon: +49-331-567 7274
> Mail: steffen.grunewald(at)aei.mpg.de
> ~~~
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Secure Boot

[Justin Cattle]

I got this working in the end.

A few key takeaways if anyone else is looking at this.

If you are using debian for the nfsroot, you have to use buster, as Secure
Boot was not supported properly on any release before that.

The nfsroot should contain these packages: shim-signed,
grub-efi-amd64-signed

You have to use grub as the boot loader, not syslinux.

Make sure the following files are copied into your tftp dir:

NFSROOT/usr/lib/shim/shimx64.efi.signed -> bootx64.efi
NFSROOT/usr/lib/grub/x86_64-efi-signed/grubnetx64.efi.signed -> grubx64.efi
NFSROOT/usr/share/grub/unicode.pf2 -> grub/fonts/unicode.pf2
NFSROOT/usr/lib/grub/x86_64-efi -> grub/x86_64-efi

You may also need to symlink the grub dir back from the tftp root, in our
case that looked like: TFTP_DIR/grub -> TFTP_DIR/fai/grub

Then you need a grub.cfg in: TFTP_DIR/fai/grub/grub.cfg

Ours looked a little like this:

set default="0"
set timeout=1

if loadfont unicode ; then
  set gfxmode=auto
  set locale_dir=$prefix/locale
  set lang=en_US
fi
terminal_output gfxterm

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
if background_color 44,0,30; then
  clear
fi

menuentry 'FAI' {
linux fai/vmlinuz console=ttyS0,115200n8 console=tty0 ip=dhcp
root=1.2.3.4:/srv/fai/nfsroot/filesystem.dir rootovl
FAI_FLAGS=verbose,sshd,createvt,reboot FAI_ACTION=install
rd.net.timeout.carrier=20 rd.net.timeout.ifup=30 BOOTIF=$net_default_mac
initrd fai/initrd.img
}




Cheers,
Just


On Thu, 25 Feb 2021 at 16:05, Justin Cattle  wrote:

> Hi FAI-ers,
>
>
> Has anyone done any work on using FAI on hardware that has Secure Boot
> enabled ?
> In particular I'm interested in what you are doing in your nfsroot.
>
>
> Cheers,
> Just
>

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Secure Boot

[Justin Cattle]

Hi FAI-ers,


Has anyone done any work on using FAI on hardware that has Secure Boot
enabled ?
In particular I'm interested in what you are doing in your nfsroot.


Cheers,
Just

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Breaking change in buster dracut with Juniper switch dhcp

[Justin Cattle]

Hi,


Just a quick not on something we found recently in case it helps any else.

After rebuilding our NFS root using debian buster for initial testing, we
found that DHCP within the initrd created by dracut did not work, it just
timed out and never got a lease back from the switch [ Juniper Switches,
providing DHCP ].

We traced it back to this change in
/usr/lib/dracut/modules.d/40network/dhclient.conf:

send dhcp-client-identifier = hardware;

Adding that from the client, meant that due to the strictness of the
Juniper DHCP bindings, while the previous binding existed a lease would not
be given to this request.

The best solution for us is to patch the dracut package to remove that.

Actually it looks like the future for dracut is not using that set of
scripts anymore anyway, they are now referred to as legacy.  But for now,
this may help anyone else who finds the same issue using buster for the NFS
root.


Cheers,
Just

-- 


Notice: 
This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group.

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses.

References to the 
"Ocado Group" are to Ocado Group plc (registered in England and Wales with 
number 7098618) and its subsidiary undertakings (as that expression is 
defined in the Companies Act 2006) from time to time. The registered office 
of Ocado Group plc is Buildings One & Two, Trident Place, Mosquito Way, 
Hatfield, Hertfordshire, AL10 9UL.

Re: Short circuit dracut?

[Justin Cattle]

If you're using PXELINUX to boot, you can use the ipappend option like this
in your pxe config:

ipappend 2

That will make it add this to the kernel comandline for the device it pxe'd
from:

BOOTIF=

So dracut _should_ only use that interface and ignore the others.


If that's not working for you for some reason, another approach would be to
just change the timeouts.
This one is 60 seconds by default:

rd.net.timeout.iflink=10

..or even less if that will still work for the interface you do want
configured :)



Cheers,
Just


On Tue, 2 Jul 2019 at 07:29, Andrew Ruthven  wrote:

> Hey,
>
> I think this has been asked before (possibly by me), but is it possible
> to short circuit dracut so it stops processing interfaces as soon as it
> has a successful DHCP responses?
>
> I'm doing some test builds on a machine with 10 interfaces and dracut
> takes 60 seconds to timeout on each interface...
>
> Cheers,
> Andrew
> --
> Andrew Ruthven, Wellington, New Zealand
> and...@etc.gen.nz  | linux.conf.au 2020, Gold Coast, AU
> Catalyst Cloud:|https://lca2020.linux.org.au/
>https://catalystcloud.nz|
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: fai-cd - encryption possibilities

[Justin Cattle]

Yes, those are good alternatives.

I've also been considering a debian package with an encrypted blob, and a
postinst script that requires user input to decrypt it.

Thanks for your suggestions - time for some cogitating :)


Cheers,
Just


On Sat, 23 Mar 2019 at 11:37, Thomas Lange 
wrote:

> >>>>> On Sat, 23 Mar 2019 11:08:59 +0000, Justin Cattle 
> said:
>
> > Is there any way of customising the fai-cd process? [ without
> hacking the scripts ] ?
> > Does it use FAI hooks at all ?
> No, fai-cd is not using hooks.
> A simple idea would be, as Andrew said, just to add an encrypted file
> to the config space which then get onto the FAI CD. You can then write
> a hook, that decrypts this file and maybe loop mount it.
>
> --
> regards Thomas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: fai-cd - encryption possibilities

[Justin Cattle]

Hi Thomas,

Is there any way of customising the fai-cd process? [ without hacking the
scripts ] ?
Does it use FAI hooks at all ?
Or do you have any other ideas ?


Cheers,
Just


On Fri, 22 Mar 2019 at 22:40, Thomas Lange 
wrote:

> setup-storage is not used during the creation of fai-cd.
> --
> regards Thomas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: fai-cd - encryption possibilities

[Justin Cattle]

No problem, thanks anyway Étienne.


Cheers,
Just


On Sat, 23 Mar 2019 at 08:51, Étienne Mollier 
wrote:

> Good Day All,
>
> Justin Cattle, on 2019-03-22 :
> > Re-reading my email, I don't think I was clear enough.  So I
> > _think_  you may have mis-understood what I was asking.
>
> Your thinking is right, I misunderstood the situation.  It could
> have come from the clarity of your email perhaps, or simply from
> the late time of day from my perspective.  :)
>
> Thomas Lange, on 2019-03-22 :
> > setup-storage is not used during the creation of fai-cd.
>
> Indeed, should ciphering the fai-cd itself occur, that would be
> by other means.  Cryptsetup could be involved I guess, but
> concerning the details of implementation, I must admit I have no
> idea for the moment.
>
> Sorry for the confusion,
> Kind Regards,
> --
> Étienne Mollier 
>
> All opinions are my own.
>
>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: fai-cd - encryption possibilities

[Justin Cattle]

Hi Étienne,


Thanks very much for your response.

Re-reading my email, I don't think I was clear enough.  So I _think_  you
may have mis-understood what I was asking.

What I actually want to do is, produce an ISO using the fai-cd utility, but
make some of the contents of that ISO encrypted.
So, not encrypting the disks in the server, but part or all of the ISO used
to build the server.

It maybe that's what you meant too, in which case I'd like more detail on
how I would use setup-storage as part of the ISO creation :)

So, apologies if I was a mis-understanding, and I look forward to any
further responses :)

Thanks!


Cheers,
Just


On Fri, 22 Mar 2019 at 21:28, Étienne Mollier 
wrote:

> Good Day Just,
>
> Justin Cattle, on 2019-03-22 :
> > Has anyone ever though about or implemented some form of
> > encrypted volume within the fai images created using fai-cd ?
>
> Yes!
>
> > I'm thinking about some encryption, such that the initramfs or
> > systemd prompts for a password to unlock it.
> >
> > I'm not sure quite how to easy it is to hook into the fai-cd
> > image build process though to customise it.
> > I presume I can't really do anything like "disk_config" ?
>
> It would actually be quite possible, have a look at the
> setup-storage manual:
>
> http://fai-project.org/doc/man/setup-storage.html
>
> There is a chapter named "Crypt example" in which a few
> disk_config entries are shown setting up cryptsetup volumes.
> If you are not afraid to put your passphrases in cleartext in
> the disk_config/, you can even define it at installation time.
> Otherwise, the key is left in a temporary file during
> installation, using crypttab and stuff; I am not that well aware
> of the details for this part unfortunately.
>
> Anyway, the thing is definitely possible, and relatively simple.
>
> Kind Regards
> --
> Étienne Mollier 
>
> All opinions are my own.
>
>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

fai-cd - encryption possibilities

[Justin Cattle]

Hi,


Has anyone ever though about or implemented some form of encrypted volume
within the fai images created using fai-cd ?

I'm thinking about some encryption, such that the initramfs or systemd
prompts for a password to unlock it.

I'm not sure quite how to easy it is to hook into the fai-cd image build
process though to customise it.
I presume I can't really do anything like "disk_config" ?
Perhaps I can do hooks ?

Any ideas welcome :)



Cheers,
Just

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: install_packages seems to be excluding a lot of pakcages

[Justin Cattle]

Hi Thomas,

So, this was the issue.  It was very hard to track down, as there was
nothing in the FAI logs to indicate were the problem was, but eventually I
found it :)

Someone had uploaded a package with bad control data into one of our
repos.  That is what the parsing of the available packages.  I'm guessing
that the packages that did manage to get included still, where before the
bad control data for the package [ in the file that's parsed ].

I think I found the code that add all the available packages to the dpkg
file as well:

commit 8963c7268a68dc3ed29aa4208d3d7e800b5d2eb4
Author: Thomas Lange 
Date:   Mon Dec 12 09:19:23 2016 +0100

create /var/lib/dpkg/available during packge installation

diff --git a/bin/fai-make-nfsroot b/bin/fai-make-nfsroot
index 5321d54d..8aebd6da 100755
--- a/bin/fai-make-nfsroot
+++ b/bin/fai-make-nfsroot
@@ -474,6 +474,7 @@ add_packages_nfsroot() {
 local err

 setclasses
+$ROOTCMD apt-cache dumpavail > $NFSROOT/var/lib/dpkg/available
 install_packages -l -p$cfdir > $NFSROOT/var/tmp/packages.nfsroot
 echo "Adding additional packages to $NFSROOT:"
 cat $NFSROOT/var/tmp/packages.nfsroot
@@ -483,6 +484,7 @@ add_packages_nfsroot() {
 if [ $err -ne 0 ]; then
die 12 "install_packages had exit code: $err"
 fi
+rm $NFSROOT/var/lib/dpkg/available
 set -e
 }


I presume fai-make-nfsroot is called during FAI build at some point ?
That's what I was missing, because whenever I
check $NFSROOT/var/lib/dpkg/available it only had about 200 packages in it
anyway.

I added a hack to a class for insoft to do:

chroot /target/ apt-cache dumpavail > $NFSROOT/var/lib/dpkg/available

..and that fixed the included packages, but I then found that aptitude was
failing, due to an error on some control data in
$NFSROOT/var/lib/dpkg/available
That's how I found the package with bad control data.

I removed it from the repos, and removed my hacks, and everything was back
to normal :)

Would it be possible to somehow pass any errors through to fai.log if the
process to a that parses the available packages errors?
Or do you think it was just failing silently anyway?



Cheers,
Just


On Fri, 8 Mar 2019 at 11:25, Justin Cattle  wrote:

> I think I have found the cause, so please don't check anything Thomas.
> I'll update with my findings soon :)
> Hopefully you can help me understand the issue I've found.
>
>
> Cheers,
> Just
>
>
> On Fri, 8 Mar 2019 at 09:02, Justin Cattle  wrote:
>
>> I've also just tried the latest version of install_packages from git,
>> Same thing.
>>
>>
>> Like I said, I'm not a perl guy, but it seems like this is happening:
>>
>> The hash pname is built from packages listed in
>> $FAI_ROOT/var/lib/dpkg/available , using the function create_debian_pkg_list
>>
>> clean_pkg_list parses all the files, using insert_pkg.  If insert_pkg
>> doesn't find the file in pmap, it will return 0 and package falls
>> into @unknown.
>>
>>
>> So, I think this can be summarised as, if the package isn't declared in
>> $FAI_ROOT/var/lib/dpkg/available, it will be removed from the installation
>> list.
>>
>> If that's correct, then I'm slightly confused, because
>> /var/lib/dpkg/available isn't a full list of packages available for
>> installation via apt.
>> Is there something I'm missing in FAI that normally populates that file
>> with more of the packages from apt sources?
>>
>>
>> Cheers,
>> Just
>>
>>
>> On Fri, 8 Mar 2019 at 08:02, Justin Cattle  wrote:
>>
>>> Apologies, forgot to add that info.
>>>
>>> FAI 5.3.6
>>> nfsroot is debian squeeze, and building ubuntu trusty in this case
>>>
>>>
>>> Cheers,
>>> Just
>>>
>>>
>>> On Thu, 7 Mar 2019 at 22:37, Thomas Lange 
>>> wrote:
>>>
>>>> >>>>> On Thu, 7 Mar 2019 21:59:09 +, Justin Cattle 
>>>> said:
>>>>
>>>> > This is a FAI system that I haven't used for a while, and I'm not
>>>> 100% how the config has changed if at all.
>>>>
>>>> > When it get to task instsoft, it fails to include most of the
>>>> packages from the package_config files.  It includes just a few.
>>>> Which FAI version are you using?
>>>>
>>>> --
>>>> regards Thomas
>>>>
>>>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 

Re: install_packages seems to be excluding a lot of pakcages

[Justin Cattle]

I think I have found the cause, so please don't check anything Thomas.
I'll update with my findings soon :)
Hopefully you can help me understand the issue I've found.


Cheers,
Just


On Fri, 8 Mar 2019 at 09:02, Justin Cattle  wrote:

> I've also just tried the latest version of install_packages from git,
> Same thing.
>
>
> Like I said, I'm not a perl guy, but it seems like this is happening:
>
> The hash pname is built from packages listed in
> $FAI_ROOT/var/lib/dpkg/available , using the function create_debian_pkg_list
>
> clean_pkg_list parses all the files, using insert_pkg.  If insert_pkg
> doesn't find the file in pmap, it will return 0 and package falls
> into @unknown.
>
>
> So, I think this can be summarised as, if the package isn't declared in
> $FAI_ROOT/var/lib/dpkg/available, it will be removed from the installation
> list.
>
> If that's correct, then I'm slightly confused, because
> /var/lib/dpkg/available isn't a full list of packages available for
> installation via apt.
> Is there something I'm missing in FAI that normally populates that file
> with more of the packages from apt sources?
>
>
> Cheers,
> Just
>
>
> On Fri, 8 Mar 2019 at 08:02, Justin Cattle  wrote:
>
>> Apologies, forgot to add that info.
>>
>> FAI 5.3.6
>> nfsroot is debian squeeze, and building ubuntu trusty in this case
>>
>>
>> Cheers,
>> Just
>>
>>
>> On Thu, 7 Mar 2019 at 22:37, Thomas Lange 
>> wrote:
>>
>>> >>>>> On Thu, 7 Mar 2019 21:59:09 +, Justin Cattle 
>>> said:
>>>
>>> > This is a FAI system that I haven't used for a while, and I'm not
>>> 100% how the config has changed if at all.
>>>
>>> > When it get to task instsoft, it fails to include most of the
>>> packages from the package_config files.  It includes just a few.
>>> Which FAI version are you using?
>>>
>>> --
>>> regards Thomas
>>>
>>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: install_packages seems to be excluding a lot of pakcages

[Justin Cattle]

I've also just tried the latest version of install_packages from git,  Same
thing.


Like I said, I'm not a perl guy, but it seems like this is happening:

The hash pname is built from packages listed in
$FAI_ROOT/var/lib/dpkg/available , using the function create_debian_pkg_list

clean_pkg_list parses all the files, using insert_pkg.  If insert_pkg
doesn't find the file in pmap, it will return 0 and package falls
into @unknown.


So, I think this can be summarised as, if the package isn't declared in
$FAI_ROOT/var/lib/dpkg/available, it will be removed from the installation
list.

If that's correct, then I'm slightly confused, because
/var/lib/dpkg/available isn't a full list of packages available for
installation via apt.
Is there something I'm missing in FAI that normally populates that file
with more of the packages from apt sources?


Cheers,
Just


On Fri, 8 Mar 2019 at 08:02, Justin Cattle  wrote:

> Apologies, forgot to add that info.
>
> FAI 5.3.6
> nfsroot is debian squeeze, and building ubuntu trusty in this case
>
>
> Cheers,
> Just
>
>
> On Thu, 7 Mar 2019 at 22:37, Thomas Lange 
> wrote:
>
>> >>>>> On Thu, 7 Mar 2019 21:59:09 +, Justin Cattle 
>> said:
>>
>> > This is a FAI system that I haven't used for a while, and I'm not
>> 100% how the config has changed if at all.
>>
>> > When it get to task instsoft, it fails to include most of the
>> packages from the package_config files.  It includes just a few.
>> Which FAI version are you using?
>>
>> --
>> regards Thomas
>>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: install_packages seems to be excluding a lot of pakcages

[Justin Cattle]

Apologies, forgot to add that info.

FAI 5.3.6
nfsroot is debian squeeze, and building ubuntu trusty in this case


Cheers,
Just


On Thu, 7 Mar 2019 at 22:37, Thomas Lange 
wrote:

> >>>>> On Thu, 7 Mar 2019 21:59:09 +0000, Justin Cattle  said:
>
> > This is a FAI system that I haven't used for a while, and I'm not
> 100% how the config has changed if at all.
>
> > When it get to task instsoft, it fails to include most of the
> packages from the package_config files.  It includes just a few.
> Which FAI version are you using?
>
> --
> regards Thomas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

install_packages seems to be excluding a lot of pakcages

[Justin Cattle]

Hi,


I'm debugging an odd issue.

This is a FAI system that I haven't used for a while, and I'm not 100% how
the config has changed if at all.

When it get to task instsoft, it fails to include most of the packages from
the package_config files.  It includes just a few.

All the packages are available to install from the configured sources [
which are local mirrors, but it makes no difference ].
If I chroot into the /target, and install them with apt-get,it works fine.

Here is an excerpt from the log.

Calling task_instsoft
Installing software may take a while
Maximum number of packages installed at a time set to 800
install_packages: reading config files from directory
/var/lib/fai/config/package_config
install_packages: read config file OCADO_NUC
install_packages: read config file FAIBASE
install_packages: read config file UBUNTU
install_packages: read config file TRUSTY
install_packages: executing chroot /target apt-get clean
WARNING: These unknown packages are removed from the installation list:
xdotool cryptsetup plymouth-label plymouth-theme-ubuntu-logo acpid
augeas-tools bc debconf-utils grub2 hdparm ifenslave iptraf
linux-image-generic lshw lsof lvm2 man-db mdadm nfs-common nload nmap nscd
openssh-client openssh-server parted pciutils python-minimal python-yaml
rdate rsync ruby-dnsruby ruby-net-ssh screen strace sysstat tcpdump tcsh
time tmux unzip usbutils wget xfsprogs grub-pc grub-legacy- lilo-
puppet-common=3.8.5-1puppetlabs1 puppet=3.8.5-1puppetlabs1 libnet-ssh2-ruby
ruby-hiera-eyaml
install_packages: executing chroot /target aptitude -R -y -o
Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confnew install
plymouth cron file initramfs-tools less locales lsb-release resolvconf


..and here is the contents of the included classes from the pcakge_config


#
# stuff that get installed on all the versions
#
PACKAGES aptitude
augeas-tools
bc
cron
debconf-utils
file
grub2
hdparm
ifenslave
initramfs-tools
iptraf
less
linux-image-generic
locales
lsb-release
lshw
lsof
lvm2
man-db
mdadm
nfs-common
nload
nmap
nscd
openssh-client
openssh-server
parted
pciutils
python-minimal
python-yaml
rdate
resolvconf
rsync
ruby-dnsruby
ruby-net-ssh
screen
strace
sysstat
tcpdump
tcsh
time
tmux
unzip
usbutils
wget
xfsprogs

PACKAGES aptitude GRUB_PC
grub-pc grub-legacy- lilo-


#
# stuff that get installed on all the versions
#
PACKAGES aptitude
puppet-common=3.8.5-1puppetlabs1
puppet=3.8.5-1puppetlabs1
libnet-ssh2-ruby
ruby-hiera-eyaml

#
# stuff that get installed on all the versions
#
PACKAGES aptitude
xdotool
cryptsetup
plymouth
plymouth-label
plymouth-theme-ubuntu-logo
acpid



I have tried debuging the install_packages script. but perl isn't my forte
by a long shot! :)

Can you think of any reason most, but not all pakcages would be excluded?



Cheers,
Just

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: Tip: Run a script after FAI install

[Justin Cattle]

Systemd is pretty good at this.

Get FAI to install a unit file to run your thing, and add a unit option
like ConditionPathExists=/foo/bar or ConditionPathExists=!/foo/bar
Then make your thing touch or delete a flag file [ /foo/bar ] if
successfully run.
There you have it, a service/script that only runs once.


Cheers,
Just


On Thu, 10 Jan 2019 at 17:14, John G Heim  wrote:

> Well, it's not really to the point. Maybe my example was bad but there
> are lots of other reasons one might want to run a script after the
> install is finished. Actually, the reason this came up is that I want
> the linux cli screen reader to run on the first boot and then get
> disabled. But I just added it to the things my fai setup does after the
> first reboot.
>
>
>
>
> On 1/10/19 6:55 AM, Brian Kroth wrote:
> > You can configure the nfsroot with your ldap configs so you can have
> > them available during fai. I used this (at another UW department) for
> > rescue consoles to support natural logins from admins.
> >
> > As far as the sudo config, why not just copy a sudoers (.d) snippet down
> > that references the user during fai/config management time? It can still
> > reference an ldap user without them being available yet. They don't need
> > to be in the local sudo group to privelege them. You can also add host
> > match restrictions if you want. It's quite customizable.
> >
> > On Thu, Jan 10, 2019, 00:29 Martin Krämer  > > wrote:
> >
> > Hi John,
> >
> > if you are using LDAP - why not permitting a LDAP group (which
> > already exists during install) and then configure sudo via LDAP?
> >
> > Thats how I solved it for my soho environment.
> >
> > See: https://www.sudo.ws/man/1.8.17/sudoers.ldap.man.html
> >
> > Kind Regards
> >
> > Martin
> >
> > On Wed, Jan 9, 2019, 22:06 John G Heim  >  wrote:
> >
> > So I had this problem. I want to configure certain users to have
> > sudo on
> > the workstations I manage. Problem we do ldap authenticaition --
> > so the
> > users don't exist during the install. I can easily write an fai
> > script
> > to do an adduser but it doesn't work because the user doesn't
> exist
> > during the install. What I needed to do is to run a script once
> > after
> > the system reboots into the newly installed operating system. I
> > thought
> > about putting a script on there that would run at boot time and
> > delete
> > itself. But that's ugly and failure prone. But I came up with a
> > solution
> > that is much more reliable and flexible.
> >
> > 1. Create a crontab file to be copied to the target system
> > during the
> > install. For example, during my fai installs, I create a class
> > called
> > INSTALL. So I created a crontab file
> > /srv/fai/config/files/etc/crontab/INSTALL.
> >
> > Put a command like this in this file:
> >
> > @reboot root fai --class/dev/null=POSTINST softupdate
> >
> > 2. Add an fcopy command to one of your installation scripts to
> > copy the
> > crontab file:
> >
> > fcopy -Bi /etc/crontab
> >
> > 3. Create another, normal crontab file without the above line
> > and call
> > it POSTINST or whatever you called the class in the first
> > crontab. In
> > this example, it would be
> > /srv/fai/config/files/etc/crontab/POSTINST.
> >
> > 4. in your fai script space, create a directory called POSTINST
> >
> > mkdir /srv/fai/config/scripts/POSTINST
> >
> > 5. Put a script in there to install the normal crontab file
> >
> > fcopy -Bi /etc/crontab
> >
> > 6. Put scripts to do whatever else you want into that same
> > directory.
> > These scripts will be run just once when the system reboots
> > after the
> > original fai install. The target machine will look completely
> > normal and
> > there won't be any extra programs/scripts on it (unless you
> > count fai
> > itself).
> >
> > Verstehst du?
> >
> > --
> > --
> > John G. Heim; jh...@math.wisc.edu ;
> > sip://jh...@sip.linphone.org 
> >
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of 

Re: Ubuntu 18.04: netplan support

[Justin Cattle]

Personally - I like yaml and use it a lot for many things :)

I've never liked xml on the other hand!


Cheers,
Just


On Wed, 17 Oct 2018 at 16:07, John G Heim  wrote:

> It' may be worse than you know. Netplan uses something called yaml for
> it's config files. YAML is a newish config language, sort of a
> replacement and/or competitor for XML.  I am no fan of XML, believe me.
> I'm blind, you sighted people think you have it hard editing XML config
> files, try doing it by listening to the code. At the same time, I am not
> looking forward to learning yet another protocol that is essentially a
> programming language for config files.
>
> I understand the drive. Every system has it's own syntax that ammounts
> to a programming language for it's config files. Knowing the syntax for
> bind9 config files doesn't help you one bit when writing a systemd
> config file. Samba uses a syntax borrowed from Windows INI files. About
> the only place I can think of where I didn't have to learn a new config
> file language is with apache and podcast feeds. Both use XML so if you
> understand the syntax for apache config files, you also understand the
> syntax for podcast feed files.
>
> So I don't know what to think about netplan & YAML. Haven't made up my
> mind whether to hate it or love it.
>
> On 10/17/18 9:19 AM, Justin Cattle wrote:
> > We have quite a lot invested in ifupdown - we won't be using netplan for
> > servers on bionic.
> >
> > I know that's not a helpful response, but if you decide not to use
> > netplan you won't be the only people :)
> >
> >
> > Cheers,
> > Just
> >
> >
> > On Wed, 17 Oct 2018 at 15:11, John G Heim  > <mailto:jh...@math.wisc.edu>> wrote:
> >
> > Well, I pulled a Bart Simpson. I tried for about 10 minutes and gave
> up.
> >
> > Bakc with Ubuntu 16.04, I tried modifying the old networking script
> > using the ifclass command. If the UBUNTU class was defined, it
> > generated
> > a /etc/netplan/10-interfaces file. Otherwise, it ran the old code to
> > generate a /etc/networking/interfaces file.
> >
> > I think the reason I couldn't get it to work was because of a bug in
> > NetworkManager. It looked like NetworkManager would neither make that
> > interface work nor release it so it could be configured by netplan.
> > This
> > was a long time ago and my memory is a little fuzzy but I recall
> typing
> > in network manager commands to try to get it to stop managing the
> > interface, it saying it wasn't managing the interface, while at the
> > same
> > time, a listing of the interfaces it managed showed the interface.
> Now
> > that I think about it, I believe I worked on it for way more than 10
> > minutes because I recall doing a file-by-file comparison of the
> configs
> > from a regular Ubuntu install and an FAI install and finding no
> > difference. The files in /etc were identical yet it worked in a
> normal
> > install of Ubuntu but not in the FAI install.
> >
> > I finally just added ifupdown to the packages installed during an
> > Ubuntu
> > install. So I've been subverting Ubuntu's normal network config
> process
> > for years now. I just don't do it the way Ubuntu would normally do
> > it. I
> > figure when/if Debian switched to netplan, the FAI developers will
> > modify FAI to account for it.
> >
> > Now that you've brought it up though, I might give it another try
> > when I
> > upgrade all of my workstations to Ubuntu 18.10 during semester break
> in
> > January. If I get it to work, I can post a howto here.
> >
> >
> >
> >
> > On 10/17/18 8:05 AM, Robert Markula wrote:
> >  > Hi,
> >  >
> >  > I'm currently in the process of updating tried-and-trusted FAI
> > 4.2.5 to
> >  > 5.7.2 and completely reworking the config space in the process,
> > starting
> >  > with the examples provided by fai-doc (which, btw, has been quite
> a
> >  > surprisingly pleasant experience so far, as fewer customization is
> >  > necessary in order to support different distributions as it was
> > the case
> >  > with the ancient 4.2.5 version. Nice!)
> >  >
> >  > Two questions arose so far:
> >  >
> >  > 1. Is the 'UBUNTU' class intended to be complementing the
> > 'DEBIAN' class
> >  > or does it completely replace the DEBIAN class?
> &g

Re: Ubuntu 18.04: netplan support

[Justin Cattle]

We have quite a lot invested in ifupdown - we won't be using netplan for
servers on bionic.

I know that's not a helpful response, but if you decide not to use netplan
you won't be the only people :)


Cheers,
Just


On Wed, 17 Oct 2018 at 15:11, John G Heim  wrote:

> Well, I pulled a Bart Simpson. I tried for about 10 minutes and gave up.
>
> Bakc with Ubuntu 16.04, I tried modifying the old networking script
> using the ifclass command. If the UBUNTU class was defined, it generated
> a /etc/netplan/10-interfaces file. Otherwise, it ran the old code to
> generate a /etc/networking/interfaces file.
>
> I think the reason I couldn't get it to work was because of a bug in
> NetworkManager. It looked like NetworkManager would neither make that
> interface work nor release it so it could be configured by netplan. This
> was a long time ago and my memory is a little fuzzy but I recall typing
> in network manager commands to try to get it to stop managing the
> interface, it saying it wasn't managing the interface, while at the same
> time, a listing of the interfaces it managed showed the interface. Now
> that I think about it, I believe I worked on it for way more than 10
> minutes because I recall doing a file-by-file comparison of the configs
> from a regular Ubuntu install and an FAI install and finding no
> difference. The files in /etc were identical yet it worked in a normal
> install of Ubuntu but not in the FAI install.
>
> I finally just added ifupdown to the packages installed during an Ubuntu
> install. So I've been subverting Ubuntu's normal network config process
> for years now. I just don't do it the way Ubuntu would normally do it. I
> figure when/if Debian switched to netplan, the FAI developers will
> modify FAI to account for it.
>
> Now that you've brought it up though, I might give it another try when I
> upgrade all of my workstations to Ubuntu 18.10 during semester break in
> January. If I get it to work, I can post a howto here.
>
>
>
>
> On 10/17/18 8:05 AM, Robert Markula wrote:
> > Hi,
> >
> > I'm currently in the process of updating tried-and-trusted FAI 4.2.5 to
> > 5.7.2 and completely reworking the config space in the process, starting
> > with the examples provided by fai-doc (which, btw, has been quite a
> > surprisingly pleasant experience so far, as fewer customization is
> > necessary in order to support different distributions as it was the case
> > with the ancient 4.2.5 version. Nice!)
> >
> > Two questions arose so far:
> >
> > 1. Is the 'UBUNTU' class intended to be complementing the 'DEBIAN' class
> > or does it completely replace the DEBIAN class?
> >
> > 2. Ubuntu 18.04 now uses a different network configuration utility
> > called 'netplan' [1]. However, I don't see support for that in the
> > example configspace. So while a Ubuntu Bionic host can be successfully
> > installed using the 'FAIBASE UBUNTU DEMO' classes, it has no network
> > connectivity as the netplan configuration is missing from the examples.
> > Has anybody successfully integrated netplan support yet?
> >
> >
> > Robert
> >
> > [1] https://netplan.io
> >
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: Alternatives in package_lists?

[Justin Cattle]

This is pretty niche, and requires some logic not directly supported in apt.
You could just create a shell script with the logic in FAI, instead of the
package_config.




Cheers,
Just

On 27 April 2018 at 08:18, Steffen Grunewald 
wrote:

> On Thu, 2018-04-26 at 21:20:44 +0200, Thomas Lange wrote:
> > > On Thu, 26 Apr 2018 17:34:23 +0200, Steffen Grunewald <
> steffen.grunew...@aei.mpg.de> said:
> >
> > > Is it possible to specify "a | b" in a package_config file, so
> that "a"
> > > gets installed if available, and "b" otherwise?
> > FAI only creates a long list of package names and then calls apt-get,
> > aptitude or apt. If one of those tools provides a function like install
> > a if available or b otherwise, then FAI cloud use it. FAI itself does
> > not have this function yet. Patches are welcome ;-)
>
> apt-get install "a | b" doesn't work.
>
> > I wonder if any of the config management tools provides such a function?
>
> Not directly, I'm afraid - but what seems to be feasible is to create a
> metapackage that has "a | b" as its only install dependency, and add that
> to the package list.
> Of course, the primary goal is to fix the underlying problem - which may
> take longer than the time I have to come up with a working setup though.
>
> Thanks,
>  S
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us 
immediately and delete all copies of this message. Please note that it is 
your responsibility to scan this message for viruses. 

 

Fetch and Sizzle 
are trading names of Speciality Stores Limited and Fabled is a trading name 
of Marie Claire Beauty Limited, both members of the Ocado Group.

 


References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time.  
The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: setup-storage fails on blank disk

[Justin Cattle]

Hi,


Are you 100% sure you tried it on a "fresh" disk that was really clean?
It defiantly feels like there is some metadata or something remaining in
some blocks on the disk.

When FAI fails, are you able check for things like md info, dm info, lvm
info and the like?

You may have to do some dmsetup remove, vgremove or pvremove.
wiptefs is also a good utility at clearing metadata, run it against any
partitions before removing them.
Then remove partitions with dd, put some zeros on the first few Mg of the
disk.




Cheers,
Just

On 4 January 2018 at 13:47, Andreas Heinlein  wrote:

> Am 03.01.2018 um 17:28 schrieb Holger Parplies:
> > Hi,
> >
> > Andreas Heinlein wrote on 2018-01-03 13:53:40 +0100 [setup-storage fails
> on blank disk]:
> >> [...]
> >> I have encountered a problem with setup-storage which occurs only when
> >> the disk is blank, i.e. wiped with nwipe/dban or brand new. It then
> >> fails on creating the LVM; running 'pvcreate' returns 'cannot open
> >> /dev/sda5 exclusively'.
> > this is probably unrelated, but is there any reason to put the LVM PV
> inside
> > a "logical" volume? DOS extended partitions seem to be the worst hack
> ever
> > invented to get around a limitation in a bad design, yet they repeatedly
> > and apparently unnecessarily pop up in quoted disk_configs:
> >
> >> [...]
> >> This is your disk_config file:
> >> # generic disk configuration for one small disk
> >> # disk size from 500Mb up to what you can buy today
> >> #
> >> #   [extra
> options]
> >>
> >> disk_config disk1 disklabel:msdos bootable:1 preserve_lazy:6
> align-at:1M fstabkey:uuid
> >> primary  /boot  300  ext4rw  createopts="-O
> ^64bit,^metadata_csum"
> >> logical  -  29500-3  -   -
> >> logical  /media/daten  1024- ext4acl createopts="-O
> ^64bit,^metadata_csum -L Daten"
> > I count three partitions, which would work perfectly with primary
> partitions
> > (furthermore, you are using LVM to have an arbitrary number of named and
> > dynamic "volumes" (i.e. partitions) anyway, so if you needed more, LVM
> would
> > be the superior mechanism; of course, your specific requirements may
> vary).
> > Ok, you are preserving a logical partition, so in this particular case
> you'd
> > actually need to stick with logical partitions, but the partition in
> question
> > is ext4, not FAT-based, so it doesn't appear to be a legacy Windoze
> issue.
> >
> > My point: am I missing something, and there is some obscure benefit of
> putting
> > an LVM container within an extended-partition-container (such as hiding
> it
> > from something), or is it simply a common misconception that you for some
> > reason cannot or should not put an LVM PV (or even several individual
> native
> > Linux partitions - such as /, /var and /tmp) into primary partitions -
> > assuming you only need upto four of them (and, obviously, assuming you
> are
> > still using MSDOS partition tables)?
> >
> > Or, differently: for a *blank disk*, you obviously won't be preserving
> sda6,
> > and you probably aren't referencing it by partition number
> ("fstabkey:uuid"),
> > so does using 'primary' instead of 'logical' for all three partitions
> change
> > anything concerning the error you are experiencing?
> >
> > Hope that helps someone (perhaps me ;-) ...
> >
> > Regards,
> > Holger
> Hello,
>
> yes, you are right - in some way, this *is* a legacy windows issue that
> has developed over time. In fact, the preserved partition once was a FAT
> partition as long as we had dualboot installations on these machines. We
> finally removed the dualboot two or three years ago and chose to format
> this partition ext4 instead. Why we didn't move to primary partitions or
> put it within the LVM at that time - I don't know.
>
> On the other hand, up to now we had no problems with that, so no urge to
> change anything. If you think it might help, I could try changing this.
>
> Bye,
> Andreas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: FAI server on Ubuntu

[Justin Cattle]

He have xenial FAI servers, but we had to use a stretch nfsroot.

The problem is centred around dractut, which can't be used on xenial
properly , due to a hard coding of the "update-initramfs" command in the
kernel packages postinst script.

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1660587




Cheers,
Just

On 8 November 2017 at 08:27, Thomas Lange 
wrote:

> On IRC we had the question, if FAI can be installed on Ubuntu. Does
> anyone has a FAI server running on Ubuntu Xenial? I remember that
> there were major problems creating the nfsroot on a Ubuntu system,
> because of dracut and upstart in the past. But I'm not sure if these
> problems still apply or not. Any help appreciated.
>
> --
> regards Thomas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: fai 5.3 + debian stretch NFS issues

[Justin Cattle]

nfs4 is know not to work with overlayfs, and I think you may be mixing your
pxe configs.
I'm using a stretch nfs root via nfs3.

I have an exports like this:

/srv/fai/nfsroot *(async,ro,no_subtree_check,no_root_squash)
/srv/fai/config *(async,ro,no_subtree_check,no_root_squash)

..and a pxe append line like this:

append initrd=initrd.img ip=dhcp root=1.2.3.4:/srv/fai/nfsroot rootovl
FAI_FLAGS=verbose,sshd,createvt,reboot FAI_ACTION=install

It all works fine.





Cheers,
Just

On 28 September 2017 at 22:18, Mark Devaney 
wrote:

> I've searched all the relevant postings and cannot find an answer that
> works for our installation.
>
> Is anyone else having issues with debian stretch?
>
> Our previous jessie configuration works fine, but after updating the
> nfsroot to wheezy we either get "mount.nfs: Protocol not supported" in
> dracut (with NFS 4)  or overlay FS issues if we use NFS 3.
>
> FAI configs are default values other than file locations.
>
> I can mount using NFS V4 from the dracut shell but not using NFS V3 - our
> server supports both and I can mount either version from other machines so
> it's not the server side.
>
> FAI_VERSION="5.3.6~bpo8+1"
>
> FAI_DEBOOTSTRAP="stretch http://httpredir.debian.org/debian; (also set in
> apt/sources.list)
>
> dracut NFS protocol error
>
> exports:
>
> /srv/nfs4  *(fsid=0,async,ro,no_subtree_check)
> /srv/fai *(async,ro,no_subtree_check)
>
> pxelinux.cfg:
>
> append initrd=initrd.img-4.9.0-3-amd64 ip=dhcp
> root=/srv/fai/nfsroot,nfsvers=3 rootovl FAI_FLAGS=verbose,sshd,createvt,reboot
> FAI_CONFIG_SRC=nfs://srv/fai/config,nfsvers=3 FAI_ACTION=install
>
> overlay issues
>
> exports:
>
> /srv  *(fsid=0,async,ro,no_subtree_check)
>
> pxelinux.cfg:
>
> append initrd=initrd.img-4.9.0-3-amd64 ip=dhcp
> root=/fai/nfsroot,nfsvers=4 rootovl FAI_FLAGS=verbose,sshd,createvt,reboot
> FAI_CONFIG_SRC=nfs://fai/config,nfsvers=4 FAI_ACTION=install
>
>
> errors :
>
> touch: cannot touch '/var/log/dmesg.new' : Operation not supported
>
> etc.
>
> ends with FAI: installation aborted
>
> Thank you.
>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: make nfs root dies on dracut setup in stretch build

[Justin Cattle]

It works ok for us using stretch , but we do manually include curl in the
NFSROOT config.



Cheers,
Just

On 23 June 2017 at 21:37, Nat Sincheler  wrote:

> We are trying to build a stretch NFS root. We keep getting this error:
>
> #
> Setting up dracut (044+241-3) ...
> dracut: Generating /boot/initrd.img-4.9.0-3-amd64
> dracut: dracut module 'livenet' depends on 'url-lib', which can't be
> installed
> dracut: dracut module 'livenet' cannot be found or installed.
> dpkg: error processing package dracut (--configure):
>  subprocess installed post-installation script returned error exit status 1
> Errors were encountered while processing:
>  dracut
> ERROR: 256 256
> ERROR: chroot /srv/fai/stretch-amd64 dpkg --configure --pending return
> code 1
> install_packages: executing chroot /srv/fai/stretch-amd64 dpkg -C
> The following packages are only half configured, probably due to problems
> configuring them the first time.  The configuration should be retried using
> dpkg --configure  or the configure menu option in dselect:
>  dracut   Low-level tool for generating an initramfs image
> (automat
>
> install_packages: executing chroot /srv/fai/stretch-amd64 apt-get clean
> 2 errors during executing of install_packages
> ERROR: install_packages had exit code: 3
> Log file written to /var/log/fai/fai-make-nfsroot.log
> #
>
> We are using FAI version 5.3.6 on Debian jessie.
>
>
> Here are the packages that we install:
>
> PACKAGES install
>
> # Administrative and debugging utilities
> deborphan dnsutils locales numactl psmisc rsync wipe
> syslinux pxelinux
>
> cryptsetup
>
> # Base
> dialog iproute lsb-release xz-utils
>
> # Disk and file system
> dosfstools dump e2fsprogs hdparm lvm2 mdadm parted xfsprogs xfsdump
>
> # FAI (Perl sends log messages after build)
> fai-nfsroot libmime-lite-perl lftp perl
>
> # Hardware
> hwinfo lshw pciutils procinfo read-edid smartmontools usbutils
>
> # Kernel
> firmware-bnx2 firmware-bnx2x firmware-linux firmware-myricom
> firmware-netxen firmware-qlogic firmware-realtek kmod udev
>
> # BusyBox
> busybox-static
>
> # Install dracut and remove initramfs-tools:
> dracut-network dracut live-boot- initramfs-tools-
> dracut-config-generic
>
> systemd-sysv- sysvinit-core
>
> # Networking
> dnsutils isc-dhcp-client ntpdate openssh-client openssh-server rdate
>
> # NFS client
> nfs-common rpcbind
>
>
> PACKAGES aptitude I386
> grub-pc
> linux-image-686-pae
>
> PACKAGES aptitude AMD64
> grub-pc
> linux-image-amd64
>
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Thanks Afif - that's good extra info.



Cheers,
Just

On 9 June 2017 at 04:52, Afif Elghraoui  wrote:

> I had asked the same question here a few months ago here; I know you got
> it figured out already, so I'm just posting the link for reference:
>
> https://lists.uni-koeln.de/pipermail/linux-fai/2016-November/011565.html
>
> regards
>
> Afif
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Found an old thread on this as well:

https://lists.uni-koeln.de/pipermail/linux-fai/2014-April/010368.html

I really cant believe Dracut has this - I may create an issue on github,
see what they are saying..




Cheers,
Just

On 8 June 2017 at 21:46, Justin Cattle <j...@ocado.com> wrote:

> Aha - it is dracut - I've just found this!
>
> https://bugzilla.redhat.com/attachment.cgi?id=535417=diff
>
> Disappointing - hostnames with with a "." aren't even valid are they ?
>
>
>
>
> Cheers,
> Just
>
> On 8 June 2017 at 21:43, Justin Cattle <j...@ocado.com> wrote:
>
>> Thanks for the input John.
>> Glad to here it's not just me :)
>>
>>
>> I could implement some workarounds in the various scripts, that's true.
>>
>>
>> I've been doing some digging, and although I don't have the real answer
>> yet, I don't believe it's a FAI problem as such, it's something about the
>> NFSROOT OS.  In my case, debian stretch.
>>
>> This script does get the correct info from DHCP:
>>
>> /usr/share/fai/dhclient-fai-script
>>
>>
>> If you change the content to log to a file:
>>
>>
>> perl /usr/lib/fai/dhclient-perl >/tmp/foo 2>&1
>>
>>
>> The file contains the correct HOSTNAME value.
>>
>>
>> Likewise, if you change the perl script itself:
>>
>> --- dhclient-perl 2017-06-08 14:06:00.857442761 +0100
>> +++ usr/lib/fai/dhclient-perl 2017-06-08 16:28:03.015219588 +0100
>> @@ -29,7 +29,7 @@
>>  # map dhcp names to bootp names
>>  %names = qw/
>>ip_address  IPADDR
>> -  host_name   HOSTNAME
>> +  host_name   FAI_HOSTNAME
>>
>>
>> The variable FAI_HOSTNAME appears with the correct short name you would
>> expect to find in HOSTNAME.
>>
>> This makes me to think that something else is setting the HOSTNAME
>> varialbe in the environment, and that the FAI scripts don't then override
>> that.
>>
>> That part might be handled by the perl script, but my perl is
>> non-existent, and I don't really know if that's the case.
>>
>>
>> If that is all true, the question is what is setting the HOSTNAME
>> variable.  I've tried disabling /etc/init.d/hostname.sh, and
>> lib/systemd/system/systemd-hostnamed.service, but it doesn't seem to
>> help.  Maybe something else in systemd, or even dracut.  But I can't find
>> it...
>>
>> In fact, just been looking some more, and the hostname is set before
>> /usr/sbin/fai runs , /proc/sys/kernel/hostname is set , so maybe it is
>> dracut?
>>
>>
>> I'd really like to get to the bottom of this before I implement
>> workarounds.
>>
>>
>>
>>
>> Cheers,
>> Just
>>
>> On 8 June 2017 at 17:30, John G Heim <jh...@math.wisc.edu> wrote:
>>
>>> I posted about this problem approximately a year ago. I poked around in
>>> the source code a little but finally just made a workaround by adding a
>>> line to my class script:
>>>
>>> echo "${HOSTNAME}" | sed s/\.math\.wisc\.edu//
>>>
>>> So you end up with classes for both the fqdn and the hostname. The only
>>> problem with that seems to be that logs are created in /var/log/fai//
>>> instead of /var/logs/fai//.
>>>
>>> I am a little surprised you're getting this in 5.3.6. I was running
>>> 5.0.3 when I had the problem. I create a whole new FAI setup for 5.3.6 and
>>> I don't have the problem any more. Are you using 5.3.6 in the nfsroot? I'm
>>> not sure how FAI installs FAI itself inside the nfsroot but you might be
>>> running an older version there. I am also installing ubuntu zesty instead
>>> of xenial but I don't see how that could matter.
>>>
>>>
>>>
>>> On 06/08/2017 04:40 AM, Justin Cattle wrote:
>>>
>>>> Hi,
>>>>
>>>>
>>>> I've been testing migrating to fai 5 from 4.
>>>> I'm currently using fai-5.3.6, on Ubuntu xenial, but with a Debian
>>>> stretch nfsroot [ due to lack of dracut support in ubuntu ].
>>>>
>>>> When I build a host, the boot all works ok and the build process starts
>>>> fine.
>>>> However, the hostname of the instance is somehow getting set to the
>>>> FQDN.
>>>>
>>>> It sounds like a small problem, but has a knock on effect on some of
>>>> our custom class assignment scripts.
>>>>
>>>> I know that dhcp is sending the correct host-name.

Re: HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Aha - it is dracut - I've just found this!

https://bugzilla.redhat.com/attachment.cgi?id=535417=diff

Disappointing - hostnames with with a "." aren't even valid are they ?




Cheers,
Just

On 8 June 2017 at 21:43, Justin Cattle <j...@ocado.com> wrote:

> Thanks for the input John.
> Glad to here it's not just me :)
>
>
> I could implement some workarounds in the various scripts, that's true.
>
>
> I've been doing some digging, and although I don't have the real answer
> yet, I don't believe it's a FAI problem as such, it's something about the
> NFSROOT OS.  In my case, debian stretch.
>
> This script does get the correct info from DHCP:
>
> /usr/share/fai/dhclient-fai-script
>
>
> If you change the content to log to a file:
>
>
> perl /usr/lib/fai/dhclient-perl >/tmp/foo 2>&1
>
>
> The file contains the correct HOSTNAME value.
>
>
> Likewise, if you change the perl script itself:
>
> --- dhclient-perl 2017-06-08 14:06:00.857442761 +0100
> +++ usr/lib/fai/dhclient-perl 2017-06-08 16:28:03.015219588 +0100
> @@ -29,7 +29,7 @@
>  # map dhcp names to bootp names
>  %names = qw/
>ip_address  IPADDR
> -  host_name   HOSTNAME
> +  host_name   FAI_HOSTNAME
>
>
> The variable FAI_HOSTNAME appears with the correct short name you would
> expect to find in HOSTNAME.
>
> This makes me to think that something else is setting the HOSTNAME
> varialbe in the environment, and that the FAI scripts don't then override
> that.
>
> That part might be handled by the perl script, but my perl is
> non-existent, and I don't really know if that's the case.
>
>
> If that is all true, the question is what is setting the HOSTNAME
> variable.  I've tried disabling /etc/init.d/hostname.sh, and
> lib/systemd/system/systemd-hostnamed.service, but it doesn't seem to
> help.  Maybe something else in systemd, or even dracut.  But I can't find
> it...
>
> In fact, just been looking some more, and the hostname is set before
> /usr/sbin/fai runs , /proc/sys/kernel/hostname is set , so maybe it is
> dracut?
>
>
> I'd really like to get to the bottom of this before I implement
> workarounds.
>
>
>
>
> Cheers,
> Just
>
> On 8 June 2017 at 17:30, John G Heim <jh...@math.wisc.edu> wrote:
>
>> I posted about this problem approximately a year ago. I poked around in
>> the source code a little but finally just made a workaround by adding a
>> line to my class script:
>>
>> echo "${HOSTNAME}" | sed s/\.math\.wisc\.edu//
>>
>> So you end up with classes for both the fqdn and the hostname. The only
>> problem with that seems to be that logs are created in /var/log/fai//
>> instead of /var/logs/fai//.
>>
>> I am a little surprised you're getting this in 5.3.6. I was running 5.0.3
>> when I had the problem. I create a whole new FAI setup for 5.3.6 and I
>> don't have the problem any more. Are you using 5.3.6 in the nfsroot? I'm
>> not sure how FAI installs FAI itself inside the nfsroot but you might be
>> running an older version there. I am also installing ubuntu zesty instead
>> of xenial but I don't see how that could matter.
>>
>>
>>
>> On 06/08/2017 04:40 AM, Justin Cattle wrote:
>>
>>> Hi,
>>>
>>>
>>> I've been testing migrating to fai 5 from 4.
>>> I'm currently using fai-5.3.6, on Ubuntu xenial, but with a Debian
>>> stretch nfsroot [ due to lack of dracut support in ubuntu ].
>>>
>>> When I build a host, the boot all works ok and the build process starts
>>> fine.
>>> However, the hostname of the instance is somehow getting set to the FQDN.
>>>
>>> It sounds like a small problem, but has a knock on effect on some of our
>>> custom class assignment scripts.
>>>
>>> I know that dhcp is sending the correct host-name.
>>>
>>>
>>> It looks like FAI set's the HOSTNAME variable from this script passed
>>> from dhclient:
>>>
>>> root  1531  0.0  0.0  21484  1076 ?Ss   10:21   0:00
>>> dhclient -lf /dev/null -cf /usr/share/fai/dhclient-fai.conf -sf
>>> /usr/share/fai/dhclient-fai-script eth0
>>>
>>>
>>> Am I right in thinking that?
>>>
>>> Any ideas on how I can fix or debug this so HOSTNAME is set to the
>>> host-name requested from DHCP?
>>>
>>> It's not even obvious to me where the dhcp config for fai that calls
>>> that script is set.
>>>
>>>
>>> Any help or info is greatly appreciated.
>>> Than

Re: HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Thanks for the input John.
Glad to here it's not just me :)


I could implement some workarounds in the various scripts, that's true.


I've been doing some digging, and although I don't have the real answer
yet, I don't believe it's a FAI problem as such, it's something about the
NFSROOT OS.  In my case, debian stretch.

This script does get the correct info from DHCP:

/usr/share/fai/dhclient-fai-script


If you change the content to log to a file:


perl /usr/lib/fai/dhclient-perl >/tmp/foo 2>&1


The file contains the correct HOSTNAME value.


Likewise, if you change the perl script itself:

--- dhclient-perl 2017-06-08 14:06:00.857442761 +0100
+++ usr/lib/fai/dhclient-perl 2017-06-08 16:28:03.015219588 +0100
@@ -29,7 +29,7 @@
 # map dhcp names to bootp names
 %names = qw/
   ip_address  IPADDR
-  host_name   HOSTNAME
+  host_name   FAI_HOSTNAME


The variable FAI_HOSTNAME appears with the correct short name you would
expect to find in HOSTNAME.

This makes me to think that something else is setting the HOSTNAME varialbe
in the environment, and that the FAI scripts don't then override that.

That part might be handled by the perl script, but my perl is non-existent,
and I don't really know if that's the case.


If that is all true, the question is what is setting the HOSTNAME
variable.  I've tried disabling /etc/init.d/hostname.sh, and
lib/systemd/system/systemd-hostnamed.service, but it doesn't seem to help.
Maybe something else in systemd, or even dracut.  But I can't find it...

In fact, just been looking some more, and the hostname is set before
/usr/sbin/fai runs , /proc/sys/kernel/hostname is set , so maybe it is
dracut?


I'd really like to get to the bottom of this before I implement workarounds.




Cheers,
Just

On 8 June 2017 at 17:30, John G Heim <jh...@math.wisc.edu> wrote:

> I posted about this problem approximately a year ago. I poked around in
> the source code a little but finally just made a workaround by adding a
> line to my class script:
>
> echo "${HOSTNAME}" | sed s/\.math\.wisc\.edu//
>
> So you end up with classes for both the fqdn and the hostname. The only
> problem with that seems to be that logs are created in /var/log/fai//
> instead of /var/logs/fai//.
>
> I am a little surprised you're getting this in 5.3.6. I was running 5.0.3
> when I had the problem. I create a whole new FAI setup for 5.3.6 and I
> don't have the problem any more. Are you using 5.3.6 in the nfsroot? I'm
> not sure how FAI installs FAI itself inside the nfsroot but you might be
> running an older version there. I am also installing ubuntu zesty instead
> of xenial but I don't see how that could matter.
>
>
>
> On 06/08/2017 04:40 AM, Justin Cattle wrote:
>
>> Hi,
>>
>>
>> I've been testing migrating to fai 5 from 4.
>> I'm currently using fai-5.3.6, on Ubuntu xenial, but with a Debian
>> stretch nfsroot [ due to lack of dracut support in ubuntu ].
>>
>> When I build a host, the boot all works ok and the build process starts
>> fine.
>> However, the hostname of the instance is somehow getting set to the FQDN.
>>
>> It sounds like a small problem, but has a knock on effect on some of our
>> custom class assignment scripts.
>>
>> I know that dhcp is sending the correct host-name.
>>
>>
>> It looks like FAI set's the HOSTNAME variable from this script passed
>> from dhclient:
>>
>> root  1531  0.0  0.0  21484  1076 ?Ss   10:21   0:00
>> dhclient -lf /dev/null -cf /usr/share/fai/dhclient-fai.conf -sf
>> /usr/share/fai/dhclient-fai-script eth0
>>
>>
>> Am I right in thinking that?
>>
>> Any ideas on how I can fix or debug this so HOSTNAME is set to the
>> host-name requested from DHCP?
>>
>> It's not even obvious to me where the dhcp config for fai that calls that
>> script is set.
>>
>>
>> Any help or info is greatly appreciated.
>> Thanks!
>>
>>
>>
>> Cheers,
>> Just
>>
>> Notice:  This email is confidential and may contain copyright material of
>> members of the Ocado Group. Opinions and views expressed in this message
>> may not necessarily reflect the opinions and views of the members of the
>> Ocado Group.
>>
>> If you are not the intended recipient, please notify us immediately and
>> delete all copies of this message. Please note that it is your
>> responsibility to scan this message for viruses.
>>
>> Fetch and Sizzle are trading names of Speciality Stores Limited and
>> Fabled is a trading name of Marie Claire Beauty Limited, both members of
>> the Ocado Group.
>>
>> References to the “Ocado Group” are to Ocado Gro

Re: HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Actually, I just found that HOSTNAME is not in /tmp/fai/boot.log

That is strange, as I can see it's configured on the dhcp server, and the
dhcp server hasn't changed.
The only change is a FAI 5 nfsroot.




Cheers,
Just

On 8 June 2017 at 10:40, Justin Cattle <j...@ocado.com> wrote:

> Hi,
>
>
> I've been testing migrating to fai 5 from 4.
> I'm currently using fai-5.3.6, on Ubuntu xenial, but with a Debian stretch
> nfsroot [ due to lack of dracut support in ubuntu ].
>
> When I build a host, the boot all works ok and the build process starts
> fine.
> However, the hostname of the instance is somehow getting set to the FQDN.
>
> It sounds like a small problem, but has a knock on effect on some of our
> custom class assignment scripts.
>
> I know that dhcp is sending the correct host-name.
>
>
> It looks like FAI set's the HOSTNAME variable from this script passed from
> dhclient:
>
> root  1531  0.0  0.0  21484  1076 ?Ss   10:21   0:00 dhclient
> -lf /dev/null -cf /usr/share/fai/dhclient-fai.conf -sf
> /usr/share/fai/dhclient-fai-script eth0
>
>
> Am I right in thinking that?
>
> Any ideas on how I can fix or debug this so HOSTNAME is set to the
> host-name requested from DHCP?
>
> It's not even obvious to me where the dhcp config for fai that calls that
> script is set.
>
>
> Any help or info is greatly appreciated.
> Thanks!
>
>
>
> Cheers,
> Just
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

HOSTNAME in nfsroot boot is FQDN

[Justin Cattle]

Hi,


I've been testing migrating to fai 5 from 4.
I'm currently using fai-5.3.6, on Ubuntu xenial, but with a Debian stretch
nfsroot [ due to lack of dracut support in ubuntu ].

When I build a host, the boot all works ok and the build process starts
fine.
However, the hostname of the instance is somehow getting set to the FQDN.

It sounds like a small problem, but has a knock on effect on some of our
custom class assignment scripts.

I know that dhcp is sending the correct host-name.


It looks like FAI set's the HOSTNAME variable from this script passed from
dhclient:

root  1531  0.0  0.0  21484  1076 ?Ss   10:21   0:00 dhclient
-lf /dev/null -cf /usr/share/fai/dhclient-fai.conf -sf
/usr/share/fai/dhclient-fai-script eth0


Am I right in thinking that?

Any ideas on how I can fix or debug this so HOSTNAME is set to the
host-name requested from DHCP?

It's not even obvious to me where the dhcp config for fai that calls that
script is set.


Any help or info is greatly appreciated.
Thanks!



Cheers,
Just

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.