Compatibility when installing APT keys

[Derek Poon]

Thomas,

I'd like to revisit the issue of APT keys that I had mentioned earlier.

FAI 5.5.3 changed the task_repository() function in /usr/lib/fai/subroutines 
<https://github.com/faiproject/fai/commit/140c23b45f351291a76b59bdd3f588dadd9e4711#diff-66f2709eb62796fdd9d538c83ad12389>,
 such that it copies *.asc files into the target OS's /etc/apt/trusted.gpg.d 
instead of calling `apt-key add`.

This broke APT keys for target OSes that have apt < 1.4~beta1, which only look 
for /etc/apt/trusted.gpg.d/*.gpg files.  Since 1.4~beta1 
<https://salsa.debian.org/apt-team/apt/commit/2906182db398419a9c59a928b7ae73cf7c7aa307#d14080a187d34907415113d1099a756db02ae709_260_267>,
 APT will also look for /etc/apt/trusted.gpg.d/*.asc.

FAI 5.6.1 introduces a workaround 
<https://github.com/faiproject/fai/commit/13234e697b833f7d1057e3e2138f067abd69#diff-66f2709eb62796fdd9d538c83ad12389>:
 for each relevant class, it copies a $class.gpg file if it exists, and falls 
back to copying a $class.asc file.  However, the FAI 5.6.1 behaviour still 
imposes an unreasonable burden of maintaining redundant files in the 
package_config directory: .asc files for FAI clients older than 5.5.3, and .gpg 
files for FAI clients since 5.6.1.

To summarize, FAI has traditionally supported ASCII-armored .asc key files, and 
APT has traditionally supported dearmored .gpg key files.  I see two possible 
good solutions:

(a) Revert to the pre-5.5.3 behaviour, and call `apt-key add`. A disadvantage 
is that it requires a working apt-key command in the target OS, which may 
require the gnupg package to be installed (by debootstrap).  Unfortunately, it 
is neither easy nor desirable for FAI to demand that gnupg be installed in the 
target OS.

(b) Run gpg in FAI to convert .asc files to .gpg files.  This requires the 
gnupg package to be installed in the NFS root (or wherever FAI is running 
from).  We can ensure that the gpg command is available by declaring that the 
fai-client package depends on (or recommends) gnupg, and by adding the gnupg 
package to the sample NFSROOT configuration.

The following excerpt from /usr/lib/fai/subroutines implements option (b).

# add apt keys for all classes
for keyfile in ${classes:-}; do
[ ! -f $FAI/package_config/$keyfile.asc ] && continue
echo -n "Copying APT key from $keyfile.asc to $keyfile.asc.gpg "
[ -f $FAI_ROOT/etc/apt/trusted.gpg.d/$keyfile.asc.gpg ] || touch 
$FAI_ROOT/etc/apt/trusted.gpg.d/$keyfile.asc.gpg
gpg --batch -o $FAI_ROOT/etc/apt/trusted.gpg.d/$keyfile.asc.gpg 
--dearmor $FAI/package_config/$keyfile.asc
done

I hope that you will incorporate it, since this solution offers maximum 
compatibility without introducing .gpg files into the package_config directory 
and without requiring that gnupg be installed into the target OS.  (It has the 
disadvantage of making assumptions about how apt-key interacts internally with 
its key files, but you already started down that path by removing the apt-key 
call in 5.5.3.)

Derek
 

> On May 18, 2018, at 1:13 PM, Derek Poon <derekp+...@ece.ubc.ca> wrote:
> 
> In my experience, I have found two potential issues when installing Ubuntu 
> 18.04 using FAI.
> 
> The first is that if using FAI <= 5.5, then FAI_DEBOOTSTRAP_OPTS needs 
> '--include gnupg'.  Otherwise, the `apt-key add` in /usr/lib/fai/subroutines 
> would fail.  (At our site, we always run deboostrap instead of relying on 
> basefiles, and we often need backwards compatibility for dirinstalls.)
> 
> The second is that netplan (see https://netplan.io/examples) is the new 
> preferred configuration format for networking [...]
> 
> 
> 
>> On May 18, 2018, at 11:04, Thomas Lange <la...@informatik.uni-koeln.de> 
>> wrote:
>> 
>> After building the basefile for Ubuntu 18.04 LTS aka Bionic, my first
>> tests look fine. Setting the release name in class/UBUNTU.var to
>> ubuntudist=bionic
>> and copying the basefile from
>> https://fai-project.org/download/basefiles/BIONIC64.tar.xz
>> works. Oh, I didn't test yet if the network is running after the
>> installation. I will do this later.
> 

Re: FAI 5.6 and Ubuntu 18.04 LTS bionic

[Derek Poon]

In my experience, I have found two potential issues when installing Ubuntu 
18.04 using FAI.

The first is that if using FAI <= 5.5, then FAI_DEBOOTSTRAP_OPTS needs 
'--include gnupg'.  Otherwise, the `apt-key add` in /usr/lib/fai/subroutines 
would fail.  (At our site, we always run deboostrap instead of relying on 
basefiles, and we often need backwards compatibility for dirinstalls.)

The second is that netplan (see https://netplan.io/examples) is the new 
preferred configuration format for networking; /etc/network/interfaces works 
only if you install the ifupdown package.  I've included excerpts from our 
local scripts/DEBIAN/30-interface file below.  It generates either 
/etc/netplan/*.yaml or /etc/network/interfaces.d/* or /etc/network/interfaces 
configuration files as appropriate.


###

netplan_yaml() {
local IFNAME="$1"
local METHOD="$2"
echo "Generating netplan configuration for $IFNAME ($METHOD)" >&2
echo "# generated by FAI"
echo "network:"
echo "version: 2"
ifclass SERVER ||
echo "renderer: NetworkManager"
echo "ethernets:"
echo "$IFNAME:"
case "$METHOD" in
  dhcp)
echo "dhcp4: true"
;;
  static)
echo "addresses: [$CIDR]"
echo "gateway4: $GATEWAYS_1"
echo "nameservers:"
echo "search: [$DOMAIN]"
echo "addresses: [${DNSSRVS// /, }]"
;;
esac
}

iface_stanza() {
local IFNAME="$1"
local METHOD="$2"
echo "Generating interface configuration for $IFNAME ($METHOD)" >&2
echo "# generated by FAI"
echo "auto $IFNAME"
echo "iface $IFNAME inet $METHOD"
case "$METHOD" in
  static)
echo "address $IPADDR"
echo "netmask $NETMASK"
echo "broadcast $BROADCAST"
echo "gateway $GATEWAYS"
;;
esac
}

…

case "$FAI_ACTION" in
  install|dirinstall)
ifclass DHCPC && METHOD=dhcp || METHOD=static

if [ -d $target/etc/netplan ]; then
# Ubuntu >= 17.10 with netplan.io
if [ -n "$NIC1" ]; then
netplan_yaml $NIC1 $METHOD > $target/etc/netplan/01-${NIC1}.yaml
fi
elif [ -d $target/etc/network/interfaces.d ]; then
# ifupdown >= 0.7.41 (Debian >= 8, Ubuntu >= 14.04)
iface_stanza lo loopback > $target/etc/network/interfaces.d/lo
if [ -n "$NIC1" ]; then
iface_stanza $NIC1 $METHOD > \
$target/etc/network/interfaces.d/$NIC1
fi
else
(
iface_stanza lo loopback
iface_stanza $NIC1 $METHOD
) > $target/etc/network/interfaces
fi

if ! ifclass DHCPC ; then
[ -n "$NETWORK" ] && echo "localnet $NETWORK" > $target/etc/networks
if [ ! -L $target/etc/resolv.conf -a -e /etc/resolv.conf ]; then
cp -p /etc/resolv.conf $target/etc
fi
fi
;;
esac


###



> On May 18, 2018, at 11:04, Thomas Lange  wrote:
> 
> After building the basefile for Ubuntu 18.04 LTS aka Bionic, my first
> tests look fine. Setting the release name in class/UBUNTU.var to
> ubuntudist=bionic
> and copying the basefile from
> https://fai-project.org/download/basefiles/BIONIC64.tar.xz
> works. Oh, I didn't test yet if the network is running after the
> installation. I will do this later.

Re: setup-storage fails on blank disk

[Derek Poon]

Me too.  I have been using the following script, placed in 
hooks/partition.LINUX, as a workaround.


#!/bin/bash

##
# If setup-storage encounters a disk to be partitioned, but the disk
# contains no partition table at all, it will give up.  To prevent
# such failures, this hook creates an empty partition table of the
# type specified in the setup-storage configuration file on any disk
# whose first 512 bytes are all null.
##

. /usr/lib/fai/subroutines

# Prints the absolute path of the disk configuration file, chosen based on
# the last applicable class to have a configuration file in
# /var/lib/fai/config/disk_config.
disk_config_file() {
echo $classes | tr ' ' '\n' | tac | while read class ; do
if [ -f "$FAI/disk_config/$class" ]; then
echo "$FAI/disk_config/$class"
break
fi
done
}

# Output contains one line per disk, with the device followed by the partition
# table type that it is supposed to have:
#
# /dev/sda gpt
# /dev/sda gpt-bios
# /dev/sdc msdos
want_partition_tables() {
config_file=`disk_config_file`
if [ -z "$config_file" ]; then
# No suitable setup-storage config file
exit 1
fi
awk -v disks="`/usr/lib/fai/fai-disk-info | sort`" '
BEGIN {
split(disks, dev);
}

$1 == "disk_config" && $2 ~ "disk.*" {
sub("^disk", "", $2);
for (i = 3; i <= NF; i++) {
if (split($i, optionpair, ":")) {
if (optionpair[1] == "sameas") {
sub("^disk", "", optionpair[2]);
partition_table[$2] = partition_table[optionpair[2]];
if (dev[$2]) print "/dev/" dev[$2], partition_table[$2];
next;
} else if (optionpair[1] == "disklabel") {
partition_table[$2] = optionpair[2];
if (dev[$2]) print "/dev/" dev[$2], partition_table[$2];
next;
}
}
}

# If no disklabel option is given, setup-storage uses "msdos" as
# the default.
if (dev[$2]) print "/dev/" dev[$2], "msdos";
}
' "$config_file"
}

# Returns 0 if the first 512 bytes of the specified device are all null.
has_partition_table() {
local dev="$1"
perl -e "
open D, '<', '$dev' and
read D, \$buf, 512 and
\$buf ne \"\\000\" x 512 or
exit 1
"
}

# For every disk in the setup-storage configuration, if there is no partition
# table apparent in the first 512 bytes, then we will create one of the
# appropriate type.
ensure_partition_tables_exist() {
set -o pipefail
want_partition_tables |
while read dev table ; do
if ! has_partition_table "$dev" ; then
[ "$table" = "gpt-bios" ] && table=gpt
echo "$dev has no partition table; making one of type $table"
parted --script "$dev" mktable "$table"
fi
done
}

ensure_partition_tables_exist



> On Jan 4, 2018, at 7:12 AM, Brian Kroth  wrote:
> 
> Ages ago I recall having a similar problem to that as well.  I think it was a 
> bug with the preserver_reinstall flag not working correctly with LVM setups 
> on fresh disks.  Pretty sure I ended up writing a hack that executed early on 
> in the classes phase to figure out if the disk was lacking a label (partition 
> table) and then creating one and setting the reinstall flag to force the disk 
> layout down.  Unfortunately I don't easily have access to that code anymore, 
> else I'd share it.  Also, it may no longer be applicable.  setup-storage has 
> changed a lot since then.
> 
> Cheers,
> Brian

Re: FAI server on Ubuntu

[Derek Poon]

Thomas,

It seems like http://ppa.launchpad.net/fai/ubuntu isn't being updated anymore?  
The latest version available for 16.04 is still 5.3.2-0~6669~ubuntu16.04.1.

Derek


> On Nov 8, 2017, at 12:27 AM, Thomas Lange  
> wrote:
> 
> On IRC we had the question, if FAI can be installed on Ubuntu. Does
> anyone has a FAI server running on Ubuntu Xenial? I remember that
> there were major problems creating the nfsroot on a Ubuntu system,
> because of dracut and upstart in the past. But I'm not sure if these
> problems still apply or not. Any help appreciated.
> 
> -- 
> regards Thomas
> 

Re: FAI going to the cloud

[Derek Poon]

On Sep 27, 2016, at 8:24 PM, Thomas Lange  wrote:
> 
> thanks a lot for all your ideas. fai-diskimage was written as a
> wrapper script for several reasons. First I tried to implement it as a
> new action like 'fai install' but I had some problems with that. The
> wrapper script approach worked much better during the
> development.

As long as you've thought it through, that's fine.


> Which flexibility is missing in the wrapper? All you can do in FAI,
> can also be done when calling fai-diskimage.

At the moment, one example I can think of is creating more than one disk image 
at once, such as an OS image and a data image.


> Do you think .raw is the right extension or is .img better? Which other
> formats should be supported?

Other common formats include VDI, VMDK, and VHD.[1]  Personally, I would make 
it convert to any recognized extension; any other extension on the 
user-supplied image name (whether .raw, .img, or .whatever) would case the 
result not to be converted.

I can also imagine that some users might want the installation process to 
produce the result packaged in an OVA, in which case the output would be an 
entire virtual machine rather than just a disk image.  In my opinion, a hook 
for such local requirements would be preferable to a wrapper around a wrapper 
around fai.

[1]: https://spin.atomicobject.com/2013/06/03/ovf-virtual-machine/

Re: FAI going to the cloud

[Derek Poon]

On Sep 22, 2016, at 5:48 PM, Thomas Lange <la...@informatik.uni-koeln.de> wrote:
> 
>>>>>> On Thu, 22 Sep 2016 17:31:54 -0700, Derek Poon <derekp+...@ece.ubc.ca> 
>>>>>> said:
> 
>> Instead of fai-diskimage as a separate command, have you considered making 
>> it a mode of the fai command itself, analogous to `fai dirinstall`?  It 
>> would be nice to be able to write hooks for diskimage installs.
> fai-diskimage is a seperate script which calls "fai install"
> internally. You can write hooks for fai-diskinstall the same way as in
> fai itself, because it uses the same config space as fai.


I was actually trying to make two observations.

The first issue is user experience consistency.  We have:

   fai install
   fai sysinfo
   fai dirinstall /path/to/target/directory
   fai-diskimage -S 32G /path/to/image

The last one seems conceptually similar to dirinstall.  Why should it invoked 
differently, as "fai-diskimage" rather than "fai diskimage"?  Furthermore, the 
fai-diskimage command is hard-coded to do the "install" action, so these 
choices are mutually exclusive.

To me, "fai-diskimage" is closer in nature to "fai install" than to "fai-cd" or 
"fai-kvm", and should therefore behave similarly.


The second issue is flexibility.  The fai-diskimage script doesn't really do 
that much: it creates and mounts an image at the beginning, then possibly 
converts the resulting image to QCOW2 format afterwards.  If the fai command 
were tweaked to allow a size to be specified, then the image creation could be 
accomplished with a hook for the partition task (or perhaps a new diskimage 
task).

Whether the format conversion is needed could be inferred from whether the 
image filename ends in ".qcow2".  The conversion to QCOW2 could be done using a 
hook for faiend.

Personally, I would prefer the configuration approach over the wrapper script, 
since it allows for customization better.  (For example, it would be easy to 
add support for image formats other than QCOW2.)


On top of that, there would be one fewer man page to write.

I hope you will give these thoughts some consideration before making the next 
FAI release.

Derek

Re: FAI going to the cloud

[Derek Poon]

On Jul 29, 2016, at 7:27 AM, Thomas Lange <la...@informatik.uni-koeln.de> wrote:
> 
> A new beta version of FAI is available. It now includes a new command
> for creating cloud^Wdisk images. fai-diskimage can create a raw or
> qcow2 disk image without starting a virtual machine. This image is
> ready-to-boot for your VM or cloud instance. Currently it has no man
> page, but a help is printed when calling with -h.
> 
> I plan to release FAI 5.2 in september, so any feedback is welcome.
> 
> Here's a log from a call of fai-diskimage:
> 
> # fai-diskimage -u cloudhost -S900M 
> -cDEFAULT,DEBIAN,AMD64,FAIBASE,DEMO,GRUB_PC,CLOUD /tmp/disk

Thomas,

Instead of fai-diskimage as a separate command, have you considered making it a 
mode of the fai command itself, analogous to `fai dirinstall`?  It would be 
nice to be able to write hooks for diskimage installs.

Derek Poon
University of British Columbia, IT Services

Re: preseeded value of "locales/locales_to_be_generated"

[Derek Poon]

Andy,

I was just investigating this issue a few days ago, and came to the conclusion 
that Ubuntu 16.04 ignores debconf when it comes to selecting locales.  The 
/var/lib/dpkg/info/locales.config file was introduced between 14.04 and 16.04, 
and its presence causes /usr/share/debconf/frontend to ignore 
/var/lib/dpkg/info/locales.postinst, which used to do the debconf integration.

The /var/lib/dpkg/info/locales.config script, on the other hand, only reads its 
settings from /var/lib/locales/supported.d/local.  I have therefore devised 
this hook to use debconf to write out /var/lib/locales/supported.d/local.

hooks/updatebase.UBUNTU
==
# In Ubuntu 16.04 (but not 14.04), the locales configuration mechanism has
# changed.  There is a /var/lib/dpkg/info/locales.config file, which
# overrides /var/lib/dpkg/info/locales.postinst and consults
# /var/lib/locales/supported.d/local instead of the debconf system.  (See
# the code in /usr/share/debconf/frontend that prefers locales.config.) This
# hook applies the debconf setting.  It must run after FAI's debconf task
# but before dpkg gets a chance to clobber debconf with an empty setting.

if [ ! -f "$target/var/lib/locales/supported.d/local" ]; then
$ROOTCMD debconf --owner=locales sh -c '
. /usr/share/debconf/confmodule
db_version 2.0
db_get locales/locales_to_be_generated &&
mkdir -p /var/lib/locales/supported.d &&
echo "$RET" > /var/lib/locales/supported.d/local' &&
$ROOTCMD dpkg-reconfigure locales
fi

==

I consider it to be a workaround for a bug.

Derek Poon
University of British Columbia



> On Jul 10, 2016, at 19:39, andrew bezella <abeze...@archive.org> wrote:
> 
> hi -
> 
> i've been having some issues w/locales on a xenial system, installed
> from a jessie nfsroot with fai 5.1.  my initial problem was that i'd not
> been including the 'language-pack-en' but it still doesn't seem to be
> working as expected.
> 
> i'd logged https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1598326
> but at one point it appeared that might be a red herring.
> the locales excerpts from debconf.data ([2]) are: 
> locales locales/default_environment_locale select None
> locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
> 
> it's unclear to me why the run of `/var/lib/dpkg/info/locales.config
> reconfigure 2.23-0ubuntu3` behaves as it does.  beyond the above
> "Substitution failed" error, while looking at the debug output i noticed
> other confusing bits.  e.g., after finding $target/etc/locale.gen empty,
> the locales.config script (via GEN_LOCALES and SELECTED_LOCALES
> variables)  appears to *un*set locales/locales_to_be_generated: 
> + SELECTED_LOCALES=
> + db_set locales/locales_to_be_generated 
> + _db_cmd SET locales/locales_to_be_generated 
> + _db_internal_IFS= 
> 
> + IFS= 
> + printf %s\n SET locales/locales_to_be_generated 
> + IFS=  
> 
> + IFS=
> read -r _db_internal_line
> debconf (developer): <-- SET locales/locales_to_be_generated 
> debconf (developer): --> 0 value set
> + RET=value set
> + return 0
> 
> similarly, it seems that the script tries to substitute the
> DEFAULT_LOCALES that it finds (from on-disk files) into
> locales/default_environment_locale (cf. the debconf error above) without
> regard to its existing value.
> 
> while investigating i did also find this unexpected bit during fai's
> debconf task (full logs[1]): 
> read -r _db_internal_line
> debconf (developer): <-- SUBST locales/default_environment_locale locales 
> en_US.UTF-8
> debconf (developer): --> 100 Substitution failed
> + RET=100 Substitution failed
> + return 100
> 
> but i'm guessing some of my confusion there may have to do with: 
> DEBCONF_DB_OVERRIDE="File{$tmpdb readonly:true}"
> and its interaction with `dpkg-reconfigure`
> 
> because despite all the above, the resultant system ends up with the
> desired debconf settings: 
> locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
> locales locales/default_environment_locale  select  None
> 
> *however*, the on-disk /etc/locale.gen only has commented lines and
> while /etc/default/locale is empty when i set "None", it is also empty
> when i set locales/default_environment_locale to en_US.UTF-8 via fai's
> debconf.
> 
> then there's a bit that leads me back to that bug.  on the live system,
> attempting to noninteractively reconfigure locales again *un*sets
> locales/locales_to_be_generated (full output [3]): 
> ~# debconf-show locales
> * locales/locales_to_be_generated: en_US.UTF-8 U