Hi,
= me =
I'm Marc. I have not been present much on debian-boot in the last
decades, but I have been around for a while. I do remember Debian when
d-i was not yet availebl. You might know me as the person taking care of
adduser and sudo.
In my other life, I am not only the can-opener for four lovely cats, but
also a freelance IT consultant in Germany. I do quite some work in
larger Linux environments, and customers using Debian get a (small)
discount. My largest (sadly, ex-) customer is running a mid-size
five-digit number of Debian installations, the majority of those on bare
metal.
= Why this mail =
First let me thank you to provide d-i. Debian would not be here without
you guys providing the way to get Debian on systems and running. But I
am kind of concerned about recent developments in Debian, for example
debian-live growing more and more into "the method to install Debian on
Laptop and desktop" recommended in the press, social media and user
groups, leaving d-i as the method to install servers, moving into the
role of a second-class citizen.
Whenever I talk with people running mid-size to large numbers of Debian
installations, people roll their eyes when one talks about the
Installer, in particular its way to partition disks. I must admit that I
am not partman's best friend either. Most of those installation don't
use d-i to install Debian on their systems (rolling their own installer
or using a golden image to clone new systems from), and I have seen
sites migrate away from Debian because Anaconda is so much better
automatable.
The big installation mentioned above is the only larger installation I
have ever been working with who actually uses d-i: They boot d-i from
the network, using a preseed-file that is generated from their CMDB and
after installation hand over the system to puppet, using a
400-or-so-character late_command that made my eyes water. In this
installation, the inflexibility of disk partitioning has been constant
gripe of the other workgroups that end up using the systems installed by
this method. They all want more flexibility. I haven't worked with them
in eight years so I don't know whether they still use this.
Myself, I developed my own Debian installation method that involves
booting a live linux¹ and then running debootstrap and and a number of
idempotent scripts. That was before d-i was invented and I just never
got around to migrate to d-i beause my method was still working
reasonably well. Nowadays, I use an ansible playbook to install Debian
(which allows me to reuse code from the normal configuration management
that takes care of my Debian systems during their lifecycle). I am
therefore not very familiar with current d-i mechanisms. I do
occasionally use d-i to install Debian on notebooks and desktop
computers that don't run in a datacenter and in installations where only
a handful of Debian systems are in use (or when the customer explicitly
doesn't want automation).
¹ you might remember Linuxcare's Bootable Business Card from the early
2000 years, and there is a reason why the first grml-small release had
the codename "Zugschlus" back in - i think - 2005.
Whenever I use d-i manually, partman shows up as my nemesis. I don't
think that I ever was able to get partman non-trivial partitioning right
on the first try.
I would like to give you input to improve this. Sadly, I neither do have
time nor the knowledge to be of actual help, but I'd like to offer my
input, wishlist reports to help you improve and I am also willing to
write docs and specs. I might be able to contribute code, but please
don't expect my code to be directly useable. I am way too bad a
programmer for that. But I am willing to test.
I'd like to provide input about the following topics.
= Find preseeding data, make it easy to place preseeding data =
Preseeding usually means network boot or rolling your own .iso. As far
as I know, if you do neither, you have to give a (network) path to the
preseed file on the boot prompt otherwise. That doesn't scale, and it is
even not nice for just installing a single system. I am wondering
whether you have considered more default places to have the installer
look for preseed files. For example, I'd like to have the following
default places for preseed files in addition to the places where the
installer already looks for:
* (any EFI partition on any hdd)/d-i/preseed.cfg
* that is likely to already exist or can trivially be pre-created
* /d-i/preseed.cfg on the (virtual) cdrom on the second CD-ROM drive
* having a second CD-ROM drive is trivially achievable in
virtualization
* /d-i/preseed.cfg on any filesystem carrying a certain label or
partlabel.
* that can be on the target disk as well, or on an USB key, ...
* /d-i/preseed.cfg on a second filesystem on the installer .iso
* not sure whether this will actually work with an (emulated) cd-rom
drive, but live linuxes use this mechanism on usb boot media to
allow customization of system boot with an unchanged .iso
The file names searched for could be mutated using a firmware asset tag,
MAC address prefixes of different length (like PXE clients already do),
ISO release numbers, etc bla foo.
The installer then could put up a dialog "preseed file foo found on
(path), do you want to use it?" by default. Of course, to facilitate a
fully automated install, there must be a (preseedable) option to silence
this.
Gold Plating: If no preseed file was found, the (expert) installer could
put up a dialog "No preseed file found, do you want to enter the path to
one" to allow the user to add a preseed file after missing the timeout
boot prompt (that could also be used as an indicator that the search for
a preseed file failed).
= verification of preseed files =
There is the option to add a checksum to the preseed file option. The
fact that md5 is ths only option offered here suggests that this part
hasnt been touched in quite a while. Have there been thoughts of
introducing signed preseed files, putting the public part(s) of key(s)
in there and just processing preseed files that have the right
signature?
= Kickstart, anyone? =
Other installers after a manual install leave a file in the installed
system that can directly be used to preseed another installation, which
then will run identically to the one that was just finished, just
unattended. Searching for this on the Web suggests that this was left
out of d-i intentionally, but I didn't find the rationale for that. I
one was in charge of a classroom setting with 15 identical machines, and
I would have been really nice be able to tell a tutor "Just do the
install manually on machine 1, and I can then identically repeat the
installation on machines 2-15".
The method outlined in the installation manual B.3 is a bit arcane, I am
not very happy about that. There could be a framework to automate this,
probably removing those items that "should not be preseeded".
Also the recommendation to use an editor in a running installation to
check for possible options sounds like an excuse. Nobody should be
forced to manually read debconf template files.
There could be a framework that reads the templates when the installer
is built and as a corollary generates a nice web page or an .md document
giving the templates and the possible values. Or, the example preseed
file (like https://www.debian.org/releases/trixie/example-preseed.txt)
could be automatically augmented with the valid values.
= Which preseed setting corresponds to which question in d-i =
In addition, I have found it quite hard to see the connection between
the questions that the installer asked and the respective configuration
being put into the debconf database. If the Installer would write its
own preseed file instead of referring people to debconf-get-selections
--installer, it would be possible to put the title of the respective
installer dialog in a comment, making it way easier to find out which
answer to the installer dialogs has resulted in this particular preseed
option being generated.
= Interactive Preseed Preparation =
I would love to have a possibility to run through a virtual installer
inside a running Debian system in a window, and get a preseed file that
contains my answers. I could live with running an actual installation in
a VM and having some side-channel into that VM to interactively and
directly see what is being written to the installer's debconf database
without having to change virtual console hundreds of times. Maybe
another virtual serial port that I could attach to would be a good
method.
= syslog =
Can the Installer be made to optionally log to syslog once the network
is up? Maybe I could give a syslog=syslogserver.example.com on the
kernel command line? In an ideal world, dmesg and everything that is
already logged at this time would be (optionally) dumped to the syslog
server at the beginning, and a meaningful host name would be given in the
syslog messages as well.
= Make early_command easier =
Would it make sense to automatically search for an
(preseed|partman)/early_command in the same way than it looks for the
preseed file? That might be an easier way to influence installation and
partitioning without having to do a full preseed? I guess that it would
be easier for many people to just drop a shell script in some
pre-defined place than having to grok the preseeding file format.
When this is addressed, a similar process could be used for the
late_command that many installations use to hand over the system to
puppet, ansible or relatives.
= Partitioning =
This is a huge issue in Linux installation, always. I have yet to see a
Linux installation tool for any distribution that does this as flexibly
as big installations want. Most installations that I am aware of that
have rolled their own installation procedure did that because of the
inflexibility of the distribution-provided partitioning tools. This is
probably caused by the fact that partitioning naturally happens before
anything is installed and that a partitioning process therefore is
spartanic at best.
I don't expect Debian to be the first distribution to provide a perfect
partitioning tool. But I would like to have some methods to
bypass/replace partman while still being able to use Debian Installer
proper.
I think there is too much documentation out there that explains how to
use partman. Therefore, partman as it is should stay. While part of me
is all for a rewrite (see above), I would also be happy with alternative
ways to partition for a Debian installation, while still making it
possible to make use of the Installer.
== early_command ==
I could for example imagine having an early_command (or even a partman
early_command) that does the partitioning in the way I want it. Probably
the easiest way I can imaging would be making partman and the rest of
the Installer assume that if something is mounted on /target at the
point when partman gets invoked, that is already the way the system
should be installed. That way, partman could (preseedably) ask "do you
want to use /target as already mounted" and if yes, just advance in d-i
wihout even doing anything.
== pre-partition ==
Just in case one is too lazy to write a proper early_command (which
probably must run in busybox with a rather limited userland), the system
could first be booted to a rescue system with more capabilities to do
the partitioning and building the file systems, with a much simpler
early_command that only needs to mount the prepared filesystems once the
installer is booted.
== fstab import ==
Developing the use cases outlined above further leads to the idea of the
preparation system leaving an /etc/fstab file around that the installer
could directly use to mount /target and as /etc/fstab for the installed
system. That would completely eliminate the need for an early_command if
partitioning is done before the actual installer is booted.
== YAML Import ==
Partman could be extended in a form that allows it to read YAML, which
could be
* pasted into a free-text field of the installer
* read from the network
* searched for in a similar way like preseed and (early|late)_command
That YAML input could be pre-crafted or generated to allow the
partitioning to be exactly like people want it, without having to learn
partman's preseed partitioning syntax, and without having to live with
partman-auto's limitations.
After having written this, I feel that I begin to be a fan of the
early_command approach.
That's what I have for you today, I hope that we can discussion the
options and what I missed. Once the discussion is finished, I intend to
file wishlist bug requests against the appropraite parts of d-i to
suggest implementation of our results, if feasible.
I will join #debian-boot later today and will be available there in the
next weeks.
Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
