[ Please note the cross-post and Reply-To ]
Hi folks,
As promised, here's a summary of what was discussed at the BoF session
at DebConf19 in Curitiba. Apologies for the huge delay in posting -
life... :-/
Thanks to the awesome efforts of our video team, video of the session
is online [1]. I've taken a copy of the Gobby notes too, alongside my
small set of slides for the session. [2]
=====================================================================
Secure Boot. In Debian. In Buster. Really.
Sunday, 2019-07-21, 13:30
What is Secure Boot?
====================
Simply, it is a way for computer firmware to validate that the
boot-time binaries it runs are the right ones - they have not been
tampered with, e.g. infested by malware. This works using a chain of
signatures, based on public key crypto.
It is typically *not* designed to totally lock down what software a
user can run on their system. It *is* designed to stop boot-time
malware - verifying the boot loader and kernel means that the
operating system, virus checkers, (etc.) can be started in a
known-clean state.
UEFI Secure Boot is included and enabled by default in just about
every modern x86 machine. It can be disabled if desired, but it's
genuinely useful technology and we should be part of it.
In the Linux world
==================
Most of the Linux distro community have decided to use shim as part of
their UEFI Secure Boot support. Shim is a tiny, really simple
first-stage boot loader started by Matthew Garrett. It is well
audited, well understood, etc. It has a simple job: it is a single
piece of signed software that verifies and runs the next stage
bootloader (Grub). It's a collaborative project, with contributions
from many people in lots of the distributions.
Microsoft act as the Certification Authority (CA) for UEFI SB here -
for obvious reasons their key is present on almost all x86 computers
as shipped. They *don't* want to sign Grub, as it's too large and
complex and (importantly) they don't like the GPLv3 licence
there. Instead, Microsoft will sign the simple, small, BSD-licenced
shim. A team of cross-distro people are responsible for reviewing the
shim patches and builds that are presented for signing. Microsoft
trust us to do that job - they don't pretend to be experts in how shim
works.
Each distro builds a version of shim with their own key(s) embedded,
and those key(s) are used for validating further things that are
loaded: Grub, Linux, firmware update programs, etc.
Users can also enrol their own Machine Owner Key(s) (MOK) on the
system, and shim will also use those key(s) for validation. This is an
important feature - end users have control over the keys that are used
here.
In Debian, we use our own key to sign some of the binaries generated
by a small number of source packages: Grub, shim (helper binaries),
Linux (and modules) and fwupd/fwupdate. We *can* sign other things if
we need to, but we want to keep the list of signed packages small and
controlled. Anything that is signed here must have a good security
story, for obvious reasons.
There are restrictions applied *by default* to systems booted with SB
enabled. Examples:
1. Grub will refuse to boot unsigned kernels.
[ Ubuntu's signed version of Grub initially would allow booting
with unsigned kernels, but this was widely agreed to be a
mistake and things have now been changed here. ]
2. No non-free/out-of-tree kernel modules will be loaded by Debian's
kernel, as they will not be signed. It is possible for users to
sign them (via MOK, for example); Ubuntu have methods to do module
signing in a more automated fashion, but we haven't yet set up
similar in Debian. More on this later.
3. Hibernation is disabled. All the state of the hibernated system is
put into the hibernation space (swap); it's very difficult to
secure and validate that data so that a cleanly-booted system can
know to trust it. This might be solved in the future, but not yet.
Is this supported in Debian?
============================
YES! Finally!
Everything is deployed (since earlier in 2019) so that everything
should work on three architectures: amd64, i386, arm64. Most users are
likely to be on amd64, but the others should work too.
On arm64, a minor issue is that there has not been a single CA for
arm64 machines to use until very recently, so a lot of current
machines will need configuration to be able to secure
things. Microsoft has recently agreed to act as a CA here also, but it
may take time for this setup to turn up on common shipping hardware.
Our signed packages are pulled in using Recommends, to still allow
people to *not* have them installed if that is preferred. This means
that standard installation methods will be secure by default, but be
careful during installation of packages when using apt
--no-install-recommends.
An important target here is that SB should Just Work for people,
invisibly. Any problems are bugs that should be reported. As of the
Buster release, all of our debian-installer and live media for our
supported arches should work with SB enabled, and the secure setup
will also be propagated through to systems installed using either.
Debian's cloud images are not quite all set up with SB yet, but this
should hopefully be done soon. Lots of the cloud providers are very
much interested in this - they want to see trusted verification of
software too!
Debian infrastructure
=====================
Working out the best way to build signed packages using Debian
infrastructure has been the most "fun" part of this whole process. We
had *many* proposals over the last few years for how to do stuff, each
of which was found to be problematic.
In the end, binary packages that are signed in Debian will have two
variants, one signed and one unsigned. In most cases, the *signed*
binary package will be called $something-signed; for the linux kernel
packages, they instead are called linux-image-* (signed) and
linux-image-*-unsigned (for the unsigned version).
Building the signed packages for the Debian archive is slightly
complex, but here's how it works:
1. Each source package that needs to build a signed binary package
($package-$arch-signed) now also generates pseudo-source template
packages called $package-$arch-signed-template. The template
package contains two things:
a. metadata to describe exactly what binaries should be signed,
and how;
b. a skeleton source package to describe how to build
$package-$arch-signed
2. There are now 2 buildd passes for source packages that are wanting
to build signed binary packages, and in the middle of that a new
service that we have called the "signing service".
a. [buildd pass 1]
The existing buildd pass to make binary packages, just as we
had previously.
b. [signing service]
*Iff* the source package builds -signed-template packages *and*
the package is on an allowed list (controlled by ftpmaster),
then those -signed-template packages will be queued up for the
signing service (SS for short here).
The SS is utterly locked down, by design. The signing keys are
all stored in an HSM for extra security. It also runs *no* code
or scripts from the -signed-template packages that it is
processing, to minimise any attack surface here.
The SS parses the configuration from the -signed-template and
generates appropriate detached signatures for the desired
binaries. (These binaries will come from the normal buildd
output.)
The detached signatures are then combined with the skeleton
source package to make another source package
($package-$arch-signed). This new source package is then
uploaded into the buildd queue again.
c. [buildd pass 2]
The buildds pick up on the new version of $package-$arch-signed
and build binary packages to match. This involves simply
grabbing binaries again and attaching the included signatures.
This second buildd pass allows us to have the separation of
code and data here - it's how we can generate signed binary
packages without having to run debian/rules (etc.) on the
signing service.
We want 100% reproducibility for everything that we sign, of
course. All of the code we have for the signing service (etc.) is of
course open and freely available. We want total transparency for what
we're doing here - that's how we can all trust it.
More details about the design of the signing service and our process
can be found online at https://wiki.debian.org/SecureBoot/Discussion
The awkward bits
================
Time
----
It's taken a *very* long time to get this into Debian. We've been
talking about this since ~2012, only in the archive in 2019. Why has
it taken seven years?
This is a very complex topic that required cross-team collaboration
from (at least!) 5 different teams in Debian: Kernel, EFI, FTP, DSA,
buildd.
Debian works really well when people can work independently - it's how
we have thousands of contributors working on tens of thousands of
different source packages without forever blocking each other.
But here we had busy people (and teams) waiting on each other,
multiple times. There there were several different proposals and we
needed many rounds of discussions before we eventually got to our
solution. Huge progress was finally made during a sprint in April 2018
in Germany when we had people from *all* the relevant teams together
in a room for the first time. It's amazing how much better things can
work when then feedback loop is measured in minutes rather than weeks!
Tooling
-------
Tooling is still not very friendly in the Secure Boot area. It's
security-related software, so it apparently has to be difficult to
use. Also, very few people use it and understand it well, so it's
still quite easy to find bugs and edge cases that others have not
yet. Docs are rare and out of date. This is not an attempt to be nasty
about the developers here - good software needs a certain number of
users to provide feedback and help find the bugs.
Bugs
----
Unfortunately, we will also now be using more firmware features so
we'll definitely discover more bugs, just as we found with switching
from BIOS boot to UEFI initially. Please report things as you find
them - that's always the best way to get them fixed.
Supporting non-free drivers
---------------------------
We still need to come up with a good solution to support SB users
needing non-free drivers (like the Nvidia binary graphics drivers). We
don't have anything really put together yet, but we can borrow
liberally from people like Ubuntu who have solutions for this.
The good bits
=============
Now, with UEFI Secure Boot, it's much easier (and safer) to install
Debian. We don't need to disable secure boot in the BIOS, or (worse!)
explain to other users how to do it on their systems.
We have *better* support for secured systems. We're still far from
supporting locking things down end-to-end here like some people
want/need, but it's progress.
User freedom really matters, of course. End users have the option to
disable all of the SB features if they desire, and that has been an
important consideration all along this process. Users also have the
ability to use their own keys alongside (or instead of) the default
keys on their system - full control of everything is possible here.
We have deliberately made all of our work here fully open so that
users can follow our process and even use our code to generate their
own signed packages if desired. This gives options for Free Software
end-to-end.
Thanks to everyone!
===================
I was doing the last pieces to tie all this together, but I was only
one of a number of people who spent effort to make SB a reality in
Debian. It was a lot of hard work for a number of people across
various teams in Debian! In the session, I particularly called out:
* Tollef Fog Heen
* Helen Koike
* Luke Faraone
* Ben Hutchings
* Joerg Jaspert
* Ansgar Burchardt
* Aurelien Jarno
* patient people in the release team :-)
for contributing to this effort. Hopefully this is a good feature that
people will appreciate!
Questions and discussion
=======================
Purism are using coreboot, not UEFI SB. Needs a different setup.
We've only done the common case here to start with (UEFI SB on
x86/arm64). Adding support for different secure methods is of course
possible! Patches are welcomed, especially for different
architectures. It's hard to test booting without access to
hardware. We designed our infrastructure to be flexible and
extensible. The current setup allows for more algorithms, signing
variants, etc. U-Boot with SB is happening - we'd like to support SB
on embedded devices too, not just servers and laptops.
We could also support other Roots of Trust if needed, not just the
Microsoft CA signature on our shim. There are potential worries about
compatibility and process delays if we tried to get multiple CA
signatures on a single shim binary. But we could easily have multiple
shim binaries, each signed by a different CA as the root for a SB
system.
Quite a few people are going to need support for non-free drivers for
their machines. Adding better support here is a big piece of work that
we should target. For now, such users will need to disable SB or work
out how to do it. Helping us work out the best process and documenting
it / automating it is a great way to get involved.
We had Some proposals from the audience about ways to do this
(e.g. integrating with DKMS). but we need to be careful about
risks. Needs more discussions, particularly on security trade-offs.
SB and lockdown will block people from doing some things, particularly
low-level debugging. By their very nature, security and low-level
access are basically incompatible! Lockdown design and patch work is
still ongoing in the kernel, so things here are fluid.
Secure boot and key setup will vary massively, depending on
platform. Small systems like Arm-based IoT things will most likely use
Arm Trusted Firmware (ATF) for a secure base and may have keys
embedded. Interfacing to those and giving a good SB story is still
work in progress.
Discussion on how to compile own kernel and use it with SB:
1. Build own kernel, add your own keys
2. Add your key to shim, or in the MOK database
3. Might require separate chain of keys
Debian has the signing service setup, which we needed to avoid the
requirement for manual signing (and giving access for to many people
for signing). When running on your own machine, manual signing might
be good enough. We don't know of anybody else trying to run a signing
service using our code, so expect to find rough edges for now!
Further questions and problems
==============================
Please talk to us by mail (debian-...@lists.debian.org) or IRC
(#debian-efi), and definitely file bugs as you find them. We want to
make this work for people.
[1]
https://meetings-archive.debian.net/pub/debian-meetings/2019/DebConf19/secure-boot-in-debian-in-buster-really.webm
[2] https://www.einval.com/~steve/talks/Debconf19-SecureBoot/
--
Steve McIntyre, Cambridge, UK. st...@einval.com
"Further comment on how I feel about IBM will appear once I've worked out
whether they're being malicious or incompetent. Capital letters are forecast."
Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html
