Hello,
As a result of FOSDEM conversations today I felt inspired to put some
thoughts on paper about an area where I think we currently run into
complications.
Not sure if it is appropriate to write this blog-post-ish contribution
here, but as I don't have a blog, and it's about Guix development, I
figured it might be OK.
Best wishes,
Alex
1 Introduction
══
Guix is amazing. A large part of why it continues to be amazing is
because it provides strong guarantees to the end-user and developer.
As such it makes reasoning about packages and deployments relatively
straight-forward.
The functional paradigm cleary works fantastically for packages.
Unfortunately it is not quite clear that it works just as well for
services. The reason for this is that too many useful services are
inherently stateful. Their statefulness means they have side-effects,
which in turn cause issues when relying on Guix features such as
roll-back or automated deployment & guaranteed reproducability.
Disciplined developers can implement many services in such a way that
their statefulness is delegated to other dedicated services. In this
way the problem of statefulness can be isolated. However, many
existing useful services /have not/ been implemented in such a way.
These notes are an attempt to think through ways of formalising how we
mitigate stateful services in Guix.
The notes below are organised around the example of a popular PHP
content management system, Drupal. Similar problems will apply to
many other end-user services.
My intent here is to communicate my thinking in the hopes that others
can point to obvious flaws in my reasoning — or to stimulate
conversation about the topic. I hope at the very least that it is an
interesting read!
2 Learning by Doing: Packaging & Deploying Drupal
═
2.1 Problem (1): the Drupal tarball is a binary blob!
─
Released Drupal tarballs are shipped with a bunch of PHP dependencies,
as well as compiled JS files. A fully source-distributed installation
of Drupal would:
1) delete all shipped PHP dependencies
2) independently build all PHP dependencies and make them available to
Drupal (through it's vendor directory as symlinks?)
3) delete all compiled JS files & libraries
4) independently build the JS dependencies and make them avalable to
Drupal.
2.2 Packages must always be stateless!
──
What does this mean in practice? Let's look at Drupal again. When
you download Drupal, the resulting tarball contains the source code,
and an empty sites/ folder. The sites folder is intended to contain
/state/ files. The normal installation procedure is to simply drop
the drupal distribution in your web root directory, and for state
files to live underneath sites/ within your webroot.
In Guix, Drupal is packaged so that it is installed in the store.
The store is read-only and hence no /state/ files can live under the
sites/ directory in the store.
How do we get around this?
1) either we patch drupal to expect the sites/ directory outside of
its own folder tree.
2) or we symlink /gnu/store/…drupal…/sites/ to a different location on
the filesystem (e.g. /var/lib/drupal/sites/)
The latter solution, while easy, means that a *successful*
installation of the package Drupal in Guix results in an installation
in the store with a *broken symlink* pointing outside the store.
But the package itself has been rendered stateless!
2.3 Drupal: a stateful service
──
Services in Guix are rich and multidimensional entities. At core they
are promises of things that will have happened when a system is up and
running. These promises can be arbitrary, like generating a
configuration file every boot; or they can be an extension of other,
already existing services.
A particularly popular service to extend is the shepherd service,
which ensures that particular daemons are started as soon as possible
after the system has started.
What would a Drupal service look like? Essentially the software is
just a bunch of files in a webroot — so at it's heart it simply
extends a web service (e.g. nginx) with new location directives.
On the other hand it also requires that tha web server, a sql backend
and php-fpm are running, so that drupal can actually function.
These two requirements are easily met with the usual service
infrastructure: simply extend nginx with a location definition
pointing to the Drupal folder in the store as the webroot and extend
shepherd to require mysql, nginx and php-fpm.
2.3.1 Enter the state dragon
But hang-on… if Mysql service is a dependency, and we store state in a
Mysql DB, what happens when we upgrade to a newer version of Drupal,
