Colin Watson has proposed merging ~cjwatson/launchpad:doc-charm-workflow into launchpad:master.
Commit message: doc: Add explanation of charm development Requested reviews: Launchpad code reviewers (launchpad-reviewers) For more details, see: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/439223 This is doubtless incomplete, but I thought I'd at least get everything I could immediately think of written down as a starting point for others. -- Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:doc-charm-workflow into launchpad:master.
diff --git a/doc/conf.py b/doc/conf.py index f43f4fd..3f6130a 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -38,7 +38,7 @@ master_doc = "index" # General information about the project. project = "Launchpad" -copyright = "2004-2022, Canonical Ltd." +copyright = "2004-2023, Canonical Ltd." # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the diff --git a/doc/explanation/charms.rst b/doc/explanation/charms.rst new file mode 100644 index 0000000..6fe52d3 --- /dev/null +++ b/doc/explanation/charms.rst @@ -0,0 +1,113 @@ +================= +Charm development +================= + +The direction of our official deployments is to use `Juju charms +<https://juju.is/docs/sdk>`_. (We still have a number of manually-deployed +systems, so we aren't there yet.) + +To get an overview of how this works, you'll need to look in the ``charm/`` +directory of Launchpad itself, as well as `ols-layers +<https://git.launchpad.net/ols-charm-deps>`_, `launchpad-layers +<https://git.launchpad.net/launchpad-layers>`_, and `launchpad-mojo-specs +<https://git.launchpad.net/launchpad-mojo-specs>`_. Each of the +subdirectories of ``charm/`` represents a single logical function which can +be deployed as a Juju `application <https://juju.is/docs/olm/application>`_ +with one or more `units <https://juju.is/docs/olm/unit>`_. The layers +provide common code used by multiple charms. The specs are used with `Mojo +<https://mojo.canonical.com/>`_ to coordinate whole deployments of multiple +applications; they contain configuration of individual applications and +`integrations <https://juju.is/docs/olm/integration>`_ between applications. + +Principles +========== + +Wherever possible, charm code should live in the same repository as the code +it deploys (the payload). This makes it easier to evolve both in parallel. + +Launchpad is open source. Its charms and its configuration (aside from a +small number of necessary secrets) should also be open source. As well as +being the right thing to do, this also allows using machinery such as +Launchpad's charm recipes that upload to `Charmhub <https://charmhub.io/>`_. +When used in combination with Charmhub, Juju can easily be instructed to +upgrade charms and update configuration using a single `bundle +<https://juju.is/docs/olm/bundle>`_, allowing the top-level spec to be +relatively simple. + +Each charm should correspond to a deployment of a single top-level payload. +On the other hand, it's fine for a single payload to have multiple charms +corresponding to different ways in which it can be deployed: for example, +Launchpad itself will have charms for the appservers, buildd-manager, +publishers, and so on. + +If a legacy deployment bundled multiple logical functions onto a single +machine purely for economic convenience, don't be afraid to split those up +in a way that makes sense. However, there's no need to go to extremes: if +multiple cron jobs all have broadly the same system requirements, then we're +unlikely to want each cron job to be deployed using its own Juju +application. + +It will not always make sense to expose every single configuration option of +the payload directly in the charm. Some configuration options may only +exist for internal testing purposes or for backward-compatibility, and some +may make sense for the charm to use internally but not expose in ``juju +config``. A good rule of thumb is to consider whether a given configuration +option needs to differ between deployments; if it doesn't, there's probably +no need to expose it. + +`DRY <https://en.wikipedia.org/wiki/Don%27t_repeat_yourself>`_ applies to +configuration as well as code, and can help to avoid gratuitous differences +between deployments. `Jinja <https://jinja.palletsprojects.com/>`_ +templates are widely used in both charms and Mojo specs as part of this. + +Keep multi-datacentre operation in mind where possible. We don't have +enough experience with this yet to know what we'll need to do, but it's +likely to involve deploying parts of an application in different datacentres +from other parts, so loose coupling will help: for example, it may be useful +to allow configuring connections using explicit configuration as well as or +instead of Juju integrations. + +Workflow +======== + +You can run test deployments using `Juju <https://juju.is/docs/olm>`_ and +`LXD <https://linuxcontainers.org/lxd/introduction/>`_. If you don't +already have a suitable testbed, then see the `Juju tutorial +<https://juju.is/docs/olm/get-started-with-juju>`_ for how to set one up; +you should use the non-Kubernetes approach here. + +Each Mojo spec has a ``README.md`` file explaining how to deploy it, and +that's usually the easiest way to get started. You should normally use the +corresponding ``devel`` stage, as that's intended for local deployments: for +example, it will normally deploy fewer units, and doesn't assume that parts +of Canonical's internal infrastructure will be available. + +Once you've successfully deployed an environment, you will probably want to +iterate on it in various ways. You can build a new charm using ``charmcraft +pack`` in the appropriate subdirectory, and then use ``juju refresh`` to +upgrade your local deployment to that. You can change configuration items +using ``juju config``. Alternatively, you can make a local clone of the +Mojo spec and point ``mojo run`` at that rather than at a repository on +``git.launchpad.net``, and then you can iterate by changing the spec. + +Use ``juju debug-log`` and ``juju status`` liberally to observe what's +happening as you make changes. See `How to debug a charm +<https://juju.is/docs/sdk/debug-a-charm>`_ for more specific advice on that +topic. + +Secrets +======= + +Cryptographic secrets should not be stored in Mojo specs, and nor should +some other pieces of information (such as configuration relevant to +preventing spam). These are instead stored in a secrets file on the +relevant deployment host (``launchpad-bastion-ps5.internal`` or +``is-bastion-ps5.internal`` for official deployments), and are updated +manually. The ``bundle`` command in the Mojo manifest will have a +``local=`` parameter pointing to this file, relative to +``$MOJO_ROOT/LOCAL/$MOJO_PROJECT/$MOJO_STAGE``. + +Managing secrets like this is more cumbersome than updating Mojo specs, so +try to keep it to a minimum. In some cases there may be automation +available to help, such as the `autocert charm +<https://charmhub.io/autocert>`_. diff --git a/doc/explanation/index.rst b/doc/explanation/index.rst index 9b66e6d..21299fd 100644 --- a/doc/explanation/index.rst +++ b/doc/explanation/index.rst @@ -13,3 +13,4 @@ Explanation architecture pip favicon + charms
_______________________________________________ Mailing list: https://launchpad.net/~launchpad-reviewers Post to : launchpad-reviewers@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-reviewers More help : https://help.launchpad.net/ListHelp
