                            Configuration management

revision 7.0

   Wichert Akkerman

    The Debian Project

   wakkerma@debian.org

   Joey Hess

    The Debian Project

   joeyh@debian.org

   Copyright (c) 1998 , 1999 , 2000 by Wichert Akkerman and Joey Hess

    This text is copyright by the authors under the terms of the BSD
   license, sans advertising clause.

     ----------------------------------------------------------------------

   Table of Contents

   Introduction

   Configuration Data

   Templates

   Configuration frontends

   Communication with the frontend

   Debian install-time configuration

                                  Introduction

    Configuration management is quickly becoming a very important issue.
   Having programs which do cool stuff is great, but we need to store their
   configuration as well. We see more and more different configuration
   systems being introduced all the time, which is not very practical. This
   text introduces a general configuration management system which flexible
   enough to be used for all kinds of applications.

     ----------------------------------------------------------------------

                               Configuration Data

The configuration space

    All configuration information is stored in what I call the configuration
   space. This is a database with a special design which resembles the method
   we look at configuration information. This is done by defining a hierarchy
   of information. Each package receives its own space in the hierarchy. Each
   package is free to use a flat space, or divide its space further into
   sub-hierarchies. If multiple packages share a common purpose they may use
   a shared toplevel hierarchy, preferably with the same name as a shared
   (virtual) package name (for example, both mutt and elm can use
   mail-reader, strn and nn could use news-reader). This shared tree can also
   be used as a default, ie a variable news-reader/nntpserver can be used by
   strn if strn/nntpserver does not exist.

    Each variable in the configuration space has some information associated
   with it. Most importantly, it has a value. It also may have a set of flags
   and a set of substitution data.

     ----------------------------------------------------------------------

                                   Templates

    Each variable in the configuration space is associated with some
   meta-data. The minimum meta-data associated with a variable is: long and
   short description, type, and default value. The meta-data is essentially
   static; the protocol described below does not allow it to be changed.

    The meta-data exists in a space with similar naming properties to the
   configuration space described above, and typically one variable in the
   configuration space will have associated with it metadata with the same
   name in the meta-data space. However, this need not be the case; many
   different variables can all be associated with the same meta-data. In
   effect the meta-data serves as a template for the configuration variable.

     ----------------------------------------------------------------------

Template information

    So, what do we need to store in a variable template? Of course we need a
   name to identify the template. Template names are made up of components
   separated by the character `/' (slash). Each component is limited to
   alphanumerics and `+' `-' `.' (plus, minus, full stop).

    A type is also needed so data can be verified. Here is a table of common
   types; implementations are free to make up more.

   Table 1. Available data types

   +------------------------------------------------------------------------+
   | Type        | Description                                              |
   |-------------+----------------------------------------------------------|
   | string      | Holds any arbitrary string of data.                      |
   |-------------+----------------------------------------------------------|
   | boolean     |  Holds "true" or "false".                               |
   |-------------+----------------------------------------------------------|
   |             |  Holds one of a finite number of possible values. These |
   | select      | values must be specified in a field named Choices:.      |
   |             | Separate the possible values with commas and spaces,     |
   |             | like this:  Choices: yes, no, maybe                     |
   |-------------+----------------------------------------------------------|
   |             |  Just like the select data type, except the user can    |
   |             | choose any number of items from the list. This means     |
   |             | that the Default: field and the actual value of the      |
   |             | question may be a comma and space delimited list of      |
   | multiselect | values, just like the Choices: field.                    |
   |             |                                                          |
   |             |   Note: For compatability with old versions of debconf,  |
   |             |   if you use this data type, please make your package    |
   |             |   conflict with debconf (<< 0.2.26)                      |
   |-------------+----------------------------------------------------------|
   |             |  This template is a note that can be displayed to the   |
   |             | user. As opposed to text, it is something important,     |
   | note        | that the user really should see. If it is not possible   |
   |             | to display it, it might be saved to a log file or        |
   |             | mailbox for them to see later.                           |
   |-------------+----------------------------------------------------------|
   |             |  This template is a scrap of text that can be displayed |
   |             | to the user. It's intended to be used for mostly         |
   |             | cosmetic reasons, touching up around other questions     |
   | text        | that are asked at the same time. Unlike a note, it isn't |
   |             | treated as something the user should definitely see.     |
   |             | Less complex frontends may refuse to ever display this   |
   |             | type of element.                                         |
   |-------------+----------------------------------------------------------|
   |             |  Holds a password. Use with caution. Be aware that the  |
   | password    | password the user enters will be written to a database.  |
   |             | You should consider clearing that value out of the       |
   |             | database as soon as is possible.                         |
   +------------------------------------------------------------------------+

    Of course a default value is useful as well, and finally we need a
   description of the variable. We actually use two descriptions: a short one
   (limited to 50 characters or so) and an extended one.

    The extended description may be word-wrapped by the FrontEnd. To make
   separate paragraphs in it, use . on a line by itself to separate them.
   Text in the extended description that is prefaced by additional whitespace
   will not be wordwrapped. Both the description and extended description may
   have substitutions embeded in them. Ie, ${foo}. These will be expanded
   when the descriptions are displayed.

    This information is stored in a template file that consists of stanzas
   in a rfc-822 compliant format, separated by blank lines. Here is an
   example:

 Template: hostname
 Type: string
 Default: debian
 Description: unqualified hostname for this computer
  This is the name by which this computer will be known on the network. It
  has to be a unique name in your domain.

 Template: domain
 Type: string
 Description: domain for this computer
   This is the domain your computer is a member of. Typically it is
   something like "mycompany.com" or "myuniversity.edu".
   

     ----------------------------------------------------------------------

                            Configuration frontends

    Of course applications can use the database and meta-database directly.
   But there should be a simple system to interact with the user that is
   simple and modular enough to be used with systems ranging from
   shell-scripts to Fortran programs. To do this we define a general frontend
   that can be driven using the simplest and most common form of
   communication: stdin and stdout.

    Using this simple form of communication gives us a great advantage: it
   becomes easy to change the frontend. That means the user can switch
   between a console, a graphical or even a web-interface at will.

    Besides being able to switch between types of frontends there is another
   important aspect of a good user interface: user friendliness. We have to
   account for the fact that some users know more then others and change the
   information we show or ask from the user. We do this by giving everything
   a priority and giving the user control over what kind of questions he
   wants to see. Experts can request to see everything, while novices get the
   option of only seeing only important questions. Finally there is an option
   to simply skip all questions, so it becomes possible to do automatic
   configuration using default values or values that are downloaded into the
   database from a remote location. This makes it simple for example to
   install and manage clusters or lab rooms or do installs for dummies.

     ----------------------------------------------------------------------

                        Communication with the frontend

    This communication between the frontend and the application should be as
   simple as possible. Since most IO implementations default to line-buffered
   IO, so we use a simple language where each command is exactly one line.

    After sending each command to stdout, the client should read one line
   from stdin. This is the response to the command, and it will be in the
   form of a number followed by whitespace and an optional string of text.
   The number is the status code, while the text provides additional
   information.

   Table 2. Numeric status codes

   +-----------------------------------------+
   | Range   | Description                   |
   |---------+-------------------------------|
   | 0       | success                       |
   |---------+-------------------------------|
   | 1-9     | reserved                      |
   |---------+-------------------------------|
   | 10-19   | invalid parameters            |
   |---------+-------------------------------|
   | 20-29   | syntax errors                 |
   |---------+-------------------------------|
   | 30-99   | command-specific return codes |
   |---------+-------------------------------|
   | 100-109 | internal errors               |
   |---------+-------------------------------|
   | 110-255 | reserved                      |
   +-----------------------------------------+

    Here are the currently supported commands.

     *  VERSION number

        This exchanges with the frontend the protocol version number that is
       being used. The current version is 2.0. Versions in the 2.x series
       will be backwards-compatible. You may specify the protocol version
       number you are speaking. The frontend will return the version of the
       protocol it speaks. If the version you specify is too low, this
       command will return the numeric return code 30.

     *  CAPB capabilities

        This exchanges with the frontend a list of supported capabilities
       Capabilities both the frontend and your confmodule support may be
       used; the capabilities supported by the frontend are returned by this
       command.

       Table 3. Currently used capabilities

       +--------------------------------------------------------------------+
       | capability  | description                                          |
       |-------------+------------------------------------------------------|
       | backup      |  Backing up to a previous step is supported.        |
       |-------------+------------------------------------------------------|
       |             |  The multiselect data type is supported. For        |
       | multiselect | compatability reasons, you should not ask questions  |
       |             | of this type unless this capability is returned.     |
       +--------------------------------------------------------------------+

     *  TITLE string

        You can use this command to set a title in the frontend. This may
       appear in different ways, depending on the frontend being used, for
       example it might change the title of the frontend's window. If you
       don't specify anything, a title will automatically be generated.

     *  STOP

        This command tells the frontend you're done talking to it. Typically
       the frontend can detect the termination of your program and this
       command is not necessary.

     *  INPUT priority question

        This tells the frontend to display a question (or other type of
       item) to the user. question is the name of the item to display, all
       other information about the item is retrieved from the templates
       described previously. priority is how important it is that the user be
       prompted. The frontend need only ask this question if the priority is
       high enough. The question is not displayed until a go command is
       given. This allows us to ask multiple questions in a single screen.
       Once a question has been displayed to the user and the user has
       provided input, the frontend will set the seen flag.

       Table 4. Supported priorities

       +--------------------------------------------------------------------+
       | Priority | Description                                             |
       |----------+---------------------------------------------------------|
       | low      |  Very trivial items that have defaults that will work  |
       |          | in the vast majority of cases.                          |
       |----------+---------------------------------------------------------|
       | medium   |  Normal items that have reasonable defaults.           |
       |----------+---------------------------------------------------------|
       | high     |  Items that don't have a reasonable default.           |
       |----------+---------------------------------------------------------|
       | critical |  Items that will probably break the system without     |
       |          | user intervention.                                      |
       +--------------------------------------------------------------------+

        Note that the frontend decides if the user is actually prompted or
       not. If the user has already answered a question, they are normally
       not asked it again even if input is called again. And if the user is
       ignoring low priority items, they will not see them. In either of
       these cases, this command returns the numeric return code 30.

     *  BEGINBLOCK

     *  ENDBLOCK

        Some frontends are able to display a number of items to the user at
       once. To do this, they need to be given blocks of input commands,
       enclosed in the BEGINBLOCK and ENDBLOCK commands. Blocks can be nested
       and very advanced frontends may use this as a user interface hint.

         Note: There is an implicit block around any set of INPUT commands
         that are not enclosed in an explicit block.

     *  GO

        Shows the current set of accumulated items to the user and lets them
       fill in values, etc. If the backup capability is supported and the
       user indicates they want to back up a step, this command returns the
       numeric return code 30.

     *  CLEAR

        Clears the accumulated set of INPUT commands without displaying them
       to the user.

     *  GET question

        Ask the frontend to tell you how the user answered a question. The
       value is returned to you.

     *  SET question value

        Set the answer of a question to a value.

     *  RESET question

        Reset the question to its default value. This includes resetting
       flags to their defaults.

     *  SUBST question key value

        Questions (and other items) can have substitutions embedded in their
       descriptions. These substitutions look like "${key}". When the
       question is displayed, the substitutions are replaced with their
       values. This command can be used to set the value of a substitution.

     *  FGET question flag

        Questions (and other items) can have flags associated with them. The
       flags have a value of "true" or "false". This command returns the
       value of a flag.

     *  FSET question flag value

        This sets the state of a flag on a question. Valid states for the
       flag are "true" and "false".

        One common flag is the "seen" flag. It is normally only set if a
       user already seen a question. Typically, frontends only display
       questions to users if they have the seen flag set to "false".
       Sometimes you want the user to see a question again -- in these cases
       you can set the seen flag to false to force the frontend to redisplay
       it.

     *  METAGET question field

        This returns the value of any field of a question (the description,
       for example).

     *  REGISTER template question

        This creates a new question that is bound to a template. By default
       each template has an associated question with the same name. However,
       any number of questions can really be associated with a template, and
       this lets you create more such questions.

     *  UNREGISTER question

        This removes a question from the database.

     *  PURGE

        Call this in your postinst when your package is purged. It removes
       all templates and questions your package has generated.

     ----------------------------------------------------------------------

                       Debian install-time configuration

    Debian has had an excellent packaging system for a long time now. There
   is one thing missing though: a system to handle the configuration of
   packages so we don't have to stop the installation every time a package
   needs some data from the user or wants to show some information.

    We want to make a package which does not break older dpkg's, and we want
   to be able to get the configuration information before the package is
   unpacked. To do this we add two new files, config and templates, to the
   control.tar.gz of a .deb package. Since all installation-software (apt,
   dselect, dpkg) download the package before installing it, we can extract
   this before the package is unpacked.

    The templates file lists the templates for variables that this package
   uses. This is done using the format as used in the example in the section
   on templates.

    The config-file contains a new element, which I call the configmodule.
   This is a program that will determine the configuration before the package
   is unpacked. This means it is run before the preinst, and before the
   package is unpacked! This is done to make sure that we can use the desired
   configuration in the preinst if necessary.

    How does the configmodule get its information? The configmodule needs a
   way to retrieve information from the configuration space, ask the user for
   information if necessary, etc. But we don't want to implement a user
   interface for each package. To solve this we use a separate frontend as
   specified in the section on frontends.
