Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Ian Jackson]

Gioele Barabucci writes ("Bug#907313: Lack of guidelines on purging conffiles 
in stateless packages"):
> Most of the current base programs still work in this way, but I suppose 
> that it is only a matter of time before everything will switch to the 
> stateless paradigm (that is in need of a better name). It is just too 
> convenient. At that point the state "configured" would have little 
> sense, same thing for the "apt --purge remove" command.

I think we are lookng at this slightly from the wrong perspective.

Why might a user (sysadmin) purge a package ?

I can think of the following reasons:

(i) The package was removed but some of its leftover config content
(eg hook files provided in other packages' .d directories - init
scripts are an example) have bugs which are now causing lossage.

(ii) The user intents to reinstall it but wants a completely fresh,
vanilla, default install.

(iii) The package, even when removed, uses an inconvenient amount of
disk space.

(iv) The package's content in /etc is clutter (getting in the way of
listings, tab completion, etckeeper diffs, etc. etc.)

The idea that config supplied by the user by creating files should be
kept, but the config supplied by the user by editing files should be
removed, doesn't seem to fit into the above.

I think it only makes sense in a world where the latter don't exist.
Debian is not such a world, even though some packages are.

(Of course a package foobar must not delete its /etc/foobar.d
directory completely, even on purge, because another package may have
put a dropping into it.)

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Gioele Barabucci]

On 28/08/2018 05:21, Russ Allbery wrote:

Guillem Jover  writes:


I think the distinguishing factor here is whether a pathname is a
configuration file or a configuration fragments directory. So, I'd
say:



  * configuration file → /etc/foo/foo.conf → remove on purge, even if
[…]
  * configuration fragment directory → /etc/foo/foo.d/* → do not remove
[…]


This makes sense to me, and your first case would apply if the package has
a specific number of configuration files that can be shadowed in /etc.


I wonder if stateless programms aren't, in a certain sense, using `/etc` 
as a conf fragment directory. Take, for instance, a systemd directory 
like `/etc/systemd/system/`. It is no conventional ".d" directory, but 
it works exactly like that.


After a bit of reflection, I came to the conclusion that purging is a 
sort of relic of a bygone era where sysadmins configured applications by 
_changing_ files instead of _adding_ files.


Most of the current base programs still work in this way, but I suppose 
that it is only a matter of time before everything will switch to the 
stateless paradigm (that is in need of a better name). It is just too 
convenient. At that point the state "configured" would have little 
sense, same thing for the "apt --purge remove" command.


Regards,

--
Gioele Barabucci 

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Russ Allbery]

Guillem Jover  writes:

> I think the distinguishing factor here is whether a pathname is a
> configuration file or a configuration fragments directory. So, I'd
> say:

>  * configuration file → /etc/foo/foo.conf → remove on purge, even if
>the package did not ship a file there, because this is "virtually"
>owned by the program/package (and I can see in the future these
>being marked as ghost conffiles in the dpkg metadata manifest, for
>example).
>  * configuration fragment directory → /etc/foo/foo.d/* → do not remove
>on purge, unless the package ships or creates these itself. This
>preserves local admin changes, and changes from 3rd party packages.

This makes sense to me, and your first case would apply if the package has
a specific number of configuration files that can be shadowed in /etc.

My subjective impression (and I might be wrong here) is that it's more
common for packages that support this style of configuration to also
support fragment directories, for all the reasons that fragment
directories are a good idea anyway.  So I think this policy would lead to
normally not removing most administrator configuration.  Does that sound
right?

-- 
Russ Allbery (r...@debian.org)   

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Guillem Jover]

On Sun, 2018-08-26 at 12:17:23 -0700, Russ Allbery wrote:
> Gioele Barabucci  writes:
> > For instance, apache (the application) is configured by some stub conf
> > in `/etc/apache` that loads *.conf files from directories such as
> > `/etc/apache2/sites-enabled/`. The real files are usually in
> > `/etc/apache2/sites-available/`.
> 
> > The purge process for the apache (the package) removes the configuration
> > files it has installed and the symlinks it has created but leaves the
> > configuration files written by the sysadmin alone.
> 
> Yeah, this is a very interesting example.  If the administrator puts a
> bunch of local configuration in /etc/apache2/sites-available and related
> directories, those are pretty clearly intended to be configuration files
> of Apache, but should we delete everything in those directories on purge?
> I can imagine someone being *quite* surprised by that.
> 
> Another thing that makes this less obvious is that this mechanism is
> frequently used for cross-package cooperation.  In a sense, everything
> under /etc/apache2 is a configuration file for Apache, but a bunch of
> other packages do install files into that hierarchy (including things that
> don't strongly depend on Apache), so running rm -rf in postrm purge isn't
> necessarily correct.

I think the distinguishing factor here is whether a pathname is a
configuration file or a configuration fragments directory. So, I'd
say:

 * configuration file → /etc/foo/foo.conf → remove on purge, even if
   the package did not ship a file there, because this is "virtually"
   owned by the program/package (and I can see in the future these
   being marked as ghost conffiles in the dpkg metadata manifest, for
   example).
 * configuration fragment directory → /etc/foo/foo.d/* → do not remove
   on purge, unless the package ships or creates these itself. This
   preserves local admin changes, and changes from 3rd party packages.

Thanks,
Guillem

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Holger Levsen]

On Sun, Aug 26, 2018 at 12:11:55PM -0700, Russ Allbery wrote:
> > I don't think this (nice) new paradigm changes anything and do think
> > that all types of configuration files should be treated the same.
> 
> > I'd suggest to close this bug as 'wontfix'.
> 
> If we do want to recommend removing those files, we shouldn't close this
> bug as wontfix, since that's not particularly obvious from Policy at
> present.  Those files are not conffiles of the package, and we're not
> entirely explicit that either they are configuration files of the package
> (I think they are, but it feels useful to clearly say this) or that all
> configuration files should be removed by the postrm script on purge (we
> sort of say this, but not as clearly as we could, at least in a quick
> check).

agreed. thanks for pointing this out.


-- 
cheers,
Holger

---
holger@(debian|reproducible-builds).org


signature.asc
Description: PGP signature

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Russ Allbery]

Gioele Barabucci  writes:

> For instance, apache (the application) is configured by some stub conf
> in `/etc/apache` that loads *.conf files from directories such as
> `/etc/apache2/sites-enabled/`. The real files are usually in
> `/etc/apache2/sites-available/`.

> The purge process for the apache (the package) removes the configuration
> files it has installed and the symlinks it has created but leaves the
> configuration files written by the sysadmin alone.

Yeah, this is a very interesting example.  If the administrator puts a
bunch of local configuration in /etc/apache2/sites-available and related
directories, those are pretty clearly intended to be configuration files
of Apache, but should we delete everything in those directories on purge?
I can imagine someone being *quite* surprised by that.

Another thing that makes this less obvious is that this mechanism is
frequently used for cross-package cooperation.  In a sense, everything
under /etc/apache2 is a configuration file for Apache, but a bunch of
other packages do install files into that hierarchy (including things that
don't strongly depend on Apache), so running rm -rf in postrm purge isn't
necessarily correct.

-- 
Russ Allbery (r...@debian.org)   

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Russ Allbery]

Julien Cristau  writes:

> Second, policy 10.7.3 seems pretty clear to me that purge ought to
> remove the package's configuration files, whether they're conffiles or
> not.

I just looked at this, and I think that's the *intent*, but I don't think
we succeed in being quite as clear as we might have intended.  The key
sentence seems to be this one:

configuration files must be preserved when the package is removed, and
only deleted when the package is purged.

That doesn't quite say that configuration files must be deleted when the
package is purged, only that they *shouldn't* be deleted when the package
*isn't* purged.

-- 
Russ Allbery (r...@debian.org)   

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Russ Allbery]

Holger Levsen  writes:

> I don't think this (nice) new paradigm changes anything and do think
> that all types of configuration files should be treated the same.

> I'd suggest to close this bug as 'wontfix'.

If we do want to recommend removing those files, we shouldn't close this
bug as wontfix, since that's not particularly obvious from Policy at
present.  Those files are not conffiles of the package, and we're not
entirely explicit that either they are configuration files of the package
(I think they are, but it feels useful to clearly say this) or that all
configuration files should be removed by the postrm script on purge (we
sort of say this, but not as clearly as we could, at least in a quick
check).

-- 
Russ Allbery (r...@debian.org)   

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Gioele Barabucci]

On 26/08/2018 18:37, Julien Cristau wrote:

On Sun, Aug 26, 2018 at 12:41:48 +0200, Gioele Barabucci wrote:

the policy lacks guidelines on how to treat user-provided configuration
files during configuration purging in packages for programs that follow the
"stateless" paradigm (default in `/usr`, override in `/etc`). Should they be
removed? Should they be kept?


Second, policy 10.7.3 seems pretty clear to me that purge ought to
remove the package's configuration files, whether they're conffiles or
not.


The question is: what to do with the configuration files of the 
_application_ that has not been installed by the _package_?


For instance, apache (the application) is configured by some stub conf 
in `/etc/apache` that loads *.conf files from directories such as 
`/etc/apache2/sites-enabled/`. The real files are usually in 
`/etc/apache2/sites-available/`.


The purge process for the apache (the package) removes the configuration 
files it has installed and the symlinks it has created but leaves the 
configuration files written by the sysadmin alone.


That seems sensible to me, at least in the cases where there is a clear 
cut between package-provided configuration files and user-provided 
configuration files for the same application.


Shouldn't it be documented?

Regards,

--
Gioele Barabucci 

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Julien Cristau]

On Sun, Aug 26, 2018 at 12:41:48 +0200, Gioele Barabucci wrote:

> Package: debian-policy
> 
> Hello,
> 
> the policy lacks guidelines on how to treat user-provided configuration
> files during configuration purging in packages for programs that follow the
> "stateless" paradigm (default in `/usr`, override in `/etc`). Should they be
> removed? Should they be kept?
> 
First, there seems to be some confusion on your part, the bug title says
"conffiles" and the above seems to refer to non-conffile configuration
files.

Second, policy 10.7.3 seems pretty clear to me that purge ought to
remove the package's configuration files, whether they're conffiles or
not.

Cheers,
Julien

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Sean Whitton]

Hello,

On Sun 26 Aug 2018 at 12:41PM +0200, Gioele Barabucci wrote:

> For example, apticron (the program) has recently switched to a stateless
> configuration. The default configuration is shipped in
> `/usr/lib/apticron/apticron.conf` and the local configuration is written
> by the sysadmin (if they want) in `/etc/apticron.conf`. At the same time
> the maintscript of apticron (the package) says
>
>
>  purge)
>  [...]
>  if [ -d /etc/apticron ] ; then
>  rm -rf /etc/apticron || true
>  fi
>  [...]
>
> That means that purging the package will remove also all the
> configuration files provided by the sysadmin. Should this be the
> normality or should this be avoided?
>
> When there was only one configuration file for both the default and
> sysadmin's settings this could had been a sensible way to purge the
> package. With these new stateless packages I'm not so sure that removing
> user-created configuration files makes sense. They are purely
> user-provided data and will never conflict with future
> (re-)installations of the same package.

Thank you for raising this.

An important factor, since we're talking about the deletion of files, is
what local administrators will expect to happen.  It's difficult to say!
On the one hand, people are used to the idea that purging a Debian
package nukes its configuration in /etc.  On the other hand, people are
used to the idea that purging a Debian package does not touch files the
package did not install.  I am really not sure.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Holger Levsen]

On Sun, Aug 26, 2018 at 12:41:48PM +0200, Gioele Barabucci wrote:
> the policy lacks guidelines on how to treat user-provided configuration
> files during configuration purging in packages for programs that follow the
> "stateless" paradigm (default in `/usr`, override in `/etc`).

btw, I think the term 'stateless' is a misnomer here, clearly there's
state being kept in /etc.

then, policy says what to do with configuration files when a package is
purged: they should be removed. :-D

> That means that purging the package will remove also all the configuration
> files provided by the sysadmin. Should this be the normality or should this
> be avoided?

yes, that should be the normality and this is how purge worked since
basically forever, and I don't see a new compelling reason to change
this. 

If you want to keep your configuration files/modifications, don't purge
a package. And do backups.

> Could the policy be more explicit on what maintscripts are supposed to do in
> packages for stateless programs?

I don't think this (nice) new paradigm changes anything and do think
that all types of configuration files should be treated the same.

I'd suggest to close this bug as 'wontfix'.


-- 
cheers,
Holger

---
holger@(debian|reproducible-builds).org


signature.asc
Description: PGP signature

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Gioele Barabucci]

Package: debian-policy

Hello,

the policy lacks guidelines on how to treat user-provided configuration 
files during configuration purging in packages for programs that follow 
the "stateless" paradigm (default in `/usr`, override in `/etc`). Should 
they be removed? Should they be kept?


For example, apticron (the program) has recently switched to a stateless 
configuration. The default configuration is shipped in 
`/usr/lib/apticron/apticron.conf` and the local configuration is written 
by the sysadmin (if they want) in `/etc/apticron.conf`. At the same time 
the maintscript of apticron (the package) says



purge)
[...]
if [ -d /etc/apticron ] ; then
rm -rf /etc/apticron || true
fi
[...]

That means that purging the package will remove also all the 
configuration files provided by the sysadmin. Should this be the 
normality or should this be avoided?


When there was only one configuration file for both the default and 
sysadmin's settings this could had been a sensible way to purge the 
package. With these new stateless packages I'm not so sure that removing 
user-created configuration files makes sense. They are purely 
user-provided data and will never conflict with future 
(re-)installations of the same package.


The policy [1] says nothing explicit on this subject, only

> conffiles and any backup files (~-files, #*# files, %-files, 
.dpkg-{old,new,tmp}, etc.) are removed.


In a message to debian-policy [2], Russ Allbery provided a precedent:
optional files in `/etc/default` haven't historically been removed. (He 
is not convinced, however, that this is correct solution.)


Could the policy be more explicit on what maintscripts are supposed to 
do in packages for stateless programs?


Regards,

[1] https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#id6
[2] https://lists.debian.org/debian-policy/2018/08/msg00182.html

Regards,

--
Gioele Barabucci