Re: Debian Configuration Packaging System
On 11-Apr-08, 16:07 (CDT), Steve Langasek [EMAIL PROTECTED] wrote: I humbly offer up samba-common in unstable as an example of a package that DTRT with ucf to manage a config file. Is it just me, or does everyone get prompted twice about smb.conf on samba/samba-common upgrades? (I've looked through the various install and config scripts, but not closely enough to figure out why...) Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Sat April 12 2008 10:48:01 Steve Greenland wrote: Is it just me, or does everyone get prompted twice about smb.conf on samba/samba-common upgrades? (I've looked through the various install and config scripts, but not closely enough to figure out why...) It happened here too. I haven't figured out why. --Mike Bird -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Sat, Apr 12, 2008 at 12:48:01PM -0500, Steve Greenland wrote: On 11-Apr-08, 16:07 (CDT), Steve Langasek [EMAIL PROTECTED] wrote: I humbly offer up samba-common in unstable as an example of a package that DTRT with ucf to manage a config file. Is it just me, or does everyone get prompted twice about smb.conf on samba/samba-common upgrades? (I've looked through the various install and config scripts, but not closely enough to figure out why...) On your first upgrade to the ucf-based samba-common, you may get two prompts - first to reconcile the differences with your smb.conf which doesn't match the known old template, and then to merge in the smb.conf from the current package. There's room for improvement on the first pass by doing a better job of matching up the user's config with past smb.conf templates (which mostly means doing some scary package archaeology), but the history of this config file spans more than a decade, so there will definitely be users for whom we can't help but prompt twice on the first upgrade. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ [EMAIL PROTECTED] [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
(oh, hey, look at that, someone drawing my attention back to this thread that I meant to follow up to.) On Mon, Feb 25, 2008 at 10:43:55AM -0800, Russ Allbery wrote: Josselin Mouette [EMAIL PROTECTED] writes: On lun, 2008-02-25 at 10:23 -0800, Russ Allbery wrote: ucf, from its DESCRIPTION in its man page, seems to handle the case of shipping a configuration file upstream that may also be locally modified, but I don't see where it handles merging in the results of debconf prompting. Am I just missing something? ucf can easily be fed with the debconf-generated file. When told to with the proper option, it will even do a three-way merge of the proposed (maintainer + debconf) changes to the locally modified file. Oh! Okay, that was the piece that I was missing. That's a better solution than what I have now. Thanks, I'll try to find time to look at this in the not-too-distant future. I humbly offer up samba-common in unstable as an example of a package that DTRT with ucf to manage a config file. I wish the three-way merge support in ucf was better, but for all that, the status quo is still pretty darn slick. :) -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ [EMAIL PROTECTED] [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Anders Kaseorg and I created a system of CDBS modules (which we've tentatively packaged as the config-package-dev package) for creating Debian configuration packages. I'm designing a hosting service at Penn State which involves configuring a big pile of Debian machines (https://weblion.psu.edu/trac/weblion/wiki/WebLionHosting ). I was midway through inventing/accumulating something very similar when I discovered your framework, and I now plan to use it for almost all our config packages. Bravo! Please count my vote toward polishing dpkg's divert behavior, adding config hooks, or whatever the best solution turns out to be for officially supporting config packages. This is too useful a feature to leave teetering on the edge of acceptance! Erik Rose Core Developer and General Incorrigible The WebLion Group Pennsylvania State University -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Timothy G Abbott writes (Re: Debian Configuration Packaging System): So, our goal is to provide our users with the same opportunities to override our configuration defaults as they would have had if Debian had been providing them instead. Using the Debian packaging system for this configuration is a good way to achieve this. The Debian packaging system and configuration setups are not designed to provide more than the one level of override for the contents of system files. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Thu, 28 Feb 2008, Ian Jackson wrote: Timothy G Abbott writes (Re: Debian Configuration Packaging System): So, our goal is to provide our users with the same opportunities to override our configuration defaults as they would have had if Debian had been providing them instead. Using the Debian packaging system for this configuration is a good way to achieve this. The Debian packaging system and configuration setups are not designed to provide more than the one level of override for the contents of system files. Configuration packages made using our system distribute only conffiles. End-users of the configuration packages can modify (e.g.) /etc/ldap/ldap.conf by changing /etc/ldap/ldap.conf.debathena, which is a normal Debian conffile (and is what users will change if they try to modify /etc/ldap/ldap.conf using any editor). While this may not have been something envisioned by the designers of the Debian packaging system, our system's use of standard pieces of the Debian packaging system certainly supports this. -Tim Abbott -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Sun, 24 Feb 2008 17:15:48 -0800, Russ Allbery [EMAIL PROTECTED] wrote: Timothy G Abbott [EMAIL PROTECTED] writes: Anders Kaseorg and I created a system of CDBS modules (which we've tentatively packaged as the config-package-dev package) for creating Debian configuration packages. By configuration packages, we mean packages that configure an existing Debian system by applying dpkg-divert to configuration files. Our configuration package system makes the process of creating configuration packages efficient. It's generally accepted wisdom that dpkg-divert doesn't work properly with configuration files and isn't safe. Which is actually a big pity since I have been looking for a method to roll out _configuration_ to a lot of systems by means of the already in-place, apt-based software distribution system. I have once experimented with using ucf to force in new configuration, but that has shown to have interesting quirks. A different method would be to background a job which waits until no apt or dpkg jobs are running and more and does the conffile change then, but that's a very bad hack. We need a documented way to do those things. Greetings Marc -- -- !! No courtesy copies, please !! - Marc Haber |Questions are the | Mailadresse im Header Mannheim, Germany | Beginning of Wisdom | http://www.zugschlus.de/ Nordisch by Nature | Lt. Worf, TNG Rightful Heir | Fon: *49 621 72739834
Re: Debian Configuration Packaging System
Timothy G Abbott tabbott at MIT.EDU writes: On Mon, 25 Feb 2008, Frank Küster wrote: Uh, you can dpkg-divert conffiles, but not generally configuration files, since many won't even be known to dpkg. I must admit I'm a bit sceptical about a proposal on configuration, written by someone who lets this important distinction slip by... No, I really meant configuration files. While our system certainly applies to all conffiles, it also applies to various other classes of files: But in these cases you can't use dpkg-divert. 3) Scripts that are not marked as conffiles but which cannot be configured in any way other than by modifying the script. If those scripts live below /etc, they definitely should be marked as conffiles. If they live elsewhere, no package should edit them. I probably should have said this explicitly earlier, but our system is currently only an 80% solution, because it cannot override any package's configuration file handling system that does not preserve manual changes. Such a package has a RC bug anyway. It turns out that these are fairly rare, and can be handled with some annoyance (e.g. releasing a new version of our configuration package whenever a new release of such a package comes out, so that the configuration package wins). Of file a bug... So, yes, our system uses both symlinks and dpkg-divert. Ah, I understand. I think here you have a much larger problem than with the small number of RC-buggy packages that don't keep manual changes: Larger because I fear there are more packages with such problems, because less people are aware that it is a problem, and maybe even because there might be some debate whether respecting a symlink state actually is required by policy. One idea [...] would be for dpkg to run postinst scripts in a special environment that rewrites filenames according to the diversion database I would rather contemplate a new interface for configuration packages, something like a hook which has to be executed by each postinst script and allows the configuration package to bring its settings into effect. If you want to be able to remove configuration packages, the original file needs to be preserved somehow, but the current dpkg-divert is not sufficient for this: Consider someone installing the configuration package in stable+0, then upgrading to stable+1 and keeping it, and then upgrading to stable+2 and subsequently removing the configuration package. Any special code in the maintainer scripts which dealt with incompatible conf(iguration )file changes in the stable-to-stable+1 upgrade is likely to have been removed by now, so the old, incompatible, file from stable+0 will be used after the removal, and the package is in a broken state. Regards, Frank -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Timothy G Abbott wrote: There are really two problems with debconf in our system. The first is that debconf asks questions which our configuration package system will override. Using 'DEBCONF_PRIORITY=critical apt-get install' limits them, but some packages we configure prompt for information even with critical DEBCONF_PRIORITY (is this a bug?). No. If you want to avoid all prompts you should use the noninteractive frontend. Cheers, FJP -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Tue, 26 Feb 2008, Frank K??ster wrote: [...] 3) Scripts that are not marked as conffiles but which cannot be configured in any way other than by modifying the script. If those scripts live below /etc, they definitely should be marked as conffiles. If they live elsewhere, no package should edit them. I think it's totally reasonable for the MIT site configuration packages to (e.g.) divert chsh to replace it with a wrapper that supports handling both inherently local user accounts (like root or postfix) by calling chsh.debathena-orig and network accounts (through some other mechanism). While some might argue that instead MIT should create a modified version of the passwd package to make this change, I think our solution is substantially cleaner. It's important to keep in mind is that while the system for creating configuration packages should be part of Debian, the configuration packages themselves are not distributed as part of Debian, but should instead be considered as modifications to Debian. It makes sense for them to be able to change some standard programs in a way that only makes sense for that particular site, but doesn't break other uses of those programs, as well as changing configuration file defaults. [...] One idea [...] would be for dpkg to run postinst scripts in a special environment that rewrites filenames according to the diversion database I would rather contemplate a new interface for configuration packages, something like a hook which has to be executed by each postinst script and allows the configuration package to bring its settings into effect. If you want to be able Would deploying this new interface require modifying every Debian package's postinst scripts to add support for this new hook? I'm not enthusiastic about the deployment costs of that kind of sweeping change if there are more localized changes to Debian that would work. to remove configuration packages, the original file needs to be preserved somehow, but the current dpkg-divert is not sufficient for this: Consider someone installing the configuration package in stable+0, then upgrading to stable+1 and keeping it, and then upgrading to stable+2 and subsequently removing the configuration package. Any special code in the maintainer scripts which dealt with incompatible conf(iguration )file changes in the stable-to-stable+1 upgrade is likely to have been removed by now, so the old, incompatible, file from stable+0 will be used after the removal, and the package is in a broken state. That's certainly a potential problem with our current system. The filename rewriting idea does not have this problem. During the upgrades, the maintainer scripts for the package would update /etc/ldap/ldap.conf.debathena-orig (rather than the running version), and when the configuration package is uninstalled, /etc/ldap/ldap.conf.debathena-orig will be moved back to /etc/ldap/ldap.conf, leaving the package in the state it would have been in had the configuration package never been installed in the first place. The filename rewriting plan is really a natural extension to the current dpkg-divert behavior; rather than just unpacking files in locations as determined by the diversion database, all actions by the packaging system would have their filenames transformed according to the diversion database. I also can't think of anything that doing this would break -- its primary cost would be the added complexity in dpkg (and perhaps some performance penalty, depending on implementation). But it's also no silver bullet. There are packages that want to restart their daemon after changing their configuration file, and it's unclear to me how to prevent packages from restarting their daemon (or regerating their database, etc.) in the filename rewriting environment. -Tim Abbott
Re: Debian Configuration Packaging System
Timothy G Abbott writes (Debian Configuration Packaging System): applying dpkg-divert to configuration files. Yikes. Tim Abbott writes (Re: Debian Configuration Packaging System): I'll note that we wrap our dpkg-divert calls with a bunch of error-handling code that we found quite important for correctly recovering from people hitting ^C in the middle of installation (see http://debathena/config-packages/code/config-package-dev-4.2/divert.sh.in for the code). Earlier iterations that did not do this were plagued with problems whenever there were errors in installation. Yikes. I think this is a bad idea. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Tue, 26 Feb 2008, Ian Jackson wrote: Tim Abbott writes (Re: Debian Configuration Packaging System): applying dpkg-divert to configuration files. Yikes. At the moment, there are no choices for doing site configuration that are general enough to change arbitrary configuration options and don't destroy whatever configuration state the machine had before configuring it (as well as suffering from a version of every bad interaction with existing maintainer scripts that our system has). All of the important problems with our dpkg-divert based configuration package system that have been discussed in this forum are problems for any configuration mechanism other than debconf preseeding. Debconf preseeding is unacceptable for site configuration because it is insufficiently general (often things need to be configured that are not set up as debconf options). Any system that is not based on debconf pre-seeding (or maintaining a fork of all the relevant packages containing the relevant configuration files, which is unmaintainable) will have all the problems with maintainer scripts that our system has. Further, any configuration system that does not use dpkg-divert will have to fight with dpkg over control of conffiles (a problem that our system does not have). It turns out that our system also works for almost all the other configuration files that we've wanted to change (in particular, those whose packages don't overwrite manual changes with answers from the debconf database on package upgrades, something the Debian Policy Manual forbids anyway). So, anything attempting to achieve the functionality we want has to use dpkg-divert, and I think the best way to remove the problems with dealing with configuration files that are not conffiles should probably involve making maintainer scripts that do overwrite existing configuration files follow diversions when doing so (though one mechanism or another). Tim Abbott writes (Re: Debian Configuration Packaging System): I'll note that we wrap our dpkg-divert calls with a bunch of error-handling code that we found quite important for correctly recovering from people hitting ^C in the middle of installation (see http://debathena.mit.edu/config-packages/code/config-package-dev-4.2/divert.sh.in for the code). Earlier iterations that did not do this were plagued with problems whenever there were errors in installation. Yikes. Those earlier iterations date to 2004 and were never deployed beyond a testing environment. The particular shell code we're using now has been basically unchanged since we wrote it back in Summer 2006, and has not been a problem for us since. -Tim Abbott -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Timothy G Abbott writes (Re: Debian Configuration Packaging System): All of the important problems with our dpkg-divert based configuration package system that have been discussed in this forum are problems for any configuration mechanism other than debconf preseeding. Debconf preseeding is unacceptable for site configuration because it is insufficiently general (often things need to be configured that are not set up as debconf options). I think a better approach is just to have a package which writes stuff over the configuration files without diverting them. Such a package is putting itself in the position of the system administrator, which is presumably what you intend. That's not appropriate for a Debian package but for a site-specific configuration setting system it's OK I think. You do have to deal with conffile prompts and other kinds of things that happen when software (package maintainer scripts, and dpkg) spot you've modified the files. But if you do all of your installations in a noninteractive mode (with a noninteractive debconf frontend and with dpkg configured not to prompt about conffiles) you get predictable behaviour which you can override as you like. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Wed, 27 Feb 2008, Ian Jackson wrote: Timothy G Abbott writes (Re: Debian Configuration Packaging System): All of the important problems with our dpkg-divert based configuration package system that have been discussed in this forum are problems for any configuration mechanism other than debconf preseeding. Debconf preseeding is unacceptable for site configuration because it is insufficiently general (often things need to be configured that are not set up as debconf options). I think a better approach is just to have a package which writes stuff over the configuration files without diverting them. Such a package is putting itself in the position of the system administrator, which is presumably what you intend. That's not appropriate for a Debian package but for a site-specific configuration setting system it's OK I think. If we were running a cluster of identical machines, such a system would probably work. But we actually don't intend to be in the position of the system administrator; we just want to provide defaults in a decentralized environment with many different sysadmins. This is a harder setting, and I think our users find it useful that they can run etch or sid or even Ubuntu as needed to get their work done and benefit from the site defaults. Our current deployment is on a few hundred machines, primarily personal machines owned by individuals who want to be able to access MIT services, but who do their own system administration. We offer various metapackages with different levels of login integration, and we support all Debian and Ubuntu releases from sarge until present from a single set of source packages, a valuable feature of the configuration package creator interface of our system. So, our goal is to provide our users with the same opportunities to override our configuration defaults as they would have had if Debian had been providing them instead. Using the Debian packaging system for this configuration is a good way to achieve this. You do have to deal with conffile prompts and other kinds of things that happen when software (package maintainer scripts, and dpkg) spot you've modified the files. But if you do all of your installations in a noninteractive mode (with a noninteractive debconf frontend and with dpkg configured not to prompt about conffiles) you get predictable behaviour which you can override as you like. We've found it pretty useful to be able to install our configuration on an existing system. -Tim Abbott -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Timothy G Abbott tabbott at MIT.EDU writes: Anders Kaseorg and I created a system of CDBS modules (which we've tentatively packaged as the config-package-dev package) for creating Debian configuration packages. By configuration packages, we mean packages that configure an existing Debian system by applying dpkg-divert to configuration files. Our configuration package system makes the process of creating configuration packages efficient. Uh, you can dpkg-divert conffiles, but not generally configuration files, since many won't even be known to dpkg. I must admit I'm a bit sceptical about a proposal on configuration, written by someone who lets this important distinction slip by... Later on you wrote something about symlinks, do you use them, too? Here's one more complication: While I think that preserve local changes also includes the symlink-or-not state of a file and thus must be respected by maintainer scripts, I do not think that this is usually done. In particular, every configuration file handling that was first written for sarge (where sed -i wouldn't work) probably used tempfiles and mv, and I have not seen a single case where the script checked whether it was dealing with a symlink. [...] Since this package is adding a new feature to Debian itself, we think our system should be discussed before we submit an ITP bug. There are some changes to Debian that would enhance the effectiveness of this system, (such as having all packages include md5sums and making ucf interact well with dpkg-divert'ed configuration files), which should perhaps be discussed in this context as well. It's not just ucf, it's also perfectly possible that a maintainer script edits a configuration file, e.g. after debconf prompting, in order to take over user changes from a related package's configuration file, or similar. How do you deal with that? Regards, Frank -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Le dimanche 24 février 2008 à 19:46 -0800, Russ Allbery a écrit : The ones that are overwritten completely that I'm aware of contain only settings managed by debconf, or (as is the case for krb5-kdc and krb5-admin-server) explicitly ask whether you want to manage the configuration file through debconf or want to manage it manually so that you can set more obscure options. This is really a bad idea, because you always end up modifying a debconf-managed file by hand. The solution to this problem is to use ucf instead. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée
Re: Debian Configuration Packaging System
Josselin Mouette [EMAIL PROTECTED] writes: Le dimanche 24 février 2008 à 19:46 -0800, Russ Allbery a écrit : The ones that are overwritten completely that I'm aware of contain only settings managed by debconf, or (as is the case for krb5-kdc and krb5-admin-server) explicitly ask whether you want to manage the configuration file through debconf or want to manage it manually so that you can set more obscure options. This is really a bad idea, because you always end up modifying a debconf-managed file by hand. The solution to this problem is to use ucf instead. Reading the ucf man page, I don't understand how it would help me for the case of those two packages. But I may be suffering from insufficient imagination. The problem is this: both krb5-kdc and krb5-admin-server have /etc/default files that control various aspects of the startup of the servers, such as whether a krb524d is run and what level of Kerberos v4 support is enabled in the KDC. All of those parameters are prompted for by debconf if you select manage my configuration with Debconf and a corresponding /etc/default file is written out. This lets people get the benefit of translated prompts for the possible options. However, people have also requested an escape hatch to pass arbitrary options to the daemon without having to modify the init script, using the semi-standard DAEMON_ARGS setting in the /etc/default file. ucf, from its DESCRIPTION in its man page, seems to handle the case of shipping a configuration file upstream that may also be locally modified, but I don't see where it handles merging in the results of debconf prompting. Am I just missing something? The approach taken right now is that the user chooses whether the file is debconf-managed or not debconf-managed, and a comment at the top of the file tells the user to pick. Debconf-managed configuration seems to be pretty common in Debian, although I know that some things that are common still aren't good ideas and I'm happy to switch to something better. (There are various bits of debconf cleanup in the krb5 packages that needs to be done, so now's a good time to tell me how the whole thing *should* work so that I can do it all at once.) -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: Debian Configuration Packaging System
Josselin Mouette [EMAIL PROTECTED] writes: On lun, 2008-02-25 at 10:23 -0800, Russ Allbery wrote: ucf, from its DESCRIPTION in its man page, seems to handle the case of shipping a configuration file upstream that may also be locally modified, but I don't see where it handles merging in the results of debconf prompting. Am I just missing something? ucf can easily be fed with the debconf-generated file. When told to with the proper option, it will even do a three-way merge of the proposed (maintainer + debconf) changes to the locally modified file. Oh! Okay, that was the piece that I was missing. That's a better solution than what I have now. Thanks, I'll try to find time to look at this in the not-too-distant future. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On lun, 2008-02-25 at 10:23 -0800, Russ Allbery wrote: The problem is this: both krb5-kdc and krb5-admin-server have /etc/default files that control various aspects of the startup of the servers, such as whether a krb524d is run and what level of Kerberos v4 support is enabled in the KDC. All of those parameters are prompted for by debconf if you select manage my configuration with Debconf and a corresponding /etc/default file is written out. This lets people get the benefit of translated prompts for the possible options. However, people have also requested an escape hatch to pass arbitrary options to the daemon without having to modify the init script, using the semi-standard DAEMON_ARGS setting in the /etc/default file. This is one of the standard ucf use cases. ucf, from its DESCRIPTION in its man page, seems to handle the case of shipping a configuration file upstream that may also be locally modified, but I don't see where it handles merging in the results of debconf prompting. Am I just missing something? ucf can easily be fed with the debconf-generated file. When told to with the proper option, it will even do a three-way merge of the proposed (maintainer + debconf) changes to the locally modified file. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée
Re: Debian Configuration Packaging System
On Mon, 25 Feb 2008, Frank K??ster wrote: Uh, you can dpkg-divert conffiles, but not generally configuration files, since many won't even be known to dpkg. I must admit I'm a bit sceptical about a proposal on configuration, written by someone who lets this important distinction slip by... No, I really meant configuration files. While our system certainly applies to all conffiles, it also applies to various other classes of files: 1) Those that are accidentally not marked as conffiles (e.g. /etc/adduser.conf). 2) Those that are initially created via a system like debconf but that preserve user changes, often because they support other configuration options than those in debconf (e.g. /etc/krb5.conf or /etc/ssh/sshd_config) [the dpkg-divert doesn't do anything here, but the system still works]. 3) Scripts that are not marked as conffiles but which cannot be configured in any way other than by modifying the script. I probably should have said this explicitly earlier, but our system is currently only an 80% solution, because it cannot override any package's configuration file handling system that does not preserve manual changes. It turns out that these are fairly rare, and can be handled with some annoyance (e.g. releasing a new version of our configuration package whenever a new release of such a package comes out, so that the configuration package wins). I would like to see it become a 100% solution, since I think support for easily creating configuration packages would be really valuable for a lot of organizations using Debian and Ubuntu. The primary difficulties are interacting nicely with systems like ucf and debconf; I don't think that it is possible to achieve 100% without some changes to Debian itself. Because of ucf's architecture, it would be easy to make ucf interact nicely with our system (and our system should perhaps use ucf itself, since it seems to be a strict improvement over the just marking things as conffiles). There are really two problems with debconf in our system. The first is that debconf asks questions which our configuration package system will override. Using 'DEBCONF_PRIORITY=critical apt-get install' limits them, but some packages we configure prompt for information even with critical DEBCONF_PRIORITY (is this a bug?). The second problem is that some packages override any manual changes to the relevant configuration file. Because the code to take action based on debconf selections in so decentralized, this second problem is relatively hard. For the second debconf problem, the easiest solution would be to add to our system support for pre-seeding the debconf database for package X and then running the configure script for package X. We have the tools to do this, and it would work, but it would lose a property of our system that I really like: support for uninstalling configuration packages. I would prefer to come up with a solution that does not lose this feature. I can think of several solutions preserving this feature that require changing every package that blindly overwrites user configuration on package upgrades to do some sort of check before doing so; I would love to hear ideas for mechanisms for achieving our goals (see http://debathena.mit.edu/config-packages/#philosophy) without having to change all such packages. Later on you wrote something about symlinks, do you use them, too? Here's one more complication: While I think that preserve local changes also includes the symlink-or-not state of a file and thus must be respected by maintainer scripts, I do not think that this is usually done. In particular, every configuration file handling that was first written for sarge (where sed -i wouldn't work) probably used tempfiles and mv, and I have not seen a single case where the script checked whether it was dealing with a symlink. To be precise, after installing a package that intends to configure /etc/ldap/ldap.conf, the system will be in the following configuration: $ ls -l /etc/ldap/ldap.conf* [irrelevant stat information removed] /etc/ldap/ldap.conf - ldap.conf.debathena /etc/ldap/ldap.conf.debathena /etc/ldap/ldap.conf.debathena-orig $ /usr/sbin/dpkg-divert --list | grep ldap diversion of /etc/ldap/ldap.conf to /etc/ldap/ldap.conf.debathena-orig by debathena-ldap-config So, yes, our system uses both symlinks and dpkg-divert. Simply placing files at /etc/ldap/ldap.conf (rather than using a symlink) doesn't work because it would require running dpkg-divert at preinst rather than postinst, which results in needing to pre-depend the packages whose configuration one is changing, and lots of other problems. It's not just ucf, it's also perfectly possible that a maintainer script edits a configuration file, e.g. after debconf prompting, in order to take over user changes from a related package's configuration file, or similar. How do you deal with that? Indeed, there's almost nothing
Re: Debian Configuration Packaging System
Timothy G Abbott [EMAIL PROTECTED] writes: Anders Kaseorg and I created a system of CDBS modules (which we've tentatively packaged as the config-package-dev package) for creating Debian configuration packages. By configuration packages, we mean packages that configure an existing Debian system by applying dpkg-divert to configuration files. Our configuration package system makes the process of creating configuration packages efficient. It's generally accepted wisdom that dpkg-divert doesn't work properly with configuration files and isn't safe. I admit to have done something similar in the past, but I have noticed odd things that didn't matter for my particular use, but which meant that the support didn't work right. That's likely to be an issue for a general package. Fixing dpkg-divert to work correctly with configuration files (if possible) would probably be a good idea. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
I'll note that we wrap our dpkg-divert calls with a bunch of error-handling code that we found quite important for correctly recovering from people hitting ^C in the middle of installation (see http://debathena/config-packages/code/config-package-dev-4.2/divert.sh.in for the code). Earlier iterations that did not do this were plagued with problems whenever there were errors in installation. We also ran into a few packages which will overwrite configuration files that they manage via debconf, overwriting our symlink every time the relevant package is upgraded. But I think that's a bug in those Debian packages, since the same problem would occur for any manual changes to those configuration files as well (I think in the cases I've seen it is a failure to check whether an upgrade is occuring when generating the configuration file in postinst). What other problems have you experienced? -Tim Abbott On Sun, 24 Feb 2008, Russ Allbery wrote: Timothy G Abbott [EMAIL PROTECTED] writes: Anders Kaseorg and I created a system of CDBS modules (which we've tentatively packaged as the config-package-dev package) for creating Debian configuration packages. By configuration packages, we mean packages that configure an existing Debian system by applying dpkg-divert to configuration files. Our configuration package system makes the process of creating configuration packages efficient. It's generally accepted wisdom that dpkg-divert doesn't work properly with configuration files and isn't safe. I admit to have done something similar in the past, but I have noticed odd things that didn't matter for my particular use, but which meant that the support didn't work right. That's likely to be an issue for a general package. Fixing dpkg-divert to work correctly with configuration files (if possible) would probably be a good idea. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Tim Abbott [EMAIL PROTECTED] writes: We also ran into a few packages which will overwrite configuration files that they manage via debconf, overwriting our symlink every time the relevant package is upgraded. But I think that's a bug in those Debian packages, since the same problem would occur for any manual changes to those configuration files as well (I think in the cases I've seen it is a failure to check whether an upgrade is occuring when generating the configuration file in postinst). Configuration files generated by debconf may not be manually changed without running this risk, including by humans. Generally, this is documented in the file. I have several of those in packages I maintain. This is a pretty widely accepted way of dealing with configuration files, and the right way to modify those files is with debconf pre-seeding rather than by trying to overwrite the file, IMO. I don't agree that this is a bug in the package in the general case, although there may be packages that are more aggressive about overwriting than need to be. What other problems have you experienced? I've seen the diverted configuration file disappear, making it impossible to undo the diversion, and never did track down why that happened. I haven't run into any problems in cases where the diversion is never dropped, though. (But renaming the package that manages the diversions is something that dpkg-divert doesn't deal with at all well.) We're using Puppet to handle this instead of Debian packages, and I'm much happier with that solution. I wouldn't recommend doing what you're doing, but of course you may have improved the tools sufficiently that it's a good idea. :) Also, I know that most of the alternatives involve some central management system, and there are a lot of use cases that don't allow for that. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
On Sun, 24 Feb 2008, Russ Allbery wrote: Tim Abbott [EMAIL PROTECTED] writes: We also ran into a few packages which will overwrite configuration files that they manage via debconf, overwriting our symlink every time the relevant package is upgraded. But I think that's a bug in those Debian packages, since the same problem would occur for any manual changes to those configuration files as well (I think in the cases I've seen it is a failure to check whether an upgrade is occuring when generating the configuration file in postinst). Configuration files generated by debconf may not be manually changed without running this risk, including by humans. Generally, this is documented in the file. I have several of those in packages I maintain. This is a pretty widely accepted way of dealing with configuration files, and the right way to modify those files is with debconf pre-seeding rather than by trying to overwrite the file, IMO. Many such files have configuration options other than those managed by debconf (/etc/krb5.conf would be a good example). For those files, pre-seeding debconf doesn't suffice. What other problems have you experienced? I've seen the diverted configuration file disappear, making it impossible to undo the diversion, and never did track down why that happened. I haven't run into any problems in cases where the diversion is never dropped, though. (But renaming the package that manages the diversions is something that dpkg-divert doesn't deal with at all well.) Renaming a package requires no special effort in our system (the new package and the old package will conflict because they divert the same file, and the old package will remove the diversion on prerm because it knows it is being removed, and then the new package will install the new diversion on postinst). -Tim Abbott -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Debian Configuration Packaging System
Tim Abbott [EMAIL PROTECTED] writes: On Sun, 24 Feb 2008, Russ Allbery wrote: Configuration files generated by debconf may not be manually changed without running this risk, including by humans. Generally, this is documented in the file. I have several of those in packages I maintain. This is a pretty widely accepted way of dealing with configuration files, and the right way to modify those files is with debconf pre-seeding rather than by trying to overwrite the file, IMO. Many such files have configuration options other than those managed by debconf (/etc/krb5.conf would be a good example). For those files, pre-seeding debconf doesn't suffice. And krb5.conf is not a configuration file managed in the way that you describe for exactly that reason. It uses a conservative approach, only installing a file based on debconf prompts if the file doesn't already exist and otherwise trying to do automated updates of only the configured data where applicable. (You can also make the file a symlink to disable all automated modification.) The ones that are overwritten completely that I'm aware of contain only settings managed by debconf, or (as is the case for krb5-kdc and krb5-admin-server) explicitly ask whether you want to manage the configuration file through debconf or want to manage it manually so that you can set more obscure options. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
