Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over
Wookey writes ("Re: Bug#557730: /etc/{protocols,network,services} not schroot's
to scribble over"):
> On 2023-02-14 16:02 +, Ian Jackson wrote:
> > The fake netbase.deb could be contained within schroot.deb, in
> > /usr/share, so schroot wouldn't need to gain runtime build-deps on
> > dpkg tooling.
>
> Except that it has to build this package live in order to contain the
> /etc/protocols and /etc/services files from the host
> environment. Having these default files with standard contents in a
> pre-built .deb is pointless, isn't it?
No, I thinko it's fine. They mostly contain standardised constants.
schroot could build-depend on netbase and copy the files from there,
or just have fixed copies which would be updated manually once a
release or something.
These files, especially services, do change, but it is rare for
low-level basic things (or build systems) to actually depend on the
new entries.
> > > Whilst having the passwd database reflected in the chroot is
> > > incredibly useful it's not clear how often the services and protocols
> > > are needed, but I assume people do find that functionality useful.
> >
> > I had a package that failed its build-time tests due to lack of
> > /etc/protocols. The missing build-dep was detected in the buildds,
> > because my own local sid build chroot has netbase installed, precisely
> > because of this bug.
>
> Right, which gets back to having a proper minimal environment used by
> sbuild to do a clean build. I have that (and it doesn't mount home,
> using the 'sbuild' profile). I use it once things are working
> reasonably well to get at least one clean build before uploading. This
> bug is a problem in the 'less clean, but more useful' 'default' chroot
> environment which is best for diagnostic work and builds of various
> sorts where some file persistence (of user files) is needed.
IME having a somewhat-dirty build environment is both convenient and
not a problem.
For example, my own sid chroot which *does* mount /home and import my
passwd and so on, reproducibily-builds the same .debs as the buildds
even for src:xen, whose build system is hardly straightforward.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over
On 2023-02-14 16:02 +, Ian Jackson wrote:
> Wookey writes ("Bug#557730: /etc/{protocols,network,services} not schroot's
> to scribble over"):
> > 2) netbase could be installed in base chroots then the problem would not
> > arise (or only arise once).
> >
> A downside is that missing build-dependencies on netbase would no
> longer be detected. One alternative would be to declare netbase
> build-essential. TBH I'm not sure why that hasn't been done already.
Good point. And related to recent debian-devel discussion about bare
build chroots actually being bare so that missing build-deps are
indeed discovered. I don't think that just adding more things to
'essential' is actually helpful here. netbase is not essential,
especially for builds that explicitly mustn't use the (external)
network.
To be fair schroot has a 'config=buildd' setup where /etc/protocols
and /etc/services are not copied over (and configs 'minimal', 'sbuild'
and 'mini-buildd'). The problematic situation is 'config=default' (and
'desktop') where they are. But I use the 'default' setup a lot because
it mounts /home as well and that's hugely helpful for doing various
sort of work, as opposed to a totally-clean sbuild build. And I think
it may be the default for sbuild-createchroot, but I could be wrong
about that.
For nearly everything I do a config without /etc{protocols|services}
but with /home would work great, but none of the supplied configs:
minimal, buildd, mini-buildd, sbuild, default, desktop do that. Do we
really need a 7th config (and if so what should it be
called?). Obviously I can just make one, but the fact that problem
affects people so often with the 'default' config seems to me to be a
problem we should try to fix.
> I have a 4th option:
>
> scroot could create this file by installing and then removing (but not
> purging) a fake netbase.deb ("Version: 0~~). Then, when the new
> netbase appears, the usual conffile mechanism would DTRT, since the
> file would not have been "locally created" (or indeed "locally
> modified").
That is a neat idea.
> The fake netbase.deb could be contained within schroot.deb, in
> /usr/share, so schroot wouldn't need to gain runtime build-deps on
> dpkg tooling.
Except that it has to build this package live in order to contain the
/etc/protocols and /etc/services files from the host
environment. Having these default files with standard contents in a
pre-built .deb is pointless, isn't it?
> > Whilst having the passwd database reflected in the chroot is
> > incredibly useful it's not clear how often the services and protocols
> > are needed, but I assume people do find that functionality useful.
>
> I had a package that failed its build-time tests due to lack of
> /etc/protocols. The missing build-dep was detected in the buildds,
> because my own local sid build chroot has netbase installed, precisely
> because of this bug.
Right, which gets back to having a proper minimal environment used by
sbuild to do a clean build. I have that (and it doesn't mount home,
using the 'sbuild' profile). I use it once things are working
reasonably well to get at least one clean build before uploading. This
bug is a problem in the 'less clean, but more useful' 'default' chroot
environment which is best for diagnostic work and builds of various
sorts where some file persistence (of user files) is needed.
Wookey
--
Principal hats: Debian, Wookware, ARM
http://wookware.org/
signature.asc
Description: PGP signature
Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over
Wookey writes ("Bug#557730: /etc/{protocols,network,services} not schroot's to
scribble over"):
> 2) netbase could be installed in base chroots then the problem would not
> arise (or only arise once).
>
> The problem here is that chroots can be made by many tools, and the
> usual tools (debootstrap and mmdebstrap) do not put netbase in by
> default. It would be very easy to make sbuild-createchroot just add
> --include=netbase to the invocation, and I'm not sure there is any
> real downside to doing that? Documenting the reason for including this
> package so that people using other tools to make chroots know it's a
> good idea would also be easy and helpful.
A downside is that missing build-dependencies on netbase would no
longer be detected. One alternative would be to declare netbase
build-essential. TBH I'm not sure why that hasn't been done already.
I have a 4th option:
scroot could create this file by installing and then removing (but not
purging) a fake netbase.deb ("Version: 0~~). Then, when the new
netbase appears, the usual conffile mechanism would DTRT, since the
file would not have been "locally created" (or indeed "locally
modified").
The fake netbase.deb could be contained within schroot.deb, in
/usr/share, so schroot wouldn't need to gain runtime build-deps on
dpkg tooling.
> Whilst having the passwd database reflected in the chroot is
> incredibly useful it's not clear how often the services and protocols
> are needed, but I assume people do find that functionality useful.
I had a package that failed its build-time tests due to lack of
/etc/protocols. The missing build-dep was detected in the buildds,
because my own local sid build chroot has netbase installed, precisely
because of this bug.
Ian.
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over
Source: schroot Version: 1.6.13-3 Followup-For: Bug #557730 Just discovered this bug from 2009. This problem has been annoying me regularly since about then, and I finally got round to working out what was actually going on, which led me here. The primary practical issue is that builds in the chroot are quite likely to bring in netbase, and that package contains /etc/protocols and /etc/services so when dpkg finds that they are already present in the chroot it stops with: Setting up netbase (6.4) ... Configuration file '/etc/protocols' ==> File on system created by you or by a script. ==> File also in package provided by package maintainer. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** protocols (Y/I/N/O/D/Z) [default=N] ? So I often find that a build has just got stuck at the beginning unless I inject y\ny\n. This has caused endless hassle over the years, and it's a bit sad to see it's not been fixed in 13 years. I always assumed that someone would sort this out at some point and the hassle would go away. There are various ways we could deal with this. Sbuild-createchroot could set 1) Dpkg::Options::force=--force-confnew in its chroots. Are there reasons you might not want to set this in general? People would expect conf changes they made in their chroots to be preserved just the way they are in a normal system. So this seems like it would be intrusive. 2) netbase could be installed in base chroots then the problem would not arise (or only arise once). The problem here is that chroots can be made by many tools, and the usual tools (debootstrap and mmdebstrap) do not put netbase in by default. It would be very easy to make sbuild-createchroot just add --include=netbase to the invocation, and I'm not sure there is any real downside to doing that? Documenting the reason for including this package so that people using other tools to make chroots know it's a good idea would also be easy and helpful. 3) schroot could just not copy those files over. Whilst having the passwd database reflected in the chroot is incredibly useful it's not clear how often the services and protocols are needed, but I assume people do find that functionality useful. The actual issue here is that schroot is copying them over even when they don't exist in the chroot, then the thing that is supposed to be installing them gets tripped up (correctly) by dpkg's checks. So a simple fix that keeps the functionality when it's useful, but stops this breakage, would be to only copy over the files in the nssdatabases list _if they are already present in the chroot_. Possibly that should apply to all the files in the list? I'll see if I can come up with a patch to do that.
Bug#557730: /etc/{protocols,network,services} not schroot's to scribble over
Package: schroot Version: 1.3.1-1 Severity: normal /etc/schroot/setup.d/20nssdatabases unconditionally copies the databases listed in /etc/schroot/nssdatabases-defaults into the chroot. However, /etc/protocols, /etc/network, and /etc/services are all owned by netbase, which isn't installed in build chroots. This means that if a package build-deps on netbase, dpkg will present a conffile conflict for those three files, which is highly undesirable in a non-interactive builder. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#557730: [buildd-tools-devel] Bug#557730: /etc/{protocols, network, services} not schroot's to scribble over
On Mon, Nov 23, 2009 at 07:19:46PM -0500, Evan Broder wrote: /etc/schroot/setup.d/20nssdatabases unconditionally copies the databases listed in /etc/schroot/nssdatabases-defaults into the chroot. However, /etc/protocols, /etc/network, and /etc/services are all owned by netbase, which isn't installed in build chroots. OK. I would suggest removing these from nssdatabases-defaults for the time being if this is causing problems, as a workaround. This means that if a package build-deps on netbase, dpkg will present a conffile conflict for those three files, which is highly undesirable in a non-interactive builder. I wonder if (for buildd chroots) we should be running dpkg with --force-confnew to unconditionally force replacement with new conffiles? If this is configurable via APT such as Dpkg::Options::force=--force-confnew then maybe we should be setting this by default (this applies more generally than just this specific case). We could remove these files from nssdatabases-defaults to avoid this issue. However, the idea is that all the NSS databases are slaved to the host and automated updating is wanted in almost all cases. We were considering creating a separate set of buildd defaults, which would still allow the current default behaviour on non- buildd systems, while allowing the defaults to be restricted on buildd setups. Does anyone else have any further comments? Thanks for reporting this! Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
