Bug#1037946: mmdebstrap: Create /var/lib/dpkg/available for Debian releases <=buster
Hello, Quoting Guillem Jover (2023-06-14 20:29:17) > > I'm quite certain about the version requirement because I did tests using > > debbisect and using the suggestion below I'm able to create chroots for all > > of unstable since 2006-08-10. > > So just to clarify, old and new dpkg in normal conditions works fine > for a long time when the available file is not present (since 1.17.11 as > you point out). The problem here as hinted at below is that apt tries to > set the package selections via «dpkg --set-selections» after having > refreshed the available file via «dpkg --merge-avail» where both these > commands fail if the available file is missing with versions earlier > than 1.20.0. aha, then I tested the wrong thing! I'm now able to verify the correct dpkg version required for dpkg --set-selections without /var/lib/apt/available using: debbisect --depends=dpkg --cache=./cache 20200627T025456Z 20200627T085709Z '! echo | /usr/sbin/chroot "$1" dpkg --set-selections' ... bisection finished successfully last good timestamp: 20200627T025456Z first bad timestamp: 20200627T085709Z only one package differs: dpkg 1.19.7 -> 1.20.1 > > Try running mmdebstrap with > > > > --hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older > > > > which will unconditionally do certain things only necessary for old > > releases. > > > > If you instead use > > > > --hook-dir=/usr/share/mmdebstrap/hooks/maybe-jessie-or-older > > > > some detection code will only run the hook if necessary (in case you do not > > know upfront which release you are creating a chroot for). > > Yeah I mentioned this also to Mika, but also encouraged him to file > this report as I think it might make sense for these hooks to be > adapted to cover up to buster out of the box. > > Initially I thought that perhaps only the variants that end up with > apt installed so that it can work correctly, but given that with older > dpkg versions some of these actions will always fail, then perhaps it > makes sense to still add those unconditionally so that users do not > need to create the file manually if they use the other dpkg actions. > > (Hope this clarifies things! :) yes, thank you for finding and reporting this bug! I'm unsure how to fix it... maybe by introducing a maybe-buster-or-older hook which automatically pulls in the jessie-or-older hook? Thanks! cheers, josch signature.asc Description: signature
Bug#1037946: mmdebstrap: Create /var/lib/dpkg/available for Debian releases <=buster
Hi!
On Wed, 2023-06-14 at 19:09:35 +0200, Johannes Schauer Marin Rodrigues wrote:
> Quoting Michael Prokop (2023-06-14 17:19:37)
> > since version 1.20.0, dpkg no longer creates /var/lib/dpkg/available (see
> > #647911).
>
> I see that the bug talks about 1.20.0 but I've had code creating
> /var/lib/dpkg/available in mmdebstrap for a while and determined that it
> wasn't
> necessary anymore since dpkg 1.17.11. Did you do practical tests or are you
> only basing your analysis on that bug?
>
> I'm quite certain about the version requirement because I did tests using
> debbisect and using the suggestion below I'm able to create chroots for all of
> unstable since 2006-08-10.
So just to clarify, old and new dpkg in normal conditions works fine
for a long time when the available file is not present (since 1.17.11 as
you point out). The problem here as hinted at below is that apt tries to
set the package selections via «dpkg --set-selections» after having
refreshed the available file via «dpkg --merge-avail» where both these
commands fail if the available file is missing with versions earlier
than 1.20.0.
> > But on Debian releases until and including buster, dpkg fails to operate
> > with
> > e.g. `dpkg --set-selections`, if /var/lib/dpkg/available doesn't exist:
> >
> > | The following NEW packages will be installed:
> > | [...]
> > | debconf: delaying package configuration, since apt-utils is not installed
> > | dpkg: error: failed to open package info file '/var/lib/dpkg/available'
> > for reading: No such file or directory
> >
> > My workaround to work around this is to run:
> >
> > chroot "${TARGET}" /usr/lib/dpkg/methods/apt/update /var/lib/dpkg
> >
> > As mmdebstrap seems to rely on dpkg of the host system, it would be
> > nice if mmdebstrap could ensure to create an environment where apt/dpkg of
> > the installed system doesn't stumble upon. :)
>
> this already exists as a hook. :)
> Try running mmdebstrap with
>
> --hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older
>
> which will unconditionally do certain things only necessary for old releases.
>
> If you instead use
>
> --hook-dir=/usr/share/mmdebstrap/hooks/maybe-jessie-or-older
>
> some detection code will only run the hook if necessary (in case you do not
> know upfront which release you are creating a chroot for).
Yeah I mentioned this also to Mika, but also encouraged him to file
this report as I think it might make sense for these hooks to be
adapted to cover up to buster out of the box.
Initially I thought that perhaps only the variants that end up with
apt installed so that it can work correctly, but given that with older
dpkg versions some of these actions will always fail, then perhaps it
makes sense to still add those unconditionally so that users do not
need to create the file manually if they use the other dpkg actions.
(Hope this clarifies things! :)
Thanks,
Guillem
Bug#1037946: mmdebstrap: Create /var/lib/dpkg/available for Debian releases <=buster
Hi,
* Johannes Schauer Marin Rodrigues [Wed Jun 14, 2023 at 07:09:35PM +0200]:
> Quoting Michael Prokop (2023-06-14 17:19:37)
> > since version 1.20.0, dpkg no longer creates /var/lib/dpkg/available (see
> > #647911).
>
> I see that the bug talks about 1.20.0 but I've had code creating
> /var/lib/dpkg/available in mmdebstrap for a while and determined that it
> wasn't
> necessary anymore since dpkg 1.17.11. Did you do practical tests or are you
> only basing your analysis on that bug?
This showed up in an actual environment, where we automatically
deploy systems on a daily base. The deployment of a buster based
system failed as soon as we upgraded the host system (a Grml based
live system) to bookworm, which provides dpkg v1.21.22 and
mmdebstrap v1.3.5-7. (Deploying bullseye where dpkg v1.20.12 exists
works fine, FTR)
> I'm quite certain about the version requirement because I did tests using
> debbisect and using the suggestion below I'm able to create chroots for all of
> unstable since 2006-08-10.
mmdebstrap itself does *not* fail, but the installation of e.g. the
`nullmailer` package inside the target buster environment then fails:
| Error: Execution of '/usr/bin/apt-get -q -y -o
DPkg::Options::=--force-confold install nullmailer' returned 100: Reading
package lists...
| Building dependency tree...
| Reading state information...
| The following packages were automatically installed and are no longer
required:
| libevent-2.1-6 libgnutls-dane0 libunbound8
| Use 'apt autoremove' to remove them.
| The following packages will be REMOVED:
| exim4-base exim4-config exim4-daemon-light
| The following NEW packages will be installed:
| nullmailer
| 0 upgraded, 1 newly installed, 3 to remove and 0 not upgraded.
| Need to get 109 kB of archives.
| After this operation, 3560 kB disk space will be freed.
| [...]
| [master dbda30d] saving uncommitted changes in /etc prior to apt run
| 6 files changed, 115 insertions(+), 575 deletions(-)
| rewrite locale.gen (99%)
| create mode 100644 needrestart/conf.d/ignore.conf
| debconf: delaying package configuration, since apt-utils is not installed
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for
reading: No such file or directory
| Fetched 109 kB in 0s (542 kB/s)
| E: Sub-process dpkg --set-selections returned an error code (2)
| E: Couldn't record the approved state changes as dpkg selection states[0m
(FTR, this is part of an automated puppet run)
I can manually reproduce the problem with:
| # mmdebstrap buster /tmp/buster
| [...]
| # chroot /tmp/buster apt update
| [...]
| # chroot /tmp/buster apt install nullmailer
| [...]
| # chroot /tmp/buster apt purge nullmailer
| [...]
| perl: warning: Falling back to the standard locale ("C").
| locale: Cannot set LC_CTYPE to default locale: No such file or directory
| locale: Cannot set LC_MESSAGES to default locale: No such file or directory
| locale: Cannot set LC_ALL to default locale: No such file or directory
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for
reading: No such file or directory
| E: Sub-process dpkg --set-selections returned an error code (2)
| E: Couldn't record the approved state changes as dpkg selection states
> > But on Debian releases until and including buster, dpkg fails to operate
> > with
> > e.g. `dpkg --set-selections`, if /var/lib/dpkg/available doesn't exist:
> >
> > | The following NEW packages will be installed:
> > | [...]
> > | debconf: delaying package configuration, since apt-utils is not installed
> > | dpkg: error: failed to open package info file '/var/lib/dpkg/available'
> > for reading: No such file or directory
> >
> > My workaround to work around this is to run:
> >
> > chroot "${TARGET}" /usr/lib/dpkg/methods/apt/update /var/lib/dpkg
> >
> > As mmdebstrap seems to rely on dpkg of the host system, it would be
> > nice if mmdebstrap could ensure to create an environment where apt/dpkg of
> > the installed system doesn't stumble upon. :)
>
> this already exists as a hook. :)
>
> Try running mmdebstrap with
>
> --hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older
>
> which will unconditionally do certain things only necessary for old releases.
This indeed solves the problem, /var/lib/dpkg/available exists in
the target system then (as empty file, FTR).
> If you instead use
>
> --hook-dir=/usr/share/mmdebstrap/hooks/maybe-jessie-or-older
>
> some detection code will only run the hook if necessary (in case you do not
> know upfront which release you are creating a chroot for).
This does *not* help/work, though, FYI.
(FTR, we already have a local workaround, I just thought that it
would make sense to have a decent and nice out-of-the-box experience
also for other users. :))
Thanks! :)
regards
-mika-
signature.asc
Description: PGP signature
Bug#1037946: mmdebstrap: Create /var/lib/dpkg/available for Debian releases <=buster
Hi,
Quoting Michael Prokop (2023-06-14 17:19:37)
> since version 1.20.0, dpkg no longer creates /var/lib/dpkg/available (see
> #647911).
I see that the bug talks about 1.20.0 but I've had code creating
/var/lib/dpkg/available in mmdebstrap for a while and determined that it wasn't
necessary anymore since dpkg 1.17.11. Did you do practical tests or are you
only basing your analysis on that bug?
I'm quite certain about the version requirement because I did tests using
debbisect and using the suggestion below I'm able to create chroots for all of
unstable since 2006-08-10.
> But on Debian releases until and including buster, dpkg fails to operate with
> e.g. `dpkg --set-selections`, if /var/lib/dpkg/available doesn't exist:
>
> | The following NEW packages will be installed:
> | [...]
> | debconf: delaying package configuration, since apt-utils is not installed
> | dpkg: error: failed to open package info file '/var/lib/dpkg/available' for
> reading: No such file or directory
>
> My workaround to work around this is to run:
>
> chroot "${TARGET}" /usr/lib/dpkg/methods/apt/update /var/lib/dpkg
>
> As mmdebstrap seems to rely on dpkg of the host system, it would be
> nice if mmdebstrap could ensure to create an environment where apt/dpkg of
> the installed system doesn't stumble upon. :)
this already exists as a hook. :)
Try running mmdebstrap with
--hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older
which will unconditionally do certain things only necessary for old releases.
If you instead use
--hook-dir=/usr/share/mmdebstrap/hooks/maybe-jessie-or-older
some detection code will only run the hook if necessary (in case you do not
know upfront which release you are creating a chroot for).
Thanks!
cheers, josch
signature.asc
Description: signature
Bug#1037946: mmdebstrap: Create /var/lib/dpkg/available for Debian releases <=buster
Package: mmdebstrap
Version: 1.3.5-7
Severity: normal
Hi,
since version 1.20.0, dpkg no longer creates /var/lib/dpkg/available
(see #647911).
But on Debian releases until and including buster, dpkg fails to
operate with e.g. `dpkg --set-selections`, if
/var/lib/dpkg/available doesn't exist:
| The following NEW packages will be installed:
| [...]
| debconf: delaying package configuration, since apt-utils is not installed
| dpkg: error: failed to open package info file '/var/lib/dpkg/available' for
reading: No such file or directory
My workaround to work around this is to run:
chroot "${TARGET}" /usr/lib/dpkg/methods/apt/update /var/lib/dpkg
As mmdebstrap seems to rely on dpkg of the host system, it would be
nice if mmdebstrap could ensure to create an environment where
apt/dpkg of the installed system doesn't stumble upon. :)
Thanks for mmdebstrap!
regards
-mika-
