Processed: Re: Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Debian Bug Tracking System]

Processing control commands:

> severity -1 serious
Bug #994042 [libc6] libc6: debconf text fallback failed to accept input, had to 
be killed leading to broken dist-upgrade
Severity set to 'serious' from 'normal'

-- 
994042: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994042
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Aurelien Jarno]

control: severity -1 serious

On 2021-09-10 16:02, Neil Williams wrote:
> Package: libc6
> Version: 2.31-13
> Severity: normal
> X-Debbugs-Cc: codeh...@debian.org
> 
> This is related to #984533 - in my case, there was no effect on the initramfs.
> 
> I am attaching the section of the apt log. (gzipped)
> I am also attaching the dpkg -l output for the package list (after upgrade).
> 
> The apt log includes the details of the --purge autoremove operation I 
> completed
> after a reboot, so those packages were also installed on buster too.
> 
> This was an upgrade from buster to bullseye.
> apt upgrade worked fine (first part of the log).
> 
> When dist-upgrade started, I got:

[snip]

> No matter what I typed to the Do you want to upgrade prompt, the process did 
> not continue.
> I needed to open a terminal tab, find the preinst process & kill it.

Not being able to perform the upgrade without having to kill the preinst
process from another tab (and having the skills do so) looks a serious
issue to me. I am therefore upgrading the severity.

Regards,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Aurelien Jarno]

Hi,

On 2021-09-15 21:54, Colin Watson wrote:
> On Fri, Sep 10, 2021 at 09:59:44PM +0200, Aurelien Jarno wrote:
> > On 2021-09-10 20:39, Colin Watson wrote:
> > > On Fri, Sep 10, 2021 at 09:03:32PM +0200, Aurelien Jarno wrote:
> > > > I gave a try with debconf-show instead. I have attached a totally
> > > > untested patch to check that we agree on the way to do it.
> > > 
> > > I think you forgot to attach the patch?
> > 
> > Dooh. Please find a new version attached.
> 
> The general idea of this looks good to me.  I've left some detailed
> review comments below, and IMO we should test it against my reproducer
> (especially since this preinst patch needs to end up in bullseye).

Thanks for the fixes.

> > > I managed to reproduce the reported bug by taking Neil's full package
> > > list, mangling it to roughly make sense on buster, installing all of
> > > that, and then doing "apt upgrade && apt full-upgrade" (my own habit is
> > > just to do "apt full-upgrade", but in this case the initial "apt
> > > upgrade" is crucial).  I'm now trying to more or less bisect the package
> > > list to find something rather more minimal; this is a slow process, but
> > > no roadblocks so far, and I'll let you know when I have something.
> > 
> > Thanks a lot for your help.
> 
> OK, it took much longer than I expected because I wasn't able to do it
> by just bisecting the package list, but here's a reproducer.  I ran this
> in a fresh container produced by "lxc launch images:debian/buster"; I
> expect other container tools can be made to exhibit this too, though it
> may be sensitive to exactly which packages are in the base image.
> 
>   # apt update && apt -y install gimp libc6-dev postgresql whiptail
>   # cat >/etc/apt/sources.list <   deb http://deb.debian.org/debian bullseye main
>   deb http://security.debian.org/debian-security bullseye-security main
>   # apt update && apt -y upgrade && apt -y dist-upgrade

Thanks for the reproducer. I have been able to use it to test the fixes,
both with upgrade + dist-upgrade to check that the problem is fixed, and
with dist-upgrade only to check that things still work when debconf is
usable.

Regards,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Colin Watson]

On Fri, Sep 10, 2021 at 09:59:44PM +0200, Aurelien Jarno wrote:
> On 2021-09-10 20:39, Colin Watson wrote:
> > On Fri, Sep 10, 2021 at 09:03:32PM +0200, Aurelien Jarno wrote:
> > > I gave a try with debconf-show instead. I have attached a totally
> > > untested patch to check that we agree on the way to do it.
> > 
> > I think you forgot to attach the patch?
> 
> Dooh. Please find a new version attached.

The general idea of this looks good to me.  I've left some detailed
review comments below, and IMO we should test it against my reproducer
(especially since this preinst patch needs to end up in bullseye).

> > I managed to reproduce the reported bug by taking Neil's full package
> > list, mangling it to roughly make sense on buster, installing all of
> > that, and then doing "apt upgrade && apt full-upgrade" (my own habit is
> > just to do "apt full-upgrade", but in this case the initial "apt
> > upgrade" is crucial).  I'm now trying to more or less bisect the package
> > list to find something rather more minimal; this is a slow process, but
> > no roadblocks so far, and I'll let you know when I have something.
> 
> Thanks a lot for your help.

OK, it took much longer than I expected because I wasn't able to do it
by just bisecting the package list, but here's a reproducer.  I ran this
in a fresh container produced by "lxc launch images:debian/buster"; I
expect other container tools can be made to exhibit this too, though it
may be sensitive to exactly which packages are in the base image.

  # apt update && apt -y install gimp libc6-dev postgresql whiptail
  # cat >/etc/apt/sources.list  diff --git a/debian/debhelper.in/libc.preinst 
> b/debian/debhelper.in/libc.preinst
> index d679db4f..e7808a44 100644
> --- a/debian/debhelper.in/libc.preinst
> +++ b/debian/debhelper.in/libc.preinst
> @@ -21,23 +21,22 @@ kfreebsd_compare_versions () {
>  
>  if [ "$type" != abort-upgrade -a -z "$DPKG_ROOT" ]
>  then
> -# Load debconf module if available and usable
> +# Check if the debconf module is available and usable

I'd suggest initializing "USE_DEBCONF=" here to avoid potential
weirdness in case somebody happens to have that variable set in their
environment for some reason.

>  if [ -f /usr/share/debconf/confmodule ]; then
>  # cdebconf has a working fallback mechanism in case dialog
>  # is not usable, so do not try to do anything smart here
>  if [ "$DEBCONF_USE_CDEBCONF" ] ; then
> -. /usr/share/debconf/confmodule
>  USE_DEBCONF=1
>  # debconf requires perl
>  elif perl -e "" 2>/dev/null ; then
> -. /usr/share/debconf/confmodule
>  # Check that the selected frontend will work
>  if [ -n "$DEBIAN_FRONTEND" ] ; then
>  frontend="$DEBIAN_FRONTEND"
>  else
> -db_version 2.0
> -db_get debconf/frontend || RET="Dialog"
> -frontend="$RET"
> +# Query the frontend without first sourcing the confmodule 
> to avoid
> +# loosing control of the tty. This snipped must not be 
> copied blindly.

s/loosing/losing/, and s/snipped/snippet/.

> +frontend="$(echo 'GET debconf/frontend' | 
> debconf-communicate | sed '/^0 /!d;s/^0 //')"
> +frontend="${frontend:-Dialog}"
>  fi
>  frontend=`echo $frontend | tr '[:upper:]' '[:lower:]'`
>  case "$frontend" in
> @@ -61,6 +60,11 @@ then
>  fi
>  fi
>  
> +# Load debconf module if available and usable
> +if [ "$USE_DEBCONF" ]

This needs a "; then".

Thanks,

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Aurelien Jarno]

On 2021-09-10 20:39, Colin Watson wrote:
> On Fri, Sep 10, 2021 at 09:03:32PM +0200, Aurelien Jarno wrote:
> > On 2021-09-10 16:51, Colin Watson wrote:
> > > The only way to fix what libc.preinst is currently trying to do would
> > > be:
> > > 
> > >  * Fetch the current debconf frontend *without* first sourcing the
> > >confmodule, e.g. using something like "echo GET debconf/frontend |
> > >debconf-communicate" which I *think* is safe as long as you haven't
> > >sourced the confmodule yet;
> > > 
> > >  * Decide whether to use debconf based on this and other information;
> > > 
> > >  * Only source the confmodule if you've positively decided to use it.
> > 
> > debconf-communicate might be safe, but its manpage explicitly says it
> > should not be used in maintainer scripts.
> 
> Strictly, it says "a maintainer script of a package that uses debconf".
> I think what that really means is that it shouldn't be used after
> sourcing the debconf confmodule (which is normally the same as using
> debconf since nearly everything sources it right at the top - but this
> is a weird edge case).  The point is to avoid having multiple processes
> with the debconf database open at the same time.
> 
> Put another way, for this purpose, libc6.preinst isn't really a
> maintainer script that uses debconf until it sources the confmodule.
> 
> > I gave a try with debconf-show instead. I have attached a totally
> > untested patch to check that we agree on the way to do it.
> 
> I think you forgot to attach the patch?

Dooh. Please find a new version attached.

> I wouldn't completely veto using debconf-show in this very specialized
> situation, as long as it came with a substantial comment explaining
> what's going on so that nobody else is tempted to copy it.  However, the
> output format of debconf-show isn't guaranteed, so I'm not very happy
> about it being used mechanically like this.  I'd prefer
> debconf-communicate if we can ensure that it works in this context,
> notwithstanding its documentation.

Ok, I have updated my patch to use that method.

> > > But a simpler approach might be to update debconf in buster with the
> > > change from 1.5.76 to check whether whiptail/dialog is usable before
> > > trying to use it, and then remove at least some of this fragile and
> > > broken code from libc.preinst.  At the very least, USE_DEBCONF=1 must
> > > always be set if (and only if) the debconf confmodule has been sourced.
> > 
> > While it is probably a good idea to backport that change in buster to
> > limit to reduce the risk, we don't require people to upgrade to the
> > latest buster release before starting an upgrade.
> > 
> > On the other hand, given that bullseye has a fixed debconf, I fully
> > agree that we should drop that fragile code for bookworm.
> 
> We probably agree on both points here.

Great.

> > > I'm currently seeing if I can construct a reduced reproduction recipe
> > > based on Neil's logs, since it evidently depends on exactly which order
> > > apt chooses to unpack things early on, and it would be very helpful to
> > > be able to test fixes properly.
> > 
> > Thanks, tell me if you need help on that.
> 
> I managed to reproduce the reported bug by taking Neil's full package
> list, mangling it to roughly make sense on buster, installing all of
> that, and then doing "apt upgrade && apt full-upgrade" (my own habit is
> just to do "apt full-upgrade", but in this case the initial "apt
> upgrade" is crucial).  I'm now trying to more or less bisect the package
> list to find something rather more minimal; this is a slow process, but
> no roadblocks so far, and I'll let you know when I have something.
> 

Thanks a lot for your help.

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net
commit d67e52a7d1997a3e461b6971b00ce94a136a5b2d
Author: Aurelien Jarno 
Date:   Fri Sep 10 20:58:48 2021 +0200

first fix for #994042

diff --git a/debian/debhelper.in/libc.preinst b/debian/debhelper.in/libc.preinst
index d679db4f..e7808a44 100644
--- a/debian/debhelper.in/libc.preinst
+++ b/debian/debhelper.in/libc.preinst
@@ -21,23 +21,22 @@ kfreebsd_compare_versions () {
 
 if [ "$type" != abort-upgrade -a -z "$DPKG_ROOT" ]
 then
-# Load debconf module if available and usable
+# Check if the debconf module is available and usable
 if [ -f /usr/share/debconf/confmodule ]; then
 # cdebconf has a working fallback mechanism in case dialog
 # is not usable, so do not try to do anything smart here
 if [ "$DEBCONF_USE_CDEBCONF" ] ; then
-. /usr/share/debconf/confmodule
 USE_DEBCONF=1
 # debconf requires perl
 elif perl -e "" 2>/dev/null ; then
-. /usr/share/debconf/confmodule
 # Check that the selected frontend will work
 if [ -n "$DEBIAN_FRONTEND" ] ; then
 frontend="$DEBIAN_FRONTEND"
 else
-   

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Colin Watson]

On Fri, Sep 10, 2021 at 09:03:32PM +0200, Aurelien Jarno wrote:
> On 2021-09-10 16:51, Colin Watson wrote:
> > The only way to fix what libc.preinst is currently trying to do would
> > be:
> > 
> >  * Fetch the current debconf frontend *without* first sourcing the
> >confmodule, e.g. using something like "echo GET debconf/frontend |
> >debconf-communicate" which I *think* is safe as long as you haven't
> >sourced the confmodule yet;
> > 
> >  * Decide whether to use debconf based on this and other information;
> > 
> >  * Only source the confmodule if you've positively decided to use it.
> 
> debconf-communicate might be safe, but its manpage explicitly says it
> should not be used in maintainer scripts.

Strictly, it says "a maintainer script of a package that uses debconf".
I think what that really means is that it shouldn't be used after
sourcing the debconf confmodule (which is normally the same as using
debconf since nearly everything sources it right at the top - but this
is a weird edge case).  The point is to avoid having multiple processes
with the debconf database open at the same time.

Put another way, for this purpose, libc6.preinst isn't really a
maintainer script that uses debconf until it sources the confmodule.

> I gave a try with debconf-show instead. I have attached a totally
> untested patch to check that we agree on the way to do it.

I think you forgot to attach the patch?

I wouldn't completely veto using debconf-show in this very specialized
situation, as long as it came with a substantial comment explaining
what's going on so that nobody else is tempted to copy it.  However, the
output format of debconf-show isn't guaranteed, so I'm not very happy
about it being used mechanically like this.  I'd prefer
debconf-communicate if we can ensure that it works in this context,
notwithstanding its documentation.

> > But a simpler approach might be to update debconf in buster with the
> > change from 1.5.76 to check whether whiptail/dialog is usable before
> > trying to use it, and then remove at least some of this fragile and
> > broken code from libc.preinst.  At the very least, USE_DEBCONF=1 must
> > always be set if (and only if) the debconf confmodule has been sourced.
> 
> While it is probably a good idea to backport that change in buster to
> limit to reduce the risk, we don't require people to upgrade to the
> latest buster release before starting an upgrade.
> 
> On the other hand, given that bullseye has a fixed debconf, I fully
> agree that we should drop that fragile code for bookworm.

We probably agree on both points here.

> > I'm currently seeing if I can construct a reduced reproduction recipe
> > based on Neil's logs, since it evidently depends on exactly which order
> > apt chooses to unpack things early on, and it would be very helpful to
> > be able to test fixes properly.
> 
> Thanks, tell me if you need help on that.

I managed to reproduce the reported bug by taking Neil's full package
list, mangling it to roughly make sense on buster, installing all of
that, and then doing "apt upgrade && apt full-upgrade" (my own habit is
just to do "apt full-upgrade", but in this case the initial "apt
upgrade" is crucial).  I'm now trying to more or less bisect the package
list to find something rather more minimal; this is a slow process, but
no roadblocks so far, and I'll let you know when I have something.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Aurelien Jarno]

Hi Colin,

On 2021-09-10 16:51, Colin Watson wrote:
> On Fri, Sep 10, 2021 at 04:02:06PM +0100, Neil Williams wrote:
> > This is related to #984533 - in my case, there was no effect on the 
> > initramfs.
> > 
> > I am attaching the section of the apt log. (gzipped)
> > I am also attaching the dpkg -l output for the package list (after upgrade).
> > 
> > The apt log includes the details of the --purge autoremove operation I 
> > completed
> > after a reboot, so those packages were also installed on buster too.
> > 
> > This was an upgrade from buster to bullseye.
> > apt upgrade worked fine (first part of the log).
> > 
> > When dist-upgrade started, I got:
> > 
> > Preparing to unpack .../92-libc6_2.31-13_amd64.deb ...
> > debconf: unable to initialize frontend: Dialog
> > debconf: (No usable dialog-like program is installed, so the dialog based 
> > frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm 
> > line 78.)
> > debconf: falling back to frontend: Readline
> > Checking for services that may need to be restarted...
> > Checking init scripts...
> > Do you want to upgrade glibc now?
> > 
> > Running services and programs that are using NSS need to be restarted,
> > otherwise they might not be able to do lookup or authentication any more.
> > The installation process is able to restart some services (such as ssh or
> > telnetd), but other programs cannot be restarted automatically.  One such
> > program that needs manual stopping and restart after the glibc upgrade by
> > yourself is xdm - because automatic restart might disconnect your active
> > X11 sessions.
> > 
> > This script detected the following installed services which must be
> > stopped before the upgrade: postgresql 
> > 
> > If you want to interrupt the upgrade now and continue later, please
> > answer No to the question below.
> > 
> > Do you want to upgrade glibc now? [Y/n] y
> > Y
> 
> The code in libc.preinst that attempts to fall back to text mode,
> introduced in 2.31-10, is clearly incorrect; apologies for not noticing
> this earlier.  It sources the debconf confmodule and then conditionally
> sets USE_DEBCONF; but since the debconf confmodule has already been
> sourced by this point, text mode won't work properly since standard
> input and output refer to the debconf frontend.  (In particular, reading
> from stdin can't work.)

Thanks for the fast analysis, and apologies for introducing that bug.

> The only way to fix what libc.preinst is currently trying to do would
> be:
> 
>  * Fetch the current debconf frontend *without* first sourcing the
>confmodule, e.g. using something like "echo GET debconf/frontend |
>debconf-communicate" which I *think* is safe as long as you haven't
>sourced the confmodule yet;
> 
>  * Decide whether to use debconf based on this and other information;
> 
>  * Only source the confmodule if you've positively decided to use it.

debconf-communicate might be safe, but its manpage explicitly says it
should not be used in maintainer scripts. I gave a try with debconf-show
instead. I have attached a totally untested patch to check that we
agree on the way to do it.

> But a simpler approach might be to update debconf in buster with the
> change from 1.5.76 to check whether whiptail/dialog is usable before
> trying to use it, and then remove at least some of this fragile and
> broken code from libc.preinst.  At the very least, USE_DEBCONF=1 must
> always be set if (and only if) the debconf confmodule has been sourced.

While it is probably a good idea to backport that change in buster to
limit to reduce the risk, we don't require people to upgrade to the
latest buster release before starting an upgrade.

On the other hand, given that bullseye has a fixed debconf, I fully
agree that we should drop that fragile code for bookworm.

> I'm currently seeing if I can construct a reduced reproduction recipe
> based on Neil's logs, since it evidently depends on exactly which order
> apt chooses to unpack things early on, and it would be very helpful to
> be able to test fixes properly.

Thanks, tell me if you need help on that.

Regards,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Bug#994042: libc6: debconf text fallback failed to accept input, had to be killed leading to broken dist-upgrade

[Colin Watson]

On Fri, Sep 10, 2021 at 04:02:06PM +0100, Neil Williams wrote:
> This is related to #984533 - in my case, there was no effect on the initramfs.
> 
> I am attaching the section of the apt log. (gzipped)
> I am also attaching the dpkg -l output for the package list (after upgrade).
> 
> The apt log includes the details of the --purge autoremove operation I 
> completed
> after a reboot, so those packages were also installed on buster too.
> 
> This was an upgrade from buster to bullseye.
> apt upgrade worked fine (first part of the log).
> 
> When dist-upgrade started, I got:
> 
> Preparing to unpack .../92-libc6_2.31-13_amd64.deb ...
> debconf: unable to initialize frontend: Dialog
> debconf: (No usable dialog-like program is installed, so the dialog based 
> frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 
> 78.)
> debconf: falling back to frontend: Readline
> Checking for services that may need to be restarted...
> Checking init scripts...
> Do you want to upgrade glibc now?
> 
> Running services and programs that are using NSS need to be restarted,
> otherwise they might not be able to do lookup or authentication any more.
> The installation process is able to restart some services (such as ssh or
> telnetd), but other programs cannot be restarted automatically.  One such
> program that needs manual stopping and restart after the glibc upgrade by
> yourself is xdm - because automatic restart might disconnect your active
> X11 sessions.
> 
> This script detected the following installed services which must be
> stopped before the upgrade: postgresql 
> 
> If you want to interrupt the upgrade now and continue later, please
> answer No to the question below.
> 
> Do you want to upgrade glibc now? [Y/n] y
> Y

The code in libc.preinst that attempts to fall back to text mode,
introduced in 2.31-10, is clearly incorrect; apologies for not noticing
this earlier.  It sources the debconf confmodule and then conditionally
sets USE_DEBCONF; but since the debconf confmodule has already been
sourced by this point, text mode won't work properly since standard
input and output refer to the debconf frontend.  (In particular, reading
from stdin can't work.)

The only way to fix what libc.preinst is currently trying to do would
be:

 * Fetch the current debconf frontend *without* first sourcing the
   confmodule, e.g. using something like "echo GET debconf/frontend |
   debconf-communicate" which I *think* is safe as long as you haven't
   sourced the confmodule yet;

 * Decide whether to use debconf based on this and other information;

 * Only source the confmodule if you've positively decided to use it.

But a simpler approach might be to update debconf in buster with the
change from 1.5.76 to check whether whiptail/dialog is usable before
trying to use it, and then remove at least some of this fragile and
broken code from libc.preinst.  At the very least, USE_DEBCONF=1 must
always be set if (and only if) the debconf confmodule has been sourced.

I'm currently seeing if I can construct a reduced reproduction recipe
based on Neil's logs, since it evidently depends on exactly which order
apt chooses to unpack things early on, and it would be very helpful to
be able to test fixes properly.

-- 
Colin Watson (he/him)  [cjwat...@debian.org]