Bug#901185: [Debconf-devel] Bug#901185: me too

[Luca Boccassi]

On Tue, 12 Jun 2018 06:22:09 +0100 Colin Watson 
wrote:
> On Mon, Jun 11, 2018 at 08:52:11PM +0200, Andreas Metzler wrote:
> > On 2018-06-11 Colin Watson  wrote:
> > > On Sun, Jun 10, 2018 at 04:04:13PM +0200, Andreas Metzler wrote:
> > > > yes, resetting IFS to its default value in confmodule helps.
> > 
> > > Thanks.  For completeness, could you try this variant patch?  If
shells
> > > are going to be unreliable about setting IFS in this way, then
I'd
> > > rather just separate out the assignments altogether.
> > [snip]
> > 
> > Yes, this variant of the patch also works.
> > 
> > Thanks to your handholding we now have a minimal reproducer for the
> > issue:
> 
> Thanks.  Now that you have a reproducer that doesn't involve debconf
and
> so I won't be interfering with debugging, I've uploaded debconf
1.5.67
> with that confmodule patch.  I know it's a bash5 bug and will
hopefully
> be fixed, but we might as well be conservative in debconf when it's
easy
> to do so.
> 
> -- 
> Colin Watson   [cjwatson@debian.o
rg]

Hi,

FYI, this was fixed in bash 5.0 beta1:

$ ./bash --version
GNU bash, version 5.0.0(1)-beta (x86_64-pc-linux-gnu)
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ ./bash -o posix -c 'v=ok; v=bug read x 

signature.asc
Description: This is a digitally signed message part

Bug#901185: [Debconf-devel] Bug#901185: me too

[Colin Watson]

On Mon, Jun 11, 2018 at 08:52:11PM +0200, Andreas Metzler wrote:
> On 2018-06-11 Colin Watson  wrote:
> > On Sun, Jun 10, 2018 at 04:04:13PM +0200, Andreas Metzler wrote:
> > > yes, resetting IFS to its default value in confmodule helps.
> 
> > Thanks.  For completeness, could you try this variant patch?  If shells
> > are going to be unreliable about setting IFS in this way, then I'd
> > rather just separate out the assignments altogether.
> [snip]
> 
> Yes, this variant of the patch also works.
> 
> Thanks to your handholding we now have a minimal reproducer for the
> issue:

Thanks.  Now that you have a reproducer that doesn't involve debconf and
so I won't be interfering with debugging, I've uploaded debconf 1.5.67
with that confmodule patch.  I know it's a bash5 bug and will hopefully
be fixed, but we might as well be conservative in debconf when it's easy
to do so.

-- 
Colin Watson   [cjwat...@debian.org]

Bug#901185: me too

[Eduardo Bustamante]

On Mon, Jun 11, 2018 at 1:52 PM, Andreas Metzler  wrote:
[..]
> Looking at the Open Group Base Specifications Issue 7 I also think this
> behavior is a bug, since "read" is not a "a special built-in utility or
> function".

FWIW, this is a bug in bash5, and has been reported to upstream
already: https://lists.gnu.org/archive/html/bug-bash/2018-05/msg00077.html

Bug#901185: me too

[Andreas Metzler]

On 2018-06-11 Colin Watson  wrote:
> On Sun, Jun 10, 2018 at 04:04:13PM +0200, Andreas Metzler wrote:
[...] 
> > yes, resetting IFS to its default value in confmodule helps.

> Thanks.  For completeness, could you try this variant patch?  If shells
> are going to be unreliable about setting IFS in this way, then I'd
> rather just separate out the assignments altogether.
[snip]

Yes, this variant of the patch also works.

Thanks to your handholding we now have a minimal reproducer for the
issue:

(sid)ametzler@argenau:~$ cat /tmp/reproducer.sh
#!/bin/sh
set -e

list="item1 item2"

ls -l /bin/sh
printf "IFS=[%s]" "$IFS" | od -a

IFS='
' read -r dummy

printf "IFS=[%s]" "$IFS" | od -a
for i in $list ; do
echo $i
done
# Error with bash5:
(sid)ametzler@argenau:~$ echo xxx | /tmp/reproducer.sh
lrwxrwxrwx 1 root root 4 Jun 11 18:41 /bin/sh -> bash
000   I   F   S   =   [  sp  ht  nl   ]
011
000   I   F   S   =   [  nl   ]
007
item1 item2
#normal behavior with bash4 or dash:
(sid)ametzler@argenau:~$ echo xxx | /tmp/reproducer.sh
lrwxrwxrwx 1 root root 4 Jun 11 18:41 /bin/sh -> bash
000   I   F   S   =   [  sp  ht  nl   ]
011
000   I   F   S   =   [  sp  ht  nl   ]
011
item1
item2


> (This does still seem like a bash bug though, unless there's some arcane
> interpretation of the standards that I'm unaware of.)

Looking at the Open Group Base Specifications Issue 7 I also think this
behavior is a bug, since "read" is not a "a special built-in utility or
function".

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: [Debconf-devel] Bug#901185: me too

[Colin Watson]

On Sun, Jun 10, 2018 at 04:04:13PM +0200, Andreas Metzler wrote:
> On 2018-06-10 Colin Watson  wrote:
> > On Sun, Jun 10, 2018 at 03:03:28PM +0200, Andreas Metzler wrote:
> >> I have tried debugging the issue to get a minimal testcase. I have had
> >> some success. The issue is triggered by debconf.
> >> /usr/share/debconf/confmodule changes IFS to \n and with bash5 this
> >> change is persistent and breaks the maintainerscript.
> 
> > Does this patch help?
> 
> yes, resetting IFS to its default value in confmodule helps.

Thanks.  For completeness, could you try this variant patch?  If shells
are going to be unreliable about setting IFS in this way, then I'd
rather just separate out the assignments altogether.

diff --git a/confmodule b/confmodule
index 8347c0b0..3ef7a309 100644
--- a/confmodule
+++ b/confmodule
@@ -42,10 +42,11 @@ _db_cmd () {
_db_internal_IFS="$IFS"
IFS=' '
printf '%s\n' "$*" >&3
-   IFS="$_db_internal_IFS"
# Set to newline to get whole line.
IFS='
-' read -r _db_internal_line
+'
+   read -r _db_internal_line
+   IFS="$_db_internal_IFS"
# Disgusting, but it's the only good way to split the line,
# preserving all other whitespace.
RET="${_db_internal_line#[! ][  ]}"

(This does still seem like a bash bug though, unless there's some arcane
interpretation of the standards that I'm unaware of.)

Cheers,

-- 
Colin Watson   [cjwat...@debian.org]

Bug#901185: [Debconf-devel] Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 Colin Watson  wrote:
> On Sun, Jun 10, 2018 at 03:03:28PM +0200, Andreas Metzler wrote:
>> I have tried debugging the issue to get a minimal testcase. I have had
>> some success. The issue is triggered by debconf.
>> /usr/share/debconf/confmodule changes IFS to \n and with bash5 this
>> change is persistent and breaks the maintainerscript.

> Does this patch help?

> diff --git a/confmodule b/confmodule
> index 8347c0b0..e6157520 100644
> --- a/confmodule
> +++ b/confmodule
> @@ -46,6 +46,7 @@ _db_cmd () {
>   # Set to newline to get whole line.
>   IFS='
>  ' read -r _db_internal_line
> + IFS="$_db_internal_IFS"
>   # Disgusting, but it's the only good way to split the line,
>   # preserving all other whitespace.
>   RET="${_db_internal_line#[! ][  ]}"

> (If so, that would suggest that the problem might be specifically in
> variable assignments preceding builtins.)

Hello Colin,

yes, resetting IFS to its default value in confmodule helps.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: [Debconf-devel] Bug#901185: me too

[Colin Watson]

On Sun, Jun 10, 2018 at 03:03:28PM +0200, Andreas Metzler wrote:
> I have tried debugging the issue to get a minimal testcase. I have had
> some success. The issue is triggered by debconf.
> /usr/share/debconf/confmodule changes IFS to \n and with bash5 this
> change is persistent and breaks the maintainerscript.

Does this patch help?

diff --git a/confmodule b/confmodule
index 8347c0b0..e6157520 100644
--- a/confmodule
+++ b/confmodule
@@ -46,6 +46,7 @@ _db_cmd () {
# Set to newline to get whole line.
IFS='
 ' read -r _db_internal_line
+   IFS="$_db_internal_IFS"
# Disgusting, but it's the only good way to split the line,
# preserving all other whitespace.
RET="${_db_internal_line#[! ][  ]}"

(If so, that would suggest that the problem might be specifically in
variable assignments preceding builtins.)

-- 
Colin Watson   [cjwat...@debian.org]

Bug#901185: me too

[Andreas Metzler]

Hello bash maintainers (debconf Cced),

I have tried debugging the issue to get a minimal testcase. I have had
some success. The issue is triggered by debconf.
/usr/share/debconf/confmodule changes IFS to \n and with bash5 this
change is persistent and breaks the maintainerscript. i.e. installing a
package using the following.postinst (simply invoking the script as
unprivildged user also works, no dummy-package necessary)


#!/bin/sh
set -e
. /usr/share/debconf/confmodule
db_version 2.0
list="item1 item2"
for i in $list ; do
echo $i 1>&2
done


will
a) print
item1 item2
with bash5 and
b) print
item1
item2
with bash4 and dash.

Unfortunately I have not been able to get a simpler testcase.

cu Andreas

Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 Eric Valette  wrote:
> On 6/10/18 11:57 AM, Eric Valette wrote:
>> On 6/10/18 11:35 AM, Andreas Metzler wrote:
 
>>> exports both dc_directives and any variable listed in dc_directives,
>>> i.e. any external command will see both ${dc_directives} and
>>> ${dc_eximconfig_configtype} et al.

>> right but did you intent that? The rest of the script seems not to use
>> explicitly anything but dc_directives in to awk script. Remains that the
>> script is broken with bash and not sure only from experimental (but as
>> the config file was already broken cannot be tottaly sure).

> Why don't you reassign the bug to bash experimental if you are so sure?

Because as it is the bug report is less than helpful for the bash
maintainers.

> Will do until you tell me why not

Please be a little bit more cooperative. Discuss first, and act later
when there is a consensus.

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 Eric Valette  wrote:
> On 6/10/18 11:35 AM, Andreas Metzler wrote:

> > exports both dc_directives and any variable listed in dc_directives,
> > i.e. any external command will see both ${dc_directives} and
> > ${dc_eximconfig_configtype} et al.

> right but did you intent that?

Yes I did,...

> The rest of the script seems not to use
> explicitly anything but dc_directives in to awk script.

... because the awk script references both each entry in dc_directives
and its value (directives[i] and ENVIRON[directives[i]).

> Remains that the
> script is broken with bash and not sure only from experimental (but as the
> config file was already broken cannot be tottaly sure).

I had checked that before closing the bug report, bash/sid works.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: me too

[Eric Valette]

On 6/10/18 11:57 AM, Eric Valette wrote:

On 6/10/18 11:35 AM, Andreas Metzler wrote:


exports both dc_directives and any variable listed in dc_directives,
i.e. any external command will see both ${dc_directives} and
${dc_eximconfig_configtype} et al.


right but did you intent that? The rest of the script seems not to use 
explicitly anything but dc_directives in to awk script. Remains that the 
script is broken with bash and not sure only from experimental (but as 
the config file was already broken cannot be tottaly sure).



Why don't you reassign the bug to bash experimental if you are so sure? 
Will do until you tell me why not


-- eric

Bug#901185: me too

[Eric Valette]

On 6/10/18 11:35 AM, Andreas Metzler wrote:


exports both dc_directives and any variable listed in dc_directives,
i.e. any external command will see both ${dc_directives} and
${dc_eximconfig_configtype} et al.


right but did you intent that? The rest of the script seems not to use 
explicitly anything but dc_directives in to awk script. Remains that the 
script is broken with bash and not sure only from experimental (but as 
the config file was already broken cannot be tottaly sure).


-- eric

Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 Eric Valette  wrote:
> On 6/10/18 10:51 AM, Eric Valette wrote:
>> On 6/10/18 10:34 AM, Eric Valette wrote:
 
>>> line 330 :export dc_directives ${dc_directives}

>> Shodul be export dc_directives=${dc_directives} as far as I understand.
[...]
> and it worked. But I still think there is a non portable syntax line 330 of
> exim4-config.postins.

Hello,

That is not invalid syntax, export takes a list of shell-local variables
(and *optionally* value assignments like foo=bar) and exports the list
of local varaiables. dc_directives contains a list of variables
(dc_eximconfig_configtype et al.) and

export dc_directives ${dc_directives}

exports both dc_directives and any variable listed in dc_directives,
i.e. any external command will see both ${dc_directives} and
${dc_eximconfig_configtype} et al.

cu Andreas
-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 Eric Valette  wrote:
> On 6/10/18 10:59 AM, Eric Valette wrote:
> > On 6/10/18 10:51 AM, Eric Valette wrote:

> >  2) mv /etc/exim4/update-exim4.conf.conf 
> > /etc/exim4/update-exim4.conf.conf.old

exactly what I wrote ;-)
"installs/configures successfully once the broken
/etc/exim4/update-exim4.conf.conf has been removed."

Bug#901185: me too

[Eric Valette]

On 6/10/18 10:59 AM, Eric Valette wrote:

On 6/10/18 10:51 AM, Eric Valette wrote:



 2) mv /etc/exim4/update-exim4.conf.conf 
/etc/exim4/update-exim4.conf.conf.old

Bug#901185: me too

[Eric Valette]

On 6/10/18 10:51 AM, Eric Valette wrote:

On 6/10/18 10:34 AM, Eric Valette wrote:


line 330 :export dc_directives ${dc_directives}


Shodul be export dc_directives=${dc_directives} as far as I understand.


To fix I did :
1) use dash temporarilly
2) /etc/exim4/update-exim4.conf.conf.old
3) rerun install

and it worked. But I still think there is a non portable syntax line 330 
of exim4-config.postins.


-- eric

Bug#901185: me too

[Eric Valette]

On 6/10/18 10:34 AM, Eric Valette wrote:


line 330 :export dc_directives ${dc_directives}


Shodul be export dc_directives=${dc_directives} as far as I understand.

-- eric

Bug#901185: closed by Andreas Metzler (Re: Bug#901185: me too)

[Eric Valette]

On 6/10/18 10:33 AM, Debian Bug Tracking System wrote:

This is an automatic notification regarding your Bug report
which was filed against the exim4-config package:



But now how do we fix the problem? I will reopen until I have a fix. 
Putting dash as /bin/sh does fix anything BTW.


-- eric

Bug#901185: me too

[Eric Valette]

On 6/10/18 7:07 AM, Andreas Metzler wrote:

On 2018-06-10 積丹尼 Dan Jacobson  wrote:

Oops. I mean just:
Setting up exim4-config (4.91-5) ...
/etc/exim4/update-exim4.conf.conf: line 26: dc_eximconfig_configtype: command 
not found


Hello,

Are you running bash (or other stuff) from experimental? IIRC there is
some breakage there.



I was runnin experimental bash , reverted to unstable but it does not 
change anything (see below).


I'm just wondering with this line in exim4-config.postinst

line 330 :export dc_directives ${dc_directives}


apt-get -f install
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
3 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up exim4-config (4.91-5) ...
/etc/exim4/update-exim4.conf.conf: line 32: dc_eximconfig_configtype: 
command not found
/etc/exim4/update-exim4.conf.conf: line 32: dc_eximconfig_configtype: 
command not found

dpkg: error processing package exim4-config (--configure):
 installed exim4-config package post-installation script subprocess 
returned error exit status 127

Setting up exim4-daemon-light (4.91-5) ...
Job for exim4.service failed because the control process exited with 
error code.

See "systemctl status exim4.service" and "journalctl -xe" for details.
invoke-rc.d: initscript exim4, action "start" failed.
● exim4.service - LSB: exim Mail Transport Agent
   Loaded: loaded (/etc/init.d/exim4; generated)
   Active: failed (Result: exit-code) since Sun 2018-06-10 10:28:22 
CEST; 12ms ago

 Docs: man:systemd-sysv-generator(8)
  Process: 5225 ExecStart=/etc/init.d/exim4 start (code=exited, status=127)

Jun 10 10:28:22 tri-yann4 systemd[1]: Starting LSB: exim Mail Transport 
Agent...
Jun 10 10:28:22 tri-yann4 exim4[5225]: Starting 
MTA:/etc/exim4/update-exim4.conf.conf: line 32: 
dc_eximconfig_configtype: command not found
Jun 10 10:28:22 tri-yann4 systemd[1]: exim4.service: Control process 
exited, code=exited status=127
Jun 10 10:28:22 tri-yann4 systemd[1]: exim4.service: Failed with result 
'exit-code'.
Jun 10 10:28:22 tri-yann4 systemd[1]: Failed to start LSB: exim Mail 
Transport Agent.

dpkg: error processing package exim4-daemon-light (--configure):
 installed exim4-daemon-light package post-installation script 
subprocess returned error exit status 1

dpkg: dependency problems prevent configuration of exim4:
 exim4 depends on exim4-daemon-light | exim4-daemon-heavy | 
exim4-daemon-custom; however:

  Package exim4-daemon-light is not configured yet.
  Package exim4-daemon-heavy is not installed.
  Package exim4-daemon-custom is not installed.

dpkg: error processing package exim4 (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 exim4-config
 exim4-daemon-light
 exim4
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)
tri-yann4:/home/valette#

# /etc/exim4/update-exim4.conf.conf
#
# Edit this file and /etc/mailname by hand and execute update-exim4.conf
# yourself or use 'dpkg-reconfigure exim4-config'
#
# Please note that this is _not_ a dpkg-conffile and that automatic changes
# to this file might happen. The code handling this will honor your local
# changes, so this is usually fine, but will break local schemes that mess
# around with multiple versions of the file.
#
# update-exim4.conf uses this file to determine variable values to replace
# the DEBCONFsomethingDEBCONF strings in the configuration template files.
#
# Most settings found in here do have corresponding questions in the
# Debconf configuration, but not all of them.
#
# This is a Debian specific file

dc_eximconfig_configtype=''
dc_other_hostnames=''
dc_local_interfaces=''
dc_readhost=''
dc_relay_domains=''
dc_minimaldns=''
dc_relay_nets=''
dc_smarthost=''
CFILEMODE=''
dc_use_split_config=''
dc_hide_mailname=''
dc_mailname_in_oh=''
dc_localdelivery=''
dc_eximconfig_configtype dc_other_hostnames dc_local_interfaces dc_readhost 
dc_relay_domains dc_minimaldns dc_relay_nets dc_smarthost CFILEMODE 
dc_use_split_config dc_hide_mailname dc_mailname_in_oh dc_localdelivery=''

Bug#901185: me too

[Andreas Metzler]

On 2018-06-10 積丹尼 Dan Jacobson  wrote:
> Oops. I mean just:
> Setting up exim4-config (4.91-5) ...
> /etc/exim4/update-exim4.conf.conf: line 26: dc_eximconfig_configtype: command 
> not found

Hello,

Are you running bash (or other stuff) from experimental? IIRC there is
some breakage there.

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

Bug#901185: me too

[積丹尼 Dan Jacobson]

Oops. I mean just:
Setting up exim4-config (4.91-5) ...
/etc/exim4/update-exim4.conf.conf: line 26: dc_eximconfig_configtype: command 
not found

Bug#901185: me too

[積丹尼 Dan Jacobson]

Setting up exim4-config (4.91-5) ...
/var/lib/dpkg/info/exim4-config.postinst: line 330: export: 
`dc_eximconfig_configtype dc_other_hostnames dc_local_interfaces dc_readhost 
dc_relay_domains dc_minimaldns dc_relay_nets dc_smarthost CFILEMODE 
dc_use_split_config dc_hide_mailname dc_mailname_in_oh dc_localdelivery': not a 
valid identifier
dpkg: error processing package exim4-config (--configure):