bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-19 Thread Pádraig Brady 
On 17/08/17 21:55, A. Wilcox wrote:
> On 13/08/17 11:22, Jim Meyering wrote:
>> On Sun, Aug 13, 2017 at 1:07 AM, Pádraig Brady 
>> wrote:
>>> On 11/08/17 11:49, A. Wilcox wrote:
>>>
 FAIL: tests/misc/csplit-io-err ==
>>> This was due to an inconsistency in the errors output by seq. A
>>> fix for that buglet is attached.
>>>
 FAIL: tests/misc/printf-surprise 
 
>>> Depending on exit 141 couples the script to the value of SIGPIPE 
>>> and to the shell implementation (ksh will return 269 here for
>>> e.g.). So I've attached a solution that should hopefully work in
>>> all situations.
>>>
 FAIL: tests/misc/sort-debug-warn 
 
>>> This was due to an assumption that for "missing" locales that
>>> sort would fail to C rather than C.UTF8. I've adjusted sort to
>>> not assume that in the attached.
> 
>> Nice work. All three patches looks fine.  You might want to insert
>> "in https://bugs.gnu.org/28054"; at the end of each log message.
> 
>> Considering the use of "yes | :", even though there's an identical 
>> prior use in tests/misc/seq-epipe.sh, I wondered if there exists a 
>> shell/system where that would infloop. Maybe paranoid overkill,
>> but inserting a timeout would avoid the possibility.
> 
> 
> I can confirm that seq and printf-surprise are fixed by these patches.
>  The sort-debug-warn test still fails, because "failed to set locale"
> is not printed on musl (missing locales are redirected to C.UTF8).
> 
> Also, the seq-epipe patch fixes the grep but not the comment directly
> above it.
> 
> Thank you for your efforts and the fixes.

Ah Assaf already mentioned this musl issue on this list,
and followed up with musl at:
http://www.openwall.com/lists/musl/2016/04/02/1

I've installed a patch to handle these issues at:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=4b5bc54

thanks,
Pádraig

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-17 Thread A. Wilcox 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 13/08/17 11:22, Jim Meyering wrote:
> On Sun, Aug 13, 2017 at 1:07 AM, Pádraig Brady 
> wrote:
>> On 11/08/17 11:49, A. Wilcox wrote:
>> 
>>> FAIL: tests/misc/csplit-io-err ==
>> This was due to an inconsistency in the errors output by seq. A
>> fix for that buglet is attached.
>> 
>>> FAIL: tests/misc/printf-surprise 
>>> 
>> Depending on exit 141 couples the script to the value of SIGPIPE 
>> and to the shell implementation (ksh will return 269 here for
>> e.g.). So I've attached a solution that should hopefully work in
>> all situations.
>> 
>>> FAIL: tests/misc/sort-debug-warn 
>>> 
>> This was due to an assumption that for "missing" locales that
>> sort would fail to C rather than C.UTF8. I've adjusted sort to
>> not assume that in the attached.
> 
> Nice work. All three patches looks fine.  You might want to insert
> "in https://bugs.gnu.org/28054"; at the end of each log message.
> 
> Considering the use of "yes | :", even though there's an identical 
> prior use in tests/misc/seq-epipe.sh, I wondered if there exists a 
> shell/system where that would infloop. Maybe paranoid overkill,
> but inserting a timeout would avoid the possibility.
> 

I can confirm that seq and printf-surprise are fixed by these patches.
 The sort-debug-warn test still fails, because "failed to set locale"
is not printed on musl (missing locales are redirected to C.UTF8).

Also, the seq-epipe patch fixes the grep but not the comment directly
above it.

Thank you for your efforts and the fixes.

Best,
- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJZlnMzAAoJEMspy1GSK50UUsIQALmcw3yfChX9yrY/RwGTTKtP
FX3K6h1IoTOh9pQRN3cO/AitUYTrQ0CMF3HyOw1eY41g2Cy7l5uFf36bZuXYOghc
WkWk8K3uuvlvkJbIDbszsVzXkHjhoigAC0mcljORXPrTXCS4CpmcRLlaH/HI0niV
p1pk64dvNJhgdD/O7M9vatj8s0RH86HleWf0Jl87Vl7wBXfPnB5V4Uu9hpCxRWy6
j0pfZpaOmqbAYLxjWDWfOlfJq0BOFKrtTy4SHhUF1r2CG88WVVSpr1RD/bM+SVkr
R9LT/XMXw1tZ82z76AIn2+4KFaunRGhSbOulmhs2yCHORHhgc8jyomSvS8oh47yK
EmMgT//VN6hJsKDBYQ2fkiFvrRd4gbf2tXKU92J1KfewoB23wgHvG1EahiSCFUA5
la9xSIZ4X8IIGIiNnHvooxITITdC8oGct8XaLcBfiHYz2GXihAyI0gX+Sc4gL7PI
uyK9C2/I3Tw6zJslU+lPtCCwGVUpwLH1LWRxy0YAQn8ejI7eOeK6QNvmsX/tJXEf
5NjauAA5Hca4d11Nq6hRZxkTAsEdbvxupvOQ9EcWoRyLqIj2ayaCvP7VuBjrNYLF
huL7oXUGPD81duPaDE1ALPyflG9O70NEmQdXVqv3mLSpkdmbKNzjZZJJ80UV6lXw
eLgWSdw8MIQqXUNYMnmo
=h0SV
-END PGP SIGNATURE-

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-13 Thread Pádraig Brady 
On 13/08/17 09:22, Jim Meyering wrote:
> On Sun, Aug 13, 2017 at 1:07 AM, Pádraig Brady  wrote:
>> On 11/08/17 11:49, A. Wilcox wrote:
>>
>>> FAIL: tests/misc/csplit-io-err
>>> ==
>> This was due to an inconsistency in the errors output by seq.
>> A fix for that buglet is attached.
>>
>>> FAIL: tests/misc/printf-surprise
>>> 
>> Depending on exit 141 couples the script to the value of SIGPIPE
>> and to the shell implementation (ksh will return 269 here for e.g.).
>> So I've attached a solution that should hopefully work in all situations.
>>
>>> FAIL: tests/misc/sort-debug-warn
>>> 
>> This was due to an assumption that for "missing" locales
>> that sort would fail to C rather than C.UTF8.
>> I've adjusted sort to not assume that in the attached.
> 
> Nice work. All three patches looks fine.  You might want to insert "in
> https://bugs.gnu.org/28054"; at the end of each log message.

Good point. I'll adjust.

> Considering the use of "yes | :", even though there's an identical
> prior use in tests/misc/seq-epipe.sh, I wondered if there exists a
> shell/system where that would infloop. Maybe paranoid overkill, but
> inserting a timeout would avoid the possibility.

Also good point, and I vaguely remember there being such an issue
on systems where pipes were simulated using files.
Though timeout would have the same portability issues as such pipes I think.
I'll look at refactoring this pattern in a separate patch
and cater for all uses of this pattern with a
require_regular_pipe_ or equivalent.

thanks,
Pádraig

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-13 Thread Jim Meyering 
On Sun, Aug 13, 2017 at 1:07 AM, Pádraig Brady  wrote:
> On 11/08/17 11:49, A. Wilcox wrote:
>
>> FAIL: tests/misc/csplit-io-err
>> ==
> This was due to an inconsistency in the errors output by seq.
> A fix for that buglet is attached.
>
>> FAIL: tests/misc/printf-surprise
>> 
> Depending on exit 141 couples the script to the value of SIGPIPE
> and to the shell implementation (ksh will return 269 here for e.g.).
> So I've attached a solution that should hopefully work in all situations.
>
>> FAIL: tests/misc/sort-debug-warn
>> 
> This was due to an assumption that for "missing" locales
> that sort would fail to C rather than C.UTF8.
> I've adjusted sort to not assume that in the attached.

Nice work. All three patches looks fine.  You might want to insert "in
https://bugs.gnu.org/28054"; at the end of each log message.

Considering the use of "yes | :", even though there's an identical
prior use in tests/misc/seq-epipe.sh, I wondered if there exists a
shell/system where that would infloop. Maybe paranoid overkill, but
inserting a timeout would avoid the possibility.

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-13 Thread Pádraig Brady 
On 11/08/17 11:49, A. Wilcox wrote:

> FAIL: tests/misc/csplit-io-err
> ==
This was due to an inconsistency in the errors output by seq.
A fix for that buglet is attached.

> FAIL: tests/misc/printf-surprise
> 
Depending on exit 141 couples the script to the value of SIGPIPE
and to the shell implementation (ksh will return 269 here for e.g.).
So I've attached a solution that should hopefully work in all situations.

> FAIL: tests/misc/sort-debug-warn
> 
This was due to an assumption that for "missing" locales
that sort would fail to C rather than C.UTF8.
I've adjusted sort to not assume that in the attached.

thanks,
Pádraig
From 1e1dbbe0f95c7170009f3ba4d8380c1b98ff53c8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= 
Date: Sun, 13 Aug 2017 00:18:43 -0700
Subject: [PATCH] seq: produce consistent error messages upon write error

* src/seq.c (io_error): Use the same error message as would
be generated at exit time when closing the stdout stream.
The inconsistency was added with commit v8.25-26-gc92585b.
This was noticed due to an inconsistency in the expected
error message generated by seq on musl libc.
Reported by A. Wilcox.
---
 src/seq.c   | 2 +-
 tests/misc/seq-epipe.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/seq.c b/src/seq.c
index bb191ba..5e5b381 100644
--- a/src/seq.c
+++ b/src/seq.c
@@ -284,7 +284,7 @@ io_error (void)
 {
   /* FIXME: consider option to silently ignore errno=EPIPE */
   clearerr (stdout);
-  die (EXIT_FAILURE, errno, _("standard output"));
+  die (EXIT_FAILURE, errno, _("write error"));
 }
 
 /* Actually print the sequence of numbers in the specified range, with the
diff --git a/tests/misc/seq-epipe.sh b/tests/misc/seq-epipe.sh
index edbd563..69dd75d 100755
--- a/tests/misc/seq-epipe.sh
+++ b/tests/misc/seq-epipe.sh
@@ -33,7 +33,7 @@ compare exp code || fail=1
 
 # The error message must begin with "standard output:"
 # (but don't hard-code the strerror text)
-grep '^seq: standard output: ' err \
+grep '^seq: write error: ' err \
   || { warn_ "seq emitted incorrect error on EPIPE"; \
cat err;\
fail=1; }
-- 
2.9.3

From ad5c14dcadd4603363e68da960ed0101b61439bd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= 
Date: Sat, 12 Aug 2017 23:53:33 -0700
Subject: [PATCH] tests: fix false failure with large printf formats

* tests/misc/printf-surprise.sh: With musl libc the
large printf format does succeed, outputting data.
To avoid SIGPIPE being generated we ignore that signal
and then handle the subsequent EPIPE error.
Reported by A. Wilcox
---
 tests/misc/printf-surprise.sh | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/misc/printf-surprise.sh b/tests/misc/printf-surprise.sh
index 2edd34c..46c0d8f 100755
--- a/tests/misc/printf-surprise.sh
+++ b/tests/misc/printf-surprise.sh
@@ -50,6 +50,9 @@ vm=$(get_min_ulimit_v_ env $prog %20f 0) \
 
 mkfifo_or_skip_ fifo
 
+(trap '' PIPE && yes | :) 2>&1 | grep -qF 'Broken pipe' ||
+skip_ 'trapping SIGPIPE is not supported'
+
 # Disable MALLOC_PERTURB_, to avoid triggering this bug
 # https://bugs.debian.org/481543#77
 export MALLOC_PERTURB_=0
@@ -60,11 +63,11 @@ cleanup_() { kill $pid 2>/dev/null && wait $pid; }
 head -c 10 fifo > out & pid=$!
 
 # Trigger large mem allocation failure
-( ulimit -v $vm && env $prog %2000f 0 2>err-msg > fifo )
+( trap '' PIPE && ulimit -v $vm && env $prog %2000f 0 2>err-msg > fifo )
 exit=$?
 
 # Map this longer, and rarer, diagnostic to the common one.
-# printf: cannot perform formatted output: Cannot allocate memory" \
+# printf: cannot perform formatted output: Cannot allocate memory"
 sed 's/cannot perform .*/write error/' err-msg > k && mv k err-msg
 err_msg=$(tr '\n' : < err-msg)
 
@@ -81,6 +84,7 @@ n_out=$(wc -c < out)
 
 case $n_out:$diagnostic:$exit in
   10:n:0) ;; # ok, succeeds w/no diagnostic: FreeBSD 6.1
+  10:y:1) ;; # ok, fails with EPIPE diagnostic: musl libc
   0:y:1)  ;; # ok, glibc-2.8 and newer, when printf(3) fails with ENOMEM
 
   # With MALLOC_PERTURB_=0, this no longer happens.
-- 
2.9.3

From b938d8966c3d1afec9730d403fd6a6eb4d856b85 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= 
Date: Sun, 13 Aug 2017 00:45:32 -0700
Subject: [PATCH] sort: handle musl locale differences in --debug reporting

* src/sort.c (main): Don't assume hard_LC_COLLATE implies
a successful setting of the locale as musl defaults to
UTF8 when failing to set the specified locale.
* tests/misc/sort-debug-warn.sh: Adjust for the now
separated locale debug info and map the musl specific
message back to the common case.
Reported by A. Wilcox.
---
 src/sort.c| 17 +
 tests/misc/sort-debug-warn.sh |  9 +++--
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 015e40e..ba6ceac 100644
--- a/src/sort.c
+++

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-11 Thread A. Wilcox 
Probing further at printf-surprise.sh, it appears that the problem is:

* musl is unaffected by the printf bug being tested

* therefore, it does successfully output 20 MiB of blank space

* `head -c 10 fifo` will read the first 10 characters of the fifo

* the rest of the 20 MiB cannot be written, so SIGPIPE is received

* 'exit=141' (- 128 = 13 = SIGPIPE)


I have attached a patch that fixes both csplit-io-err and
printf-surprise on my test box.

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
--- coreutils-8.27/tests/misc/csplit-io-err.sh.old	2017-01-01 22:34:24.0 +
+++ coreutils-8.27/tests/misc/csplit-io-err.sh	2017-08-11 20:07:31.094710635 +
@@ -54,6 +54,6 @@
 # Get the wording of the OS-dependent ENOSPC message
 returns_ 1 seq 1 >/dev/full 2>msgt || framework_failure_
-sed 's/seq: write error: //' msgt > msg || framework_failure_
+sed 's/seq: .*: //' msgt > msg || framework_failure_
 
 # Create the expected error message
 { printf "%s" "csplit: write error for 'xx01': " ; cat msg ; } > exp \
--- coreutils-8.27/tests/misc/printf-surprise.sh.old	2017-01-21 14:53:43.0 +
+++ coreutils-8.27/tests/misc/printf-surprise.sh	2017-08-11 20:20:00.769708921 +
@@ -80,6 +80,7 @@
 n_out=$(wc -c < out)
 
 case $n_out:$diagnostic:$exit in
+  10:n:141) ;; # ok, succeeds w/SIGPIPE due to lots of output: Linux
   10:n:0) ;; # ok, succeeds w/no diagnostic: FreeBSD 6.1
   0:y:1)  ;; # ok, glibc-2.8 and newer, when printf(3) fails with ENOMEM
 


signature.asc
Description: OpenPGP digital signature

bug#28054: coreutils 8.27 test failure on x86_64-foxkit-linux-musl

2017-08-11 Thread A. Wilcox 
The first test failure, csplit-io-err, seems to be a simple error
message format change.

The third test failure, sort-debug-warn, seems to be sensitive to
locales.  I am not sure the best way to fix this.

Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org

   GNU coreutils 8.27: ./tests/test-suite.log


# TOTAL: 592
# PASS:  453
# SKIP:  136
# XFAIL: 0
# FAIL:  3
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: tests/misc/csplit-io-err
==

++ initial_cwd_=/usr/src/aports/main/coreutils/src/coreutils-8.27
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /usr/src/aports/main/coreutils/src/coreutils-8.27 gt-csplit-io-err.sh.
+++ case $# in
+++ destdir_=/usr/src/aports/main/coreutils/src/coreutils-8.27
+++ template_=gt-csplit-io-err.sh.
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ destdir_slash_=/usr/src/aports/main/coreutils/src/coreutils-8.27/
+++ case $template_ in
 unset TMPDIR
+++ d=/usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
+++ case $d in
+++ :
+++ test -d /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
 ls -dgo /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
+++ perms='drwx--S--- 2 4096 Aug 11 18:43 /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5'
+++ case $perms in
+++ :
+++ echo /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
+++ return
++ test_dir_=/usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
++ cd /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
++ gl_init_sh_nl_='
'
++ IFS=' 	
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ ./src
+ test 1 '!=' 0
+ path_dir_=./src
+ case $path_dir_ in
+ abs_path_dir_=/usr/src/aports/main/coreutils/src/coreutils-8.27/./src
+ case $abs_path_dir_ in
+ PATH=/usr/src/aports/main/coreutils/src/coreutils-8.27/./src:/usr/src/aports/main/coreutils/src/coreutils-8.27/src:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin:/sbin
+ create_exe_shims_ /usr/src/aports/main/coreutils/src/coreutils-8.27/./src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ print_ver_ csplit
+ require_built_ csplit
+ skip_=no
+ for i in '"$@"'
+ case " $built_programs " in
+ test no = yes
+ test yes = yes
+ local i
+ for i in '$*'
+ env csplit --version
csplit (GNU coreutils) 8.27
Packaged by Adélie
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Stuart Kemp and David MacKenzie.
+ require_gcc_shared_
+ gcc_shared_ - d.so -xc
+ local in=-
+ local out=d.so
+ shift 2
+ gcc -Wall -shared --std=gnu99 -fPIC -O2 -xc - -o d.so -ldl
+ rm -f d.so
+ test -w /dev/full
+ test -c /dev/full
+ LC_ALL=C
+ export LC_ALL
+ cat
+ returns_ 1 seq 1
+ sed 's/seq: write error: //' msgt
+ printf %s 'csplit: write error for '\''xx01'\'': '
+ cat msg
+ gcc_shared_ k.c k.so
+ local in=k.c
+ local out=k.so
+ shift 2
+ gcc -Wall -shared --std=gnu99 -fPIC -O2 k.c -o k.so -ldl
+ export LD_PRELOAD=:./k.so
+ LD_PRELOAD=:./k.so
+ returns_ 1 csplit - 1 4
+ seq 10
0
+ test -e preloaded
+ compare exp out
+ compare_dev_null_ exp out
+ test 2 = 2
+ test xexp = x/dev/null
+ test xout = x/dev/null
+ return 2
+ case $? in
+ compare_ exp out
+ diff -u exp out
--- exp	2017-08-11 18:43:48.292960030 +
+++ out	2017-08-11 18:43:48.339626571 +
@@ -1 +1 @@
-csplit: write error for 'xx01': seq: standard output: No space left on device
+csplit: write error for 'xx01': No space left on device
+ fail=1
+ Exit 1
+ set +e
+ exit 1
+ exit 1
+ remove_tmp_
+ __st=1
+ cleanup_
+ :
+ test '' = yes
+ cd /usr/src/aports/main/coreutils/src/coreutils-8.27
+ chmod -R u+rwx /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
+ rm -rf /usr/src/aports/main/coreutils/src/coreutils-8.27/gt-csplit-io-err.sh.OPG5
+ exit 1
FAIL tests/misc/csplit-io-err.sh (exit status: 1)

FAIL: tests/misc/printf-surprise


++ initial_cwd_=/usr/src/aports/main/coreutils/src/coreutils-8.27
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /usr/src/aports/main/coreutils/src/coreutils-8.27 gt-printf-surprise.sh