Re: [lfs-dev] grep : test fails after XFAIL, gnulib tests not run

[Ken Moffat]

On Sun, Aug 12, 2018 at 09:48:19PM +0800, Xi Ruoyao wrote:
> On 2018-08-12 13:03 +0100, Ken Moffat wrote:
> > On Sun, Aug 12, 2018 at 02:19:39PM +0800, Xi Ruoyao wrote:
> 
> It "should" abort because a Glibc bug in 2015.  In Glibc-2.28 the bug
> is finally fixed.
> 
> > [...]
> > > > 
> > > > Looking at the log from the same version (3.1) when I last built it,
> > > > the main tests had two XFAIL, backref-alt and triple-backref but
> > > > WITHOUT an assertion failure.
> > > 
> > > I rebuilt grep-3.1 on my LFS-8.2 with Glibc-2.27.  Both
> > > triple-backref and backref-alt aborted because of fail assertions.
> > > Please recheck your result.
> > > 
> > 
> > On my 20180726 system the results were as I said, no sign of any
> > messages for them except XFAIL, and the tests completed with normal
> > status.
> 
> `grep -B1 Aborted test-suite.log` then you'll see the two tests fail
> because of assertion fail.
> 
I only have the log of what was in stdout and stderr from that
previous build and 'Aborted' does not appear.
> > > 
> > > The "main" test "failed" (XPASS in fact) then `make` just bailed out.
> > > With "make check -k" gnulib-test would run.
> > > 
> > > We can
> > > 
> > > sed '/^am__append_2/d' -i tests/Makefile
> > > 
> > > after configuring Grep, or just use "make check -k" and tell the
> > > audiences there would be a XPASS test with Glibc-2.28.
> 
> > Using 'make -k check' sounds plausible.
> 
> The issue is grep test suite considers XPASS as failure.  I think it makes
> the test suite a fragile "status detector". In the next release of grep
> backref-alt has been removed from XPASS list.
> 
> If I recall, several other packages also have XPASS but they do not
> consider XPASS as failure.

Hmm, in the detail of my current build backref-alt ends with status
0

XPASS backref-alt (exit status: 0)

but the triple-backref is reported as dumping core

+ grep -E '(.?)(.?)(.?)\3\2\1' in
grep: regexec.c:1359: pop_fail_stack: Assertion `num >= 0' failed.
./triple-backref: line 15:  8025 Aborted (core dumped) grep -E 
'(.?)(.?)(.?)\3\2\1' in > out

leading to
XFAIL triple-backref (exit status: 1)

ĸen
-- 
   Entropy not found, thump keyboard to continue

-- 
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page

Re: [lfs-dev] grep : test fails after XFAIL, gnulib tests not run

[Xi Ruoyao]

On 2018-08-12 13:03 +0100, Ken Moffat wrote:
> On Sun, Aug 12, 2018 at 02:19:39PM +0800, Xi Ruoyao wrote:
> > On 2018-08-11 18:07 +0100, Ken Moffat wrote:
> > > I'm running all the tests (except for vim - I think that dislikes my
> > > use of urxvt) and I wasn't intending to look at the results until
> > > later (this build is experimental, to see how much, if any, damage
> > > is caused by various potential BLFS updates).  But for grep 'make
> > > check' unexpectedly failed (status 1) after an XFAIL.
> > > 
> > > XFAIL triple-backref (exit status: 1)
> > 
> > It's a good thing.  Glibc-2.28 fixed a bug so this test should not
> > FAIL now.  See thread
> > .
> > 
> 
> The other backref test (alt-backref) should not fail and it didn't.
> 
> But I don't pay any attention to unexpected passes (XPASS) and I
> don't think we normally mention them.

It "should" abort because a Glibc bug in 2015.  In Glibc-2.28 the bug
is finally fixed.

> [...]
> > > 
> > > Looking at the log from the same version (3.1) when I last built it,
> > > the main tests had two XFAIL, backref-alt and triple-backref but
> > > WITHOUT an assertion failure.
> > 
> > I rebuilt grep-3.1 on my LFS-8.2 with Glibc-2.27.  Both
> > triple-backref and backref-alt aborted because of fail assertions.
> > Please recheck your result.
> > 
> 
> On my 20180726 system the results were as I said, no sign of any
> messages for them except XFAIL, and the tests completed with normal
> status.

`grep -B1 Aborted test-suite.log` then you'll see the two tests fail
because of assertion fail.

> > 
> > The "main" test "failed" (XPASS in fact) then `make` just bailed out.
> > With "make check -k" gnulib-test would run.
> > 
> > We can
> > 
> > sed '/^am__append_2/d' -i tests/Makefile
> > 
> > after configuring Grep, or just use "make check -k" and tell the
> > audiences there would be a XPASS test with Glibc-2.28.

> Using 'make -k check' sounds plausible.

The issue is grep test suite considers XPASS as failure.  I think it makes
the test suite a fragile "status detector". In the next release of grep
backref-alt has been removed from XPASS list.

If I recall, several other packages also have XPASS but they do not
consider XPASS as failure.
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page

Re: [lfs-dev] grep : test fails after XFAIL, gnulib tests not run

[Ken Moffat]

On Sun, Aug 12, 2018 at 02:19:39PM +0800, Xi Ruoyao wrote:
> On 2018-08-11 18:07 +0100, Ken Moffat wrote:
> > I'm running all the tests (except for vim - I think that dislikes my
> > use of urxvt) and I wasn't intending to look at the results until
> > later (this build is experimental, to see how much, if any, damage
> > is caused by various potential BLFS updates).  But for grep 'make
> > check' unexpectedly failed (status 1) after an XFAIL.
> > 
> > XFAIL triple-backref (exit status: 1)
> 
> It's a good thing.  Glibc-2.28 fixed a bug so this test should not
> FAIL now.  See thread
> .
> 
The other backref test (alt-backref) should not fail and it didn't.

But I don't pay any attention to unexpected passes (XPASS) and I
don't think we normally mention them.

[...]
> > 
> > Looking at the log from the same version (3.1) when I last built it,
> > the main tests had two XFAIL, backref-alt and triple-backref but
> > WITHOUT an assertion failure.
> 
> I rebuilt grep-3.1 on my LFS-8.2 with Glibc-2.27.  Both
> triple-backref and backref-alt aborted because of fail assertions.
> Please recheck your result.
> 
On my 20180726 system the results were as I said, no sign of any
messages for them except XFAIL, and the tests completed with normal
status.

> 
> The "main" test "failed" (XPASS in fact) then `make` just bailed out.
> With "make check -k" gnulib-test would run.
> 
> We can
> 
> sed '/^am__append_2/d' -i tests/Makefile
> 
> after configuring Grep, or just use "make check -k" and tell the
> audiences there would be a XPASS test with Glibc-2.28.

Using 'make -k check' sounds plausible.

ĸen
-- 
   Entropy not found, thump keyboard to continue

-- 
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page

Re: [lfs-dev] grep : test fails after XFAIL, gnulib tests not run

[Xi Ruoyao]

On 2018-08-11 18:07 +0100, Ken Moffat wrote:
> I'm running all the tests (except for vim - I think that dislikes my
> use of urxvt) and I wasn't intending to look at the results until
> later (this build is experimental, to see how much, if any, damage
> is caused by various potential BLFS updates).  But for grep 'make
> check' unexpectedly failed (status 1) after an XFAIL.
> 
> XFAIL triple-backref (exit status: 1)

It's a good thing.  Glibc-2.28 fixed a bug so this test should not
FAIL now.  See thread
.

>  The voluminous output in test-suite.log seems to be the same as
> what I logged from stdout and stderr, and looking closely I now
> notice:
> 
> grep: regexec.c:1359: pop_fail_stack: Assertion `num >= 0' failed.
> ./triple-backref: line 15:  8025 Aborted (core dumped) grep 
> -E '(.?)(.?)(.?)\3\2\1' in > out
> + fail=1
> 
> Looking at the log from the same version (3.1) when I last built it,
> the main tests had two XFAIL, backref-alt and triple-backref but
> WITHOUT an assertion failure.

I rebuilt grep-3.1 on my LFS-8.2 with Glibc-2.27.  Both
triple-backref and backref-alt aborted because of fail assertions.
Please recheck your result.

> make[3]: Entering directory '/building/grep-3.1/tests'
> 
> Testsuite summary for GNU grep 3.1
> 
> # TOTAL: 109
> # PASS:  88
> # SKIP:  19
> # XFAIL: 2
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> 
> make[3]: Leaving directory '/building/grep-3.1/tests'
> Making check in gnulib-tests
> 
> and in gnulib-tests on that build 134 tests passed and 7 were
> skipped, but on today's build gnulib is not mentioned in the test
> output.

The "main" test "failed" (XPASS in fact) then `make` just bailed out.
With "make check -k" gnulib-test would run.

We can

sed '/^am__append_2/d' -i tests/Makefile

after configuring Grep, or just use "make check -k" and tell the
audiences there would be a XPASS test with Glibc-2.28.
-- 
Xi Ruoyao 
School of Aerospace Science and Technology, Xidian University

-- 
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page

[lfs-dev] grep : test fails after XFAIL, gnulib tests not run

[Ken Moffat]

I'm running all the tests (except for vim - I think that dislikes my
use of urxvt) and I wasn't intending to look at the results until
later (this build is experimental, to see how much, if any, damage
is caused by various potential BLFS updates).  But for grep 'make
check' unexpectedly failed (status 1) after an XFAIL.

XFAIL triple-backref (exit status: 1)


Testsuite summary for GNU grep 3.1

# TOTAL: 109
# PASS:  88
# SKIP:  19
# XFAIL: 1
# FAIL:  0
# XPASS: 1
# ERROR: 0

See tests/test-suite.log
Please report to bug-g...@gnu.org

make[3]: *** [Makefile:1743: test-suite.log] Error 1
make[3]: Leaving directory '/building/grep-3.1/tests'
make[2]: *** [Makefile:1851: check-TESTS] Error 2
make[1]: *** [Makefile:2685: check-am] Error 2
make: *** [Makefile:1301: check-recursive] Error 1

 The voluminous output in test-suite.log seems to be the same as
what I logged from stdout and stderr, and looking closely I now
notice:

grep: regexec.c:1359: pop_fail_stack: Assertion `num >= 0' failed.
./triple-backref: line 15:  8025 Aborted (core dumped) grep -E 
'(.?)(.?)(.?)\3\2\1' in > out
+ fail=1

Looking at the log from the same version (3.1) when I last built it,
the main tests had two XFAIL, backref-alt and triple-backref but
WITHOUT an assertion failure.

make[3]: Entering directory '/building/grep-3.1/tests'

Testsuite summary for GNU grep 3.1

# TOTAL: 109
# PASS:  88
# SKIP:  19
# XFAIL: 2
# FAIL:  0
# XPASS: 0
# ERROR: 0

make[3]: Leaving directory '/building/grep-3.1/tests'
Making check in gnulib-tests

and in gnulib-tests on that build 134 tests passed and 7 were
skipped, but on today's build gnulib is not mentioned in the test
output.

Smells like a glibc-2.28 problem.

ĸen
-- 
   Entropy not found, thump keyboard to continue

-- 
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page