Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
On 05 Jun 2020 19:02, Petter Reinholdtsen wrote: > > I discovered what the problem is. The test [ $res ] do not work the way > you want it to. It need to compare with 0, like this: Which is kind of strange, and makes autopkgtest very hard to use since this works fine when run locally like so: # cd # autopkgtest -- null Everything passes when run this way. Easy fix now that you've pointed it out, but passing strange. > diff --git a/debian/tests/test01 b/debian/tests/test01 > index 1c0d63f..73e1fac 100755 > --- a/debian/tests/test01 > +++ b/debian/tests/test01 > @@ -1,4 +1,5 @@ > #!/bin/sh > + > cd debian/tests > > echo "info: compiling" > @@ -7,7 +8,7 @@ echo "info: compiling" > echo "info: running" > cmp -s test01.exp $AUTOPKGTEST_TMP/test01.act > res=$? > -if [ $res ] ; then > +if [ 0 = $res ] ; then > echo "success: test01 produced proper results" > else > echo "error: test01 did not produce proper results" > diff --git a/debian/tests/test02 b/debian/tests/test02 > index fb85d2e..cb4359d 100755 > --- a/debian/tests/test02 > +++ b/debian/tests/test02 > @@ -7,7 +7,7 @@ echo "info: compiling" > echo "info: running" > cmp -s test02.exp $AUTOPKGTEST_TMP/test02.act > res=$? > -if [ $res ] ; then > +if [ 0 == $res ] ; then > echo "success: test02 produced proper results" > else > echo "error: test02 did not produce proper results" > diff --git a/debian/tests/test03 b/debian/tests/test03 > index c028d8b..07d679c 100755 > --- a/debian/tests/test03 > +++ b/debian/tests/test03 > @@ -7,7 +7,7 @@ echo "info: compiling" > echo "info: running" > cmp -s test03.exp $AUTOPKGTEST_TMP/test03.act > res=$? > -if [ $res ] ; then > +if [ 0 == $res ] ; then > echo "success: test03 produced proper results" > else > echo "error: test03 did not produce proper results" > diff --git a/debian/tests/test04 b/debian/tests/test04 > index fd2a6ad..ee31d4a 100755 > --- a/debian/tests/test04 > +++ b/debian/tests/test04 > @@ -7,7 +7,7 @@ echo "info: compiling" > echo "info: running" > cmp -s test04.exp t$AUTOPKGTEST_TMP/est04.act > res=$? > -if [ $res ] ; then > +if [ 0 == $res ] ; then > echo "success: test04 produced proper results" > else > echo "error: test04 did not produce proper results" > > -- > Happy hacking > Petter Reinholdtsen > -- Ciao, al -- Al Stone Debian Developer E-mail: a...@ahs3.nethttp://www.debian.org a...@debian.org --
Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
I discovered what the problem is. The test [ $res ] do not work the way you want it to. It need to compare with 0, like this: diff --git a/debian/tests/test01 b/debian/tests/test01 index 1c0d63f..73e1fac 100755 --- a/debian/tests/test01 +++ b/debian/tests/test01 @@ -1,4 +1,5 @@ #!/bin/sh + cd debian/tests echo "info: compiling" @@ -7,7 +8,7 @@ echo "info: compiling" echo "info: running" cmp -s test01.exp $AUTOPKGTEST_TMP/test01.act res=$? -if [ $res ] ; then +if [ 0 = $res ] ; then echo "success: test01 produced proper results" else echo "error: test01 did not produce proper results" diff --git a/debian/tests/test02 b/debian/tests/test02 index fb85d2e..cb4359d 100755 --- a/debian/tests/test02 +++ b/debian/tests/test02 @@ -7,7 +7,7 @@ echo "info: compiling" echo "info: running" cmp -s test02.exp $AUTOPKGTEST_TMP/test02.act res=$? -if [ $res ] ; then +if [ 0 == $res ] ; then echo "success: test02 produced proper results" else echo "error: test02 did not produce proper results" diff --git a/debian/tests/test03 b/debian/tests/test03 index c028d8b..07d679c 100755 --- a/debian/tests/test03 +++ b/debian/tests/test03 @@ -7,7 +7,7 @@ echo "info: compiling" echo "info: running" cmp -s test03.exp $AUTOPKGTEST_TMP/test03.act res=$? -if [ $res ] ; then +if [ 0 == $res ] ; then echo "success: test03 produced proper results" else echo "error: test03 did not produce proper results" diff --git a/debian/tests/test04 b/debian/tests/test04 index fd2a6ad..ee31d4a 100755 --- a/debian/tests/test04 +++ b/debian/tests/test04 @@ -7,7 +7,7 @@ echo "info: compiling" echo "info: running" cmp -s test04.exp t$AUTOPKGTEST_TMP/est04.act res=$? -if [ $res ] ; then +if [ 0 == $res ] ; then echo "success: test04 produced proper results" else echo "error: test04 did not produce proper results" -- Happy hacking Petter Reinholdtsen
Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
[Al Stone] > Hrm. So write access seems to be more constrained than this led > me to believe; I'll try these patches out and rebuild tonight. I > did think about using AUTOPKGTEST_TMP but had not convinced myself > it was absolutely required. Note, I am not convinced this will help, as I am flying blind without the error messages, but thought it best to play it safe and use AUTOPKGTEST_TMP just in case. After all the 'hello' test seem to work, and it would try to save directly to cwd which should be the top of the source tree. :/ > Thank you for all the help, Petter! Just glad to be of assistance. :) -- Happy hacking Petter Reinholdtsen
Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
On 03 Jun 2020 14:30, Petter Reinholdtsen wrote: > > Could > https://people.debian.org/~mpitt/autopkgtest/README.package-tests.html > > contain the key to why these test are failing? It states > > "The cwd of each test is guaranteed to be the root of the source > package, which will have been unpacked but not built. However note > that the tests must test the installed version of the program. Tests > may not modify the source tree (and may not have write access to it)." Hrm. So write access seems to be more constrained than this led me to believe; I'll try these patches out and rebuild tonight. I did think about using AUTOPKGTEST_TMP but had not convinced myself it was absolutely required. Thank you for all the help, Petter! > If so, the following patch might help: > > diff --git a/debian/tests/hello b/debian/tests/hello > index 60943be..15c3985 100755 > --- a/debian/tests/hello > +++ b/debian/tests/hello > @@ -1,4 +1,5 @@ > #!/bin/sh > +cd $AUTOPKGTEST_TMP > cat > HELLO.cob< HELLO * HISTORIC EXAMPLE OF HELLO WORLD IN COBOL > IDENTIFICATION DIVISION. > diff --git a/debian/tests/test01 b/debian/tests/test01 > index 22d943e..783901b 100755 > --- a/debian/tests/test01 > +++ b/debian/tests/test01 > @@ -2,10 +2,10 @@ > cd debian/tests > > echo "info: compiling" > -(cobc test01.cob > test01.act 2>&1) > +(cobc test01.cob > $AUTOPKGTEST_TMP/test01.act 2>&1) > > echo "info: running" > -cmp -s test01.exp test01.act > +cmp -s test01.exp $AUTOPKGTEST_TMP/test01.act > res=$? > if [ $res ] ; then > echo "success: test01 produced proper results" > diff --git a/debian/tests/test02 b/debian/tests/test02 > index 2e06f74..99f0978 100755 > --- a/debian/tests/test02 > +++ b/debian/tests/test02 > @@ -2,10 +2,10 @@ > cd debian/tests > > echo "info: compiling" > -(cobc test02.cob > test02.act 2>&1) > +(cobc test02.cob > $AUTOPKGTEST_TMP/test02.act 2>&1) > > echo "info: running" > -cmp -s test02.exp test02.act > +cmp -s test02.exp $AUTOPKGTEST_TMP/test02.act > res=$? > if [ $res ] ; then > echo "success: test02 produced proper results" > diff --git a/debian/tests/test03 b/debian/tests/test03 > index f378787..701ade9 100755 > --- a/debian/tests/test03 > +++ b/debian/tests/test03 > @@ -2,10 +2,10 @@ > cd debian/tests > > echo "info: compiling" > -(cobc test03.cob > test03.act 2>&1) > +(cobc test03.cob > $AUTOPKGTEST_TMP/test03.act 2>&1) > > echo "info: running" > -cmp -s test03.exp test03.act > +cmp -s test03.exp $AUTOPKGTEST_TMP/test03.act > res=$? > if [ $res ] ; then > echo "success: test03 produced proper results" > diff --git a/debian/tests/test04 b/debian/tests/test04 > index 04b325e..c50b157 100755 > --- a/debian/tests/test04 > +++ b/debian/tests/test04 > @@ -2,10 +2,10 @@ > cd debian/tests > > echo "info: compiling" > -(cobc test04.cob > test04.act 2>&1) > +(cobc test04.cob > $AUTOPKGTEST_TMP/test04.act 2>&1) > > echo "info: running" > -cmp -s test04.exp test04.act > +cmp -s test04.exp t$AUTOPKGTEST_TMP/est04.act > res=$? > if [ $res ] ; then > echo "success: test04 produced proper results" > > -- > Happy hacking > Petter Reinholdtsen > -- Ciao, al -- Al Stone Debian Developer E-mail: a...@ahs3.nethttp://www.debian.org a...@debian.org --
Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
Could https://people.debian.org/~mpitt/autopkgtest/README.package-tests.html > contain the key to why these test are failing? It states "The cwd of each test is guaranteed to be the root of the source package, which will have been unpacked but not built. However note that the tests must test the installed version of the program. Tests may not modify the source tree (and may not have write access to it)." If so, the following patch might help: diff --git a/debian/tests/hello b/debian/tests/hello index 60943be..15c3985 100755 --- a/debian/tests/hello +++ b/debian/tests/hello @@ -1,4 +1,5 @@ #!/bin/sh +cd $AUTOPKGTEST_TMP cat > HELLO.cob< test01.act 2>&1) +(cobc test01.cob > $AUTOPKGTEST_TMP/test01.act 2>&1) echo "info: running" -cmp -s test01.exp test01.act +cmp -s test01.exp $AUTOPKGTEST_TMP/test01.act res=$? if [ $res ] ; then echo "success: test01 produced proper results" diff --git a/debian/tests/test02 b/debian/tests/test02 index 2e06f74..99f0978 100755 --- a/debian/tests/test02 +++ b/debian/tests/test02 @@ -2,10 +2,10 @@ cd debian/tests echo "info: compiling" -(cobc test02.cob > test02.act 2>&1) +(cobc test02.cob > $AUTOPKGTEST_TMP/test02.act 2>&1) echo "info: running" -cmp -s test02.exp test02.act +cmp -s test02.exp $AUTOPKGTEST_TMP/test02.act res=$? if [ $res ] ; then echo "success: test02 produced proper results" diff --git a/debian/tests/test03 b/debian/tests/test03 index f378787..701ade9 100755 --- a/debian/tests/test03 +++ b/debian/tests/test03 @@ -2,10 +2,10 @@ cd debian/tests echo "info: compiling" -(cobc test03.cob > test03.act 2>&1) +(cobc test03.cob > $AUTOPKGTEST_TMP/test03.act 2>&1) echo "info: running" -cmp -s test03.exp test03.act +cmp -s test03.exp $AUTOPKGTEST_TMP/test03.act res=$? if [ $res ] ; then echo "success: test03 produced proper results" diff --git a/debian/tests/test04 b/debian/tests/test04 index 04b325e..c50b157 100755 --- a/debian/tests/test04 +++ b/debian/tests/test04 @@ -2,10 +2,10 @@ cd debian/tests echo "info: compiling" -(cobc test04.cob > test04.act 2>&1) +(cobc test04.cob > $AUTOPKGTEST_TMP/test04.act 2>&1) echo "info: running" -cmp -s test04.exp test04.act +cmp -s test04.exp t$AUTOPKGTEST_TMP/est04.act res=$? if [ $res ] ; then echo "success: test04 produced proper results" -- Happy hacking Petter Reinholdtsen
Bug#962081: gnucobol: Failing autopkgtest scripts should report what went wrong
Package: gnucobol Version: 3.0~rc1-2 Severity: serious Tags: patch Several of the automatic tests provided in debian/tests/ currently fail. I am unable to see from the logs what go wrong, but the failure block the package from migrating into testing. When I run the scripts locally, they work. The tests should be fixed to work also in the CI infrastructure. I propose the following patch to make it easier to figure out from the logs what is going wrong: diff --git a/debian/tests/test01 b/debian/tests/test01 index 22d943e..e8f2fc1 100755 --- a/debian/tests/test01 +++ b/debian/tests/test01 @@ -11,6 +11,7 @@ if [ $res ] ; then echo "success: test01 produced proper results" else echo "error: test01 did not produce proper results" +diff -u test01.exp test01.act fi exit $res diff --git a/debian/tests/test02 b/debian/tests/test02 index 2e06f74..ee6caa9 100755 --- a/debian/tests/test02 +++ b/debian/tests/test02 @@ -11,6 +11,7 @@ if [ $res ] ; then echo "success: test02 produced proper results" else echo "error: test02 did not produce proper results" +diff -u test02.exp test02.act fi exit $res diff --git a/debian/tests/test03 b/debian/tests/test03 index f378787..fedf826 100755 --- a/debian/tests/test03 +++ b/debian/tests/test03 @@ -11,6 +11,7 @@ if [ $res ] ; then echo "success: test03 produced proper results" else echo "error: test03 did not produce proper results" +diff -u test03.exp test03.act fi exit $res diff --git a/debian/tests/test04 b/debian/tests/test04 index 04b325e..ff3a33d 100755 --- a/debian/tests/test04 +++ b/debian/tests/test04 @@ -11,6 +11,7 @@ if [ $res ] ; then echo "success: test04 produced proper results" else echo "error: test04 did not produce proper results" +diff -u test04.exp test04.act fi exit $res -- Happy hacking Petter Reinholdtsen
