[PATCH 1/1] test: exit with nonzero value when not all tests completed successfully

2013-09-06 Thread Mark Walters 

Hi

On Fri, 06 Sep 2013, Tomi Ollila  wrote:
> If any of the tests in our test system is not passing the execution
> of the test suite completes with nonzero exit value.
>
> It is better to rely on the exit value of the test system instead
> of some arbitrary strings in test output (or use both).

This is obviously a good thing to do: my only query is whether having a
single line setting ev would be clearer. Something along the lines of

if [ "$fixed" = "0 ] && [ "$failed" = "0 ] && [ "$broken" = "0 ] && ["$skipped" 
= "0 ]; then
 ev=1
else 
 ev=0
fi

But I am happy with it as is too. (I haven't actually tested it
currently).

Best wishes

Mark


> ---
>  test/aggregate-results.sh |5 +
>  test/notmuch-test |3 +++
>  2 files changed, 8 insertions(+)
>
> diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
> index 732d6ca..5193d7f 100755
> --- a/test/aggregate-results.sh
> +++ b/test/aggregate-results.sh
> @@ -49,10 +49,12 @@ pluralize () {
>  }
>  
>  echo "Notmuch test suite complete."
> +ev=1
>  if [ "$fixed" = "0" ] && [ "$failed" = "0" ]; then
>  tests=$(pluralize "test" $total)
>  printf "All $total $tests "
>  if [ "$broken" = "0" ]; then
> + ev=0
>   echo "passed."
>  else
>   failures=$(pluralize "failure" $broken)
> @@ -76,6 +78,9 @@ fi
>  
>  skipped=$(($total - $fixed - $success - $failed - $broken))
>  if [ "$skipped" != "0" ]; then
> +ev=1
>  tests=$(pluralize "test" $skipped)
>  echo "$skipped $tests skipped."
>  fi
> +
> +exit $ev
> diff --git a/test/notmuch-test b/test/notmuch-test
> index 6db7979..aa28bb0 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -97,6 +97,9 @@ trap - HUP INT TERM
>  
>  # Report results
>  ./aggregate-results.sh test-results/*
> +ev=$?
>  
>  # Clean up
>  rm -rf test-results corpus.mail
> +
> +exit $ev
> -- 
> 1.7.10.4
>
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/1] test: exit with nonzero value when not all tests completed successfully

2013-09-06 Thread Tomi Ollila 
If any of the tests in our test system is not passing the execution
of the test suite completes with nonzero exit value.

It is better to rely on the exit value of the test system instead
of some arbitrary strings in test output (or use both).
---
 test/aggregate-results.sh |5 +
 test/notmuch-test |3 +++
 2 files changed, 8 insertions(+)

diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
index 732d6ca..5193d7f 100755
--- a/test/aggregate-results.sh
+++ b/test/aggregate-results.sh
@@ -49,10 +49,12 @@ pluralize () {
 }

 echo "Notmuch test suite complete."
+ev=1
 if [ "$fixed" = "0" ] && [ "$failed" = "0" ]; then
 tests=$(pluralize "test" $total)
 printf "All $total $tests "
 if [ "$broken" = "0" ]; then
+   ev=0
echo "passed."
 else
failures=$(pluralize "failure" $broken)
@@ -76,6 +78,9 @@ fi

 skipped=$(($total - $fixed - $success - $failed - $broken))
 if [ "$skipped" != "0" ]; then
+ev=1
 tests=$(pluralize "test" $skipped)
 echo "$skipped $tests skipped."
 fi
+
+exit $ev
diff --git a/test/notmuch-test b/test/notmuch-test
index 6db7979..aa28bb0 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -97,6 +97,9 @@ trap - HUP INT TERM

 # Report results
 ./aggregate-results.sh test-results/*
+ev=$?

 # Clean up
 rm -rf test-results corpus.mail
+
+exit $ev
-- 
1.7.10.4

[PATCH 1/1] test: exit with nonzero value when not all tests completed successfully

2013-09-06 Thread Tomi Ollila 
If any of the tests in our test system is not passing the execution
of the test suite completes with nonzero exit value.

It is better to rely on the exit value of the test system instead
of some arbitrary strings in test output (or use both).
---
 test/aggregate-results.sh |5 +
 test/notmuch-test |3 +++
 2 files changed, 8 insertions(+)

diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
index 732d6ca..5193d7f 100755
--- a/test/aggregate-results.sh
+++ b/test/aggregate-results.sh
@@ -49,10 +49,12 @@ pluralize () {
 }
 
 echo Notmuch test suite complete.
+ev=1
 if [ $fixed = 0 ]  [ $failed = 0 ]; then
 tests=$(pluralize test $total)
 printf All $total $tests 
 if [ $broken = 0 ]; then
+   ev=0
echo passed.
 else
failures=$(pluralize failure $broken)
@@ -76,6 +78,9 @@ fi
 
 skipped=$(($total - $fixed - $success - $failed - $broken))
 if [ $skipped != 0 ]; then
+ev=1
 tests=$(pluralize test $skipped)
 echo $skipped $tests skipped.
 fi
+
+exit $ev
diff --git a/test/notmuch-test b/test/notmuch-test
index 6db7979..aa28bb0 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -97,6 +97,9 @@ trap - HUP INT TERM
 
 # Report results
 ./aggregate-results.sh test-results/*
+ev=$?
 
 # Clean up
 rm -rf test-results corpus.mail
+
+exit $ev
-- 
1.7.10.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/1] test: exit with nonzero value when not all tests completed successfully

2013-09-06 Thread Mark Walters 

Hi

On Fri, 06 Sep 2013, Tomi Ollila tomi.oll...@iki.fi wrote:
 If any of the tests in our test system is not passing the execution
 of the test suite completes with nonzero exit value.

 It is better to rely on the exit value of the test system instead
 of some arbitrary strings in test output (or use both).

This is obviously a good thing to do: my only query is whether having a
single line setting ev would be clearer. Something along the lines of

if [ $fixed = 0 ]  [ $failed = 0 ]  [ $broken = 0 ]  [$skipped 
= 0 ]; then
 ev=1
else 
 ev=0
fi

But I am happy with it as is too. (I haven't actually tested it
currently).

Best wishes

Mark


 ---
  test/aggregate-results.sh |5 +
  test/notmuch-test |3 +++
  2 files changed, 8 insertions(+)

 diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
 index 732d6ca..5193d7f 100755
 --- a/test/aggregate-results.sh
 +++ b/test/aggregate-results.sh
 @@ -49,10 +49,12 @@ pluralize () {
  }
  
  echo Notmuch test suite complete.
 +ev=1
  if [ $fixed = 0 ]  [ $failed = 0 ]; then
  tests=$(pluralize test $total)
  printf All $total $tests 
  if [ $broken = 0 ]; then
 + ev=0
   echo passed.
  else
   failures=$(pluralize failure $broken)
 @@ -76,6 +78,9 @@ fi
  
  skipped=$(($total - $fixed - $success - $failed - $broken))
  if [ $skipped != 0 ]; then
 +ev=1
  tests=$(pluralize test $skipped)
  echo $skipped $tests skipped.
  fi
 +
 +exit $ev
 diff --git a/test/notmuch-test b/test/notmuch-test
 index 6db7979..aa28bb0 100755
 --- a/test/notmuch-test
 +++ b/test/notmuch-test
 @@ -97,6 +97,9 @@ trap - HUP INT TERM
  
  # Report results
  ./aggregate-results.sh test-results/*
 +ev=$?
  
  # Clean up
  rm -rf test-results corpus.mail
 +
 +exit $ev
 -- 
 1.7.10.4

 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch