On (03/07/08 15:16), David Gibson didst pronounce:
> On Thu, Jul 03, 2008 at 05:41:59AM +0100, Mel Gorman wrote:
> > On (03/07/08 13:32), David Gibson didst pronounce:
> > > On Thu, Jul 03, 2008 at 03:18:29AM +0100, Mel Gorman wrote:
> > > > The recent bug with private-reserves should have been caught during
> > > > 64-bit
> > > > testing but wasn't because a manual verification of the meminfo file was
> > > > necessary and the tests otherwise seemed fine. This patch verifies the
> > > > counters are the same before and after each test in the regression suite
> > > > and aborts the test if it finds anything suspicious.
> > >
> > > Hrm. Given that typically we continue the rest of the tests on
> > > failure, I don't really like the idea of aborting. I'd instead treat
> > > this as a new sort of test result, distinct from PASS, FAIL and
> > > CONFIG.
> > >
> >
> > What would be your suggestion? To me, it's a chronic kernel failure that
> > makes all further tests meaningless. I'd hate for it to be accidently
> > missed.
>
> Eh. Same's true of a testcase SIGKILLed due to a kernel oops.
>
Fair point.
> > > print a summary of the number of passes/failures/etc. at the end of
> > > the run.
> > >
> >
> > Although I like this idea either way.
>
> Well, here you go then.
>
> libhugetlbfs: Print summary of testsuite results
>
> This patch adds some tracking to the run_tests script which counts the
> total number of tests, and the number of passes, failures and other
> results. This is printed as a summary once the script is finished, so
> you can see at a glance if the tests ran clean or not.
>
I like it as this will also be easier to check automatically (currently I
diff the output). I'll redo the counters check on top of this on Monday.
Thanks
> Signed-off-by: David Gibson <[EMAIL PROTECTED]>
>
> ---
> tests/run_tests.sh | 41 ++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 40 insertions(+), 1 deletion(-)
>
> Index: libhugetlbfs/tests/run_tests.sh
> ===================================================================
> --- libhugetlbfs.orig/tests/run_tests.sh 2008-07-03 15:15:46.000000000
> +1000
> +++ libhugetlbfs/tests/run_tests.sh 2008-07-03 15:15:48.000000000 +1000
> @@ -7,6 +7,17 @@ unset HUGETLB_MORECORE
>
> ENV=/usr/bin/env
>
> +for BITS in 32 64; do
> + tot_tests[$BITS]=0
> + tot_tests[$BITS]=0
> + tot_pass[$BITS]=0
> + tot_fail[$BITS]=0
> + tot_config[$BITS]=0
> + tot_signal[$BITS]=0
> + tot_strange[$BITS]=0
> + tot_skip[$BITS]=0
> +done
> +
> function free_hpages() {
> H=$(grep 'HugePages_Free:' /proc/meminfo | cut -f2 -d:)
> [ -z "$H" ] && H=0
> @@ -36,8 +47,22 @@ run_test_bits () {
> shift
>
> if [ -d obj$BITS ]; then
> + tot_tests[$BITS]=$[tot_tests[$BITS] + 1]
> echo -n "$@ ($BITS): "
> - PATH="obj$BITS:$PATH"
> LD_LIBRARY_PATH="$LD_LIBRARY_PATH:../obj$BITS:obj$BITS" $ENV "$@"
> + if PATH="obj$BITS:$PATH"
> LD_LIBRARY_PATH="$LD_LIBRARY_PATH:../obj$BITS:obj$BITS" $ENV "$@"; then
> + tot_pass[$BITS]=$[tot_pass[$BITS] + 1]
> + else
> + rc="$?"
> + if [ "$rc" == "1" ]; then
> + tot_config[$BITS]=$[tot_config[$BITS] + 1]
> + elif [ "$rc" == "2" ]; then
> + tot_fail[$BITS]=$[tot_fail[$BITS] + 1]
> + elif [ "$rc" -gt 127 ]; then
> + tot_signal[$BITS]=$[tot_signal[$BITS] + 1]
> + else
> + tot_strange[$BITS]=$[tot_strange[$BITS] + 1]
> + fi
> + fi
> fi
> }
>
> @@ -49,6 +74,9 @@ run_test () {
>
> skip_test () {
> echo "$@: SKIPPED"
> + for bits in $WORDSIZES; do
> + tot_skip=$[tot_skip + 1]
> + done
> }
>
> preload_test () {
> @@ -281,3 +309,14 @@ for set in $TESTSETS; do
> ;;
> esac
> done
> +
> +echo -e "********** TEST SUMMARY"
> +echo -e "* 32-bit\t64-bit"
> +echo -e "* Total testcases: ${tot_tests[32]}\t${tot_tests[64]}"
> +echo -e "* Skipped: ${tot_skip[32]}\t${tot_skip[64]}"
> +echo -e "* PASS: ${tot_pass[32]}\t${tot_pass[64]}"
> +echo -e "* FAIL: ${tot_fail[32]}\t${tot_fail[64]}"
> +echo -e "* Killed by signal: ${tot_signal[32]}\t${tot_signal[64]}"
> +echo -e "* Bad configuration: ${tot_config[32]}\t${tot_config[64]}"
> +echo -e "* Strange test result: ${tot_strange[32]}\t${tot_strange[64]}"
> +echo -e "**********"
>
>
>
> --
> David Gibson | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_
> _other_
> | _way_ _around_!
> http://www.ozlabs.org/~dgibson
>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Libhugetlbfs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
