For some reason that I don't understand, my test system doesn't seem to
reliably run the tests.
- a smattering of tests produce no output.
- a smattering of tests require retransmission of some message, and if
that gets logged on the console, the test is considered to have failed
These failures are mostly unrepeatable. So running the test again
usually succeeds.
(I notice that Andrew has commited something today that prevents some
retransmission attempts; I hope that doesn't make things worse for
me.)
Here's what I do to work around these problems. These are shell
commands.
# change this to keep different results files
N=123
# generate a results-of-each-test file
testing/utils/kvmresults.py testing/pluto/ >tests.LOG$N.results
# find all the tests that produced no output
sed -n -e 's/ unresolved.*//p' tests.LOG$N.results
# find all the tests that produced diff files with a total of
# exactly one retransmit
./JUST-ONE-RETRANS
# rerun the tests that failed for those two reasons
make kvm-check KVM_TESTS="$( sed -n -e 's/ unresolved.*//p' tests.LOG$N.results
; ./JUST-ONE-RETRANS )"
Here's JUST-ONE-RETRANS. It hasn't earned a place in testing/utils
================ JUST-ONE-RETRANS
# this script lists tests that failed with just one retransmission
set -ue
for i in testing/pluto/*/OUTPUT
do
[ "$( cat "$i"/*.diff | wc -c )" == 0 ] ||
{ awk -e '
/^---|^\+\+\+/ { next }
/^\+.*: retransmission; will wait/ && !seen { seen = 1 ; next }
/^[-+]/ { exit 99 }
' "$i"/*.diff &&
dirname "$i" ; }
done
================ end
_______________________________________________
Swan-dev mailing list
Swan-dev@lists.libreswan.org
https://lists.libreswan.org/mailman/listinfo/swan-dev
