Hi Frederic,
Op 2-10-2018 om 8:58 schreef Frederic Lecaille:
At this time I do not know how to skip the tests which are not supported
due to disable features at compilation time without running a
"haproxy -vv" command before launching the VTC files.
Made a little script (attached)..
It would allow for the .vtc files to contain what 'requirements' they
have, by including lines like these:
#EXCLUDE_TARGET=freebsd, abns sockets are not available on freebsd
#REQUIRE_OPTION=OPENSSL, this test needs ssl compiled in.
Which could then generate output like listed below. It searches for all
testcases, validates their requirements, and then puts them in a list.
After which it calls varnishtest with quiet and simultaneous job
parameters to allow for a short turnaround time..
What do you think ? Could this work for current 'requirements'? Or is
something different preferred? I though about running each test directly
from the script, and while that allows per test OK/Fail response logging
for and a nicer overall summery, it takes more time to run. So i went
for automatically creating a list of tests and letting varnishtest
itself handle the running of multiple tests at once.
Thoughts appreciated :).
Regards,
PiBa-NL (Pieter)
root@freebsd11:/usr/ports/net/haproxy-devel # ./run-regtests.sh
########################## Preparing to run tests ##########################
HA-Proxy version 1.8.14-52e4d43 2018/09/20
Copyright 2000-2018 Willy Tarreau <[email protected]>
########################## Gathering tests to run ##########################
Skip ./haproxy_test_OK_20180831/lua/b00002.vtc because option OPENSSL
not found
REASON: this test needs ssl compiled in.
Add test: ./haproxy_test_OK_20180831/lua/h00001.vtc
Add test: ./haproxy_test_OK_20180831/lua/b00000.vtc
Add test: ./haproxy_test_OK_20180831/lua/b00001.vtc
Add test: ./haproxy_test_OK_20180831/loadtest/b00000-loadtest.vtc
Add test: ./haproxy_test_OK_20180831/log/b00000.vtc
Skip ./haproxy_test_OK_20180831/reload/b00001-server-state-file.vtc
because: TARGET = freebsd
REASON: abns sockets are not available on freebsd
Skip ./haproxy_test_OK_20180831/ssl/b00000.vtc because option OPENSSL
not found
REASON: this test needs ssl compiled in.
Add test: ./haproxy_test_OK_20180831/stick-table/b00001.vtc
Add test: ./haproxy_test_OK_20180831/stick-table/b00000.vtc
Add test: ./test/b00000-loadtest.vtc
Skip ./test/b00002.vtc because option OPENSSL not found
REASON: this test needs ssl compiled in.
Add test: ./test/b00003-cpu.vtc
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/server/b00000.vtc
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/stick-table/h00000.vtc
Skip ./work/haproxy-1.8-52e4d43/reg-tests/lua/b00002.vtc because option
OPENSSL not found
REASON: this test needs ssl compiled in.
Add test: ./work/haproxy-1.8-52e4d43/reg-tests/lua/h00001.vtc
########################## Starting varnishtest ##########################
# top TEST ./test/b00003-cpu.vtc FAILED (5.281) exit=2
1 tests failed, 0 tests skipped, 11 tests passed
root@freebsd11:/usr/ports/net/haproxy-devel #
#!/usr/bin/env sh
echo "########################## Preparing to run tests
##########################"
haproxy -v
#varnishtest -j 16 -k -t 20 ./work/haproxy-*/reg-tests/*/*.vtc >
./mytest-result.log 2>&1
#varnishtest -j 16 -k -t 20 ./haproxy_test_OK_20180831/*/*.vtc >>
./mytest-result.log 2>&1
#cat ./mytest-result.log
#echo "" >> ./mytest-result.log
#haproxy -vv >> ./mytest-result.log
TARGET=$(haproxy -vv | grep "TARGET = " | sed 's/.*= //')
OPTIONS=$(haproxy -vv | grep "OPTIONS = " | sed 's/.*= //')
echo "TARGET : $TARGET"
echo "OPTIONS : $OPTIONS"
haproxy -v > testresult_summary.log
haproxy -v > testresult_detailed.log
echo -n > testlist.lst
echo "########################## Gathering tests to run
##########################"
for i in $(find ./ -name "*.vtc");
do
optionismissing=
for required in "$(cat $i | grep REQUIRE_OPTION)";
do
if [ -z "$required" ]
then
continue
fi
requiredoption=$(echo "$required" | sed 's/.*=//' | sed
's/,.*//')
#echo " Test requires: $requiredoption"
if [ -z "$( echo "$OPTIONS" | grep "USE_$requiredoption=1" )" ]
then
echo -n "Skip $i because option $requiredoption not
found! REASON: "
echo "$required" | sed 's/.*,//'
optionismissing=1
fi
done
if [ $optionismissing ]
then
continue
fi
testtarget=$(cat $i | grep "#EXCLUDE_TARGET")
#echo "testtarget : $testtarget"
if [ "$( echo "$testtarget" | grep "#EXCLUDE_TARGET=$TARGET," )" ]
then
echo "Skip $i because: TARGET = $TARGET"
echo -n " REASON: "
echo "$testtarget" | sed 's/.*,//'
continue
fi
echo "Add test: $i"
echo "$i" >> testlist.lst
done
echo "########################## Starting varnishtest
##########################"
varnishtest -q -l -j 16 -k $(cat testlist.lst)
