William, On 6/14/21 12:12 PM, William Lallemand wrote:
feature cmd "$HAPROXY_PROGRAM -vv |grep -vq BoringSSL" and then they won't run for BoringSSL (the example is untested).That's very interesting, I had a discussion with Willy last week about this kind of problems. I think that's clearly the cleanest soluton to the problem. In my opinion it only lacks a way to check the OpenSSL version from 'haproxy -cc' so we can enable ssl/set_ssl_cert_bundle.vtc again.
Note that it doesn't necessarily need to be part of '-cc'. If a simple 'grep' is sufficient, then that works as well. Of course if you need to check exact versions instead of just product names a 'grep' probably is not sufficient.
After this series is applied the output of 'make reg-tests' will change. Tests that are excluded using 'feature cmd' will still be listed as "Add test" in the test gathering section. However the final line will show that a few tests have been skipped: 0 tests failed, 4 tests skipped, 105 tests passed I don't think this is going to be an issue. But if it is, please complain!Hm the only problem I have with this, is that we won't be able to see why a test was excluded.
VTest logs the reason for the SKIP in the verbose output, but it's true that it's not readily available (not that I bypass 'make' for this example):
env HAPROXY_PROGRAM=$(pwd)/haproxy /VTest/vtest -v -k -t 1 reg-tests/ssl/new_del_ssl_crlfile.vtc reg-tests/ssl/set_ssl_crlfile.vtc reg-tests/ssl/set_ssl_cafile.vtc reg-tests/ssl/new_del_ssl_cafile.vtc reg-tests/ssl/show_ssl_ocspresponse.vtc reg-tests/http-messaging/http_abortonclose.vtc reg-tests/startup/check_condition.vtc |grep SKIPPING
Results in:
* top SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc 'feature(OPENSSL)' * top SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc 'feature(OPENSSL)' * top SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc 'feature(OPENSSL)' * top SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc 'feature(OPENSSL)' * top SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc 'feature(OPENSSL)' 0 tests failed, 5 tests skipped, 2 tests passed
Best regards Tim Düsterhus
