On Fri, Jun 11, 2021 at 07:56:14PM +0200, Tim Duesterhus wrote: > Hi! >
Hello Tim, > I hope I added all the active developers that touch the reg-tests to the 'CC' > list :-) > > This series updates the regtests to make use of VTest's 'feature cmd' syntax > to skip tests that are not supported in the current environment. > > In the long run this will should result in much cleaner tests, because they > don't need to be parsed by run-regtests.sh to extract these marker comments. > It > also allows to easily add test specific requirements without needing to invent > an entirely new syntax. An example might be the recently added tests that are > not supported on BoringSSL. These should be able to get a condition like: > > feature cmd "$HAPROXY_PROGRAM -vv |grep -vq BoringSSL" > > and then they won't run for BoringSSL (the example is untested). > That's very interesting, I had a discussion with Willy last week about this kind of problems. I think that's clearly the cleanest soluton to the problem. In my opinion it only lacks a way to check the OpenSSL version from 'haproxy -cc' so we can enable ssl/set_ssl_cert_bundle.vtc again. > After this series is applied the output of 'make reg-tests' will change. Tests > that are excluded using 'feature cmd' will still be listed as "Add test" in > the test gathering section. However the final line will show that a few tests > have been skipped: > > 0 tests failed, 4 tests skipped, 105 tests passed > > I don't think this is going to be an issue. But if it is, please complain! > Hm the only problem I have with this, is that we won't be able to see why a test was excluded. -- William Lallemand
