On Thu, Dec 13, 2018 at 10:15:05PM +0100, [email protected] wrote:
> From: Frédéric Lécaille <[email protected]>
> 
> Add a new target to the Makefile named "reg-tests-help" to have an idea
> about how to run the reg tests from haproxy Makefile.
> Handle list of levels and lists of level range passed to make with LEVEL 
> variable.
> New supported syntax:
>     LEVEL=1,4     make reg-tests
>     LEVEL=1-2,5-6 make reg-tests
> Add two new levels 5 and 6. 5 is for broken script, 6 for experimental 
> scripts.

Thanks Fred, that's really nice and it speeds up the tests compared to
iterating over the series. I'm noticing however that these 3 ones fail
for me :
#    top  TEST ./reg-tests/checks/s00000.vtc FAILED (0.018) signal=11
#    top  TEST ./reg-tests/checks/s00003.vtc FAILED (0.011) signal=11
#    top  TEST ./reg-tests/checks/s00001.vtc FAILED (2.047) exit=2

0 and 3 make my varnishtest segfault. 1 doesn't match the regex, and
I can't figure why. It's a 3kB+ line, so I'm suspecting that we could
hit some internal limits. And it's really impossible by reading it to
tell what doesn't match unfortunately. Probably that this test should
be simplified to be less precise on its matching ? For example, maybe
it could be enough to check that the status you're looking for appears
on the expected column and that's all. It would also make the test more
robust against any minor change to the state format. 

I've merged all of them anyway, at least because the range improvement
is great!

Thanks!
Willy

Reply via email to