On 11/8/18 6:11 AM, Willy Tarreau wrote:
Hi guys,

On Wed, Nov 07, 2018 at 09:37:14AM +0100, Frederic Lecaille wrote:
New 'version' of the script attached.
It now supports a set of parameters to modify its behavior a little. And
also checking for a 'version requirement'. So a H2 test doesn't have to
fail on 1.7.

Very good work.

I missed this message last week (I'm not often on the list these days
I confess), but this tool is awesome!

Should i 'ask' to delete old test result.? Or would there be a other
better way to keep previous results separated from the current run?

No.

Rename the old vtc files like that before running the tests:

bname=$(basename $i)
mv /tmp/$bname /tmp/old.$bname

with $i in $(find /tmp/ -type d -name "vtc.*")

or something like that.

It would be nice to support TMPDIR so that we can force it in a script
instead of /tmp. What I *think* could be done by default would be to
delete the old.vtc.* files there before running the tests so that we
always keep only the last results by default. I found hundreds of vtc
directories in /tmp on my laptop and that can become problematic, even
worse on shared development machines.

please *add -type d* to the find command to list the temporary varnishtest
directories.

Well, then add -maxdepth 1 or 2 because you certainly don't want to run
find into my /tmp which contains complete distros to chroot into!

And we should only keep temporaries directory of tests which have failed
with -l varnishtest option currently enabled option. They are all the
/tmp/vtc.* directories found after having run the tests.

Interesting.

So the script should list the remaining temporary directories after having
run the tests. There is no need to parse the log file of this directories.

Then I'd like to see a small change, which would be that :

   1) we rely on a user-provided base directory instead of /tmp, such as
      ${TMPDIR:-/tmp} since TMPDIR is very commonly defined ;

   2) we create our own directory there using mkstemp and we display this
      name when starting the tests ;

   3) everything is created under this temporary directory, probably using
      the test's basename, so that we don't have a mixture of old and new
      tests at the same place, instead we have a single directory per run ;

   4) we continue to delete successful entries from these directories,
      and we can rmdir -p the temporary directory so that it completely
      disappears if everything was successful.

It will then make it very convenient to figure what tests systematically
fail and compare the errors between runs.

We cannot rely on the fact there is no bug in varnishtest logging code ;)

If you could give it another look i would appreciate that. Are there any
things that need to be added/changed about it before considering to add
it to haproxy sources branch?

Yes a few details.

It seems there are remaining useless piped sed processes (sed | sed) or
perhaps I have missed something.

There is no need to use awk to remove sections of lines. Have a look to cut
command which is more lightweight.

It depends if there are multiple spaces or not.

tr command accepts escaped characters as '\n' for new line character which
is more readable than '
' ;)

Be careful, "\n" is a gnu-ism or something like this. The only portable
way to use \n is to use \012.

For me with these remaining modifications/cleanup this script could be added
to haproxy sources.

And please make the varnishtest program configurable as is done in the
makefile. This will help those of us who installed it in a location not
present in the path, and will also help support multiple versions during
development.

Thanks again for this good work Pieter.

Seconded!


Hello Pieter,

Do you intend to finalize this script? We would like to use it in haproxy sources.

Thanks again.

Fred.


Reply via email to