On 06/19/2018 10:18 AM, Willy Tarreau wrote:
On Mon, Jun 18, 2018 at 07:50:38PM +0200, Frederic Lecaille wrote:
Hello,
Here is a simple patch to add a Makefile target to run all "*.vtc"
regression testing files found in 'reg-tests' directory.
(...)
Thank you very much for this, Fred! I hope this will ignite a long
series of such tests. It may be useful to add a README in the reg-tests
directory indicating how to install varnishtest (especially the version
compatible with haproxy), and probably suggest contributing new tests
in the CONTRIBUTING file to encourage the practice.
Here is a new patch to add reg-tests/README file and update CONTRIBUTING
file.
Fred.
>From cc4ea839d4783de2e38389801d469cd908f3a469 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?=
Date: Tue, 19 Jun 2018 14:06:07 +0200
Subject: [PATCH] MINOR: reg-tests: Add reg-tests/README file.
Add reg-tests/README file about how to compile and use varnishtest, and
how to produce patches to add regression testing files to HAProxy sources.
Also update CONTRIBUTING file to encourage the contributors to write
regression testing files.
---
CONTRIBUTING | 9
reg-tests/README | 63
2 files changed, 72 insertions(+)
create mode 100644 reg-tests/README
diff --git a/CONTRIBUTING b/CONTRIBUTING
index b2c2b49..9e0d625 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -467,6 +467,12 @@ do not think about them anymore after a few patches.
harm when nobody is able to tell whether or not the patch needs to be
backported or can be reverted in case of regression.
+ When fixing a bug which is reproducible, if possible, the contributors are
+ strongly encouraged to write a regression testing VTC file for varnishtest
+ to add to reg-tests directory. More information about varnishtest may be
+ found in README file of reg-tests directory and in doc/regression-testing.txt
+ file.
+
12) Discuss on the mailing list
When submitting changes, please always CC the mailing list address so that
@@ -615,6 +621,9 @@ part is being touched. The following tags are suggested but not limitative :
- tests regression test files. No code is affected, no need to upgrade.
+ - reg-tests regression test files for varnishtest. No code is affected, no
+ need to upgrade.
+
- init initialization code, arguments parsing, etc...
- configconfiguration parser, mostly used when adding new config keywords
diff --git a/reg-tests/README b/reg-tests/README
new file mode 100644
index 000..5427ad6
--- /dev/null
+++ b/reg-tests/README
@@ -0,0 +1,63 @@
+ * Regression testing for HAProxy with varnishtest *
+
+
+This little README file is about how to compile and run varnishtest test case files (VTC files) to test HAProxy for any regression.
+
+To do so, you will have to compile varnishtest program sources which comes with
+Varnish cache application. varnishtest is a very useful program which has been
+developed to test Varnish cache application. varnishtest has been modified in
+collaboration with Varnish cache conceptor Poul-Henning Kamp to support HAProxy in
+addition to Varnish cache.
+
+
+* varnishtest compilation *
+
+ - Clone recent Varnish cache sources which may be found in this GitHub repository
+https://github.com/varnishcache/varnish-cache:
+
+$ git clone https://github.com/varnishcache/varnish-cache
+
+ - Compile Varnish cache sources:
+
+$ cd varnish-cache && ./autogen.sh && ./configure && make
+
+ Then varnishtest program may be found in bin/varnishtest directory.
+ VTC files for Varnish cache may be found in bin/varnishtest/tests directory.
+ The Varnish cache manuals are located in 'man' directory. You will have to
+ have a look at varnishtest(7) and vtc(7) manuals.
+
+ Some information may also be found in doc/regression-testing.txt in HAProxy
+ sources.
+
+
+* varnishtest execution *
+
+ You must set HAPROXY_PROGRAM environment variable to give the location
+ of the HAProxy program to test to varnishtest:
+
+$ HAPROXY_PROGRAM= varnishtest ...
+
+ The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
+ Makefile target. You must set the VARNISHTEST_PROGRAM environment variable to
+ give the location of the varnishtest program which has been previously compiled.
+
+$ VARNISHTEST_PROGRAM= HAPROXY_PROGRAM= make reg-tests
+
+ Note that varnishtest is run with -t5 and -l option. -l option is to keep
+ keep varnishtest temporary directory in case of failed test cases. core files
+ may be found in this directory (if enabled by ulimit).
+
+
+* varnishtest patches for HAProxy VTC files *
+
+ When producing a patch to add a VCT regression testing file to reg-tests directory,
+ please follow these simple rules:
+
+- Add the commit number on the first line.
+- Then, copy and paste the commit log.
+
