Hi, Yesterday I had an idea to make it possible to reliably test a DNS director, hoping to contribute one soon with a decent coverage. Today I have come back to it and finished my prototype. I have already discussed this patch's contents with phk and it's likely to be rejected unless it manages to stay simple enough.
I'm submitting my prototype for a review, it's basically my initial idea with phk's guidelines on how to work around the process isolation in varnish. I haven't polished it yet, I could see a couple areas to slightly improve, but nothing major. Feedback greatly appreciated! Best Regards, Dridi
From f5b1a95bd0babf28200082dd116858d4233a164f Mon Sep 17 00:00:00 2001 From: Dridi Boukelmoune <[email protected]> Date: Thu, 20 Aug 2015 12:23:50 +0200 Subject: [PATCH] Introduce reliable name resolution in varnishtest Reliability means that you can run it both on- and off-line, and still have deterministic tests. You can control the resolution during the test execution. It is currently used to test backend constraints with regards to .host resolution in v00045. It is implemented with a new libvarnishmock library that intercepts library calls to mock results under certain circumstances. It currently only intercepts calls to `getaddrinfo` and reads records in a `hosts` file, and the file name is read in an environment variable. Varnishtest, when run with the `-i` flag will take care of passing the environment variables to the varnishd process. --- bin/varnishtest/tests/v00045.vtc | 59 +++++++++++++ bin/varnishtest/vtc.h | 1 + bin/varnishtest/vtc_main.c | 17 ++++ bin/varnishtest/vtc_varnish.c | 7 +- configure.ac | 1 + lib/Makefile.am | 2 + lib/libvarnishmock/Makefile.am | 15 ++++ lib/libvarnishmock/fail.c | 45 ++++++++++ lib/libvarnishmock/gai.c | 173 +++++++++++++++++++++++++++++++++++++++ 9 files changed, 318 insertions(+), 2 deletions(-) create mode 100644 bin/varnishtest/tests/v00045.vtc create mode 100644 lib/libvarnishmock/Makefile.am create mode 100644 lib/libvarnishmock/fail.c create mode 100644 lib/libvarnishmock/gai.c diff --git a/bin/varnishtest/tests/v00045.vtc b/bin/varnishtest/tests/v00045.vtc new file mode 100644 index 0000000..7ed3f81 --- /dev/null +++ b/bin/varnishtest/tests/v00045.vtc @@ -0,0 +1,59 @@ +varnishtest "at most one IPv4 and one IPv6 addresses allowed for backends" + +server s1 { +} -start + +# Tests run with `varnishtest -i` will mock getaddrinfo calls and return +# a list of hosts from a hosts file for the host "varnishtest.debug". +# +# Unlike a /etc/hosts file, there's no domain-to-address mapping, the +# domain is always "varnishtest.debug". Instead, it contains one line with +# exactly one IP address and a port number, separated by a single space. +# It allows resolving to several `server` instances, like a DNS resolution, +# even though they *can't* listen to the same port. +# +# The test's host file is expected to be in ${tmpdir}/hosts if you try to +# resolve "varnishtest.debug". Empty lines will be ignored. + +shell { +cat >${tmpdir}/hosts <<EOF +127.0.0.1 ${s1_port} +::1 ${s1_port} +EOF +} + +varnish v1 -vcl { + import ${vmod_debug}; + + backend s1 { + .host = "varnishtest.debug"; + } +} + +shell { +cat >${tmpdir}/hosts <<EOF +127.0.0.1 ${s1_port} +127.0.0.2 ${s1_port} +::1 ${s1_port} +EOF +} + +varnish v1 -errvcl {resolves to too many addresses.} { + import ${vmod_debug}; + + backend s1 { + .host = "varnishtest.debug"; + } +} + +# An empty file resolves to nothing, but the file is mandatory. + +shell "echo >${tmpdir}/hosts" + +varnish v1 -errvcl {resolves to neither IPv4 nor IPv6 addresses.} { + import ${vmod_debug}; + + backend s1 { + .host = "varnishtest.debug"; + } +} diff --git a/bin/varnishtest/vtc.h b/bin/varnishtest/vtc.h index f2817d8..50eca67 100644 --- a/bin/varnishtest/vtc.h +++ b/bin/varnishtest/vtc.h @@ -71,6 +71,7 @@ extern pthread_t vtc_thread; extern int iflg; extern unsigned vtc_maxdur; extern int vtc_witness; +extern struct vsb *varnish_preload; void init_sema(void); void init_server(void); diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c index df8787c..31cc0fd 100644 --- a/bin/varnishtest/vtc_main.c +++ b/bin/varnishtest/vtc_main.c @@ -89,6 +89,7 @@ static char *tmppath; static char *cwd = NULL; int leave_temp; int vtc_witness = 0; +struct vsb *varnish_preload = NULL; /********************************************************************** * Parse a -D option argument into a name/val pair, and insert @@ -366,6 +367,18 @@ i_mode(void) AZ(putenv(strdup(VSB_data(vsb)))); /* + * Build $LD_PRELOAD with libvarnishmock + */ + AN(varnish_preload); + VSB_printf(varnish_preload, + "LD_PRELOAD=%s/lib/libvarnishmock/.libs/libvarnishmock.so", + topbuild); + + q = getenv("LD_PRELOAD"); + if (q != NULL) + VSB_printf(varnish_preload, ":%s", q); + + /* * Redefine VMOD macros */ #define VTC_VMOD(l) \ @@ -480,9 +493,13 @@ main(int argc, char * const *argv) VTAILQ_INSERT_TAIL(&tst_head, tp, list); } + varnish_preload = VSB_new_auto(); + if (iflg) i_mode(); + AZ(VSB_finish(varnish_preload)); + vb = vev_new_base(); i = 0; diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index 05ee65b..d60e5de 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -380,6 +380,8 @@ varnish_launch(struct varnish *v) const char *err; char *r; + AN(varnish_preload); + v->vd = VSM_New(); /* Create listener socket */ @@ -394,8 +396,9 @@ varnish_launch(struct varnish *v) vsb = VSB_new_auto(); AN(vsb); VSB_printf(vsb, "cd ${pwd} &&"); - VSB_printf(vsb, " exec ${varnishd} %s -d -n %s", - v->jail, v->workdir); + VSB_printf(vsb, " %s VARNISHTEST_HOSTSFILE=${tmpdir}/hosts", + VSB_data(varnish_preload)); + VSB_printf(vsb, " exec ${varnishd} %s -d -n %s", v->jail, v->workdir); if (vtc_witness) VSB_cat(vsb, " -p debug=+witness"); if (leave_temp) diff --git a/configure.ac b/configure.ac index 6c4d5f2..d0b1764 100644 --- a/configure.ac +++ b/configure.ac @@ -650,6 +650,7 @@ AC_CONFIG_FILES([ lib/Makefile lib/libvarnish/Makefile lib/libvarnishapi/Makefile + lib/libvarnishmock/Makefile lib/libvarnishtools/Makefile lib/libvarnishcompat/Makefile lib/libvcc/Makefile diff --git a/lib/Makefile.am b/lib/Makefile.am index c0c0e60..6d58e42 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -4,6 +4,7 @@ SUBDIRS = \ libvarnishcompat \ libvarnish \ libvarnishapi \ + libvarnishmock \ libvarnishtools \ libvcc \ libvgz \ @@ -15,6 +16,7 @@ DIST_SUBDIRS = \ libvarnishcompat \ libvarnish \ libvarnishapi \ + libvarnishmock \ libvarnishtools \ libvcc \ libvgz \ diff --git a/lib/libvarnishmock/Makefile.am b/lib/libvarnishmock/Makefile.am new file mode 100644 index 0000000..66dd083 --- /dev/null +++ b/lib/libvarnishmock/Makefile.am @@ -0,0 +1,15 @@ +AM_CPPFLAGS = \ + -I$(top_srcdir)/include \ + -I$(top_builddir)/include + +AM_LDFLAGS = $(AM_LT_LDFLAGS) -ldl + +pkglib_LTLIBRARIES = libvarnishmock.la + +libvarnishmock_la_LDFLAGS = $(AM_LDFLAGS) -avoid-version + +libvarnishmock_la_SOURCES = \ + fail.c \ + gai.c + +libvarnishmock_la_CFLAGS = -DVARNISH_STATE_DIR='"${VARNISH_STATE_DIR}"' diff --git a/lib/libvarnishmock/fail.c b/lib/libvarnishmock/fail.c new file mode 100644 index 0000000..e310ec8 --- /dev/null +++ b/lib/libvarnishmock/fail.c @@ -0,0 +1,45 @@ +/*- + * Copyright (c) 2015 Varnish Software AS + * All rights reserved. + * + * Author: Dridi Boukelmoune <[email protected]> + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "config.h" + +#include <stdio.h> +#include <stdlib.h> + +#include "vas.h" + +static void __attribute__((__noreturn__)) +vmck_fail(const char *func, const char *file, int line, const char *cond, + enum vas_e kind) +{ + + fprintf(stderr, "VMCK: %s %s %d %s %d\n", func, file, line, cond, kind); + abort(); +} + +vas_f *VAS_Fail __attribute__((__noreturn__)) = vmck_fail; diff --git a/lib/libvarnishmock/gai.c b/lib/libvarnishmock/gai.c new file mode 100644 index 0000000..2ea1619 --- /dev/null +++ b/lib/libvarnishmock/gai.c @@ -0,0 +1,173 @@ +/*- + * Copyright (c) 2015 Varnish Software AS + * All rights reserved. + * + * Author: Dridi Boukelmoune <[email protected]> + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "config.h" + +#include <dlfcn.h> +#include <netdb.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#include <sys/socket.h> +#include <sys/types.h> + +#include "vas.h" +#include "miniobj.h" + +// hosts records: IP space PORT + +static void +vmck_addrinfo_parse(const char *line, char **addr, char **port) +{ + char buf[128]; + char *sp, *p; + + AN(line); + AN(addr); + AN(port); + + AZ(*addr); + AZ(*port); + + if (line[0] == '\0' || line[0] == '\n') + return; + + (void)strncpy(buf, line, sizeof buf); + AZ(buf[sizeof(buf) - 1]); + + sp = memchr(buf, ' ', sizeof buf); + AN(sp); + *sp = '\0'; + + p = sp + 1; + sp = strchr(p, '\n'); + if (sp != NULL) + *sp = '\0'; + + REPLACE(*addr, buf); + REPLACE(*port, p); +} + +static void +vmck_addrinfo_append(struct addrinfo **head, struct addrinfo **tail, + struct addrinfo *ai) +{ + + AN(head); + AN(tail); + AN(ai); + + if (*head == NULL) { + AZ(*tail); + *head = *tail = ai; + } + else { + AN(*tail); + AZ((*tail)->ai_next); + (*tail)->ai_next = ai; + } + + while ((*tail)->ai_next != NULL) + *tail = (*tail)->ai_next; +} + +static int +vmck_addrinfo_read(const struct addrinfo *hints, struct addrinfo **res) +{ + const char *fn; + FILE* fp; + char *line = NULL; + size_t len = 0; + struct addrinfo *head, *tail, *ai, h; + char *node, *service; + int i; + + if (hints == NULL) + memset(&h, 0, sizeof h); + else + memcpy(&h, hints, sizeof h); + + h.ai_flags |= AI_NUMERICHOST|AI_NUMERICSERV; + hints = &h; + + fn = getenv("VARNISHTEST_HOSTSFILE"); + AN(fn); + + fp = fopen(fn, "r"); + AN(fp); + + head = tail = NULL; + + while (getline(&line, &len, fp) != -1) { + ai = NULL; + node = service = NULL; + vmck_addrinfo_parse(line, &node, &service); + + if (node == NULL) + continue; + + i = getaddrinfo(node, service, hints, &ai); + if (i == 0) + vmck_addrinfo_append(&head, &tail, ai); + else + WRONG(gai_strerror(i)); + + AN(head); + + free(node); + free(service); + } + + free(line); + fclose(fp); + + *res = head; + return (0); +} + +/*--------------------------------------------------------------------*/ + +typedef int(gai_f)(const char *, const char *, const struct addrinfo *, + struct addrinfo **); + +int +getaddrinfo(const char *node, const char *service, const struct addrinfo *hints, + struct addrinfo **res) +{ + gai_f *orig; + + orig = dlsym(RTLD_NEXT, __func__); + AN(orig); + + if (node == NULL || strcmp(node, "varnishtest.debug")) + return orig(node, service, hints, res); + + AN(res); + return vmck_addrinfo_read(hints, res); +} -- 2.1.0
_______________________________________________ varnish-dev mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
