Many tests build their input data with rte_rand(), which is seeded from the system entropy source and therefore produces a different sequence on every run. Varying the data is good for coverage, but it adds noise when comparing performance results between two builds.
Add a DPDK_TEST_SEED environment variable to the test application that seeds the generator with a fixed value. The generator is reseeded before each test is dispatched, so a test gets the same input data no matter what ran before it in the same invocation. When the variable is not set the behaviour is unchanged and each run gets a random seed. Note that rand_perf_autotest calls rte_srand() itself. It measures generator latency rather than any value derived from the sequence, so it is left alone. Signed-off-by: Stephen Hemminger <[email protected]> --- app/test/test.c | 48 ++++++++++++++++++++++++++ doc/guides/contributing/unit_test.rst | 15 ++++++++ doc/guides/rel_notes/release_26_11.rst | 2 ++ 3 files changed, 65 insertions(+) diff --git a/app/test/test.c b/app/test/test.c index c610c3588e..176e4fa1f5 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -6,6 +6,7 @@ #include <stdio.h> #include <stdint.h> #include <stdarg.h> +#include <stdbool.h> #include <stdlib.h> #include <errno.h> #include <ctype.h> @@ -21,6 +22,7 @@ extern cmdline_parse_ctx_t main_ctx[]; #include <rte_eal.h> #include <rte_cycles.h> #include <rte_log.h> +#include <rte_random.h> #include <rte_string_fns.h> #ifdef RTE_LIB_TIMER #include <rte_timer.h> @@ -107,6 +109,33 @@ do_recursive_call(void) int last_test_result; +/* Tests that use rte_rand() get a different sequence on every run, which + * is good for coverage but makes it hard to compare performance results. + * Setting DPDK_TEST_SEED gives a repeatable sequence instead. + */ +static bool use_test_seed; +static uint64_t test_seed; + +static int +get_test_seed(void) +{ + const char *env = getenv("DPDK_TEST_SEED"); + char *end; + + if (env == NULL || *env == '\0') + return 0; + + errno = 0; + test_seed = strtoull(env, &end, 0); + if (errno != 0 || end == env || *end != '\0') { + fprintf(stderr, "Invalid DPDK_TEST_SEED: '%s'\n", env); + return -1; + } + + use_test_seed = true; + return 0; +} + #define MAX_EXTRA_ARGS 32 int @@ -175,8 +204,18 @@ main(int argc, char **argv) goto out; } + if (get_test_seed() < 0) { + ret = -1; + goto out; + } + recursive_call = getenv(RECURSIVE_ENV_VAR); if (recursive_call != NULL) { + /* Child instances inherit the environment, so seed them + * too rather than leaving them randomly seeded. + */ + if (use_test_seed) + rte_srand(test_seed); ret = do_recursive_call(); goto out; } @@ -233,6 +272,12 @@ main(int argc, char **argv) } } + /* Reseed before each test so that a test gets the same + * sequence no matter what ran before it. + */ + if (use_test_seed) + rte_srand(test_seed); + snprintf(buf, sizeof(buf), "%s\n", tests[i]); if (cmdline_parse_check(cl, buf) < 0) { printf("Error: invalid test command: '%s'\n", tests[i]); @@ -260,6 +305,9 @@ main(int argc, char **argv) goto out; } + if (use_test_seed) + rte_srand(test_seed); + cmdline_interact(cl); cmdline_stdin_exit(cl); } diff --git a/doc/guides/contributing/unit_test.rst b/doc/guides/contributing/unit_test.rst index f13ddff291..5d08fe1792 100644 --- a/doc/guides/contributing/unit_test.rst +++ b/doc/guides/contributing/unit_test.rst @@ -114,6 +114,21 @@ via the ``DPDK_TEST_PARAMS`` argument, in case some tests need additional configuration. This is not currently used in the Meson test suites. +Many tests build their input data with ``rte_rand()``, +which produces a different sequence on every run. +This is useful for coverage, but it adds noise when comparing results. +Setting ``DPDK_TEST_SEED`` to an integer seeds the generator with that +value before each test command is run, so that a test gets the same input +data no matter what ran before it:: + + $ DPDK_TEST_SEED=42 ./build/app/test/dpdk-test lpm_perf_autotest + +This also works when running the tests via Meson:: + + $ DPDK_TEST_SEED=42 meson test -C build --suite perf-tests + +When the variable is not set, the generator keeps its default random seeding. + Running test cases via Meson ---------------------------- diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index 9d1238d6f3..d315d426f4 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -69,6 +69,8 @@ New Features 64 bits of the value passed to ``rte_srand()`` now affect the state, so a given seed produces a different sequence than in previous releases. + * The test application now accepts a ``DPDK_TEST_SEED`` environment + variable to make runs using ``rte_rand()`` repeatable. Removed Items -- 2.53.0
