Hi, I've attached a patch which enables all the tests in our test suite to run through valgrind This behaviour can be invoked by setting the environment variable: VALGRIND_TESTS.
Simultaneously, I also propose that we merge the new python based test suite into origin/master, to ensure that all new tests are created for the new test suite. In case a feature isn't implemented, we can add it ASAP. -- Thanking You, Darshit Shah
From 6140b1b6e3831a35fd2cf4536fda741a4dcc9019 Mon Sep 17 00:00:00 2001 From: Darshit Shah <[email protected]> Date: Wed, 23 Jul 2014 18:42:43 +0530 Subject: [PATCH] Support running tests through valgrind --- testenv/ChangeLog | 8 ++++++++ testenv/README | 6 +++++- testenv/conf/expected_ret_code.py | 11 +++++++---- testenv/test/base_test.py | 6 +++++- 4 files changed, 25 insertions(+), 6 deletions(-) diff --git a/testenv/ChangeLog b/testenv/ChangeLog index db796e8..97d2266 100644 --- a/testenv/ChangeLog +++ b/testenv/ChangeLog @@ -1,3 +1,11 @@ +2014-07-23 Darshit Shah <[email protected]> + + * test/base_test.py (BaseTest.gen_cmd_line): Add support for running all + tests through valgrind if the relevant environment variable is set + * conf/expected_ret_code (ExpectedRetCode.__call__): Valgrind returns error + code 45 when it detects a memory leak. + * Readme: Update with details about valgrind tests + 2014-07-22 Darshit Shah <[email protected]> * (README): Remove old TODO and document SERVER_WAIT variable diff --git a/testenv/README b/testenv/README index c1e4297..413e12e 100644 --- a/testenv/README +++ b/testenv/README @@ -93,6 +93,8 @@ Environment Variables: valgrind. * NO_CLEANUP: Do not remove the temporary files created by the test. This will prevent the ${testname}-test directory from being deleted +* VALGRIND_TESTS: If this variable is set, the test suite will execute all the + tests through valgrind's memcheck tool. File Structure: @@ -271,7 +273,9 @@ Makefile.am. This way the Test will be executed on running a 'make check'. If a Test is expected to fail on the current master branch, then the Test should also be added to the XFAIL_TESTS variable. This will allow expected failures to pass through. If a test mentioned in the XFAIL_TESTS variable passes, it gets -red-flagged as a XPASS. +red-flagged as a XPASS. Currently, tests expected to fail under valgrind are not +explicitly marked as XFAIL. Tests failing under valgrind must always be +considered a blocking error. Remember to always name the Test correctly using the TEST_NAME variable. This is essential since a directory with the Test Name is created and this can diff --git a/testenv/conf/expected_ret_code.py b/testenv/conf/expected_ret_code.py index 5f53f8c..febef32 100644 --- a/testenv/conf/expected_ret_code.py +++ b/testenv/conf/expected_ret_code.py @@ -9,8 +9,11 @@ class ExpectedRetCode: def __call__(self, test_obj): if test_obj.ret_code != self.expected_ret_code: - failure = "Return codes do not match.\n" \ - "Expected: %s\n" \ - "Actual: %s" % (self.expected_ret_code, - test_obj.ret_code) + if test_obj.ret_code == 45: + failure = "Memory Leak Found by Valgrind" + else: + failure = "Return codes do not match.\n" \ + "Expected: %s\n" \ + "Actual: %s" % (self.expected_ret_code, + test_obj.ret_code) raise TestFailed(failure) diff --git a/testenv/test/base_test.py b/testenv/test/base_test.py index 1c21c18..369370a 100644 --- a/testenv/test/base_test.py +++ b/testenv/test/base_test.py @@ -99,7 +99,11 @@ class BaseTest: wget_path = os.path.abspath(os.path.join(test_path, "..", '..', 'src', "wget")) - cmd_line = '%s %s ' % (wget_path, self.wget_options) + if os.getenv("VALGRIND_TESTS"): + valgrind_test = "valgrind --error-exitcode=301 --leak-check=full" + else: + valgrind_test = "" + cmd_line = '%s %s %s ' % (valgrind_test, wget_path, self.wget_options) for protocol, urls, domain in zip(self.protocols, self.urls, self.domains): -- 2.0.2
