From: Vinson Lee <[email protected]> The existing results checking checks for any appearance of the string 'FAILED' from stdout of the test run. runltp logs as information the location of the failed command file with the string 'FAILED COMMAND File'. This leads to the test being detected as a failure on every run.
runltp stdout excerpt: -e FAILED COMMAND File: /usr/local/autotest/results/default/ltp/debug/failcmdfile This patch changes the heuristic tokens used to determine a failure. If the tokens TFAIL, TBROK, and TWARN are observed, report the run as a failure. This correlates to how LTP reports the status of an individual test. See the LTP source files include/test.h and lib/tst_res.c for more information. This patch also fixes the stdout result parsing by correctly splitting out lines from stdout. It was incorrectly splitting stdout into words. Changes from v2: * Gather a list of all failed tests rather than just the first failed test that came up Signed-off-by: Lucas Meneghel Rodrigues <[email protected]> Signed-off-by: Vinson Lee <[email protected]> --- client/tests/ltp/ltp.py | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) diff --git a/client/tests/ltp/ltp.py b/client/tests/ltp/ltp.py index 404468b..bfa13e7 100644 --- a/client/tests/ltp/ltp.py +++ b/client/tests/ltp/ltp.py @@ -57,10 +57,18 @@ class ltp(test.test): cmd = os.path.join(ltpbin_dir, script) + ' ' + args result = utils.run(cmd, ignore_status=True) - # look for the first line in result.stdout containing FAIL and, - # if found, raise the whole line as a reason of the test failure. - for line in result.stdout.split(): - if 'FAIL' in line: + # Look for the first line in result.stdout containing a token + # that runltp would identify as a failure. If found, raise the + # whole line as a reason of the test failure. + # + # See include/test.h and lib/test_res.c:tst_exit of LTP for + # more information about the failure tokens. + failed_tests = [] + for line in result.stdout.splitlines(): + if set(('TFAIL', 'TBROK', 'TWARN')).intersection(line.split()): test_name = line.strip().split(' ')[0] if not test_name in ignore_tests: - raise error.TestFail(line) + failed_tests.append(test_name) + + if failed_tests: + raise error.TestFail("LTP tests failed: %s" % failed_tests) -- 1.7.7.6 _______________________________________________ Autotest mailing list [email protected] http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
