The existing results checking checks for any appearance of the string 'FAILED' from stdout of the test run. runltp logs as information the location of the failed command file with the string 'FAILED COMMAND File'. This leads to the test being detected as a failure on every run.
runltp stdout excerpt: -e FAILED COMMAND File: /usr/local/autotest/results/default/ltp/debug/failcmdfile This patch changes the heuristic tokens used to determine a failure. If the tokens TFAIL, TBROK, and TWARN are observed, report the run as a failure. This correlates to how LTP reports the status of an individual test. See the LTP source files include/test.h and lib/tst_res.c for more information. Finally, the stdout of a run is a single string and a Python split operation returns individual words and not lines. The existing code is broken in that it expects an entire line. No tests from 'ignore_tests' are ignored. This patch doesn't address this bug but adds a FIXME to note this incorrect behavior. Signed-off-by: Vinson Lee <[email protected]> --- client/tests/ltp/ltp.py | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/client/tests/ltp/ltp.py b/client/tests/ltp/ltp.py index 404468b..5fdb0c6 100644 --- a/client/tests/ltp/ltp.py +++ b/client/tests/ltp/ltp.py @@ -57,10 +57,20 @@ class ltp(test.test): cmd = os.path.join(ltpbin_dir, script) + ' ' + args result = utils.run(cmd, ignore_status=True) - # look for the first line in result.stdout containing FAIL and, - # if found, raise the whole line as a reason of the test failure. + # Look for the first line in result.stdout containing a token + # that runltp would identify as a failure. If found, raise the + # whole line as a reason of the test failure. + # + # FIXME + # result.stdout is a single string and a split operation + # gives individual words (not lines). 'test_name' is + # always going to be the same as 'line' and anything in + # 'ignore_tests' is never matched. for line in result.stdout.split(): - if 'FAIL' in line: + # See include/test.h and lib/test_res.c:tst_exit of LTP for + # more information about failure tokens and result + # determination. + if line in ('TFAIL', 'TBROK', 'TWARN'): test_name = line.strip().split(' ')[0] if not test_name in ignore_tests: raise error.TestFail(line) -- 1.7.5.4 _______________________________________________ Autotest mailing list [email protected] http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
