On 24/02/12 09:33, Vinson Lee wrote:
> The existing results checking checks for any appearance of the string
> 'FAILED' from stdout of the test run. runltp logs as information the
> location of the failed command file with the string 'FAILED COMMAND
> File'. This leads to the test being detected as a failure on every run.
>
> runltp stdout excerpt:
> -e FAILED COMMAND File:
> /usr/local/autotest/results/default/ltp/debug/failcmdfile
>
> This patch changes the heuristic tokens used to determine a failure. If
> the tokens TFAIL, TBROK, and TWARN are observed, report the run as a
> failure. This correlates to how LTP reports the status of an individual
> test. See the LTP source files include/test.h and lib/tst_res.c for more
> information.
>
> Finally, the stdout of a run is a single string and a Python split
> operation returns individual words and not lines. The existing code is
> broken in that it expects an entire line. No tests from 'ignore_tests'
> are ignored. This patch doesn't address this bug but adds a FIXME to
> note this incorrect behavior.
>
> Signed-off-by: Vinson Lee<[email protected]>
Hi Vinson,
> ---
> client/tests/ltp/ltp.py | 16 +++++++++++++---
> 1 files changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/client/tests/ltp/ltp.py b/client/tests/ltp/ltp.py
> index 404468b..5fdb0c6 100644
> --- a/client/tests/ltp/ltp.py
> +++ b/client/tests/ltp/ltp.py
> @@ -57,10 +57,20 @@ class ltp(test.test):
> cmd = os.path.join(ltpbin_dir, script) + ' ' + args
> result = utils.run(cmd, ignore_status=True)
>
> - # look for the first line in result.stdout containing FAIL and,
> - # if found, raise the whole line as a reason of the test failure.
> + # Look for the first line in result.stdout containing a token
> + # that runltp would identify as a failure. If found, raise the
> + # whole line as a reason of the test failure.
> + #
> + # FIXME
> + # result.stdout is a single string and a split operation
> + # gives individual words (not lines). 'test_name' is
> + # always going to be the same as 'line' and anything in
> + # 'ignore_tests' is never matched.
Using split('\n') can resolve this issue, I have tested.
for line in result.stdout.split('\n'):
> for line in result.stdout.split():
> - if 'FAIL' in line:
> + # See include/test.h and lib/test_res.c:tst_exit of LTP for
> + # more information about failure tokens and result
> + # determination.
> + if line in ('TFAIL', 'TBROK', 'TWARN'):
It's not precise to check by 'in', how about using
re.findall("\d\s+(TFAIL|TBROK|TWARN)\s:", result) to match failed status.
access02 43174 TFAIL : close(temp_file) failed: errno=EBADF(9): Bad
file descriptor
access02 43175 TFAIL : close(temp_file) failed: errno=EBADF(9): Bad
file descriptor
access02 43176 TFAIL : close(temp_file) failed: errno=EBADF(9): Bad
file descriptor
access02 43177 TFAIL : close(temp_file) failed: errno=EBADF(9): Bad
file descriptor
access02 43178 TFAIL : close(temp_file) failed: errno=EBADF(9): Bad
file descriptor
> test_name = line.strip().split(' ')[0]
> if not test_name in ignore_tests:
More precision match.
if re.findall("^%s\s" % testname, line):
> raise error.TestFail(line)
--
Amos.
_______________________________________________
Autotest mailing list
[email protected]
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
