On 02/25/2012 12:34 AM, Vinson Lee wrote: > The existing results checking checks for any appearance of the string > 'FAILED' from stdout of the test run. runltp logs as information the > location of the failed command file with the string 'FAILED COMMAND > File'. This leads to the test being detected as a failure on every run. > > runltp stdout excerpt: > -e FAILED COMMAND File: > /usr/local/autotest/results/default/ltp/debug/failcmdfile > > This patch changes the heuristic tokens used to determine a failure. If > the tokens TFAIL, TBROK, and TWARN are observed, report the run as a > failure. This correlates to how LTP reports the status of an individual > test. See the LTP source files include/test.h and lib/tst_res.c for more > information. > > This patch also fixes the stdout result parsing by correctly splitting > out lines from stdout. It was incorrectly splitting stdout into words.
Vinson: I suggest a slight change on the patch, to actually make a list of the failed tests and then use that list to throw the TestFail exception. I've sent an updated version of the patch (labeled as v3), would you please take a look at it and ACK it if you like the approach? Thanks, Lucas > Signed-off-by: Vinson Lee<[email protected]> > --- > client/tests/ltp/ltp.py | 12 ++++++++---- > 1 files changed, 8 insertions(+), 4 deletions(-) > > diff --git a/client/tests/ltp/ltp.py b/client/tests/ltp/ltp.py > index 404468b..0f0a27d 100644 > --- a/client/tests/ltp/ltp.py > +++ b/client/tests/ltp/ltp.py > @@ -57,10 +57,14 @@ class ltp(test.test): > cmd = os.path.join(ltpbin_dir, script) + ' ' + args > result = utils.run(cmd, ignore_status=True) > > - # look for the first line in result.stdout containing FAIL and, > - # if found, raise the whole line as a reason of the test failure. > - for line in result.stdout.split(): > - if 'FAIL' in line: > + # Look for the first line in result.stdout containing a token > + # that runltp would identify as a failure. If found, raise the > + # whole line as a reason of the test failure. > + # > + # See include/test.h and lib/test_res.c:tst_exit of LTP for > + # more information about the failure tokens. > + for line in result.stdout.splitlines(): > + if set(('TFAIL', 'TBROK', 'TWARN')).intersection(line.split()): > test_name = line.strip().split(' ')[0] > if not test_name in ignore_tests: > raise error.TestFail(line) _______________________________________________ Autotest mailing list [email protected] http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
