On Fri, 2019-09-06 at 11:10 -0400, Trevor Gamblin wrote:
> On 9/6/19 10:26 AM, Richard Purdie wrote:
> > On Fri, 2019-09-06 at 10:04 -0400, Trevor Gamblin wrote:
> > > From: Trevor Gamblin <trevor.gamb...@windriver.com>
> > >
> > > The libevent ptest used to report only a global pass or a fail
> > > result.
> > > Count individual PASS, FAIL, SKIP results. The SKIP results now
> > > include tests that are disabled in the libevent code.
> > >
> > > Signed-off-by: Trevor Gamblin <trevor.gamb...@windriver.com>
> > > ---
> > > .../libevent/libevent/run-ptest | 34 ++++++++++++-
> > > ----
> > > --
> > > 1 file changed, 22 insertions(+), 12 deletions(-)
> > >
> > > diff --git a/meta/recipes-support/libevent/libevent/run-ptest
> > > b/meta/recipes-support/libevent/libevent/run-ptest
> > > index 0241851c70..b7d945246f 100644
> > > --- a/meta/recipes-support/libevent/libevent/run-ptest
> > > +++ b/meta/recipes-support/libevent/libevent/run-ptest
> > > @@ -1,18 +1,28 @@
> > > #!/bin/sh
> > >
> > > -fail=0
> > > +# run-ptest - 'ptest' test infrastructure shell script that
> > > +# wraps the libevent test scripts
> > > +#
> > > +# Trevor Gamblin <trevor.gamb...@windriver.com>
> > > +###############################################################
> > > +LIBEVENTLIB=/usr/lib/libevent
> > > +LOG="${LIBEVENTLIB}/ptest/libevent_ptest_$(date +%Y%m%d-
> > > %H%M%S).log"
> > > +
> > > +cd ${LIBEVENTLIB}/ptest
> > > +
> > > for test in ./test/*
> > > do
> > > - $test
> > > - if [ $? -ne 0 ]
> > > - then
> > > - fail=1
> > > - fi
> > > + $test 2>&1|tee -a ${LOG}
> > > done
> > >
> > > -if [ $fail -eq 0 ]
> > > -then
> > > - echo "PASS: libevent"
> > > -else
> > > - echo "FAIL: libevent"
> > > -fi
> > > +passed=`grep OK ${LOG}|wc -l`
> > > +failed=`grep FAILED ${LOG}|wc -l`
> > > +skipped=`grep -E 'DISABLED|SKIPPED' ${LOG}|wc -l`
> > > +all=$((passed + failed + skipped))
> > > +
> > > +( echo "=== Test Summary ==="
> > > + echo "TOTAL: ${all}"
> > > + echo "PASSED: ${passed}"
> > > + echo "FAILED: ${failed}"
> > > + echo "SKIPPED: ${skipped}"
> > > +) | tee -a ${LOG}
> >
> > Does this work correctly such that ptest-runner picks up the
> > individual
> > test pass/fail results? I thought the tests would need individual
> > pass/fail lines for that to work?
> >
> > Cheers,
> >
> > Richard
> >
> >
>
> They do have individual pass/fail results in the ptest-runner output,
> but those don't match up with the pass/fail usually provided by
> ptest, so the log is parsed for OK/FAILED/SKIPPED/DISABLED and
> translated. Example output:
> write_cb: write 12
> write_cb: write -1
> === Test Summary ===
> TOTAL: 316
> PASSED: 300
> FAILED: 0
> SKIPPED: 16
> DURATION: 87
> END: /usr/lib/libevent/ptest
> 2019-09-06T14:50
> STOP: ptest-runner
> Compare with the last log from autobuilder (
> https://autobuilder.yocto.io/pub/non-release/20190829-10/testresults/qemux86-64-ptest/libevent.log
> ):
> write_cb: write 12
> write_cb: write -1
> FAIL: libevent
> DURATION: 116
> I'm working on a v2 to address multilib - I'll add the example output
> to the commit as well.
You're missing my point. There are log parsers such as:
http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/lib/oeqa/utils/logparser.py
The output has to follow the format in:
https://wiki.yoctoproject.org/wiki/Ptest
to quote it:
"""
One major point of ptest is to consolidate the output format of all
tests into a single common format. The format selected is the automake
"simple test" format:
result: testname
Where "result" is one of PASS, FAIL or SKIP and "testname" can be any
identifying string (the same format used by Automake.)
"""
which is why you'll see other run-ptest scripts using sed to add in
PASS/FAIL/SKIP strings.
I think the PASSED/FAILED counts are generated by ptest-runner itself.
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
