On 20/03/2019 13:16, Sebastian Huber wrote:
Hello,
the RTEMS project has a covoar tool:
https://git.rtems.org/rtems-tools/tree/tester/covoar
I have seen some reports generated by it, but I never used it myself.
I have trouble to find out how it works. There is nothing in the
documentation:
cd rtems-docs
grep -ri covoar .
There is some documentation available here:
https://devel.rtems.org/wiki/GCI/Documentation/CoverageAnalysis/Coverage
If I invoke the rtems-test like this in the erc32 BSP build directory,
I get an error:
rtems-test --rtems-bsp=erc32-sis --rtems-tools=/build/rtems/5
--coverage --target=sparc-rtems5 .
RTEMS Testing - Tester, 5.0.not_released
Command Line: /build/rtems/5/bin/rtems-test --rtems-bsp=erc32-sis
--rtems-tools=/build/rtems/5 --coverage --target=sparc-rtems5 .
Python: 3.6.5 (default, Mar 31 2018, 19:45:04) [GCC]
Host: Linux-4.12.14-lp150.12.48-default-x86_64-with-glibc2.3.4 (Linux
huber-nb-linux.suse 4.12.14-lp150.12.48-default #1 SMP Tue Feb 12
14:01:48 UTC 2019 (268f014) x86_64 x86_64)
Traceback (most recent call last):
File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in
<module>
test.run(sys.argv[1:], command_path = base)
File "/build/rtems/5/share/rtems/tester/rt/test.py", line 300, in run
trace = cov_trace)
File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 363,
in __init__
self.target = self.macros['target']
File "/build/rtems/5/share/rtems/rtemstoolkit/macros.py", line 181,
in __getitem__
raise IndexError('key: %s' % (key))
IndexError: key: target
I made some progress to try it out myself. The next obstacle was this:
rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
--coverage --no-clean --log=leon3-test.log
sparc-rtems5/c/leon3/testsuites/samples/hello.exe
RTEMS Testing - Tester, 5.0.not_released
[1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
Passed: 1
Failed: 0
User Input: 0
Expected Fail: 0
Indeterminate: 0
Benchmark: 0
Timeout: 0
Invalid: 0
Wrong Version: 0
Wrong Build: 0
Wrong Tools: 0
----------------
Total: 1
Average test time: 0:00:01.505144
Testing time : 0:00:01.505144
error: symbol parser write failed for score
To figure out what went wrong, I removed the try/except block from
tester/rt/coverage.py and got:
rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
--coverage --no-clean --log=leon3-test.log
sparc-rtems5/c/leon3/testsuites/samples/hello.exe
RTEMS Testing - Tester, 5.0.not_released
[1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
Passed: 1
Failed: 0
User Input: 0
Expected Fail: 0
Indeterminate: 0
Benchmark: 0
Timeout: 0
Invalid: 0
Wrong Version: 0
Wrong Build: 0
Wrong Tools: 0
----------------
Total: 1
Average test time: 0:00:01.504073
Testing time : 0:00:01.504073
Traceback (most recent call last):
File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in
<module>
test.run(sys.argv[1:], command_path = base)
File "/build/rtems/5/share/rtems/tester/rt/test.py", line 387, in run
coverage_runner.run()
File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 401, in run
parser.write_ini(sset)
File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 293, in
write_ini
object_files = [o for o in os.listdir(self.symbol_sets[sset]) if
o[-1] == 'o']
FileNotFoundError: [Errno 2] No such file or directory:
'/sparc-rtems5/c/leon3/cpukit/score/src'
This is more useful error output from my point of view. It seems you
have to use an absolute path for the test directory.
rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
--coverage --no-clean --log=leon3-test.log
/build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites/samples/hello.exe
[....]
Coverage generating reports
error: copying tree: /build/rtems/5/share/rtems/tester/covoar/covoar.css
-> /build/git-build/b-leon3/leon3-sis-coverage/score/covoar.css: [Errno
2] No such file or directory:
'/build/rtems/5/share/rtems/tester/covoar/covoar.css'
I copied some files by hand. A full test run ended up in:
rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
--coverage --no-clean --log=leon3-test.log
/build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites
RTEMS Testing - Tester, 5.0.not_released
[ 3/710] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 |
sparc/leon3-sis: ada_nsecs.exe
[...]
[710/710] p:684 f:5 u:6 e:0 I:0 B:3 t:0 i:2 W:0 |
sparc/leon3-sis: tmttest01.exe
Passed: 692
Failed: 6
User Input: 6
Expected Fail: 0
Indeterminate: 0
Benchmark: 3
Timeout: 1
Invalid: 2
Wrong Version: 0
Wrong Build: 0
Wrong Tools: 0
------------------
Total: 710
Average test time: 0:00:00.427096
Testing time : 0:05:03.237825
Running coverage analysis: score
(/build/git-build/b-leon3/leon3-sis-coverage/score)
Coverage time: 0:17:53.695293
Running coverage analysis: rtems
(/build/git-build/b-leon3/leon3-sis-coverage/rtems)
Coverage time: 0:12:55.898577
Running coverage analysis: sapi
(/build/git-build/b-leon3/leon3-sis-coverage/sapi)
error: coverage: covoar failure:: 10
--
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
