On 8/05/2014 8:15 am, Joel Sherrill wrote:
Hopefully Chris Johns will pipe up and add more details.
In particular, where his rtems-test is. I think it can be used
on the beagle board and some gdb simulators now.
The RTEMS Tester is an RTEMS Tools Project and can be found in the
rtems-tools.git repo. The web interface to the repo is
http://git.rtems.org/rtems-tools/. The tester is under the tester
directory. There is also a doc directory with some very basic doco.
It supports the gdb run command, gdb via the MI interface with gdb
simulators and actual hardware via BDM or JTAG, and qemu is support. It
provides serial port access with full termios control.
The supported BSPs can be seen here ...
http://git.rtems.org/rtems-tools/tree/tester/rtems/testing/bsps
The testing back end support can be seen here ..
http://git.rtems.org/rtems-tools/tree/tester/rtems/testing
On 5/7/2014 4:18 PM, Krzysztof Mięsowicz wrote:
Hi,
My name is Krzysztof Mięsowicz. I am interested in participating again
in SOCIS this year and I am writing to you in hope that you could
provide me some more details about possible projects related to new
rtems-test infrastructure.
As dr Joel wrote on Rtems-Users facebook group, there are some
possible improvements: (cited)
This year we want to encourage the SOCIS students to focus on
improvements to our new rtems-test Python-based framework. This
will include:
+ adding more BSPs (simulators and real boards) to rtems-test
Chris will have to point us to his rtems-test repository. I can't seem to
locate it.
See above.
It is in Python and drives real hardware debuggers
and simulators using the GDB/MI.
rtems-testing/sim-scripts supports more simulators but does not support
hardware debuggers. rtems-test supports a small subset of the simulators
supported by rtems-testing/sim-scripts. rtems-test needs to be worked
on so it can replace all uses of sim-scripts and obsolete them.
+ building more simulators with the RTEMS Source
- if successful, sim-scripts should be obsolete
That should be RTEMS Source Builder.
http://git.rtems.org/rtems-source-builder/
RSB builds the basic tools and a couple of simulator configurations.
RSB and rtems-tools work together to provide a framework to manage
build instructions and patch sets for RTEMS tools. We want to expand
it so it can build every tool one needs cross. Simulators are the next
step.
The RSB builds qemu on most hosts. Building qemu for Windows as a
Canadian cross is complex and I have not managed to sort this out so
Windows is not working. Supporting Windows is outside the scope of this
work.
There is work in tested the qemu's build and to get BSP support added to
the rtems-tester. I am sure as you point out later in this email
specific patches are needed to make things work.
Eventually, we would like to be able to even build add-on libraries
to target a specific BSP. But that is longer down the road. If someone
gets to it sooner, great!
I have added some basic support for this via pkgconfig. The RSB contains
a private implementation and that can be used to help manage this task.
I still have some things to sort out and I am not sure this is ready to
have libraries added.
RSB needs to be augmented to be able to build all the simulators
external to gdb that we currently use. This may involve some odd
cases like say an old qemu version plus a patch to do coldfire.
That would give a baseline to upgrade against.
Do you have the patches ? The RSB qemu support currently references some
zynq patches on Qemu's patchworks so if you can find them there we can
directly reference them.
+ integrating coverage runs into rtems-test
rtems-testing/covoar (C++) and rtems-testing/rtems-coverage (shell)
together do RTEMS test runs for coverage and generate the coverage
reports on http://www.rtems.org/ftp/pub/rtems/people/joel/coverage/
The rtems-test tool needs to be able to do coverage runs. It needs to know
if a target supports coverage, how to turn on coverage for a target, how
to run the report generator, etc.
This code needs to move into the rtems-tools repo under tester/covoar
and a waf build script added. This will make the rtems-tools.git repo
self contained.
+ improvements to the coverage test reporting
- reporting should be on "module" level. Too coarse now.
The report generator itself needs a refresh. Right now, it reports on two
"lumps" of code -- core and developmental. Read "critical core" and
"the kitchen sink". :)
When you say report generator do you mean coverage report generator ?
What host tools does this report generator use ? My aim is to have
rtems-tools as self contains as possible.
Ideally, the coverage reports should be generated based on modules
or source directories in cpukit/. Then we can get the coverage on
score/ separate from shell/, fatfs/, or imfs. What we have seen is that
each time we add a module to the "kitchen sink" set, the covered
percentage drops because the newly added module needs improvement
in test coverage.
Do have more details about this task and how it would proceed ?
covoar can generate .gcov output but no one has ever validated it
against "truth" and seen if gcov, lcov, etc. can produce trustworth
reports with it.
- some scripts could be migrated to Python
The older code is in bash. Migrating it to Python as it is reworked is
desirable.
I wonder which of these are of highest priorities and what will it
mean exactly. Could you provide more details about these ideas and
maybe point some starting points to get more familiar and prepare
better proposal?
Everything is high priority. :)
I guess IMO I would like to see one BSP/simulator pushed through the
entire software stack, reporting improved, and shell transitioned to
Python as it was convenient/expedient. Adding other simulators
to the RSB and rtems-test/ is a bonus. So if you want to assume we
get one student from SOCIS, that would be my suggestion.
My thinking is that if SOCIS gives us two or more students, one could
easily start at the rtems-test/ level integrating coverage runs as they
are now. Another student could focus on the coverage report generation.
Transitioning more simulators from sim-scripts/ to rtems-test/
and adding building more simulators to the RSB could be added
easily. Each simulator instance is more or less independent so
any number of students could work off a check list.
If we get two or more students, the problem can be tackled from
multiple ends.
Another and more challenging task is to add support to test networking.
This would be via the VDE support in qemu. See
http://wiki.v2.cs.unibo.it/wiki/index.php/VDE. I do not have anything
more to add on this topic other than this looks like a nice to do this.
Regards,
Krzysztof
--
Joel Sherrill, Ph.D. Director of Research & Development
joel.sherr...@oarcorp.com On-Line Applications Research
Ask me about RTEMS: a free RTOS Huntsville AL 35805
Support Available (256) 722-9985
_______________________________________________
rtems-devel mailing list
rtems-devel@rtems.org
http://www.rtems.org/mailman/listinfo/rtems-devel
_______________________________________________
rtems-devel mailing list
rtems-devel@rtems.org
http://www.rtems.org/mailman/listinfo/rtems-devel
