Hopefully Chris Johns will pipe up and add more details. In particular, where his rtems-test is. I think it can be used on the beagle board and some gdb simulators now.
On 5/7/2014 4:18 PM, Krzysztof Mięsowicz wrote: > Hi, > > My name is Krzysztof Mięsowicz. I am interested in participating again > in SOCIS this year and I am writing to you in hope that you could > provide me some more details about possible projects related to new > rtems-test infrastructure. > > As dr Joel wrote on Rtems-Users facebook group, there are some > possible improvements: (cited) > > This year we want to encourage the SOCIS students to focus on > improvements to our new rtems-test Python-based framework. This > will include: > + adding more BSPs (simulators and real boards) to rtems-test > Chris will have to point us to his rtems-test repository. I can't seem to locate it. It is in Python and drives real hardware debuggers and simulators using the GDB/MI. rtems-testing/sim-scripts supports more simulators but does not support hardware debuggers. rtems-test supports a small subset of the simulators supported by rtems-testing/sim-scripts. rtems-test needs to be worked on so it can replace all uses of sim-scripts and obsolete them. > + building more simulators with the RTEMS Source > - if successful, sim-scripts should be obsolete > That should be RTEMS Source Builder. http://git.rtems.org/rtems-source-builder/ RSB builds the basic tools and a couple of simulator configurations. RSB and rtems-tools work together to provide a framework to manage build instructions and patch sets for RTEMS tools. We want to expand it so it can build every tool one needs cross. Simulators are the next step. Eventually, we would like to be able to even build add-on libraries to target a specific BSP. But that is longer down the road. If someone gets to it sooner, great! RSB needs to be augmented to be able to build all the simulators external to gdb that we currently use. This may involve some odd cases like say an old qemu version plus a patch to do coldfire. That would give a baseline to upgrade against. > + integrating coverage runs into rtems-test > rtems-testing/covoar (C++) and rtems-testing/rtems-coverage (shell) together do RTEMS test runs for coverage and generate the coverage reports on http://www.rtems.org/ftp/pub/rtems/people/joel/coverage/ The rtems-test tool needs to be able to do coverage runs. It needs to know if a target supports coverage, how to turn on coverage for a target, how to run the report generator, etc. > + improvements to the coverage test reporting > - reporting should be on "module" level. Too coarse now. > The report generator itself needs a refresh. Right now, it reports on two "lumps" of code -- core and developmental. Read "critical core" and "the kitchen sink". :) Ideally, the coverage reports should be generated based on modules or source directories in cpukit/. Then we can get the coverage on score/ separate from shell/, fatfs/, or imfs. What we have seen is that each time we add a module to the "kitchen sink" set, the covered percentage drops because the newly added module needs improvement in test coverage. covoar can generate .gcov output but no one has ever validated it against "truth" and seen if gcov, lcov, etc. can produce trustworth reports with it. > > - some scripts could be migrated to Python > > The older code is in bash. Migrating it to Python as it is reworked is desirable. > I wonder which of these are of highest priorities and what will it > mean exactly. Could you provide more details about these ideas and > maybe point some starting points to get more familiar and prepare > better proposal? > Everything is high priority. :) I guess IMO I would like to see one BSP/simulator pushed through the entire software stack, reporting improved, and shell transitioned to Python as it was convenient/expedient. Adding other simulators to the RSB and rtems-test/ is a bonus. So if you want to assume we get one student from SOCIS, that would be my suggestion. My thinking is that if SOCIS gives us two or more students, one could easily start at the rtems-test/ level integrating coverage runs as they are now. Another student could focus on the coverage report generation. Transitioning more simulators from sim-scripts/ to rtems-test/ and adding building more simulators to the RSB could be added easily. Each simulator instance is more or less independent so any number of students could work off a check list. If we get two or more students, the problem can be tackled from multiple ends. > Regards, > Krzysztof -- Joel Sherrill, Ph.D. Director of Research & Development joel.sherr...@oarcorp.com On-Line Applications Research Ask me about RTEMS: a free RTOS Huntsville AL 35805 Support Available (256) 722-9985
_______________________________________________ rtems-devel mailing list rtems-devel@rtems.org http://www.rtems.org/mailman/listinfo/rtems-devel
