Ben Elliston wrote:
On Tue, Oct 30, 2018 at 08:58:46PM -0500, Jacob Bachmeyer wrote:
Lastly, is a soft dependency on GNU awk acceptable for DejaGnu or
would I need to rewrite my testsuite summarization tool in Tcl or
Expect before contributing it for use with the DejaGnu testsuite?
I am happy for contributed tools like this to be written in GNU
awk. Please try and use portal AWK if you can, but if not, then
perhaps give the file a .gawk extension to make it clear that you need
GNU awk?
The AWK predecessor (written as part of a testsuite for another project
that I intend to offer to GNU after the lowest-level basic functionality
is ready; other parts of that testsuite already relied on GNU awk when
this tool was written) uses both asort and asorti, so some work will be
needed to get it into portable AWK. Reading sum2junit.sh gave me a few
ideas, and I have written both shell and Tcl versions of a tool that I
am now calling "dejagnu-report-card".
An attempt to use Expect to read DejaGnu summary files had extremely
poor performance. The shell and Tcl versions have similar performance;
one spends more time in the system and the other spends more time in
user mode, but overall, they seem to have similar performance. The AWK
predecessor blows them both away, usually completing in about 1/2 (on
the DejaGnu testsuite) to 1/5 (on a larger testsuite with ~2000 results)
of the time. (This is not noticeable on the DejaGnu testsuite, but
~2000 unit test results can take about 200ms for the shell and Tcl
versions to read on my system.)
Also, I would like to see this be more than another tool in contrib; I
plan to adapt the "runtest" shell script into a "dejagnu" multi-purpose
launcher, supporting auxiliary commands in a style similar to git(1).
(The runtest(1) command itself would remain "for historical reasons".)
As such, "dejagnu report-card" would be the first such command added,
with future plans including some kind of "canned test" feature to help
with creating new testsuites and adding tools. Something like "dejagnu
create testsuite" and "dejagnu add tool" would be nice. These will need
careful review for portability, since I only have GNU close at hand for
testing.
With this approach, including multiple versions and selecting at
configure time which version to install should not be a problem. (Have
GNU awk? Install the .gawk version. Have only some other AWK? Install
the portable .awk version. Have no AWK? Install the Tcl version. The
dejagnu(1) launcher would adapt at run-time.)
This leads to a slightly more complex "structure" for DejaGnu, with an
"inner core" (runtest(1) and its components) and an "outer core" (the
dejagnu(1) subcommands) that is also installed. One of the reasons for
this is that I would like the DejaGnu testsuite itself to use the
dejagnu-report-card tool, and possibly for a future version of Automake
to use it if the DEJATOOL variable indicates to test multiple tools.
Here's what the output looks like:
__________________________________________________
/ PASS FAIL ?PASS ?FAIL UNSUP UNRES UNTEST
|--------------------------------------------------
runtest | 77 0 0 0 0 0 0
libdejagnu | 5 0 0 0 0 0 0
testrun | 0 0 0 0 0 0 0
|--------------------------------------------------
|--------------------------------------------------
| 82 0 0 0 0 0 0
\__________________________________________________
The "testrun" line will be eliminated as part of planned improvements to
the DejaGnu testsuite.
-- Jacob
[PS: A batch of your emails seems to have been "stuck" somewhere along
the way for the past five days and have just now arrived in my inbox and
at the list archives.]
_______________________________________________
Bug-dejagnu mailing list
Bug-dejagnu@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-dejagnu