Martin Rubey wrote:
> Waldek Hebisch <[email protected]> writes:
>
> >> I assume that what you add is a grand total over all testcases - or is
> >> it a grand total over all testsuites? Please make this clearer in the
> >> output.
> >>
> >>
> >> =============================================================================
> >> testsuite | testcases: failed (total) | tests: failed (total)
> >> bugs2009 0 (19) 0 (69)
> >> unexpected failures: 0
> >> expected failures: 5
> >> unexpected passes: 0
> >> total tests: 69
> >>
> >
> > I want total of all tests: if one test of a testcase is expected
> > failure and say three other pass classifying the whole testcase
> > as expected failure would be (IMHO) misleading. So I want
> > summary which ignores testsuite/testcase boundaries.
>
> OK. I agree that this was misleading. I mainly tried to copy Christian
> Aistleitner's unittest package for Aldor. So, the rules where:
>
> * if a test fails, the whole testcase fails (i.e., testcases are a very
> small unit) I guess that "expected" failures don't make a testcase
> fail now, right?
>
Yes, "expected" failures and unexpected passes do not change fail
count for a testcase. Testcase fails if fail count is positive,
that is there is at least one unexpected failure.
> * above that, there are only testsuites.
>
> > Note that my plan is to have 'make check' rule in Makefiles which
> > gives a single compact result summarizing all tests. Since we run
> > each test file separately the first step is to get summaries for
> > files.
>
> My assumption was actually that the files are all run in the same image,
> and statistics() would then output a grand total for all testsuites,
> too.
>
> But that was probably too optimistic.
>
I am not sure if we want to run all tests in a single image. Single
image has some advantages, but there are drawbacks too.
--
Waldek Hebisch
[email protected]
--
You received this message because you are subscribed to the Google Groups
"FriCAS - computer algebra system" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/fricas-devel?hl=en.
