Paul Vriens wrote:
Hi,
(I've just remembered that there is a limit on the email size, so I'm
doing it a
bit differently now. The other email, including the screenshots, will
probably come through as well)
Before sending this to wine-patches I would like people to have a look
first.
This patch makes sure that missing tests are shown in the test results,
both
when there is only one system in a group (f.e. Win98) or multiple
systems in a
group.
On 'http://www.xs4all.nl/~pvriens/200708031000.old.single/' you can see the
current state of the test results. When a test is missing a '.' is shown
(this is of course different from a missing test where we don't have
results at all but that's shown as a '.' as well).
On 'http://www.xs4all.nl/~pvriens/200708031000.old.group/' you can see the
current state when a group of systems is involved. When tests are
missing for a
group, a '.' is shown with a link to the group overview. There is
however no
missing test output in the group overview (for example gdiplus and pdh)
'http://www.xs4all.nl/~pvriens/200708031000.new.single/' shows the new
results
page for single tests. Instead of a single '.' there is a failure shown
(red
box) and a skip border (as these tests have not run). There is still a
'.' which
could maybe better be a '?'. There is also a 'onMouseOver' hint when you
go over
the '.'.
'http://www.xs4all.nl/~pvriens/200708031000.new.group/' shows the new
results
when a group is involved. The main summary is still the same as in the
old one,
but the group results now also show the missing tests (see gdiplus and pdh
again). Again there is a 'onMouseOver' hint in the group results.
Remember that this is still without winetest producing output about
missing dll's.
Next patches will:
- add a missing dll as one of the reasons for missing tests
- add some better clue's on the main summary for multiple systems in a
group
(not a single '.' but maybe the same 'failed and skipped').
As said, one of the thoughts I had was to replace the '.' with a '?' in
case of
missing tests for an unknown reason but I'm not sure about that yet.
Comments/remarks etc.. are welcome.
I'd also like to hear about other idea's for this page to make it useful
for
everybody. Apart from the fact that we probably should have more
exposure to the
fact that we have such a page/tool.
We saw an immediate rise (slowed down again) in test results as soon as
winetest
was mentioned on
'http://tweakers.net/meuktracker/15817/Wine-0.9.42.html' (in
Dutch). There it was also mentioned that some machines crashed because
of the
tests etc.. (maybe a wine-winetest component on bugzilla could help with
that
which we could reference on the main test.winehq.org/data/*/index.html
pages).
Cheers,
Paul.
Forgot to mention that only the index.html file is on the mentioned pages. No
result files (to cut down on size for me to push).
I also forgot to attach the patch. So here it is.
Cheers,
Paul
--- /wine/tools/winetest/gather 2007-08-03 17:20:09.000000000 +0200
+++ gather 2007-08-05 14:02:42.000000000 +0200
@@ -121,6 +121,22 @@
push @{$group->{tests}}, $testref;
}
+# Find missing tests. After this exercise all testresults are available
+# for all systems.
+foreach my $group (@groups) {
+ next unless exists $group->{tests};
+ foreach my $test (@{$group->{tests}}) {
+ foreach my $testname (sort keys %alltests) {
+ if (!exists $test->{results}->{$testname}) {
+ # Make sure missing tests are shown in the group results
+ $group->{digests}->{$testname} = "differ";
+ # Mark this test as missing for an unknown reason
+ $test->{results}->{$testname} = ["test missing", "-", "-",
"-"];
+ }
+ }
+ }
+}
+
# Write out the tables
my ($header); # same in thead and tfoot
@@ -201,28 +217,30 @@
# Output a single cell of a test
sub singletest {
my ($test, $testname, $groupname) = @_;
-
- if (!exists $test->{results}->{$testname}) {
- print OUT " <td class=\"note\">.</td>\n";
+ my $file = "$test->{dir}/$testname.txt";
+ my ($count, $todo, $error, $skipped) = @{$test->{results}->{$testname}};
+ if ($count eq "failed") {
+ my $msg = $todo eq "-"?"crash":
+ $todo == 258?"timeout":"failed";
+ my $fail = -r "$datadir/$build/$file"?"<a
href=\"$file\">$msg</a>":$msg;
+ print OUT " <td class=\"note\">$fail</td>\n";
+ } elsif ($count eq "test missing") {
+ print OUT <<"EOF";
+ <td class="skip_fail"><a
+ title="Test did not run for an unknown reason"
+ onMouseOver="Test did not run for an unknown reason";
+ >.</a></td>
+EOF
} else {
- my $file = "$test->{dir}/$testname.txt";
- my ($count, $todo, $error, $skipped) =
@{$test->{results}->{$testname}};
- if ($count eq "failed") {
- my $msg = $todo eq "-"?"crash":
- $todo == 258?"timeout":"failed";
- my $fail = -r "$datadir/$build/$file"?"<a
href=\"$file\">$msg</a>":$msg;
- print OUT " <td class=\"note\">$fail</td>\n";
- } else {
- my $class = $error?"fail":"pass";
- my $skip = $skipped?"skip_":"";
- print OUT <<"EOF";
+ my $class = $error?"fail":"pass";
+ my $skip = $skipped?"skip_":"";
+ print OUT <<"EOF";
<td class="$skip$class"><a
href="$file"
title="$count tests, $todo TODO, $error errors, $skipped skipped"
onMouseOver="refresh('$testname','$groupname
$test->{tag}',$count,$todo,$error,$skipped);"
>$error</a></td>
EOF
- }
}
}
