Title: [232841] trunk/Tools
- Revision
- 232841
- Author
- [email protected]
- Date
- 2018-06-14 10:49:01 -0700 (Thu, 14 Jun 2018)
Log Message
[test262-runner] Test output should summarize tests that are
unexpectedly passing/failing.
https://bugs.webkit.org/show_bug.cgi?id=186527
Patch by Valerie R Young <[email protected]> on 2018-06-14
Reviewed by Michael Saboff.
* Scripts/test262/Runner.pm:
(main):
In verbose mode, a summary of all new failing tests
and all new passing tests are printed at the end of
the script output.
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (232840 => 232841)
--- trunk/Tools/ChangeLog 2018-06-14 14:31:28 UTC (rev 232840)
+++ trunk/Tools/ChangeLog 2018-06-14 17:49:01 UTC (rev 232841)
@@ -1,3 +1,17 @@
+2018-06-14 Valerie R Young <[email protected]>
+
+ [test262-runner] Test output should summarize tests that are
+ unexpectedly passing/failing.
+ https://bugs.webkit.org/show_bug.cgi?id=186527
+
+ Reviewed by Michael Saboff.
+
+ * Scripts/test262/Runner.pm:
+ (main):
+ In verbose mode, a summary of all new failing tests
+ and all new passing tests are printed at the end of
+ the script output.
+
2018-06-14 Zan Dobersek <[email protected]>
[GTK][WPE] MiniBrowsers should be able to ignore TLS errors
Modified: trunk/Tools/Scripts/test262/Runner.pm (232840 => 232841)
--- trunk/Tools/Scripts/test262/Runner.pm 2018-06-14 14:31:28 UTC (rev 232840)
+++ trunk/Tools/Scripts/test262/Runner.pm 2018-06-14 17:49:01 UTC (rev 232841)
@@ -358,6 +358,8 @@
my $newfailcount = 0;
my $newpasscount = 0;
my $skipfilecount = 0;
+ my $newfailurereport = '';
+ my $newpassreport = '';
# Create expectation file and calculate results
foreach my $test (@results) {
@@ -381,12 +383,29 @@
}
# If an unexpected failure
- $newfailcount++ if !$expectedFailure || ($expectedFailure ne $test->{error});
+ if (!$expectedFailure || ($expectedFailure ne $test->{error})) {
+ $newfailcount++;
+ if ($verbose) {
+ my $path = $test->{path};
+ my $mode = $test->{mode};
+ my $err = $test->{error};
+ $newfailurereport .= "FAIL $path ($mode)\n$err\n\n";
+ }
+ }
+
}
elsif ($test->{result} eq 'PASS') {
# If this is an newly passing test
- $newpasscount++ if $expectedFailure;
+ if ($expectedFailure) {
+ $newpasscount++;
+
+ if ($verbose) {
+ my $path = $test->{path};
+ my $mode = $test->{mode};
+ $newpassreport .= "PASS $path ($mode)\n";
+ }
+ }
}
elsif ($test->{result} eq 'SKIP') {
$skipfilecount++;
@@ -393,6 +412,20 @@
}
}
+ # In verbose mode, the result of every test is printed, so summarize useful results
+ if ($verbose && $expect && ($newfailurereport || $newpassreport)) {
+ print "\n";
+ if ($newfailurereport) {
+ print "---------------NEW FAILING TESTS SUMMARY---------------\n\n";
+ print "$newfailurereport";
+ }
+ if ($newpassreport) {
+ print "---------------NEW PASSING TESTS SUMMARY---------------\n\n";
+ print "$newpassreport\n";
+ }
+ print "---------------------------------------------------------\n";
+ }
+
if ($saveExpectations) {
DumpFile($expectationsFile, \%failed);
print "\nSaved results in: $expectationsFile\n";
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
