Title: [258286] trunk
Revision
258286
Author
[email protected]
Date
2020-03-11 14:43:04 -0700 (Wed, 11 Mar 2020)

Log Message

Test262-runner should always consider crashes as new failures
https://bugs.webkit.org/show_bug.cgi?id=208943

Reviewed by Yusuke Suzuki.

JSTests:

* test262/expectations.yaml:

Source/_javascript_Core:

BigInt.asUintN() / BigInt.asIntN() should not crash when called even if we have
not implemented them yet...

* runtime/BigIntConstructor.cpp:
(JSC::bigIntConstructorFuncAsUintN):
(JSC::bigIntConstructorFuncAsIntN):

Tools:

Right now the runner will consider crashes with no output as passes, which is obviously bad.
Since we should never crash on a conformance test we should consider a crash as a new failure
100% of the time.

* Scripts/test262/Runner.pm:
(processFile):
(runTest):
(processResult):

Modified Paths

Diff

Modified: trunk/JSTests/ChangeLog (258285 => 258286)


--- trunk/JSTests/ChangeLog	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/JSTests/ChangeLog	2020-03-11 21:43:04 UTC (rev 258286)
@@ -1,5 +1,14 @@
 2020-03-11  Keith Miller  <[email protected]>
 
+        Test262-runner should always consider crashes as new failures
+        https://bugs.webkit.org/show_bug.cgi?id=208943
+
+        Reviewed by Yusuke Suzuki.
+
+        * test262/expectations.yaml:
+
+2020-03-11  Keith Miller  <[email protected]>
+
         Throws incorrectly a syntax error when declaring a top level catch variable the same as a parameter
         https://bugs.webkit.org/show_bug.cgi?id=189914
 

Modified: trunk/JSTests/test262/expectations.yaml (258285 => 258286)


--- trunk/JSTests/test262/expectations.yaml	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/JSTests/test262/expectations.yaml	2020-03-11 21:43:04 UTC (rev 258286)
@@ -726,6 +726,66 @@
 test/built-ins/AsyncGeneratorFunction/proto-from-ctor-realm.js:
   default: 'Test262Error: newTarget.prototype is undefined Expected SameValue(«[object AsyncGeneratorFunction]», «[object AsyncGeneratorFunction]») to be true'
   strict mode: 'Test262Error: newTarget.prototype is undefined Expected SameValue(«[object AsyncGeneratorFunction]», «[object AsyncGeneratorFunction]») to be true'
+test/built-ins/BigInt/asIntN/arithmetic.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asIntN/bigint-tobigint-errors.js:
+  default: 'Test262Error: ToBigInt: no argument => undefined => TypeError Expected a TypeError to be thrown but no exception was thrown at all'
+  strict mode: 'Test262Error: ToBigInt: no argument => undefined => TypeError Expected a TypeError to be thrown but no exception was thrown at all'
+test/built-ins/BigInt/asIntN/bigint-tobigint-toprimitive.js:
+  default: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+  strict mode: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+test/built-ins/BigInt/asIntN/bigint-tobigint-wrapped-values.js:
+  default: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asIntN/bigint-tobigint.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asIntN/bits-toindex-errors.js:
+  default: 'Test262Error: ToIndex: throw when integerIndex < 0 Expected a RangeError to be thrown but no exception was thrown at all'
+  strict mode: 'Test262Error: ToIndex: throw when integerIndex < 0 Expected a RangeError to be thrown but no exception was thrown at all'
+test/built-ins/BigInt/asIntN/bits-toindex-toprimitive.js:
+  default: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «-1») to be true'
+  strict mode: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «-1») to be true'
+test/built-ins/BigInt/asIntN/bits-toindex-wrapped-values.js:
+  default: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asIntN/bits-toindex.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asIntN/order-of-steps.js:
+  default: 'Test262Error: Expected SameValue(«0», «2») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«0», «2») to be true'
+test/built-ins/BigInt/asUintN/arithmetic.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asUintN/bigint-tobigint-errors.js:
+  default: 'Test262Error: ToBigInt: no argument => undefined => TypeError Expected a TypeError to be thrown but no exception was thrown at all'
+  strict mode: 'Test262Error: ToBigInt: no argument => undefined => TypeError Expected a TypeError to be thrown but no exception was thrown at all'
+test/built-ins/BigInt/asUintN/bigint-tobigint-toprimitive.js:
+  default: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+  strict mode: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+test/built-ins/BigInt/asUintN/bigint-tobigint-wrapped-values.js:
+  default: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asUintN/bigint-tobigint.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asUintN/bits-toindex-errors.js:
+  default: 'Test262Error: ToIndex: throw when integerIndex < 0 Expected a RangeError to be thrown but no exception was thrown at all'
+  strict mode: 'Test262Error: ToIndex: throw when integerIndex < 0 Expected a RangeError to be thrown but no exception was thrown at all'
+test/built-ins/BigInt/asUintN/bits-toindex-toprimitive.js:
+  default: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+  strict mode: 'Test262Error: ToPrimitive: @@toPrimitive takes precedence Expected SameValue(«undefined», «1») to be true'
+test/built-ins/BigInt/asUintN/bits-toindex-wrapped-values.js:
+  default: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: ToPrimitive: unbox object with internal slot Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asUintN/bits-toindex.js:
+  default: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«undefined», «0») to be true'
+test/built-ins/BigInt/asUintN/order-of-steps.js:
+  default: 'Test262Error: Expected SameValue(«0», «2») to be true'
+  strict mode: 'Test262Error: Expected SameValue(«0», «2») to be true'
 test/built-ins/BigInt/constructor-integer.js:
   default: 'RangeError: Not safe integer'
   strict mode: 'RangeError: Not safe integer'

Modified: trunk/Source/_javascript_Core/ChangeLog (258285 => 258286)


--- trunk/Source/_javascript_Core/ChangeLog	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/Source/_javascript_Core/ChangeLog	2020-03-11 21:43:04 UTC (rev 258286)
@@ -1,5 +1,19 @@
 2020-03-11  Keith Miller  <[email protected]>
 
+        Test262-runner should always consider crashes as new failures
+        https://bugs.webkit.org/show_bug.cgi?id=208943
+
+        Reviewed by Yusuke Suzuki.
+
+        BigInt.asUintN() / BigInt.asIntN() should not crash when called even if we have
+        not implemented them yet...
+
+        * runtime/BigIntConstructor.cpp:
+        (JSC::bigIntConstructorFuncAsUintN):
+        (JSC::bigIntConstructorFuncAsIntN):
+
+2020-03-11  Keith Miller  <[email protected]>
+
         Throws incorrectly a syntax error when declaring a top level catch variable the same as a parameter
         https://bugs.webkit.org/show_bug.cgi?id=189914
 

Modified: trunk/Source/_javascript_Core/runtime/BigIntConstructor.cpp (258285 => 258286)


--- trunk/Source/_javascript_Core/runtime/BigIntConstructor.cpp	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/Source/_javascript_Core/runtime/BigIntConstructor.cpp	2020-03-11 21:43:04 UTC (rev 258286)
@@ -142,8 +142,7 @@
 {
     // FIXME: [ESNext][BigInt] Implement BigInt.asIntN and BigInt.asUintN
     // https://bugs.webkit.org/show_bug.cgi?id=181144
-    CRASH();
-    return JSValue::encode(JSValue());
+    return encodedJSUndefined();
 }
 
 EncodedJSValue JSC_HOST_CALL bigIntConstructorFuncAsIntN(JSGlobalObject*, CallFrame*)
@@ -150,8 +149,7 @@
 {
     // FIXME: [ESNext][BigInt] Implement BigInt.asIntN and BigInt.asUintN
     // https://bugs.webkit.org/show_bug.cgi?id=181144
-    CRASH();
-    return JSValue::encode(JSValue());
+    return encodedJSUndefined();
 }
 
 } // namespace JSC

Modified: trunk/Tools/ChangeLog (258285 => 258286)


--- trunk/Tools/ChangeLog	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/Tools/ChangeLog	2020-03-11 21:43:04 UTC (rev 258286)
@@ -1,3 +1,19 @@
+2020-03-11  Keith Miller  <[email protected]>
+
+        Test262-runner should always consider crashes as new failures
+        https://bugs.webkit.org/show_bug.cgi?id=208943
+
+        Reviewed by Yusuke Suzuki.
+
+        Right now the runner will consider crashes with no output as passes, which is obviously bad.
+        Since we should never crash on a conformance test we should consider a crash as a new failure
+        100% of the time.
+
+        * Scripts/test262/Runner.pm:
+        (processFile):
+        (runTest):
+        (processResult):
+
 2020-03-11  Chris Dumez  <[email protected]>
 
         Assertion hit under WebPageProxy::suspendCurrentPageIfPossible() when bypassing certificate warning page

Modified: trunk/Tools/Scripts/test262/Runner.pm (258285 => 258286)


--- trunk/Tools/Scripts/test262/Runner.pm	2020-03-11 21:32:13 UTC (rev 258285)
+++ trunk/Tools/Scripts/test262/Runner.pm	2020-03-11 21:43:04 UTC (rev 258286)
@@ -684,9 +684,9 @@
         my $args = getFeatureFlags($data);
 
         foreach my $scenario (@scenarios) {
-            my ($result, $execTime) = runTest($includesfile, $filename, $scenario, $data, $args);
+            my ($exitCode, $result, $execTime) = runTest($includesfile, $filename, $scenario, $data, $args);
 
-            $resultsdata = processResult($filename, $data, $scenario, $result, $execTime);
+            $resultsdata = processResult($filename, $data, $scenario, $exitCode, $result, $execTime);
             DumpFile($resultsfh, $resultsdata);
         }
 
@@ -823,14 +823,14 @@
     chomp $result;
 
     if ($?) {
-        return ($result, $execTime);
+        return ($?, $result, $execTime);
     } else {
-        return (0, $execTime);
+        return ($?, 0, $execTime);
     }
 }
 
 sub processResult {
-    my ($path, $data, $scenario, $result, $execTime) = @_;
+    my ($path, $data, $scenario, $exitCode, $result, $execTime) = @_;
 
     # Report a relative path
     my $file = abs2rel( $path, $test262Dir );
@@ -844,12 +844,15 @@
         && $expect->{$file}
         && $expect->{$file}->{$scenario};
 
-    if ($scenario ne 'skip' && $currentfailure) {
+    my $exitSignalNumber = $exitCode & 0x7f if $scenario ne 'skip';
 
-        # We have a new failure if we haven't loaded an expectation file
-        # (all fails are new) OR we have loaded an expectation fail and
-        # (there is no expected failure OR the failure has changed).
-        my $isnewfailure = ! $expect
+    if ($scenario ne 'skip' && ($currentfailure || $exitSignalNumber)) {
+
+        # We have a new failure if the process crashed OR we haven't loaded
+        # an expectation file (all fails are new) OR we have loaded an
+        # expectation fail and (there is no expected failure OR the failure
+        # has changed).
+        my $isnewfailure = $exitSignalNumber || !$expect
             || !$expectedfailure || $expectedfailure ne $currentfailure;
 
         # Print the failure if we haven't loaded an expectation file
@@ -858,6 +861,7 @@
 
         my $newFail = '';
         $newFail = '! NEW ' if $isnewfailure;
+        $newFail = "$newFail (Exit code: $exitCode) " if $exitSignalNumber;
         my $failMsg = '';
         $failMsg = "FAIL $file ($scenario)\n";
 
@@ -871,6 +875,7 @@
 
         $resultdata{result} = 'FAIL';
         $resultdata{error} = $currentfailure;
+        $resultdata{error} = "Bad exit code: $exitCode" if $exitSignalNumber;
         $resultdata{output} = $result;
     } elsif ($scenario ne 'skip' && !$currentfailure) {
         if ($expectedfailure) {
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to