Revision: 4970
Author: [email protected]
Date: Mon Jun 28 22:56:15 2010
Log: Update the ES5 conformance tests to the latest version and fix test
expectations.
We are currently using a rather old version of the ES5 conformance
tests and there has been a range of bug fixes in the test set. This
change brings us up to date.
In addition, our current test expectations discards the all chapters
but chapter 15. I have enabled the other chapters, cleaned up the
tests that we no longer fail on, and filed bugs for tests that we fail on.
It seems some of the bugs on the es5 conformance bug-tracker has been
fixed but not marked fixed. I will file bugs for the newly discovered
test bugs.
Review URL: http://codereview.chromium.org/2840023
http://code.google.com/p/v8/source/detail?r=4970
Modified:
/branches/bleeding_edge/test/es5conform/README
/branches/bleeding_edge/test/es5conform/es5conform.status
=======================================
--- /branches/bleeding_edge/test/es5conform/README Fri Jan 15 05:40:13 2010
+++ /branches/bleeding_edge/test/es5conform/README Mon Jun 28 22:56:15 2010
@@ -4,7 +4,7 @@
https://es5conform.svn.codeplex.com/svn
-in revision 62998 as 'data' in this directory. Using later version
+in revision 71525 as 'data' in this directory. Using later version
may be possible but the tests are only known to pass (and indeed run)
with that revision.
=======================================
--- /branches/bleeding_edge/test/es5conform/es5conform.status Thu May 27
00:43:43 2010
+++ /branches/bleeding_edge/test/es5conform/es5conform.status Mon Jun 28
22:56:15 2010
@@ -29,20 +29,35 @@
def UNIMPLEMENTED = PASS || FAIL
def FAIL_OK = FAIL, OKAY
-chapter07: UNIMPLEMENTED
-chapter08: UNIMPLEMENTED
-chapter10: UNIMPLEMENTED
-chapter11: UNIMPLEMENTED
-chapter12: UNIMPLEMENTED
-chapter13: UNIMPLEMENTED
-chapter14: UNIMPLEMENTED
-chapter15/15.1: UNIMPLEMENTED
-chapter15/15.2/15.2.3/15.2.3.1: UNIMPLEMENTED
+# Non UTF8 characters in test files.
+chapter10/10.4/10.4.2/10.4.2-3-c-2-s: FAIL_OK
+chapter10/10.4/10.4.2/10.4.2-3-c-1-s: FAIL_OK
+chapter10/10.4/10.4.2/10.4.2-2-c-1: FAIL_OK
+
+# We do not implement the error chekcs specified in the production rules
+# of 11.1.5 (Object initializer).
+# We are compatible with Safari and Firefox.
+chapter11/11.1/11.1.5: UNIMPLEMENTED
+
+# Issue 759, see http://code.google.com/p/v8/issues/detail?id=759
+chapter11/11.4/11.4.1//11.4.1-4.a-5: FAIL_OK
+chapter11/11.4/11.4.1//11.4.1-4.a-7: FAIL_OK
+
+
+# We do not have a global object called 'global' as required by tests.
+chapter15/15.1: FAIL_OK
+
+# NOT IMPLEMENTED: seal
chapter15/15.2/15.2.3/15.2.3.8: UNIMPLEMENTED
+# NOT IMPLEMENTED: freeze
chapter15/15.2/15.2.3/15.2.3.9: UNIMPLEMENTED
+# NOT IMPLEMENTED: preventExtensions
chapter15/15.2/15.2.3/15.2.3.10: UNIMPLEMENTED
+# NOT IMPLEMENTED: isSealed
chapter15/15.2/15.2.3/15.2.3.11: UNIMPLEMENTED
+# NOT IMPLEMENTED: isFrozen
chapter15/15.2/15.2.3/15.2.3.12: UNIMPLEMENTED
+# NOT IMPLEMENTED: isExtensible
chapter15/15.2/15.2.3/15.2.3.13: UNIMPLEMENTED
# NOT IMPLEMENTED: seal
@@ -66,23 +81,23 @@
# NOT IMPLEMENTED: bind
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-38: FAIL_OK
-# NaN is writable
+# NaN is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-178: FAIL_OK
-# Infinity is writable
+# Infinity is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-179: FAIL_OK
-# undefined is writable
+# undefined is writable.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-180: FAIL_OK
-# Our Function object has a "arguments" property which is used as a non
-# property in in the test
+# Our Function object has an "arguments" property which is used as a
+# non-property in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-183: FAIL_OK
-# Our Function object has a "caller" property which is used as a non
-# property in in the test
+# Our Function object has a "caller" property which is used as a
+# non-property in in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-184: FAIL_OK
-# Our function object has a name property which is used as a non
-# property in the test
+# Our function object has a name property which is used as a
+# non-property in the test.
chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-188: FAIL_OK
# NOT IMPLEMENTED: RegExp.prototype.source
@@ -102,18 +117,18 @@
# than those described in the spec - but according to spec they can
# have additional properties.
# All compareArray calls in these tests could be exchanged with a
-# isSubsetOfArray call (I will upload a path to the es5conform site)
+# isSubsetOfArray call (I will upload a path to the es5conform site).
# SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-1: FAIL_OK
-# SUBSETFAIL + we do not implement all methods on Object
+# SUBSETFAIL + we do not implement all methods on Object.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-2: FAIL_OK
# SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-3: FAIL_OK
-# SUBSETFAIL + we do not implement Function.prototype.bind
+# SUBSETFAIL + we do not implement Function.prototype.bind.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-4: FAIL_OK
# SUBSETFAIL
@@ -134,28 +149,28 @@
# SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-11: FAIL_OK
-# We do not implement all methods on RegExp
+# We do not implement all methods on RegExp.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-13: FAIL_OK
# SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-14: FAIL_OK
-# EvalError.prototype does not have message property
+# EvalError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-15: FAIL_OK
-# Rangeerror.prototype does not have message property
+# Rangeerror.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-16: FAIL_OK
-# ReferenceError.prototype does not have message property
+# ReferenceError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-17: FAIL_OK
-# SyntaxError.prototype does not have message property
+# SyntaxError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-18: FAIL_OK
-# TypeError.prototype does not have message property
+# TypeError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-19: FAIL_OK
-# URIError.prototype does not have message property
+# URIError.prototype does not have message property.
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-20: FAIL_OK
# SUBSETFAIL
@@ -200,68 +215,79 @@
# SUBSETFAIL
chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-35: FAIL_OK
-
-# We fail this because Object.keys returns numbers for element indices
-# rather than strings.
-#chapter15/15.2/15.2.3/15.2.3.14/15.2.3.14-3-3: FAIL_OK
-
-chapter15/15.3: UNIMPLEMENTED
-
-chapter15/15.4/15.4.4/15.4.4.14: UNIMPLEMENTED
-chapter15/15.4/15.4.4/15.4.4.15: UNIMPLEMENTED
-chapter15/15.4/15.4.4/15.4.4.20: UNIMPLEMENTED
-chapter15/15.4/15.4.4/15.4.4.21: UNIMPLEMENTED
-chapter15/15.4/15.4.4/15.4.4.22: UNIMPLEMENTED
-
-# Wrong test - because this is not given as argument to arr.every
-# this._15_4_4_16_5_1 evaluates to undefined
-chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-5-1: FAIL_OK
-
-# In test case the element is not appended - it is added in the middle of
-# the array
-chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-7-1: FAIL_OK
-
-# We fail because the test assumes that if the reference to array is
deleted it
-# is not longer traversed
-chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-7-7: FAIL_OK
-
-# if (val>1) in test should be if (val>2)
-chapter15/15.4/15.4.4/15.4.4.16/15.4.4.16-8-10: FAIL_OK
-
-# Wrong assumption - according to spec some returns a Boolean, not a number
+# NOT IMPLEMENTED: bind on Function.prototype.
+chapter15/15.3/15.3.4/15.3.4.5/15.3.4.5-0-1: FAIL_OK
+
+# Wrong test - the spec does not say anything about throwing errors
+# on calling Array.prototype.indexOf with undefined as argument.
+chapter15/15.4/15.4.4/15.4.4.14/15.4.4.14-1-1: FAIL_OK
+
+# Wrong test - the spec does not say anything about throwing errors
+# on calling Array.prototype.indexOf with null as argument.
+chapter15/15.4/15.4.4/15.4.4.14/15.4.4.14-1-2: FAIL_OK
+
+# Wrong assumption - according to spec some returns a Boolean, not a
number.
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-4-9: FAIL_OK
-# Same as 15.4.4.16-5-1
-chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-5-1: FAIL_OK
-
-# Same as 15.4.4.16-7-1
-chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-7-1: FAIL_OK
-
# Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-7-7: FAIL_OK
# Same as 15.4.4.16-10-8
chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-8-10: FAIL_OK
-# Same as 15.4.4.16-5-1
-chapter15/15.4/15.4.4/15.4.4.18/15.4.4.18-5-1: FAIL_OK
-
# Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.18/15.4.4.18-7-6: FAIL_OK
-# Same as 15.4.4.16-5-1
-chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-5-1: FAIL_OK
-
# Same as 15.4.4.16-7-7
chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-8-7: FAIL_OK
-
-chapter15/15.5: UNIMPLEMENTED
-chapter15/15.6: UNIMPLEMENTED
-chapter15/15.7: UNIMPLEMENTED
-chapter15/15.9: UNIMPLEMENTED
-chapter15/15.10: UNIMPLEMENTED
-chapter15/15.12: UNIMPLEMENTED
+# Wrong test - uses unitialized variable a in precondition check.
+chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-9-3: FAIL_OK
+
+# We do not implement Array mapping functions correctly if array
+# entries are added for nonexistent entries smaller than length by
+# the callback function.
+# See http://code.google.com/p/v8/issues/detail?id=755
+chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-1: FAIL_OK
+
+# Wrong tests, path in test file is wrong. This will crash the test
+# script so we mark it unimplemented.
+chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4: UNIMPLEMENTED
+chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4-s: UNIMPLEMENTED
+
+# Wrong test - deleting the property on o in callbackfn will
+# have no effect on the actual array on which reduceRight is called.
+chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-7: FAIL_OK
+
+# We do not implement trim correctly on null and undefined.
+chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-1-1: FAIL_OK
+chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-1-2: FAIL_OK
+
+# We do not correctly recognize \uFEFF as whitespace
+chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-10: FAIL_OK
+chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-18: FAIL_OK
+chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-34: FAIL_OK
+
+# RegExp.prototype is not of type RegExp - we are bug compatible with JSC.
+chapter15/15.10/15.10.6/15.10.6: FAIL_OK
+
+# We do not have the properties of a RegExp instance on RegExp.prototype.
+# The spec says we should - but we are currently bug compatible with JSC.
+chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-1: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-2: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-1: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-2: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-1: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-2: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-1: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-2: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-1: FAIL_OK
+chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-2: FAIL_OK
+
+# JSON.stringify does not correctly truncate the space parameter.
+# See http://code.google.com/p/v8/issues/detail?id=753
+chapter15/15.12/15.12.3/15.12.3-6-a-2: FAIL_OK
+chapter15/15.12/15.12.3/15.12.3-6-b-1: FAIL_OK
[ $arch == mips ]
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
