(I'm going to try to shift this thread to the test262-discuss list, by bccing: es-discuss.)
I think we all agree that a web based test runner is part of what we want to have. It seems inevitable that some sort of pass % or similar score has to display even though it's not clear whether such a score is very meaningful for this style of test suite. You can get big fluctuation because a single bugs can cause many individual tests to fail while in other cases a really significant bugs might only show up in a small number. If anybody has any ideas for a meaningful metric that doesn't have those sorts of sensitivities, let's hear it. Allen > -----Original Message----- > From: Christian Plesner Hansen [mailto:[email protected]] > Sent: Tuesday, June 22, 2010 4:00 PM > To: Sam Tobin-Hochstadt > Cc: Allen Wirfs-Brock; [email protected]; Jonathan Chetwynd; es- > [email protected] > Subject: Re: Announcing TC39 ECMAScript Test Suite Project - Test262 > > Yes, there is an acidic element to this project. There will eventually be a > website > where anyone can test their browser's conformance against the test262 suite > like they can today against the acid tests. > > > -- Christian > > On Tue, Jun 22, 2010 at 3:47 AM, Sam Tobin-Hochstadt <[email protected]> > wrote: > > On Mon, Jun 21, 2010 at 1:42 PM, Allen Wirfs-Brock > > <[email protected]> wrote: > >> The intent is a suite that tests an implementation against the actual > requirement of the Ecma-262 standard. It will be based upon and similar to > what > you can already see in the Sputnik and ES5conform suites. Those tests > generally > identify a specific requirement of a specific section of the spec. and then > try to > prove that the requirement is/isn't met by an implementation. > >> > >> Give that the standard specification itself is not particularly > >> understandable > to the general public, it probably isn't reasonable to expect that all the > tests of > the specification would be understandable to the same public. > >> > >> However, reasonable understandability is a good goal. What are some of the > characteristics of the W3C suites that make them "acidic"? > > > > I believe that the various "Acid" tests for rendering are what's being > > referred to here [1]. I think the relevant characteristc is that > > anyone can run them in their browser, and they display a visual > > rendering of how "good" a job the browser does, making them into > > easily-distinguishable markers of conformance to whatever is tested. > > It's not clear to me whether this would be good for this test suite. > > > > [1] http://en.wikipedia.org/wiki/Acid3 > > -- > > sam th > > [email protected] > > _______________________________________________ > > test262-discuss mailing list > > [email protected] > > https://mail.mozilla.org/listinfo/test262-discuss > > _______________________________________________ es-discuss mailing list [email protected] https://mail.mozilla.org/listinfo/es-discuss
