[+cc John Resig since he's using this as part of dromaeo] Overall, sounds like good progress.
A couple of ideas: - can we make it so that if you try to cut-and-paste comparisons of 0.9 to 0.9.1 results, it will say "these results are from a different version"? - can we make the version more prominent in the title? - what would you think of reducing the setTimeout(..., 500) to something like setTimeout(..., 100)? This will cut the runtime of the test by ~80% :-) I'll volunteer to do any of these tasks this week if you want me to look at it. Mike On Mon, Dec 14, 2009 at 11:32 PM, Maciej Stachowiak <m...@apple.com> wrote: > > Hello folks, > > Over the past few days I made some changes to SunSpider to address some of > the more serious issues reported. I focused on only changes that seem to > make a significant difference to fairness and validity, so for example I did > not remove accidental access to global variables. I also made a small number > of harness changes that do not affect results but fix flaws in the harness. > > We are hesitant to change the SunSpider content or harness much at all, > since it's been used for cross-version and cross-brwoser comparisons for so > long. But these problems (many originally suggpointed out by Chrome or > Mozilla folks) seemed important enough to address. Also, in addition to the > patched content set, the original sunspider-0.9 content set is also > available to run through the new harness. > > The most important harness change is greatly reducing the time between > tests (as sugested by Mike Belshe) to avoid the negative impact of power > management on many systems (both Mac and Windows), and which are most > apparent for very fast browsers. > > I'm deliberately not posting this on the web site yet because I don't want > a flood of gawkers testing their browser before enough people have had a > chance to review and verify these changes. > > > Harness changes: > > In-browser SunSpider suffers excessive penalty under power management > https://bugs.webkit.org/show_bug.cgi?id=32505 > > Enable Web-hosted version of SunSpider to handle multiple versions > https://bugs.webkit.org/show_bug.cgi?id=32478 > > Use JSON.parse instead of eval for Web-hosted SunSpider results processing > https://bugs.webkit.org/show_bug.cgi?id=32490 > > Some Browser-hosted SunSpider files are not valid HTML5 > https://bugs.webkit.org/show_bug.cgi?id=32536 > > Make sunspider-0.9.1 the default content set (both command-line and hosted) > https://bugs.webkit.org/show_bug.cgi?id=32537 > > > Content changes (in sunspider-0.9.1 suite only; sunspider-0.9 is as > originally posted): > > SunSpider/tests/string-base64.js does not compute a valid base64 encoded > string > https://bugs.webkit.org/show_bug.cgi?id=16806 > > sunspider regexp-dna is inaccurate on firefox > https://bugs.webkit.org/show_bug.cgi?id=18989 > > > > Further changes I'm considering but am unsure about: > - Add correctness checking to all tests that don't use random numbers. > - Stop using array-like indexing of strings in the base64 test since that > doesn't work in IE8 and lower; but it is a standard construct now (ES5), > future IE will support it, and it's a useful thing to test. > > Changes that probably won't be considered until a 2.0 version: > - Adding new tests to cover other areas. > - Rebalancing the runtime of the existing tests. > - Considering different scoring methodology such as bigger-is-better or > geometric mean or the like. > - Removing use of random numbers from tests that do use them. > > Regards, > Maciej > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
