Thank you (and anbda of course)! I am so glad we are finally going to track test262 progress \o/. This should also make it really easy for everyone to figure out which tests are still failing.
I second Shu's opinion on defaults and excluding directories. What is the process of keeping test262 up-to-date going to be? -Tom On Wed, Jan 25, 2017 at 7:34 PM, Shu-yu Guo <s...@mozilla.com> wrote: > Woohoo, thanks for the CI work! > > Outside of CI, I am strongly in favor jstests.py running test262 by > default. Concretely, I am in favor of: > - Default set to be everything, including test262, and > - Ability to exclude entire subdirectories > > They supersede many of our tests and is the general source of compliance > truth for many corner cases. The increased running time is worth it for > correctness when implementing new features. JIT hackers can most likely get > by most of the time without running the full suite. > > On Wed, Jan 25, 2017 at 9:55 AM, Steve Fink <sf...@mozilla.com> wrote: > >> anba has done some major work to get test262 runnable as a CI test, and >> I've been looking into creating taskcluster jobs for it. One thing that >> could use input from a wider audience is how we want it to work with >> running jstests manually. The tests replace the js/src/tests/test262 >> directory, which means that if we do nothing, the runtime of jstests.py on >> an opt build goes from about 50 seconds to about 260 seconds on my machine. >> I haven't tried a debug build. >> >> One option is to suck it up. If everyone (but me) tends to use jit-tests >> for their smoke tests already, then it doesn't matter. And you can always >> explicitly choose subdirectories to run. >> >> Another option, which I've implemented but wanted some feedback before >> landing, is to say that running jstests.py with no arguments gives you "the >> default set", rather than its current "everything". And that default set >> would be "all but test262". If you want everything, you can say >> >> ./jstests.py $JS * >> >> But in order to implement that, I had to implement directory exclusions. >> Which means it wouldn't be much extra effort to implement >> --exclude=test262, in which case the first option is a little more >> tolerable: |./jstests.py $JS| would give you everything, and if you didn't >> want the huge pile of test262 tests, you could do |./jstests.py $JS >> --exclude=test262|. >> >> A third possibility that comes to mind is to have presets. |./jstests.py >> $JS --smoke| or something. >> >> Opinions? >> >> >> >> _______________________________________________ >> dev-tech-js-engine-internals mailing list >> dev-tech-js-engine-internals@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals >> > > > > -- > shu > _______________________________________________ > dev-tech-js-engine-internals mailing list > dev-tech-js-engine-internals@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals _______________________________________________ dev-tech-js-engine-internals mailing list dev-tech-js-engine-internals@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
