Woohoo, thanks for the CI work! Outside of CI, I am strongly in favor jstests.py running test262 by default. Concretely, I am in favor of: - Default set to be everything, including test262, and - Ability to exclude entire subdirectories
They supersede many of our tests and is the general source of compliance truth for many corner cases. The increased running time is worth it for correctness when implementing new features. JIT hackers can most likely get by most of the time without running the full suite. On Wed, Jan 25, 2017 at 9:55 AM, Steve Fink <sf...@mozilla.com> wrote: > anba has done some major work to get test262 runnable as a CI test, and > I've been looking into creating taskcluster jobs for it. One thing that > could use input from a wider audience is how we want it to work with > running jstests manually. The tests replace the js/src/tests/test262 > directory, which means that if we do nothing, the runtime of jstests.py on > an opt build goes from about 50 seconds to about 260 seconds on my machine. > I haven't tried a debug build. > > One option is to suck it up. If everyone (but me) tends to use jit-tests > for their smoke tests already, then it doesn't matter. And you can always > explicitly choose subdirectories to run. > > Another option, which I've implemented but wanted some feedback before > landing, is to say that running jstests.py with no arguments gives you "the > default set", rather than its current "everything". And that default set > would be "all but test262". If you want everything, you can say > > ./jstests.py $JS * > > But in order to implement that, I had to implement directory exclusions. > Which means it wouldn't be much extra effort to implement > --exclude=test262, in which case the first option is a little more > tolerable: |./jstests.py $JS| would give you everything, and if you didn't > want the huge pile of test262 tests, you could do |./jstests.py $JS > --exclude=test262|. > > A third possibility that comes to mind is to have presets. |./jstests.py > $JS --smoke| or something. > > Opinions? > > > > _______________________________________________ > dev-tech-js-engine-internals mailing list > dev-tech-js-engine-internals@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals > -- shu _______________________________________________ dev-tech-js-engine-internals mailing list dev-tech-js-engine-internals@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
