Hm. On a debug build, jstests for all but test262 takes 115sec; test262
takes 317sec. So we're talking about quadrupling the time -- of
something that really doesn't take all that much total time at all.
So if we're willing to vendor, I would suggest that we don't need any
new patches at all -- what is already in the bug is sufficient. Just
land it as-is, and let the additional tests run as part of the existing
jobs. Unless the compacting run starts timing out or something?
As a separate thing, I can land my changes to implement whole-directory
exclusions. My normal smoke test is to run a debug build on jstests, and
I'm not going to bother doing that as often if it's going to go from 2
minutes to 5 minutes. But I'm ok with having to pass --exclude=test262.
I also don't work on the frontend, so I'm a lot more interested in eg
js1_8_5/extensions than most of the stuff under test262/. That gets me
typed arrays, structured cloning, and a fair amount of GC, which are all
relevant to what I work on.
That's not a great way of dividing things up, though. I would rather
have a more rational basis for choosing smoke tests to run. Like
"interesting tests like js1_8_5/extensions" plus "really fast tests"?
Not that js1_8_5/extensions is a very good collection, either. If only
we had some magic to list out the most "useful" tests, as in ones that
have caught problems in the past -- including those that caught stuff
before it was pushed to try. Hm.
On 01/25/2017 10:34 AM, Shu-yu Guo wrote:
Woohoo, thanks for the CI work!
Outside of CI, I am strongly in favor jstests.py running test262 by
default. Concretely, I am in favor of:
- Default set to be everything, including test262, and
- Ability to exclude entire subdirectories
They supersede many of our tests and is the general source of compliance
truth for many corner cases. The increased running time is worth it for
correctness when implementing new features. JIT hackers can most likely get
by most of the time without running the full suite.
On Wed, Jan 25, 2017 at 9:55 AM, Steve Fink <sf...@mozilla.com> wrote:
anba has done some major work to get test262 runnable as a CI test, and
I've been looking into creating taskcluster jobs for it. One thing that
could use input from a wider audience is how we want it to work with
running jstests manually. The tests replace the js/src/tests/test262
directory, which means that if we do nothing, the runtime of jstests.py on
an opt build goes from about 50 seconds to about 260 seconds on my machine.
I haven't tried a debug build.
One option is to suck it up. If everyone (but me) tends to use jit-tests
for their smoke tests already, then it doesn't matter. And you can always
explicitly choose subdirectories to run.
Another option, which I've implemented but wanted some feedback before
landing, is to say that running jstests.py with no arguments gives you "the
default set", rather than its current "everything". And that default set
would be "all but test262". If you want everything, you can say
./jstests.py $JS *
But in order to implement that, I had to implement directory exclusions.
Which means it wouldn't be much extra effort to implement
--exclude=test262, in which case the first option is a little more
tolerable: |./jstests.py $JS| would give you everything, and if you didn't
want the huge pile of test262 tests, you could do |./jstests.py $JS
--exclude=test262|.
A third possibility that comes to mind is to have presets. |./jstests.py
$JS --smoke| or something.
Opinions?
_______________________________________________
dev-tech-js-engine-internals mailing list
dev-tech-js-engine-internals@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
_______________________________________________
dev-tech-js-engine-internals mailing list
dev-tech-js-engine-internals@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
