On Tue, Jun 3, 2014 at 12:41 PM, Bobby Holley <bobbyhol...@gmail.com> wrote:
> On Tue, Jun 3, 2014 at 12:31 PM, Boris Zbarsky <bzbar...@mit.edu> wrote:
>
>> I can certainly buy "it's longer than what I'm
>>> used to", and even "incremental effort is required" - just not
>>> "incremental effort is required and that effort is non-negligible
>>> given other factors"
>>>
>>
>> Purely subjectively, it's non-negligible. The more frustrating writing a
>> test is, the less I want to do it, and the more I'll skimp on testing
>> exhaustively... Maybe that's a personal failing of mine, but I suspect not.
>>
>> The way I see it, anything that makes the test-writing experience more
>> frustrating is a bar to us having better test coverage.
>
> +1 to all of this.
Agreed. I think that we should optimize our test harnesses for making
it easy to write tests that contain a large number of useful
assertions.
SimpleTest.js does this pretty well. The amount of boiler plate that
you need is really minimal.
The new names from the initial email in this thread is certainly an
improvement for xpcshell-test, but why not go all the way and align
even more with SimpleTest.js?
testharness.js still requires lots of boiler plate. Especially when
writing async tests. And especially if you try to follow the rule that
each test within a file should clean up after itself. I suspect this
is part of the reason we haven't seen more people use it for normal
regression-test writing.
However SimpleTest.js certainly has shortcomings.
* It's a pain when writing tests that trigger window.onerror.
Generally we stick those tests in an <iframe> to not make
SimpleTest.js think that we have an unhandled error. It'd be great if
SimpleTest.js had a hook for indicating "this error is expected"
* SimpleTest.js should use window.addEventListener("load") rather than
window.onload. This would enable tests to use window.onload as it sees
fit.
* Given that promises are at this point the accepted way of doing
asynchronous programming, it would be good to support something
promise based instead of (or more likely in addition to)
SimpleTest.waitForExplicitFinish().
* I think we overuse the third argument to is() and the second
argument to ok(), and under-use info(). This causes people to type out
more error logging code. This isn't so much a problem with
SimpleTest.js as with how we use it though.
* The fact that our httpd.js is completely non-standard and unlikely
to ever run in other environments. Moving to node.js or pythons
BaseHTTPServer seems much more likely to get it accepted in other
environments. Node.js seems like the better choice given that all test
writers will need to know JS anyway. Again, this isn't so much a
problem with SimpleTest.js, but rather the environment that we use it
in.
/ Jonas
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
