I am for stricter checks and crashing the test when some of the errors
discussed here are detected.
I have been puzzled when some attributes are undefined as such when I
ran tests for Thunderbird in the past.
The bugs are found non-Thunderbird-specific code often times.
Crashing the tests would be the only way to force the responsible party
to fix them.
Also, the mechanism for automatic dumping the stack during testing when
such errors are encountered will be appreciated. Running gdb under linux
and do some python magic to look at VM stack is not easy.
However, sending the stack from the user may have a privacy concern, and
should be evaluated very carefully.
(I mean, VM stack may show the URL which the user was visiting at the
time exactly, etc., or even some input.)
n 2017/10/18 18:45, Gregory Szorc wrote:
On Wed, Oct 18, 2017 at 10:28 AM, David Teller <dtel...@mozilla.com> wrote:
Hi everyone,
Yesterday, Nightly was broken on Linux and MacOS because of a typo in
JS code [1]. If I understand correctly, this triggered the usual
"undefined is not a function", which was
1/ uncaught during testing, as these things often are;
2/ basically impossible to diagnose in the wild, because there was no
error message of any kind.
I remember that we had bugs of this kind lurking for years in our
codebase, in code that was triggered daily and that everybody believed
to be tested.
I'd like to think that there is a better way to handle these bugs,
without waiting for them to explode in our user's face. Opening this
thread to see if we can find a way to somehow "solve" these bugs, either
by making them impossible, or by making them much easier to solve.
I have one proposal. Could we change the behavior of the JS VM as follows?
- The changes affect only Nightly.
- The changes affect only mozilla chrome code (including system add-ons
but not user add-ons or test code).
- Any programmer error (e.g. SyntaxError) causes a crash a crash that
displays (and attaches to the CrashReporter) both the JS stack in and
the native stack.
- Any SyntaxError is a programmer error.
- Any TypeError is a programmer error.
I expect that this will find a number of lurking errorsy, so we may want
to migrate code progressively, using a directive, say "use strict
moz-platform" and static analysis to help encourage using this directive.
What do you think?
I agree that errors like this should have better visibility in order to
help catch bugs.
I'm not sure changing behavior of the JS VM is the proper layer to
accomplish this. I think reporting messages from the JS console is a better
place to start. We could change the test harnesses to fail tests if certain
errors (like SyntaxError or TypeError) are raised. If there is a "hook" in
the JS VM to catch said errors at error time, we could have the test
harnesses run Firefox in a mode that makes said errors more fatal (even
potentially crashing as you suggest) and/or included additional metadata,
such as stacks.
Another idea would be to require all non-log output in the JS console to be
accounted for. Kinda like reftest's expected assertion count. Assuming most
JS errors/warnings are unwanted, this would allow us to fail all tests
reporting JS errors/warnings while allowing wanted/known failures to not
fail the test. A problem though is background services "randomly" injecting
their output during test execution depending on non-deterministic timing.
It could be difficult to roll this out in practice. But it feels like we
should be able to filter out messages or stacks accordingly.
But why stop there? Shouldn't Firefox installs in the wild report JS errors
and warnings in Firefox code back to Mozilla (like we do crashes)? I know
this has been discussed. I'm not sure what we're doing/planning about it
though.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
