On Fri, 19 Dec 2014 18:58:53 -0500, Ehsan Akhgari wrote: > On 2014-12-19 4:40 PM, Nils Ohlmeier wrote:
>>> On Dec 19, 2014, at 6:56 AM, Ehsan Akhgari <ehsan.akhg...@gmail.com> wrote: >>> Logging sufficiently is almost always enough to not have to >>> use these timers, as those tests have demonstrated in >>> practice. >>> >> >> I disagree. Especially as the logging of mochitest is really poor. >> - the timestamps of log messages are lost as described in bug 1020516 >> - SimpleTest only has one log level: info(). Which I think is the main >> reason the log files get big, because test writers can’t fine tune the >> logging. >> - Log messages from mochitest are not printed together with the >> stdout/stderr of the browser (so correlating these two is almost impossible) >> - Logging behaves differently on your local machine then on the build servers > Some of these (such as the latter > two) seem to be issues that you'd also have with your own custom > logging in mochitest, so I'm not sure how the timer based logging > is not affected by them. I assume multiple custom timers can provide different output that can support a diagnosis without timestamps. > >> BTW I understand that WebRTC has pretty special and probably >> unique requirements, compared to what the rest of the mochitest >> users do with it. FWIW, these issue plague test result analysis in general, not just rtc. > But thinking of this more, do you find it useful to be able to > register a callback with the harness to be called when the harness > decides to kill your test because it has timed out? (Obviously > the callback would be informational, and would not be able to > prevent the harness from killing the test and moving on.) That might be useful, but perhaps it would not be necessary if logging worked. Perhaps it is better to have an error handler that only runs when there are errors, reducing spam, but that means keeping disjoint parts of the test in sync perhaps putting more state in global scope, while logging is simple to maintain. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
