Ah, I misunderstood.
If the log line is printed or dumped directly to stdout, then the string
TEST-UNEXPECTED-FAIL actually will fail the job. It's only if we do
log.info('TEST-UNEXPECTED-FAIL ...') that we run into problems. So if
you see the string 'TEST-UNEXPECTED-FAIL' somewhere, it isn't
necessarily wrong (though I think we should stop relying on this string
anyway).
I did a quick DXR for the failure case and found one place in a rarely
used mozrunner utility function that needs to be updated. Afaict, no
problems have landed as a result, but will get this fixed asap as well.
On 29/12/16 02:55 PM, Boris Zbarsky wrote:
On 12/29/16 11:49 AM, ahalberst...@mozilla.com wrote:
Have we done any sort of audit to see whether any other tests got broken
by the structured logging changes? Had we done such an audit after bug
1321127 was fixed?
Once bug 1321127 was fixed, any other tests that were broken would
turn the job orange so would have prevented the fix from landing.
No, what I meant was this: we now have two separate instances of
harnesses not showing failures because they were relying on the
TEST-UNEXPECTED-FAIL string parsing, not the structured logger. Have we
done any auditing for _other_ cases that are relying on said string
parsing?
-Boris
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
