Gaia tests on TBPL haven't been showing any increase in intermittents.

Travis is really a bad UI for sheriffing these kinds of problems, and we're not dumping gecko.log from Travis which doesn't help either.  We should dump gecko.log at the end of the run so we can see what sorts of errors the build is sending to stdout/stderr.

Some digging shows a few details.  We haven't seen the apparent crash in the last few hours; all the failures more recent than about 7 hours ago are single-test failures, mostly of tests that we're not running in TBPL (yet).  Probably, these are flakey tests, and certainly, we should be using the same manifest in TBPL as we do in Travis so we can use TBPL to catch flakey test issues.

The crash occurred several times when the tests were using the build  http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-linux64_gecko/1385546575/b2g-28.0a1.multi.linux-x86_64.tar.bz2, and a few earlier builds, but haven't been seen with later builds...maybe this problem has already resolved itself?

Jonathan

On 11/27/2013 12:58 PM, Dale Harvey wrote:
mentioning marionette was just a stab in the dark, I was more saying that it seems like something handwavey infrastructurey than specific component failures, something crashy when going through the b2g code path seems feasible

If anyone better than me at reading tbpl could see if there was any point in the last day in which gaia tests got less stable, that would be useful information



On 27 November 2013 20:47, Jonathan Griffin <[email protected]> wrote:
Yes, that landed on Saturday, so if the problems you're seeing date back that far, that's a potential culprit.

Jonathan


On 11/27/2013 11:42 AM, Gareth Aye wrote:
Either way this is not necessarily a marionette server issue.


On Wed, Nov 27, 2013 at 2:42 PM, Gareth Aye <[email protected]> wrote:


On Wed, Nov 27, 2013 at 2:39 PM, Jonathan Griffin <[email protected]> wrote:
The only Marionette change to have gone in recently is:

http://hg.mozilla.org/mozilla-central/rev/9cc147e1d222

Jonathan


On 11/27/2013 11:28 AM, Gareth Aye wrote:
Also fwiw I don't understand how this could have landed if the python tests
are running on tbpl...

On Wed, Nov 27, 2013 at 2:16 PM, Gareth Aye <[email protected]> wrote:

Zac,

I'm a bit busy with other work right now. Would you mind bisecting gecko
to find the culprit patch for us?


On Wed, Nov 27, 2013 at 2:10 PM, Zac Campbell <[email protected]>wrote:

On 27/11/13 18:29, Dale Harvey wrote:

Tracking bug is: https://bugzilla.mozilla.org/show_bug.cgi?id=943980

Been seeing test failures on travis for the last 6 hours (
https://travis-ci.org/mozilla-b2g/gaia/builds) so we have closed the
tree

The failures wildly vary and all seem to be intermittent, so this is
likely
due to a new b2g build / marionette changes, currently debugging
(although
I will be afk for ~90minutes) so if anyone else wants to help debug that
would be great :)

Will reopen as soon as we get consistent passing builds

Cheers
Dale
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

wrt the gaia-ui-tests failures:
"Connection Refused" does look like the whole b2g/gecko is crashing. The
console trace is similar to when we run these tests on device and the
device crashes.

Also the one about:
TypeError: app is undefined ; stacktrace: execute_async_script
@gaia_test.py, line 74
inline _javascript_, line 251
src: " let result = {frame: app.frame.firstChild,"

I have witnessed on device and this is when b2g fails to start up. The
device was stuck on the splash screen, but it is conscious enough for it to
be stopped and restarted (and thus the test suite continue).


There was one test failing intermittently earlier in the day:
test_edit_contact.py
I've prepared a fix so it doesn't stop you from re-opening the tree:
https://github.com/mozilla-b2g/gaia/pull/14136


Good luck,
Zac
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g



--
Best,
Gareth







--
Best,
Gareth




--
Best,
Gareth




_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

Reply via email to