There are probably multiple reasons why this has occurred, but one of
them is the fact that bugs filed due to TBPL intermittent oranges are
usually ignored for B2G. This causes our intermittent failures to build
up in aggregate until they reach a critical point and tests become
almost unsheriffable.
I think it needs to become an engineering priority to deal with these
issues before they become critical. We need engineering owners for
these types of bugs who can be responsible for debugging and fixing them
before they get out of control.
See my recent post for a good case in point:
https://groups.google.com/forum/#!topic/mozilla.dev.b2g/YcyxeW0iuNg
One thing that could help, if some engineering resources were committed
to helping, are some additional categories in bugzilla for B2G;
currently many of these failures get filed in the "General" category,
which is busy and noisy and it's easy to see how things can fall through
the cracks. Would it make sense to add a Tests category, and file
general TBPL failures there unless they could be specifically pinned to
a particular gaia app or gecko component?
Jonathan
On 7/25/2013 8:10 AM, Fabrice Desre wrote:
Hi All,
Development for 1.1 on the b2g18 branch is approaching its term (yeah!)
which means that we will soon rely on current trunk for 1.2.
Unfortunately the builds from m-c + gaia-master have been less than
stellar lately: gfx regressions on some devices, various crashes, etc.
to the point of not being dogfoodable at all. See
https://bugzilla.mozilla.org/show_bug.cgi?id=884399 to feel the pain.
Tests are not in great shape either, and sheriffs are considering to
hide the B2G Marionnette tests later today due to the ongoing high
failure rate
(https://bugzilla.mozilla.org/buglist.cgi?keywords=intermittent-failure%2C%20&keywords_type=allwords&short_desc=socket.timeout&resolution=---&query_format=advanced&short_desc_type=allwordssubstr&list_id=7335976)
We need to improve a lot there, but it's not clear to me why we are in
this situation: are we rushing patches in? is the review-roundrip
pressure too high to ensure a decent quality level? are we just
regressing because of a lack of tests (we still have a huge technical
debt to repay there)?
I would also like QA to run daily smoketests on these builds if possible.
Thoughts?
Fabrice
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
