Yes we have the tests and they are run regularly.
The device test caught the dialer bug, but only on the twice daily
releng builds which is a rough range.
We couldn't catch it on the m-c tinderbox.ui builds because of the bug
that hamachi builds were too large for the device and thus the
automation failed for that period (since resolved).
On 03/06/14 17:24, Dave Hylands wrote:
Do we even have tests which run on a real device?
If not, how can we write a test to verify that a call can be made?
Dave Hylands
------------------------------------------------------------------------
*From: *"Julien Wajsberg" <[email protected]>
*To: *"dev-b2g" <[email protected]>
*Sent: *Tuesday, June 3, 2014 9:05:22 AM
*Subject: *Re: [b2g] Smoketest Regressions on Trunk are Out of Control
(answering on dev-b2g only, crossposting is painful)
Le 03/06/2014 17:53, L. David Baron a écrit :
> On Tuesday 2014-06-03 08:13 -0700, Jason Smith wrote:
>> I need this under control as soon as possible, as these
regressions are causing the build to move towards being completely
not functional. I'm also debating if we should close the Gaia tree
at this point since there's now risk that we can end up with a
non-functional build.
>>
>> Can we get this under control here? If closing the Gaia tree
will help this, then let's do that.
> A somewhat radical proposal:
>
> 1. Close the tree, find the patches that caused the regressions,
> back them out, and reopen the tree.
>
> 2. Don't let the patches reland until a test has been added that is
> green without the patch, but would have caused orange when
> combined with the regression.
>
> If the regression was at a platform level or other lower level
> than the gaia app, the test could be too. But if that's the
case
> it would also be good to have test coverage of the gaia apps.
>
> We're not always this strict about test coverage in Gecko-land,
> but it's been useful to be strict in areas with the problem of
> frequent regressions.
>
> Not getting tests added that catch these problems is an accumulation
> of technical debt. Perhaps now really is a bad time to deal with
> this, but there ought to be a time to do so.
>
yes, this is exactly the path we try to follow nowadays. I hope
it will
make us more resilient in the future.
--
Julien
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
