I should have noted that the existing on-device automation did catch the issue, but due to the wide range between device runs[1] the bug was filed before the investigation work to find the issue was completed.

Clint
[1] The 12 phones in automation right now run both UI and performance tests. They are triggered the moment a new tinderbox build is generated, and when they finish their job, they look to see if a new build has been generated. Therefore, at times when we have high velocity, the phones fall behind and have larger and larger ranges of commits between their runs.

We plan to solve this with the Flames by both having more devices as well as streamlining a "sanity" suite of tests to simply check some very basic functionality on the phone. This way we can keep up with our velocity.


On 6/3/2014 11:28, Clint Talbert wrote:
There are two pieces here. The first piece is that the tests for the feature were not as complete as they might have been, you can see that in the bug. Andrea jumped in and fixed that, which is great.

The most solid way to provide CI for this is to run a sanity test on Flame devices per checkin. That's what I have several of my teams engaged in building right now. It's behind schedule due to the delivery of the Flames and some IT issues that we are still working through. I'm trying hard to have this system stood up by the end of the quarter and reporting to Treeherder (tbpl v2).

Clint
On 6/2/2014 20:30, Faramarz Rashed wrote:
Hi Clint,
Is this something you can help us with? How can we help make sure we have solid CI tests around this?

Faramarz

On Jun 2, 2014, at 6:57 PM, Kyle Huey <[email protected]> wrote:

On Mon, Jun 2, 2014 at 6:47 PM, Faramarz Rashed <[email protected]> wrote:
ehsan,
how was this tested before landing and why did this break the trunk?

faramarz

On Jun 2, 2014, at 7:24 AM, Ehsan Akhgari <[email protected]> wrote:

I just backed out bug 957086. Andrea has verified that bug was the cause.

--
Ehsan
<http://ehsanakhgari.org/>


On Mon, Jun 2, 2014 at 9:46 AM, Ben Hearsum <[email protected]> wrote:
I locked our mozilla-central/master nightlies at one of the May 30th
builds until this is resolved.

On 14-06-02 09:22 AM, Dale Harvey wrote:
Is that the correct bug? its resolved for 3 days now

https://bugzilla.mozilla.org/show_bug.cgi?id=1018956 has also been filed and duped against https://bugzilla.mozilla.org/show_bug.cgi?id=1018909,
anyone know what the root cause is for this and fill us in?

Thanks
Dale


On 2 June 2014 02:27, Kevin Grandon <[email protected]> wrote:

Just wanted to send a note to let you know that current trunk is
broken,
so maybe avoid updating gecko for a little bit. I've been pinged a few times offline and didn't see an update, so thought it'd make sense to
notify everyone.

A fix is in the works and is being tracked in bug 957086.

Best,
Kevin
_______________________________________________
dev-gaia mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-gaia

_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g



_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
It was tested on our CI infrastructure for three full days. It's
pretty alarming that we don't have any testing that can catch
something as bad as "the device is completely unusable".

- Kyle

_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

Reply via email to