Hey Ben - tl;dr - Inaris won't go anywhere unless there's an exigent reason to :-) And let us know how/if our coverage balance is meeting needs!
As part of a larger effort, and as you already know from below, we're rapidly moving off Unagis (see also https://bugzilla.mozilla.org/show_bug.cgi?id=924503). Our community (in particular, but not solely -- third-party contracts, too) won't (always) have access to Hamachi, Leos, or Helix devices, though, so we have actually retained support for Inaris, and their unlocked bootloaders and RelEng builds. We'll always continue to re-evaluate the mix of devices/branches (and the combinations thereof), matched with our limited ability (few folks in Mountain View, as it stands right now) to keep the devices maintained (Hamachis are turning out to be harder to get the flashing/perf-scripts for, than were/are Inaris and Unagis). To specifically answer your question, "Do we have an alternative method for detecting hardware specific bugs?" - both the Gaia UI Tests [1] and the performance tests (perf, mozperf + fps [2]) can be run locally, so if the team has the particular device, and the appropriate build from RelEng is still available (or you can self-build one), you shouldn't be /too/ stuck. (As part of a larger discussion, going forward, it'd be great as a larger team to (re?)define what "supported" means in terms of builds, job types (UI/perf) for each branch, and each device.) Until now, I wasn't aware of a device-specific issue on the Inari (esp. wrt perf), so am happy to maintain a couple Inari nodes, as Dave suggests, as long as they're continuing to provide value, and are supported by RelEng/product/devs et al. Feedback like that (that our setup can help troubleshoot or baseline a metric) is very helpful in knowing how important and valuable the coverage we help maintain is (compared with the rest of the branches/devices), so thanks! Esp. as you and the others on the Performance team have needs like this, don't hesitate to reach out to us (email/IRC), so we can work together to figure out what sustainable coverage looks like - it's never our intention to drop something that's useful; we do so most often to free up capacity for something else :-) Thanks, - Stephen [1] http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Hamachi/ [2] http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Perf/ ----- Original Message ----- From: "Ben Kelly" <[email protected]> To: "Dave Hunt" <[email protected]>, [email protected] Cc: "Stephen Donner" <[email protected]> Sent: Tuesday, October 29, 2013 10:15:46 AM Subject: Re: [b2g] Stopping performance testing on Unagi On 10/29/2013 12:45 PM, Dave Hunt wrote: If the results we're gathering are proving useful then I would say we should continue to keep a couple of these devices attached and running the tests for now. We can always revisit this again when we have a need for testing another device, but I would say we're unlikely to want to maintain running tests for several different devices at once. Do we have an alternative method for detecting hardware specific bugs? I would think over time we would be adding more devices instead of unifying to a single device. I could see a core device where we run tests frequently and then secondary devices that run perhaps once a day or something, but I wouldn't think we would want to remove those devices completely. Thanks! Ben <blockquote> Dave On 10/29/13 4:31 PM, Ben Kelly wrote: <blockquote> On 10/18/2013 10:04 AM, Ben Kelly wrote: <blockquote> On 10/18/2013 04:15 AM, Dave Hunt wrote: <blockquote> This is a notification that the performance testing on Unagi (links below) was stopped this week in favour of Hamachi and Inari. Once we're satisfied with the reliability of Hamachi we intent to drop Inari too. Please let me know if you have any concerns. </blockquote> Thanks for the heads up Dave! Just out of curiousity, why are we dropping inari? Isn't that also a shipping platform we need to support? </blockquote> For example, bocoup was asking today in the gaia meeting about scroll performance on inari that doesn't seem to effect the other platforms. If we're only measuring hamachi, I'm worried we're going to miss this kind of stuff. Thanks again. Ben </blockquote> -- *Dave Hunt* QA, Mozilla Corporation [email protected] </blockquote> _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
