On Sun, Jun 14, 2015 at 10:17 PM, Greg Weng <[email protected]> wrote: > I think the problem is if our infrastructure or test method is not powerful > enough to test those tests in automation. For example, one smoketest I've > broken is about LockScreen doesn't update time correctly, if user pulled the > battery before rebooting (Bug 1119097). In such case, I think the key is our > automation couldn't cover some hardware related cases like battery or RIL,
I agree that covering cases like this where you need to pop the battery, or other low-level scenarios is going to be very hard. I also think there isn't going to be a single test framework that will cover all of the use cases we need to cover; we will need multiple different test harnesses to test different aspects of the code. Doing battery-popping might be much harder to do, but it shouldn't prevent us from working on some of the other things that are easier to automate. > By the way, for regressions I'm used to bisecting to find the accurate > broken patch. However from the information I've got is for Gecko or whole > B2G it's impractical to do bisecting, if considering the building time. I > wonder if we could or need to ease the pain of this to make finding a actual > broken part more easily and automatically. I believe this does help. I've had to bisections as well and it is time consuming for sure. However having inbound builds available on the pvtbuilds server helps a lot because you don't have to do builds yourself; you just need to flash and test. kats _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
