On Tue, Nov 19, 2013 at 8:40 PM, Zac Campbell <[email protected]> wrote:
> Why would a dev want to write a rigid and reliable test? It delays the > feature/bug fix, it's harder and it might have to be maintained in the > future if it fails. We all know it's easy to write a test that's more > likely to stay green over time. > Module owners and peers at Mozilla have *integrity* and put a high premium on *quality*. I trust the developers with whom I work to do the right thing when they're writing and reviewing code. I never r+ a patch until I have read (and understand at least as well as the patch writer) every line in the diff and my past Gaia reviewers have given my patches the same courtesy. > The reviewer on the other hand can take the green "OK to merge" of Travis > far to literally and not thoroughly review the tests, particularly when > under some time pressure. > They could but they don't. If they do, they are not fit to be module owners/peers. > > ----- Original Message ----- > From: Gareth Aye <[email protected]> > To: Zac Campbell <[email protected]> > Cc: Jonas Sicking <[email protected]>, [email protected], dev-b2g > <[email protected]> > Sent: Tue, 19 Nov 2013 15:34:29 -0800 (PST) > Subject: Re: [b2g] Let QA write your Gaia/Marionette UI tests! > > Zac - I didn't mean to offend! I got into this line of work by doing a lot > of theoretical math, so if I use proof-by-exhaustion to point out an issue > I have with your proposal, it doesn't mean that I don't want your help > making firefox os a quality product. > > More test coverage is always good. I also know that b2g developers haven't > always been great about writing tests. For feature/bug code that landed > without tests, late is much better than never. > > However, I think I made some valid points about why (for future > work/bugs/patches) we should strive to either have one person write the > feature/bugfix and the tests or have dev/qa work land in the same pull > request/patch. > > It's also true (and I think you sensed from my response) that I want > developers to write and maintain tests. I believe this is the only scalable > way forward. That doesn't mean that, for the test coverage we already lack, > we couldn't use some help. > > > On Tue, Nov 19, 2013 at 5:50 PM, Zac Campbell <[email protected]> > wrote: > > > Well geez i was just offering, kindly I thought, to have QA write some > > additional test coverage, especially against real device hardware. I > > thought it would be valuable. > > > > But I can read between the lines; you want Python and QA's tests to sod > > off. > > > > > > > > ----- Original Message ----- > > From: Gareth Aye <[email protected]> > > To: Zac Campbell <[email protected]> > > Cc: Jonas Sicking <[email protected]>, [email protected], > dev-b2g > > <[email protected]> > > Sent: Tue, 19 Nov 2013 14:28:43 -0800 (PST) > > Subject: Re: [b2g] Let QA write your Gaia/Marionette UI tests! > > > > Ok well that's all well and good, but I think we should be careful about > > how we do this. Will there be separate pull requests to Gaia? I posit > that > > the correct way to develop is to review and land test code alongside > > feature/bug code. Let's prove this to ourselves by exploring other, wrong > > ways in which we can do this. > > > > Case 1: Suppose we review/land our tests before the feature/bug code. > > > > - This breaks the build. > > - It's possible that we learn things while writing the feature/bug code > > about the constraints of the problem and that the feature changes. In > this > > case, our tests have gotten old and crufty before they ever passed! > > - It makes our generous reviewers take two steps out of their busy days > > instead of one. > > > > Case 2: Suppose we review/land our tests after the feature/bug code. > > > > - We check untested code into source control. Do we even know if it > works? > > - If the feature/bug code gets backed out and the test stays, then we > break > > the build. > > - We waste our reviewer's time again making them review two patches. > > > > Our proof is finished since the only options are to land our test code > > before, alongside, or after our bug code. Therefore, if we're going to > have > > people other than feature/bug code writers developing automated test > cases, > > then everyone who's working on the bug must work together to submit a > > single patch and/or pull request to Gaia. This workflow isn't what I've > > observed so far with the Python test code, and I hope that we keep to it > > for the future health of our project. > > > > Personally this seems like a lot of hassle to me (which is why I prefer > to > > write my own tests), but I am happy for others to do this so long as they > > do it thoughtfully (in a way that doesn't waste reviewer time, stop good > > tests from being written, and break the build). > > > > > > > > On Tue, Nov 19, 2013 at 3:54 AM, Zac Campbell <[email protected]> > > wrote: > > > > > Well I was trying to offer you a QA/automation team to do that for > you! > > > > > > but I am interested in new test cases for new functionality coming up > > that > > > QA haven't been let in on yet or if there's coverage that your team > needs > > > for on-device testing to use hardware that desktopb2g doesn't have > access > > > to. We can cover some space that Travis cannot. > > > > > > Julien has requested some for Messaging app that we did and Rudy also > > > asked for a keyboard/TBPL test which we did last week. > > > > > > > > > > > > > > > > > > > > > > > > On 18/11/13 16:58, Gareth Aye wrote: > > > > > > I will let you, but only if you use the js tooling so that I can review > > > and maintain the tests :) > > > > > > > > > On Sat, Nov 16, 2013 at 3:47 PM, Jonas Sicking <[email protected]> > wrote: > > > > > >> On Nov 14, 2013 3:57 AM, "Zac Campbell" <[email protected]> > wrote: > > >> > > > >> > Are you tired? is the cat bothering you for its food? Forgotten to > > shave > > >> for the last 6 days? Don't have time to write your UI tests? > > >> > > > >> > Then let QA do it for you! > > >> > > >> Hahaha, I love it! Zac++ > > >> > > >> / Jonas > > >> _______________________________________________ > > >> dev-b2g mailing list > > >> [email protected] > > >> https://lists.mozilla.org/listinfo/dev-b2g > > >> > > > > > > > > > > > > -- > > > Best, > > > Gareth > > > > > > > > > > > > > > > -- > > Best, > > Gareth > > > > > > > -- > Best, > Gareth > > -- Best, Gareth _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
