+dev-quality and qa-b2g lists Hi Kats,
Thanks for bringing this up. The Qawanted process was communicated out back around 1.3 timeframe, but it sounds like its worth revisiting. In general, qawanted investigation work is investigative and time consuming, since the purpose is to track down higher reproducible rates, applying patches to verify, and installing and bisecting device builds to narrow regression windows. Therefore, its inevitable that bugs need a priority system. Also, QAwanted work is for anyone. Other Mozilla products use this keyword also, albeit the process differs a bit on other products. Anyone is encouraged to dive in and assist. The wiki page in [1] was created specifically for B2G, and i actually have a dedicated daily test team that works on these various queries. In fact, there is a QAwanted list of results that is send daily to a public watch list [2] that anyone can follow: [email protected]. The team has been tackling an average of 15 bugs a day. The current focus of the team is to start with anything blocking the current b2g release, followed by one prior and then the master. it also has a secondary list of considerations like (is it a smoke test blocker, does it break new functionality or old regression, is it an edge case, etc..) so to answer your questions below: > > I don't know who is "officially" responsible for responding to QA requests on > these bugs, but it seems that they are falling through the cracks, and I > would like to avoid that. I see only a few options: Start with me as the contact person to initiate (bugzilla :tchung). if you flag a bug as qawanted, make sure it abides by the rules in the wiki, but you can ni? myself for now to get attention. Provide a comment on what qawanted help you need if it falls under the prioritization. Please be sure not to abuse the keyword, as again we typically get through 15 issues on average during the workday. > 1) Ensure that any bugs that need QA assistance from the B2G QA team are > moved to the Firefox OS product. This seems inefficient since in some cases > we already know that it's a bug in the core platform (e.g. a graphics bug) > and filing it in the Graphics component is more likely to get it fixed faster. I don’t agree to move the bug into another product, because that would not be true to where the issue lives. again, if its blocking a b2g issue, it’s important to make sure the flag status-b2g-<version> is set to “affected”, and even better if you can flag the blocking-b2g=? or + flag if it makes sense. if its in neither, chances are the bug will get lower priority attention from my team. but again, qawanted should have anyone can help, so don’t put all your eggs into the dedicated team if possible. > > 2) Get the B2G QA team to use a query such as the one above to find > qawanted-flagged bugs in other products, as long as the platform is "Firefox > OS". This would be my preferred solution. The backlog of such bugs is not > large; [3] is a query that searches for all qawanted bugs across all non-B2G > products with a Firefox OS platform, and as of right now it shows only 8 bugs. yes, it should include platform affected issues. i’ve already updated the outdated links on the wiki page today. thanks for pointing that out. > > 3) Start a new QA team solely responsible for dealing with bugs in other > products (e.g. Core) and having cross-platform expertise so they can deal > with bugs like these. As mentioned earlier, its really a cross product affected group. (Desktop uses it, Android team uses it, Web, maybe cloud services…. but everyone has their own rules) there was a thread way back on dev-quality that brought up variations of how to tag QA keywords in bugzilla. You’re welcomed to resurface this point in that thread with general quality team. Link to threads [3][4]. We also are building up a platform QA team and trying to define what that would look like to the rest of the world. In the meantime, if you have specific B2G requests for testing, please follow the steps i mentioned earlier, and of course you can always ni? me if its more urgently needed. Thanks, Tony > [1] https://wiki.mozilla.org/B2G/QA/Triage [2] https://mail.mozilla.org/listinfo/fxosqa-report-watchlist [3] https://groups.google.com/forum/#!topic/mozilla.dev.quality/7kiVWvgsrI8 [4] https://groups.google.com/forum/#!topic/mozilla.dev.quality/WBgsuGZNTgU On Oct 11, 2014, at 6:30 AM, Kartikaya Gupta <[email protected]> wrote: > As far as I know, the latest process for getting QA attention on B2G bugs is > documented at [1]. However, it's not clear to me which bugs are actually > checked for the qawanted flag. From my experience I feel like only bugs in > the Firefox OS product are checked, and often bugs filed in other products > get missed. > > For example, [2] is a list of bugs in the Core product whose platform is > marked as Firefox OS and have the qawanted keyword on them. > > I don't know who is "officially" responsible for responding to QA requests on > these bugs, but it seems that they are falling through the cracks, and I > would like to avoid that. I see only a few options: > > 1) Ensure that any bugs that need QA assistance from the B2G QA team are > moved to the Firefox OS product. This seems inefficient since in some cases > we already know that it's a bug in the core platform (e.g. a graphics bug) > and filing it in the Graphics component is more likely to get it fixed faster. > > 2) Get the B2G QA team to use a query such as the one above to find > qawanted-flagged bugs in other products, as long as the platform is "Firefox > OS". This would be my preferred solution. The backlog of such bugs is not > large; [3] is a query that searches for all qawanted bugs across all non-B2G > products with a Firefox OS platform, and as of right now it shows only 8 bugs. > > 3) Start a new QA team solely responsible for dealing with bugs in other > products (e.g. Core) and having cross-platform expertise so they can deal > with bugs like these. > > I would also really like it if somebody updated the bugzilla queries on [1] > since a lot of them seem out of date. I often use the bugzilla queries on [4] > to verify my bugs will get auto-uplifted and blocker nominations will get > responded to, and I would love to be able to use bugzilla queries on [1] to > verify that my QA requests will also get responded to. In a sense the > documented queries are like interfaces - if my bug doesn't show up in the > query then I tagged it wrong and I need to fix it; if it shows up but doesn't > get responded to then the fault is on the other side. Documenting these > interfaces can help tighten our process up so less things get missed. > > Does anybody have any thoughts on this issue? Any suggestions on who else > should be CC'd on this to make the appropriate decisions? > > Cheers, > kats > > [1] https://wiki.mozilla.org/B2G/QA/Triage > [2] http://bugzil.la/keywords:qawanted+product:core+os:gonk > [3] http://bugzil.la/keywords:qawanted+-product:"Firefox OS"+os:gonk > [4] https://wiki.mozilla.org/Release_Management/B2G_Landing > _______________________________________________ > dev-b2g mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-b2g
_______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
