Re: [openstack-dev] [qa] Tempest bug triage questions
On 06/22/2015 10:15 AM, Yaroslav Lobankov wrote: Hello everyone, I have some questions about the bug triage procedure for Tempest: 1. Some bugs in Tempest have status Fix committed. Should we move statuses of these bugs to Fix released? Yes, tempest doesn't have the kind of releases where Fix committed makes sense. 2. Many bugs have statuses In progress, but patches for these bugs have -1 from someone (or their workflow is -1) and it looks like these patches are abandoned, while statuses of such patches are Review in progress. What should we do with such bugs? This is kind of tricky without the project having a manager. I think we usually ping with a comment in the bug to see what the holdup is. If there is no response, we would set it back to Triaged of Confirmed. 3. What should we do with bugs like this [1]? It says that TimeoutException occurred, but the log of the test is unavailable by the link already. I don't know how to reproduce the issue, besides the log of the test is unavailable. What should I do with this bug? This bug is about how tempest should handle cases where a resource deletion fails. I think it is a legitimate issue though the answer is not clear given a gate that may be very slow at times. -David Thank you! [1] https://bugs.launchpad.net/tempest/+bug/1322011 Regards, Yaroslav Lobankov. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tempest bug triage questions
On Mon, Jun 22, 2015 at 11:41:56AM -0400, David Kranz wrote: On 06/22/2015 10:15 AM, Yaroslav Lobankov wrote: Hello everyone, I have some questions about the bug triage procedure for Tempest: 1. Some bugs in Tempest have status Fix committed. Should we move statuses of these bugs to Fix released? Yes, tempest doesn't have the kind of releases where Fix committed makes sense. Well, except for tempest-lib bugs. The tempest bug tracker is used for both tempest and tempest-lib. So if there are bugs marked as fix committed against tempest-lib we need to wait until they are included in a release before we mark it as fix released. But, in general tempest bugs should go straight from in-progress to fix-released on commit. There was a brief period where we didn't do that and you might be catching leftovers from that time. 2. Many bugs have statuses In progress, but patches for these bugs have -1 from someone (or their workflow is -1) and it looks like these patches are abandoned, while statuses of such patches are Review in progress. What should we do with such bugs? This is kind of tricky without the project having a manager. I think we usually ping with a comment in the bug to see what the holdup is. If there is no response, we would set it back to Triaged of Confirmed. Well if the patch is abandoned, I thought it does this automagically. But, if the patch is abandoned and it's still in progress I'd move it back to a different state to reflect the bug's real current state. -1 reviews are a different matter though, that's ongoing work. (although maybe slowly) 3. What should we do with bugs like this [1]? It says that TimeoutException occurred, but the log of the test is unavailable by the link already. I don't know how to reproduce the issue, besides the log of the test is unavailable. What should I do with this bug? This bug is about how tempest should handle cases where a resource deletion fails. I think it is a legitimate issue though the answer is not clear given a gate that may be very slow at times. I think the question was more about if the bug doesn't have a lot of details and the gate links are dead, because of the log server expiration window. In this case I normally mark the bug as incomplete and ask the submitter for more details. Just a stack trace and/or a link to a gate failure often isn't enough for a bug report long term. For short term fixes it's often just done as a quick way to move forward so we track it for an e-r query and fix it. But, if it ends up sitting for a while untriaged without an e-r query then it's not a good bug report and it needs more details. -Matt Treinish [1] https://bugs.launchpad.net/tempest/+bug/1322011 pgpR8LI7SthYC.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tempest Bug triage
All, As we all know bug triage rotation is working well in QA and we keep new bug count low. Thanks everyone for signing up in bug triage rotation. To continue the same strategy and keeping good progress on bugs, we need more volunteers to sign-up for coming weeks. People who want to help in bug triage, feel free to put your name in https://etherpad.openstack.org/p/qa-bug-triage-rotation Thanks gmann On Fri, Sep 12, 2014 at 4:52 AM, David Kranz dkr...@redhat.com wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Since bug triage should involve the Core review team, we propose to rotate the responsibility of triaging bugs weekly. I put up an etherpad here https://etherpad.openstack.org/p/qa-bug-triage-rotation and I hope the tempest core review team will sign up. Given our size, this should involve signing up once every two months or so. I took next week. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks Regards Ghanshyam Mann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tempest Bug triage
On Thu, Sep 11, 2014 at 03:52:56PM -0400, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Alternatively, strongly recommend people to post *contextual* logs to the bug, so they're there for reference forever and makes life less painful while triaging bugs. Many times bugs are just filed in a hurry, posting a quick bunch of logstash URLs which expires sooner or later. Sure, posting contextual logs takes time, but as you can well imagine, it results in higher quality reports (hopefully), and saves time for others who have to take a fresh look at the bug and have to begin with the maze of logs. -- /kashyap ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tempest Bug triage
On 09/12/2014 05:11 AM, Kashyap Chamarthy wrote: On Thu, Sep 11, 2014 at 03:52:56PM -0400, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Alternatively, strongly recommend people to post *contextual* logs to the bug, so they're there for reference forever and makes life less painful while triaging bugs. Many times bugs are just filed in a hurry, posting a quick bunch of logstash URLs which expires sooner or later. Sure, posting contextual logs takes time, but as you can well imagine, it results in higher quality reports (hopefully), and saves time for others who have to take a fresh look at the bug and have to begin with the maze of logs. This would be in addition to, not alternatively. Of course better bug reports with as much information as possible, with understanding of how long log files will be retained, etc. would always be better. But due to the sorry state we are now in, it is simply unrealistic to expect people to start investigating failures in code they do not understand that are obviously unrelated to the code they are trying to babysit through the gate. I wish it were otherwise, and believe this may change as we achieve the goal of focusing our test time on tests that are related to the code being tested (in-project functional testing). The purpose of rotating bug triage is that it was not happening at all. When there is a not-so-much-fun task for which every one is responsible, no one is responsible. It is better to share the load in a well understood way and know who has taken on responsibility at any point in time. -David -- /kashyap ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tempest Bug triage
On 09/11/2014 04:52 PM, David Kranz wrote: So we had a Bug Day this week and the results were a bit disappointing due to lack of participation. We went from 124 New bugs to 75. There were also many cases where bugs referred to logs that no longer existed. This suggests that we really need to keep up with bug triage in real time. Since bug triage should involve the Core review team, we propose to rotate the responsibility of triaging bugs weekly. I put up an etherpad here https://etherpad.openstack.org/p/qa-bug-triage-rotation and I hope the tempest core review team will sign up. Given our size, this should involve signing up once every two months or so. I took next week. -David +1, I'm not core team but I just assigned myself to the last week of September and first of December. Also, given the bad quality of some reports we may want to come up with a template of need to have data on the bug reports. I really haven't look at it lately but we use to have several reports with just a bunch of traces, or just a link.. -- mauro(sr) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][tempest] Bug triage and monitoring process
On 12/06/2013 03:18 PM, Adalberto Medeiros wrote: Hello all! Yesterday, during the QA meeting, I volunteer myself to help the team handling bugs and defining a better process to triage them. which is great! To accomplish that, I would like to suggest a Bug Triage Day for next week on Thursday, 12th (yup, before people leave to end-of-year holidays :) ). see you on #openstack-qa The second step, after getting a concise and triaged bug list, is to ensure we have a defined process to constant revisit the list to avoid the issues we have now. I'm would like to hear suggestions here. Please, send any thoughts about those steps and any other points you think we should address for monitoring the bugs. We may as well define in this thread what is needed for the bug triage day. a relatively simple thing we could do is to tag the bugs with the services they hit, for easier categorization and set some notifications on new tags so that one can revisit the tags on new bugs -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][tempest] Bug triage and monitoring process
Hi, -Original Message- From: Adalberto Medeiros [mailto:adal...@linux.vnet.ibm.com] Sent: Friday, December 06, 2013 11:18 PM To: OpenStack Development Mailing List Subject: [openstack-dev] [qa][tempest] Bug triage and monitoring process Hello all! Yesterday, during the QA meeting, I volunteer myself to help the team handling bugs and defining a better process to triage them. Thanks for doing this, this would help for digging problems. Investigating the current bug list, I checked we have: * 7 critical and high bugs. From those, 3 critical non-assigned: https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.importance%3Alist=CRITICALfield.importance%3Alist=HIGH assignee_option=none One HIGH report has been solved the last weekend, and I have been changed it. and I tried to solve the other one(https://bugs.launchpad.net/tempest/+bug/1213209), but I cannot find the way. This seems a libvirt problem, but we don't have some log file under /var/log/libvirt in gate tests. Can we add these log files to gate tests for digging this kind of problems? * 113 new bugs * 253 open bugs The first step here is to triage those NEW bugs and try to verify as much as possible the OPEN bugs are being addressed. One goal is to check duplicates, find assignees, confirm if the bugs are still valid and prioritize them. Another one is to ensure recheck bugs are marked correctly (critical or high) and that they have the right people looking at them. Finally, it's important to revisit old bugs in order to check they are still valid and re-prioritize them. To accomplish that, I would like to suggest a Bug Triage Day for next week on Thursday, 12th (yup, before people leave to end-of-year holidays :) ). The second step, after getting a concise and triaged bug list, is to ensure we have a defined process to constant revisit the list to avoid the issues we have now. I'm would like to hear suggestions here. Please, send any thoughts about those steps and any other points you think we should address for monitoring the bugs. We may as well define in this thread what is needed for the bug triage day. I feel these steps are nice, thanks! Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
