[openstack-dev] [qa][tempest] Bug Triage Day - Thu 12th - Prep
Hi all! We have the Tempest Bug Triage Day this Thursday and there are a few points to start digging and investigating from now on. First let me share the guidelines link (Openstack Wiki for Bug Triage): https://wiki.openstack.org/wiki/BugTriage Starting from that, there are a few issues and observations to consider for discussion: 1) Confirmed and Triaged status. There seems to not have a consistency on those status, apparently being used as synonyms. Any thoughts on how or if we should differ them? 2) New Bugs https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=NEWassignee_option=none. We have 116 (and growing) new bugs to *prioritize, confirm and identify duplicates*. *This is where the team needs to put most of the effort.* There are also 5 New bugs assigned https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=NEWorderby=assigneethat should be converted to Confirmed/ In Progress. 3) Confirmed and Triaged bugs: 13 assigned https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=CONFIRMEDfield.status%3Alist=TRIAGEDorderby=assignee should be changed to In Progress if it makes sense or removed assignees. The other 30 confirmed/triaged https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=CONFIRMEDfield.status%3Alist=TRIAGEDassignee_option=none should be prioritized. 4) 4 In Progress/ Fix Committed https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=INPROGRESSfield.status%3Alist=FIXCOMMITTEDassignee_option=none bugs should not remain unassigned. Either have someone assigned or move it back to Confirmed and ensure importance. 5) Old bugs: * 15 In Progress https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status=In%20Progressorderby=date_last_updated with last update comments in October. Assignees should verify if the status still makes sense. If not, the bug may be moved back to Confirmed/Invalid and prioritized accordingly. * Confirmed/Triaged https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.status%3Alist=CONFIRMEDfield.status%3Alist=TRIAGEDorderby=date_last_updated : review importances and assignees. Also look for duplicates. I think the focus on Thursday should be specially on item 2, but also looking at 3. I'll start looking at those points and all the help here is appreciated. The final goal is to have this process well established to avoid getting a huge list of new bugs without review. Please, all thoughts/opinions will be really appreciated. Best regards, -- Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email:adal...@linux.vnet.ibm.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa][tempest] Bug triage and monitoring process
Hello all! Yesterday, during the QA meeting, I volunteer myself to help the team handling bugs and defining a better process to triage them. Investigating the current bug list, I checked we have: * 7 critical and high bugs. From those, 3 critical non-assigned: https://bugs.launchpad.net/tempest/+bugs?search=Searchfield.importance%3Alist=CRITICALfield.importance%3Alist=HIGHassignee_option=none * 113 new bugs * 253 open bugs The first step here is to triage those NEW bugs and try to verify as much as possible the OPEN bugs are being addressed. One goal is to check duplicates, find assignees, confirm if the bugs are still valid and prioritize them. Another one is to ensure recheck bugs are marked correctly (critical or high) and that they have the right people looking at them. Finally, it's important to revisit old bugs in order to check they are still valid and re-prioritize them. To accomplish that, I would like to suggest a Bug Triage Day for next week on Thursday, 12th (yup, before people leave to end-of-year holidays :) ). The second step, after getting a concise and triaged bug list, is to ensure we have a defined process to constant revisit the list to avoid the issues we have now. I'm would like to hear suggestions here. Please, send any thoughts about those steps and any other points you think we should address for monitoring the bugs. We may as well define in this thread what is needed for the bug triage day. Regards, -- Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] list of negative tests that need to be separated from other tests.
Hi! In the QA meeting yesterday, we decide to create a blueprint specific for the negative tests in a separate file: https://blueprints.launchpad.net/tempest/+spec/negative-test-files and use it to track the patches. I added the etherpad link Ken'ichi pointed to this bp. Ken'ichi, should you be the owner of this bp? Giulio, could you mark it In Progress? Regards, Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com On Mon 02 Dec 2013 10:23:07 PM BRST, Christopher Yeoh wrote: On Tue, Dec 3, 2013 at 9:43 AM, Kenichi Oomichi oomi...@mxs.nes.nec.co.jp mailto:oomi...@mxs.nes.nec.co.jp wrote: Hi Sean, David, Marc I have one question about negative tests. Now we are in moratorium on new negative tests in Tempest: http://lists.openstack.org/pipermail/openstack-dev/2013-November/018748.html Is it OK to consider this kind of patch(separating negative tests from positive test file, without any additional negative tests) as an exception? I don't have a strong opinion on this, but I think it's ok given it will make the eventual removal of hand coded negative tests in the future easier even though it costs us a bit of churn now. Chris Thanks Ken'ichi Ohmichi --- -Original Message- From: Adalberto Medeiros [mailto:adal...@linux.vnet.ibm.com mailto:adal...@linux.vnet.ibm.com] Sent: Monday, December 02, 2013 8:33 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [tempest] list of negative tests that need to be separated from other tests. Thanks Ken'ichi. I added my name to a couple of them in that list. Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com mailto:adal...@linux.vnet.ibm.com On Mon 02 Dec 2013 07:36:38 AM BRST, Kenichi Oomichi wrote: Hi Adalberto, -Original Message- From: Adalberto Medeiros [mailto:adal...@linux.vnet.ibm.com mailto:adal...@linux.vnet.ibm.com] Sent: Saturday, November 30, 2013 11:29 PM To: OpenStack Development Mailing List Subject: [openstack-dev] [tempest] list of negative tests that need to be separated from other tests. Hi! I understand that one action toward negative tests, even before implementing the automatic schema generation, is to move them to their own file (.py), thus separating them from the 'positive' tests. (See patch https://review.openstack.org/#/c/56807/ as an example). In order to do so, I've got a list of testcases that still have both negative and positive tests together, and listed them in the following etherpad link: https://etherpad.openstack.org/p/bp_negative_tests_list The idea here is to have patches for each file until we get all the negative tests in their own files. I also linked the etherpad to the specific blueprint created by Marc for negative tests in icehouse (https://blueprints.launchpad.net/tempest/+spec/negative-tests ). Please, send any comments and whether you think this is the right approach to keep track on that task. We have already the same etherpad, and we are working on it. Please check the following: https://etherpad.openstack.org/p/TempestTestDevelopment Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] list of negative tests that need to be separated from other tests.
Thanks Ken'ichi. I added my name to a couple of them in that list. Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com On Mon 02 Dec 2013 07:36:38 AM BRST, Kenichi Oomichi wrote: Hi Adalberto, -Original Message- From: Adalberto Medeiros [mailto:adal...@linux.vnet.ibm.com] Sent: Saturday, November 30, 2013 11:29 PM To: OpenStack Development Mailing List Subject: [openstack-dev] [tempest] list of negative tests that need to be separated from other tests. Hi! I understand that one action toward negative tests, even before implementing the automatic schema generation, is to move them to their own file (.py), thus separating them from the 'positive' tests. (See patch https://review.openstack.org/#/c/56807/ as an example). In order to do so, I've got a list of testcases that still have both negative and positive tests together, and listed them in the following etherpad link: https://etherpad.openstack.org/p/bp_negative_tests_list The idea here is to have patches for each file until we get all the negative tests in their own files. I also linked the etherpad to the specific blueprint created by Marc for negative tests in icehouse (https://blueprints.launchpad.net/tempest/+spec/negative-tests ). Please, send any comments and whether you think this is the right approach to keep track on that task. We have already the same etherpad, and we are working on it. Please check the following: https://etherpad.openstack.org/p/TempestTestDevelopment Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Tempest whitebox tests in nova
Hello! I'm looking at where would be the most appropriate place to have the tempest whitebox tests in nova unit tests. At first look, the nova/tests/db/test_db_api.py seems to be an appropriate place. As previously in tempest, I can work directly with the db and change states accordingly. However, the logic to allow certain actions depending on instance states seems not to be covered at this level. For example, one of the logic tested is try to delete an instance in vm_state = 'resized' and task_state='resize_prep' . This should raise an Exception, but that does not happen considering only the db level. It would require to import manager methods in this case. On the other hand, having the whitebox tests on the manager test level, we have most of db methods stubbed or use of fakes, so it wouldn't really be doing what are expected in terms of whitebox. I think one option is to import the manager in the db level to apply the needed logic, but I'm looking for more advice from the nova team and to understand if my assumptions are correct so far. More information about the whitebox tests for servers in tempest (from the patch that removes those tests): https://review.openstack.org/#/c/46116/3/tempest/whitebox/test_servers_whitebox.py The nova db tests: https://github.com/openstack/nova/blob/master/nova/tests/db/test_db_api.py Regards, -- Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Grenade issues
Hi! I would suggest to add somewhere on code best practices that changes on requirements (pip) and configuration files (conf, paste-ini) might cause upgrade issues when running grenade. A few fixes that I needed to apply on grenade were usually related to those. Regards, Adalberto Medeiros Linux Technology Center Openstack and Cloud Development IBM Brazil Email: adal...@linux.vnet.ibm.com On 29-07-2013 08:02, Davanum Srinivas wrote: I see lots of green. Looks like reverting the jsonschema did the trick. https://jenkins.openstack.org/job/gate-grenade-devstack-vm/ -- dims On Sun, Jul 28, 2013 at 9:47 PM, Sean Dague s...@dague.net wrote: On 07/28/2013 09:42 PM, Sean Dague wrote: On 07/28/2013 09:28 PM, Davanum Srinivas wrote: Monty, I picked up a latest run https://jenkins.openstack.org/job/gate-grenade-devstack-vm/22236/ which lead me to http://logs.openstack.org/51/38951/2/check/gate-grenade-devstack-vm/22236/logs/new/screen-c-vol.txt.gz So could the problem be 38810 itself? 2013-07-29 00:46:37.562 27821 TRACE cinder.service Stderr: 'Traceback (most recent call last):\n File /usr/local/bin/cinder-rootwrap, line 4, in module\nfrom pkg_resources import require; require(\'cinder==2013.2.a78.g465eb62\')\n File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 2707, in module\nworking_set.require(__requires__)\n File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 686, in require\nneeded = self.resolve(parse_requirements(requirements))\n File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 584, in resolve\nraise DistributionNotFound(req)\npkg_resources.DistributionNotFound: jsonschema=0.7,3\n' Well, it's a think regardless. Tempest is definitely downgrading jsonschema when it hits it's requirements phase: Gr... s/think/thing/... reasons why I shouldn't do this on a Sunday night... :) -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
