Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 14:14 Thu 02 Apr , Marcus Vinícius Ramires do Nascimento wrote: > Hi Mike, > > I'm working on test coverage improvement for HDS/Hitachi drivers. As I > talked to you in #openstack-cinder channel, I'm facing troubles with 3 > tests (apparently those fails are not related to the driver) and I'm trying > to discover if it's a bug to report it or if it's a > infrastructure/configuration problem. > > I'll switch back the CI to check mode and I'll continue working with this > failures isolated to investigate the problem. Is it OK? The CI is running > 300 tests now (http://177.84.241.119:1/27/164527/3/silent/). Yes. -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Hi Mike, I'm working on test coverage improvement for HDS/Hitachi drivers. As I talked to you in #openstack-cinder channel, I'm facing troubles with 3 tests (apparently those fails are not related to the driver) and I'm trying to discover if it's a bug to report it or if it's a infrastructure/configuration problem. I'll switch back the CI to check mode and I'll continue working with this failures isolated to investigate the problem. Is it OK? The CI is running 300 tests now (http://177.84.241.119:1/27/164527/3/silent/). Thanks, On Thu, Apr 2, 2015 at 12:10 PM, Alex Meade wrote: > We believe we have satisfied the required criteria [1] to have NetApp’s > fibre channel drivers included in the Kilo release. We have submitted a > revert patch [2] along with posting an ether pad [3] to provide more detail > on our progress. Thanks for your consideration. > > [1] > http://lists.openstack.org/pipermail/openstack-dev/2015-March/059990.html > > [2] https://review.openstack.org/#/c/169781/ > > [3] https://etherpad.openstack.org/p/NetApp-Kilo-Fibre-Channel > Thanks so much, > > -Alex > > On Thu, Mar 26, 2015 at 8:48 PM, Ryan Hsu wrote: > >> Thanks for clarifying! >> >> Ryan >> >> On Mar 26, 2015, at 5:29 PM, Mike Perez wrote: >> >> > On 00:24 Fri 27 Mar , Ryan Hsu wrote: >> >> Rightfully so, but it doesn't hurt to offer suggestions that might >> improve >> >> the community. It would just be nice to have exclusions reconsidered >> if there >> >> are legitimate bugs behind them. You see them all the time in the >> tempest >> >> tests ala "SKIPPED: Skipped until Bug: 1373513 is resolved" so it's >> hard to >> >> understand why we can't just apply the same principles to third-party >> CI. >> > >> > Your usage of exclusions is fine for fixing bugs in my opinion. My >> meaning of >> > exclusion was not allowing these additional tests to be discovered. >> > >> > -- >> > Mike Perez >> > >> > >> __ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- *Marcus Vinícius Ramires do Nascimento* marcus...@gmail.com Cel: (11) 97396-4018 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
We believe we have satisfied the required criteria [1] to have NetApp’s fibre channel drivers included in the Kilo release. We have submitted a revert patch [2] along with posting an ether pad [3] to provide more detail on our progress. Thanks for your consideration. [1] http://lists.openstack.org/pipermail/openstack-dev/2015-March/059990.html [2] https://review.openstack.org/#/c/169781/ [3] https://etherpad.openstack.org/p/NetApp-Kilo-Fibre-Channel Thanks so much, -Alex On Thu, Mar 26, 2015 at 8:48 PM, Ryan Hsu wrote: > Thanks for clarifying! > > Ryan > > On Mar 26, 2015, at 5:29 PM, Mike Perez wrote: > > > On 00:24 Fri 27 Mar , Ryan Hsu wrote: > >> Rightfully so, but it doesn't hurt to offer suggestions that might > improve > >> the community. It would just be nice to have exclusions reconsidered if > there > >> are legitimate bugs behind them. You see them all the time in the > tempest > >> tests ala "SKIPPED: Skipped until Bug: 1373513 is resolved" so it's > hard to > >> understand why we can't just apply the same principles to third-party > CI. > > > > Your usage of exclusions is fine for fixing bugs in my opinion. My > meaning of > > exclusion was not allowing these additional tests to be discovered. > > > > -- > > Mike Perez > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Thanks for clarifying! Ryan On Mar 26, 2015, at 5:29 PM, Mike Perez wrote: > On 00:24 Fri 27 Mar , Ryan Hsu wrote: >> Rightfully so, but it doesn't hurt to offer suggestions that might improve >> the community. It would just be nice to have exclusions reconsidered if there >> are legitimate bugs behind them. You see them all the time in the tempest >> tests ala "SKIPPED: Skipped until Bug: 1373513 is resolved" so it's hard to >> understand why we can't just apply the same principles to third-party CI. > > Your usage of exclusions is fine for fixing bugs in my opinion. My meaning of > exclusion was not allowing these additional tests to be discovered. > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Mike, I am communicating this problem with my teams and will get it resolved asap. Jay On Mar 26, 2015 1:23 PM, "Mike Perez" wrote: > On 09:39 Thu 26 Mar , Mike Perez wrote: > > As discussed in the last Cinder meeting [1], in order to have your volume > > driver readded into the Kilo release, you must have a CI reporting and > stable > > for five days prior to 4/6. > > > > This includes: > > > > 1) Providing logs to screen sessions, etc configs, tempest output [2]. > > 2) You should be running around 304 tests if you're following > instructions from > >the Cinder third party wiki [3]. If you're running less than that, > your CI > >will be *NOT* be considered satisfactory for skipping tests. > > > > I will also be emailing individuals who have already asked for > exceptions, just > > to make sure communication was clear. > > > > > > [1] - > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25-16.00.log.html#l-173 > > [2] - http://ci.openstack.org/third_party.html#requirements > > [3] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F > > The current CI's not meeting the second requirement: > > * Cloudbyte > * Dell EQL > * Dell SC FC > * Dell SC ISCSI > * EMC VMAX FC > * EMC VMX ISCSI > * EMC VNX FC > * EMC VNX ISCSI > * EMC XIO FC > * EMC XIO ISCSI > * HDS NFS > * HDS NAS > * Hitachi HBSD FC > * Hitach HBSD ISCSI > * IBM Flash Systems FC > * IBM Flash Systems ISCSI > * IBM NAS > * IBM XIV (couldn't find tempest results to verify) > * IBM Storwize FC > * IBM Storwize ISCSI > * Nimble > * OpenVStorage > * Quobyte > * XIO FC > * XIO ISCSI > * Vmware > > Pretty sure this is because the previous instructions in the wiki were > incorrect and are now corrected [1]. This is not the fault of the CI > maintainers. As mentioned, individual emails are being sent out to get > this all > sorted. > > > [1] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 00:24 Fri 27 Mar , Ryan Hsu wrote: > Rightfully so, but it doesn't hurt to offer suggestions that might improve > the community. It would just be nice to have exclusions reconsidered if there > are legitimate bugs behind them. You see them all the time in the tempest > tests ala "SKIPPED: Skipped until Bug: 1373513 is resolved" so it's hard to > understand why we can't just apply the same principles to third-party CI. Your usage of exclusions is fine for fixing bugs in my opinion. My meaning of exclusion was not allowing these additional tests to be discovered. -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Rightfully so, but it doesn't hurt to offer suggestions that might improve the community. It would just be nice to have exclusions reconsidered if there are legitimate bugs behind them. You see them all the time in the tempest tests ala "SKIPPED: Skipped until Bug: 1373513 is resolved" so it's hard to understand why we can't just apply the same principles to third-party CI. Thanks, Ryan On Mar 26, 2015, at 4:42 PM, Anita Kuno wrote: > On 03/26/2015 06:48 PM, Ryan Hsu wrote: >> Exclusions are legitimate and will always be necessary at some point. In the >> case of the linked bug, this was once a known issue for the VMware driver >> and we had excluded affected tests so that CI could continue to run. This is >> the same way we do it in Nova CI and oslo.vmware CI. > > This is Cinder, Ryan, and Mike is the PTL. It is his decision. > > Thank you, > Anita. > >> Thanks, >> Ryan >> >> On Mar 26, 2015, at 3:30 PM, Mike Perez wrote: >> >>> On 20:49 Thu 26 Mar , Ryan Hsu wrote: Hi Mike, We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are not being run due to skips and disabled features. For example, here is a sampling of tests that are skipped in a recent run (note that this is using tempest.conf with no explicit disabling of Cinder services): tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume ... SKIPPED: Block Live migration not available setUpClass (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... SKIPPED: Heat support is required setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume ... SKIPPED: SSH required for this test setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... SKIPPED: Skipped until Bug: 1205344 is resolved. setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) ... SKIPPED: The EC2 API is not available setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... SKIPPED: The EC2 API is not available tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. As we are actually running the volume suite according to the FAQ and the above skipped tests are documented by our CI, would it be possible to add an exception to the rule? I'm sure these numbers will be different for all CIs and as long as people are not abusing and hiding skipped tests, I don't see this as a problem. >>> >>> There will be no exceptions. Everyone must pass the same tests or you're >>> not an >>> approved volume driver for OpenStack Cinder. >>> >>> You should also take this bug [1] Vmware hit as a lesson of doing any >>> excluding in your CI. The driver would've been seriously broken in Kilo if >>> this >>> wasn't caught. >>> >>> [1] - https://bugs.launchpad.net/cinder/+bug/1436603 >>> >>> -- >>> Mike Perez >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.ope
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 00:11 Fri 27 Mar , Ryan Hsu wrote: > Thanks Mike. The wiki still showing the old edit when I had sent out my > earlier email but I see it's been updated now. I've tested running with "tox > -e all -- volume" and that gets us to 291 tests now. Ah gotcha. That's great to hear, thank you! -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Thanks Mike. The wiki still showing the old edit when I had sent out my earlier email but I see it's been updated now. I've tested running with "tox -e all -- volume" and that gets us to 291 tests now. -Ryan On Mar 26, 2015, at 4:28 PM, Mike Perez wrote: > On 21:45 Thu 26 Mar , Ryan Hsu wrote: >> Hmm, that's what I thought at first but when I looked at the "What tests do >> I use" FAQ, the tests that it says to use links to: >> http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume, >> which is exactly what we're running. Even so, I ran the "tox -e all -- >> volume" command and that just runs 4 more tests. > > As the email that went out, it was mentioned the wiki has been corrected. It > should not just be api.volume. > > There have been a number of drivers today in the #openstack-cinder that have > reported to me that they made the adjustments from the wiki and are now > reporting 294 tests. If this is not making a difference for you, I recommend > you reach out to the liasons mentioned in the wiki and get help: > > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#Questions > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 03/26/2015 06:48 PM, Ryan Hsu wrote: > Exclusions are legitimate and will always be necessary at some point. In the > case of the linked bug, this was once a known issue for the VMware driver and > we had excluded affected tests so that CI could continue to run. This is the > same way we do it in Nova CI and oslo.vmware CI. This is Cinder, Ryan, and Mike is the PTL. It is his decision. Thank you, Anita. > Thanks, > Ryan > > On Mar 26, 2015, at 3:30 PM, Mike Perez wrote: > >> On 20:49 Thu 26 Mar , Ryan Hsu wrote: >>> Hi Mike, >>> >>> We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this >>> runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests >>> are >>> not being run due to skips and disabled features. For example, here is >>> a sampling of tests that are skipped in a recent run (note that this is >>> using >>> tempest.conf with no explicit disabling of Cinder services): >>> >>> tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume >>> ... SKIPPED: Block Live migration not available >>> setUpClass >>> (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... >>> SKIPPED: Heat support is required >>> setUpClass >>> (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... >>> SKIPPED: Cinder multi-backend feature disabled >>> tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume >>> ... SKIPPED: SSH required for this test >>> setUpClass >>> (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... >>> SKIPPED: Cinder backup feature disabled >>> setUpClass >>> (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... >>> SKIPPED: Cinder backup feature disabled >>> setUpClass >>> (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... >>> SKIPPED: Cinder multi-backend feature disabled >>> tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern >>> ... SKIPPED: Skipped until Bug: 1373513 is resolved. >>> tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... >>> SKIPPED: Skipped until Bug: 1205344 is resolved. >>> setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) >>> ... SKIPPED: The EC2 API is not available >>> setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... >>> SKIPPED: The EC2 API is not available >>> tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern >>> ... SKIPPED: Skipped until Bug: 1373513 is resolved. >>> >>> As we are actually running the volume suite according to the FAQ and the >>> above skipped tests are documented by our CI, would it be possible to add an >>> exception to the rule? I'm sure these numbers will be different for all CIs >>> and as long as people are not abusing and hiding skipped tests, I don't see >>> this as a problem. >> >> There will be no exceptions. Everyone must pass the same tests or you're not >> an >> approved volume driver for OpenStack Cinder. >> >> You should also take this bug [1] Vmware hit as a lesson of doing any >> excluding in your CI. The driver would've been seriously broken in Kilo if >> this >> wasn't caught. >> >> [1] - https://bugs.launchpad.net/cinder/+bug/1436603 >> >> -- >> Mike Perez >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 21:45 Thu 26 Mar , Ryan Hsu wrote: > Hmm, that's what I thought at first but when I looked at the "What tests do > I use" FAQ, the tests that it says to use links to: > http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume, > which is exactly what we're running. Even so, I ran the "tox -e all -- > volume" command and that just runs 4 more tests. As the email that went out, it was mentioned the wiki has been corrected. It should not just be api.volume. There have been a number of drivers today in the #openstack-cinder that have reported to me that they made the adjustments from the wiki and are now reporting 294 tests. If this is not making a difference for you, I recommend you reach out to the liasons mentioned in the wiki and get help: https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#Questions -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 22:42 Thu 26 Mar , Ryan Hsu wrote: > Like I mentioned earlier, these numbers are going to be different for > everyone depending on their testbed set up or driver capabilities. Just by > disabling Heat in devstack you're going to miss some tests. As long as people > are transparent about this, I don't see the harm here. This has nothing to do with what services you're running (e.g. heat, sahara). If you're just running volume.api tests, you're verifying a limited number of tests due to your discovery settings. For your convenience, here's a diff between testing just volume.api versus tox -e all -- volume: http://paste.openstack.org/raw/196962/ As you'll notice there is a lot of snapshot/image tests missed that should be covered by all CI's. However Vmware is kicking off the tests, this can be corrected without enabling any additional OpenStack services by: tox -e all -- volume or if you're using devstack-gate, export this before running the tests: export DEVSTACK_GATE_TEMPEST_REGEX="volume" All of this is explained in the Cinder wiki: https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Exclusions are legitimate and will always be necessary at some point. In the case of the linked bug, this was once a known issue for the VMware driver and we had excluded affected tests so that CI could continue to run. This is the same way we do it in Nova CI and oslo.vmware CI. Thanks, Ryan On Mar 26, 2015, at 3:30 PM, Mike Perez wrote: > On 20:49 Thu 26 Mar , Ryan Hsu wrote: >> Hi Mike, >> >> We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this >> runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are >> not being run due to skips and disabled features. For example, here is >> a sampling of tests that are skipped in a recent run (note that this is using >> tempest.conf with no explicit disabling of Cinder services): >> >> tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume >> ... SKIPPED: Block Live migration not available >> setUpClass >> (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... >> SKIPPED: Heat support is required >> setUpClass >> (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... >> SKIPPED: Cinder multi-backend feature disabled >> tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume >> ... SKIPPED: SSH required for this test >> setUpClass >> (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... >> SKIPPED: Cinder backup feature disabled >> setUpClass >> (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... >> SKIPPED: Cinder backup feature disabled >> setUpClass >> (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... >> SKIPPED: Cinder multi-backend feature disabled >> tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern >> ... SKIPPED: Skipped until Bug: 1373513 is resolved. >> tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... >> SKIPPED: Skipped until Bug: 1205344 is resolved. >> setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) >> ... SKIPPED: The EC2 API is not available >> setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... >> SKIPPED: The EC2 API is not available >> tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern >> ... SKIPPED: Skipped until Bug: 1373513 is resolved. >> >> As we are actually running the volume suite according to the FAQ and the >> above skipped tests are documented by our CI, would it be possible to add an >> exception to the rule? I'm sure these numbers will be different for all CIs >> and as long as people are not abusing and hiding skipped tests, I don't see >> this as a problem. > > There will be no exceptions. Everyone must pass the same tests or you're not > an > approved volume driver for OpenStack Cinder. > > You should also take this bug [1] Vmware hit as a lesson of doing any > excluding in your CI. The driver would've been seriously broken in Kilo if > this > wasn't caught. > > [1] - https://bugs.launchpad.net/cinder/+bug/1436603 > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Like I mentioned earlier, these numbers are going to be different for everyone depending on their testbed set up or driver capabilities. Just by disabling Heat in devstack you're going to miss some tests. As long as people are transparent about this, I don't see the harm here. -Ryan On Mar 26, 2015, at 3:31 PM, Mike Perez wrote: > On 22:28 Thu 26 Mar , Tom Swanson wrote: >> I just ran it and got ... >> >> Ran: 304 tests in 412. sec. >> - Passed: 293 >> - Skipped: 11 > > That looks good to me then and might've been a mistake in looking at > everyone's > logs. Thanks Tom. > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 22:28 Thu 26 Mar , Tom Swanson wrote: > I just ran it and got ... > > Ran: 304 tests in 412. sec. > - Passed: 293 > - Skipped: 11 That looks good to me then and might've been a mistake in looking at everyone's logs. Thanks Tom. -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 20:49 Thu 26 Mar , Ryan Hsu wrote: > Hi Mike, > > We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this > runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are > not being run due to skips and disabled features. For example, here is > a sampling of tests that are skipped in a recent run (note that this is using > tempest.conf with no explicit disabling of Cinder services): > > tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume > ... SKIPPED: Block Live migration not available > setUpClass > (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... > SKIPPED: Heat support is required > setUpClass > (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... > SKIPPED: Cinder multi-backend feature disabled > tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume > ... SKIPPED: SSH required for this test > setUpClass > (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... > SKIPPED: Cinder backup feature disabled > setUpClass > (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... > SKIPPED: Cinder backup feature disabled > setUpClass > (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... > SKIPPED: Cinder multi-backend feature disabled > tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern > ... SKIPPED: Skipped until Bug: 1373513 is resolved. > tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... > SKIPPED: Skipped until Bug: 1205344 is resolved. > setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) > ... SKIPPED: The EC2 API is not available > setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... > SKIPPED: The EC2 API is not available > tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern > ... SKIPPED: Skipped until Bug: 1373513 is resolved. > > As we are actually running the volume suite according to the FAQ and the > above skipped tests are documented by our CI, would it be possible to add an > exception to the rule? I'm sure these numbers will be different for all CIs > and as long as people are not abusing and hiding skipped tests, I don't see > this as a problem. There will be no exceptions. Everyone must pass the same tests or you're not an approved volume driver for OpenStack Cinder. You should also take this bug [1] Vmware hit as a lesson of doing any excluding in your CI. The driver would've been seriously broken in Kilo if this wasn't caught. [1] - https://bugs.launchpad.net/cinder/+bug/1436603 -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
I just ran it and got ... Ran: 304 tests in 412. sec. - Passed: 293 - Skipped: 11 From: Ryan Hsu [mailto:r...@vmware.com] Sent: Thursday, March 26, 2015 4:45 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded Hmm, that's what I thought at first but when I looked at the "What tests do I use" FAQ, the tests that it says to use links to: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume, which is exactly what we're running. Even so, I ran the "tox -e all -- volume" command and that just runs 4 more tests. -Ryan [1] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F On Mar 26, 2015, at 1:56 PM, Tom Swanson mailto:tom_swan...@dell.com>> wrote: You want to run the "volume" tests and not "tempest.api.volume" tests. -Original Message- From: Ryan Hsu [mailto:r...@vmware.com] Sent: Thursday, March 26, 2015 3:50 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded Hi Mike, We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are not being run due to skips and disabled features. For example, here is a sampling of tests that are skipped in a recent run (note that this is using tempest.conf with no explicit disabling of Cinder services): tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume ... SKIPPED: Block Live migration not available setUpClass (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... SKIPPED: Heat support is required setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume ... SKIPPED: SSH required for this test setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... SKIPPED: Skipped until Bug: 1205344 is resolved. setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) ... SKIPPED: The EC2 API is not available setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... SKIPPED: The EC2 API is not available tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. As we are actually running the volume suite according to the FAQ and the above skipped tests are documented by our CI, would it be possible to add an exception to the rule? I'm sure these numbers will be different for all CIs and as long as people are not abusing and hiding skipped tests, I don't see this as a problem. Thanks, Ryan On Mar 26, 2015, at 9:39 AM, Mike Perez mailto:thin...@gmail.com>> wrote: As discussed in the last Cinder meeting [1], in order to have your volume driver readded into the Kilo release, you must have a CI reporting and stable for five days prior to 4/6. This includes: 1) Providing logs to screen sessions, etc configs, tempest output [2]. 2) You should be running around 304 tests if you're following instructions from the Cinder third party wiki [3]. If you're running less than that, your CI will be *NOT* be considered satisfactory for skipping tests. I will also be emailing individuals who have already asked for exceptions, just to make sure communication was clear. [1] - http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25- 16.00.log.html#l-173 [2] - http://ci.openstack.org/third_party.html#requirements [3] - https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_te sts_do_I_use.3F -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Hmm, that's what I thought at first but when I looked at the "What tests do I use" FAQ, the tests that it says to use links to: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume, which is exactly what we're running. Even so, I ran the "tox -e all -- volume" command and that just runs 4 more tests. -Ryan [1] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F On Mar 26, 2015, at 1:56 PM, Tom Swanson mailto:tom_swan...@dell.com>> wrote: You want to run the "volume" tests and not "tempest.api.volume" tests. -Original Message- From: Ryan Hsu [mailto:r...@vmware.com] Sent: Thursday, March 26, 2015 3:50 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded Hi Mike, We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are not being run due to skips and disabled features. For example, here is a sampling of tests that are skipped in a recent run (note that this is using tempest.conf with no explicit disabling of Cinder services): tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume ... SKIPPED: Block Live migration not available setUpClass (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... SKIPPED: Heat support is required setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume ... SKIPPED: SSH required for this test setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... SKIPPED: Skipped until Bug: 1205344 is resolved. setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) ... SKIPPED: The EC2 API is not available setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... SKIPPED: The EC2 API is not available tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. As we are actually running the volume suite according to the FAQ and the above skipped tests are documented by our CI, would it be possible to add an exception to the rule? I'm sure these numbers will be different for all CIs and as long as people are not abusing and hiding skipped tests, I don't see this as a problem. Thanks, Ryan On Mar 26, 2015, at 9:39 AM, Mike Perez mailto:thin...@gmail.com>> wrote: As discussed in the last Cinder meeting [1], in order to have your volume driver readded into the Kilo release, you must have a CI reporting and stable for five days prior to 4/6. This includes: 1) Providing logs to screen sessions, etc configs, tempest output [2]. 2) You should be running around 304 tests if you're following instructions from the Cinder third party wiki [3]. If you're running less than that, your CI will be *NOT* be considered satisfactory for skipping tests. I will also be emailing individuals who have already asked for exceptions, just to make sure communication was clear. [1] - http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25- 16.00.log.html#l-173 [2] - http://ci.openstack.org/third_party.html#requirements [3] - https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_te sts_do_I_use.3F -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@li
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
You want to run the "volume" tests and not "tempest.api.volume" tests. -Original Message- From: Ryan Hsu [mailto:r...@vmware.com] Sent: Thursday, March 26, 2015 3:50 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded Hi Mike, We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are not being run due to skips and disabled features. For example, here is a sampling of tests that are skipped in a recent run (note that this is using tempest.conf with no explicit disabling of Cinder services): tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume ... SKIPPED: Block Live migration not available setUpClass (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... SKIPPED: Heat support is required setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume ... SKIPPED: SSH required for this test setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern ... SKI PPED: Skipped until Bug: 1373513 is resolved. tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... SKIPPED: Skipped until Bug: 1205344 is resolved. setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) ... SKIPPED: The EC2 API is not available setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... SKIPPED: The EC2 API is not available tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. As we are actually running the volume suite according to the FAQ and the above skipped tests are documented by our CI, would it be possible to add an exception to the rule? I'm sure these numbers will be different for all CIs and as long as people are not abusing and hiding skipped tests, I don't see this as a problem. Thanks, Ryan On Mar 26, 2015, at 9:39 AM, Mike Perez wrote: > As discussed in the last Cinder meeting [1], in order to have your > volume driver readded into the Kilo release, you must have a CI > reporting and stable for five days prior to 4/6. > > This includes: > > 1) Providing logs to screen sessions, etc configs, tempest output [2]. > 2) You should be running around 304 tests if you're following instructions > from > the Cinder third party wiki [3]. If you're running less than that, your CI > will be *NOT* be considered satisfactory for skipping tests. > > I will also be emailing individuals who have already asked for > exceptions, just to make sure communication was clear. > > > [1] - > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25- > 16.00.log.html#l-173 [2] - > http://ci.openstack.org/third_party.html#requirements > [3] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_te > sts_do_I_use.3F > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Hi Mike, We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are not being run due to skips and disabled features. For example, here is a sampling of tests that are skipped in a recent run (note that this is using tempest.conf with no explicit disabling of Cinder services): tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume ... SKIPPED: Block Live migration not available setUpClass (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... SKIPPED: Heat support is required setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume ... SKIPPED: SSH required for this test setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... SKIPPED: Cinder backup feature disabled setUpClass (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... SKIPPED: Cinder multi-backend feature disabled tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... SKIPPED: Skipped until Bug: 1205344 is resolved. setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) ... SKIPPED: The EC2 API is not available setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... SKIPPED: The EC2 API is not available tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern ... SKIPPED: Skipped until Bug: 1373513 is resolved. As we are actually running the volume suite according to the FAQ and the above skipped tests are documented by our CI, would it be possible to add an exception to the rule? I'm sure these numbers will be different for all CIs and as long as people are not abusing and hiding skipped tests, I don't see this as a problem. Thanks, Ryan On Mar 26, 2015, at 9:39 AM, Mike Perez wrote: > As discussed in the last Cinder meeting [1], in order to have your volume > driver readded into the Kilo release, you must have a CI reporting and stable > for five days prior to 4/6. > > This includes: > > 1) Providing logs to screen sessions, etc configs, tempest output [2]. > 2) You should be running around 304 tests if you're following instructions > from > the Cinder third party wiki [3]. If you're running less than that, your CI > will be *NOT* be considered satisfactory for skipping tests. > > I will also be emailing individuals who have already asked for exceptions, > just > to make sure communication was clear. > > > [1] - > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25-16.00.log.html#l-173 > [2] - http://ci.openstack.org/third_party.html#requirements > [3] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
I'm not sure if I was clear... The driver is OK and snapshots are working as expected. I'm working on get all CI tests back (304 tests). On Thu, Mar 26, 2015 at 3:10 PM, Marcus Vinícius Ramires do Nascimento < marcus...@gmail.com> wrote: > Hi Mike, > > I'm working on it! The bug was fixed and now I'm working to get all > tempest.api.volume tests back again on our CI, including also the tests > that were missing. > > On Thu, Mar 26, 2015 at 3:00 PM, Mike Perez wrote: > >> On 14:38 Thu 26 Mar , Erlon Cruz wrote: >> >> >> > Our HBSD drivers are only running 211 because we remove the snapshots >> tests >> > that were failing due a patch that broken our driver. >> >> Whats being done about that in Kilo? That's a minimum feature required >> for all >> drivers in Cinder: >> >> >> http://docs.openstack.org/developer/cinder/devref/drivers.html#minimum-features >> >> -- >> Mike Perez >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > *Marcus Vinícius Ramires do Nascimento* > marcus...@gmail.com > > Cel: (11) 97396-4018 > -- *Marcus Vinícius Ramires do Nascimento* marcus...@gmail.com Cel: (11) 97396-4018 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Hi Mike, I'm working on it! The bug was fixed and now I'm working to get all tempest.api.volume tests back again on our CI, including also the tests that were missing. On Thu, Mar 26, 2015 at 3:00 PM, Mike Perez wrote: > On 14:38 Thu 26 Mar , Erlon Cruz wrote: > > > > Our HBSD drivers are only running 211 because we remove the snapshots > tests > > that were failing due a patch that broken our driver. > > Whats being done about that in Kilo? That's a minimum feature required for > all > drivers in Cinder: > > > http://docs.openstack.org/developer/cinder/devref/drivers.html#minimum-features > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- *Marcus Vinícius Ramires do Nascimento* marcus...@gmail.com Cel: (11) 97396-4018 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 14:38 Thu 26 Mar , Erlon Cruz wrote: > Our HBSD drivers are only running 211 because we remove the snapshots tests > that were failing due a patch that broken our driver. Whats being done about that in Kilo? That's a minimum feature required for all drivers in Cinder: http://docs.openstack.org/developer/cinder/devref/drivers.html#minimum-features -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
Hi Mike, The majority of the CIs don't run all 304 tests mostly because of these tempest problems. I remember there was a list in the Thirdparty CI Wiki with the common tests that used to fails to everyone, or at least most of people. IMO its better to have a more consistent CI than CIs with full coverage giving lots of false negatives. Don't how much of the tests from that list are fixed on tempest but we should bring that list up again and reconsider to have the list of 'known to fail' tests. Our HBSD drivers are only running 211 because we remove the snapshots tests that were failing due a patch that broken our driver. > On Thu, Mar 26, 2015 at 2:21 PM, Mike Perez wrote: > On 09:39 Thu 26 Mar , Mike Perez wrote: > > As discussed in the last Cinder meeting [1], in order to have your volume > > driver readded into the Kilo release, you must have a CI reporting and > stable > > for five days prior to 4/6. > > > > This includes: > > > > 1) Providing logs to screen sessions, etc configs, tempest output [2]. > > 2) You should be running around 304 tests if you're following > instructions from > >the Cinder third party wiki [3]. If you're running less than that, > your CI > >will be *NOT* be considered satisfactory for skipping tests. > > > > I will also be emailing individuals who have already asked for > exceptions, just > > to make sure communication was clear. > > > > > > [1] - > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25-16.00.log.html#l-173 > > [2] - http://ci.openstack.org/third_party.html#requirements > > [3] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F > > The current CI's not meeting the second requirement: > > * Cloudbyte > * Dell EQL > * Dell SC FC > * Dell SC ISCSI > * EMC VMAX FC > * EMC VMX ISCSI > * EMC VNX FC > * EMC VNX ISCSI > * EMC XIO FC > * EMC XIO ISCSI > * HDS NFS > * HDS NAS > * Hitachi HBSD FC > * Hitach HBSD ISCSI > * IBM Flash Systems FC > * IBM Flash Systems ISCSI > * IBM NAS > * IBM XIV (couldn't find tempest results to verify) > * IBM Storwize FC > * IBM Storwize ISCSI > * Nimble > * OpenVStorage > * Quobyte > * XIO FC > * XIO ISCSI > * Vmware > > Pretty sure this is because the previous instructions in the wiki were > incorrect and are now corrected [1]. This is not the fault of the CI > maintainers. As mentioned, individual emails are being sent out to get > this all > sorted. > > > [1] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F > > -- > Mike Perez > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Deadline For Volume Drivers to Be Readded
On 09:39 Thu 26 Mar , Mike Perez wrote: > As discussed in the last Cinder meeting [1], in order to have your volume > driver readded into the Kilo release, you must have a CI reporting and stable > for five days prior to 4/6. > > This includes: > > 1) Providing logs to screen sessions, etc configs, tempest output [2]. > 2) You should be running around 304 tests if you're following instructions > from >the Cinder third party wiki [3]. If you're running less than that, your CI >will be *NOT* be considered satisfactory for skipping tests. > > I will also be emailing individuals who have already asked for exceptions, > just > to make sure communication was clear. > > > [1] - > http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-03-25-16.00.log.html#l-173 > [2] - http://ci.openstack.org/third_party.html#requirements > [3] - > https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F The current CI's not meeting the second requirement: * Cloudbyte * Dell EQL * Dell SC FC * Dell SC ISCSI * EMC VMAX FC * EMC VMX ISCSI * EMC VNX FC * EMC VNX ISCSI * EMC XIO FC * EMC XIO ISCSI * HDS NFS * HDS NAS * Hitachi HBSD FC * Hitach HBSD ISCSI * IBM Flash Systems FC * IBM Flash Systems ISCSI * IBM NAS * IBM XIV (couldn't find tempest results to verify) * IBM Storwize FC * IBM Storwize ISCSI * Nimble * OpenVStorage * Quobyte * XIO FC * XIO ISCSI * Vmware Pretty sure this is because the previous instructions in the wiki were incorrect and are now corrected [1]. This is not the fault of the CI maintainers. As mentioned, individual emails are being sent out to get this all sorted. [1] - https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_tests_do_I_use.3F -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
