Re: [openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
Thx :) > Wiadomość napisana przez Matt Riedemann w dniu > 26.07.2018, o godz. 18:32: > > On 7/23/2018 4:20 AM, Slawomir Kaplonski wrote: >> Thx Artom for taking care of it. Did You made any progress? >> I think that it might be quite important to fix as it failed around 50 times >> during last 7 days: >> http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20386%2C%20in%20test_tagged_attachment%5C%22 > > I've proposed a Tempest change to skip that part of the test for now: > > https://review.openstack.org/#/c/586292/ > > We could revert that and link it to artom's debug patch to see if we can > recreate with proper debug. > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev — Slawek Kaplonski Senior software engineer Red Hat __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
On 7/23/2018 4:20 AM, Slawomir Kaplonski wrote: Thx Artom for taking care of it. Did You made any progress? I think that it might be quite important to fix as it failed around 50 times during last 7 days: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20386%2C%20in%20test_tagged_attachment%5C%22 I've proposed a Tempest change to skip that part of the test for now: https://review.openstack.org/#/c/586292/ We could revert that and link it to artom's debug patch to see if we can recreate with proper debug. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
Hi, Thx Artom for taking care of it. Did You made any progress? I think that it might be quite important to fix as it failed around 50 times during last 7 days: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20386%2C%20in%20test_tagged_attachment%5C%22 > Wiadomość napisana przez Artom Lifshitz w dniu > 19.07.2018, o godz. 19:28: > > I've proposed [1] to add extra logging on the Nova side. Let's see if > that helps us catch the root cause of this. > > [1] https://review.openstack.org/584032 > > On Thu, Jul 19, 2018 at 12:50 PM, Artom Lifshitz wrote: >> Because we're waiting for the volume to become available before we >> continue with the test [1], its tag still being present means Nova's >> not cleaning up the device tags on volume detach. This is most likely >> a bug. I'll look into it. >> >> [1] >> https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L378 >> >> On Thu, Jul 19, 2018 at 7:09 AM, Slawomir Kaplonski >> wrote: >>> Hi, >>> >>> Since some time we see that test >>> tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment >>> is failing sometimes. >>> Bug about that is reported for Tempest currently [1] but after small patch >>> [2] was merged I was today able to check what cause this issue. >>> >>> Test which is failing is in [3] and it looks that everything is going fine >>> with it up to last line of test. So volume and port are created, attached, >>> tags are set properly, both devices are detached properly also and at the >>> end test is failing as in >>> http://169.254.169.254/openstack/latest/meta_data.json still has some >>> device inside. >>> And it looks now from [4] that it is volume which isn’t removed from this >>> meta_data.json. >>> So I think that it would be good if people from Nova and Cinder teams could >>> look at it and try to figure out what is going on there and how it can be >>> fixed. >>> >>> Thanks in advance for help. >>> >>> [1] https://bugs.launchpad.net/tempest/+bug/1775947 >>> [2] https://review.openstack.org/#/c/578765/ >>> [3] >>> https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L330 >>> [4] >>> http://logs.openstack.org/69/567369/15/check/tempest-full/528bc75/job-output.txt.gz#_2018-07-19_10_06_09_273919 >>> >>> — >>> Slawek Kaplonski >>> Senior software engineer >>> Red Hat >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> -- >> Artom Lifshitz >> Software Engineer, OpenStack Compute DFG > > > > -- > -- > Artom Lifshitz > Software Engineer, OpenStack Compute DFG > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev — Slawek Kaplonski Senior software engineer Red Hat __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
I've proposed [1] to add extra logging on the Nova side. Let's see if that helps us catch the root cause of this. [1] https://review.openstack.org/584032 On Thu, Jul 19, 2018 at 12:50 PM, Artom Lifshitz wrote: > Because we're waiting for the volume to become available before we > continue with the test [1], its tag still being present means Nova's > not cleaning up the device tags on volume detach. This is most likely > a bug. I'll look into it. > > [1] > https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L378 > > On Thu, Jul 19, 2018 at 7:09 AM, Slawomir Kaplonski > wrote: >> Hi, >> >> Since some time we see that test >> tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment >> is failing sometimes. >> Bug about that is reported for Tempest currently [1] but after small patch >> [2] was merged I was today able to check what cause this issue. >> >> Test which is failing is in [3] and it looks that everything is going fine >> with it up to last line of test. So volume and port are created, attached, >> tags are set properly, both devices are detached properly also and at the >> end test is failing as in >> http://169.254.169.254/openstack/latest/meta_data.json still has some device >> inside. >> And it looks now from [4] that it is volume which isn’t removed from this >> meta_data.json. >> So I think that it would be good if people from Nova and Cinder teams could >> look at it and try to figure out what is going on there and how it can be >> fixed. >> >> Thanks in advance for help. >> >> [1] https://bugs.launchpad.net/tempest/+bug/1775947 >> [2] https://review.openstack.org/#/c/578765/ >> [3] >> https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L330 >> [4] >> http://logs.openstack.org/69/567369/15/check/tempest-full/528bc75/job-output.txt.gz#_2018-07-19_10_06_09_273919 >> >> — >> Slawek Kaplonski >> Senior software engineer >> Red Hat >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > -- > Artom Lifshitz > Software Engineer, OpenStack Compute DFG -- -- Artom Lifshitz Software Engineer, OpenStack Compute DFG __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
Because we're waiting for the volume to become available before we continue with the test [1], its tag still being present means Nova's not cleaning up the device tags on volume detach. This is most likely a bug. I'll look into it. [1] https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L378 On Thu, Jul 19, 2018 at 7:09 AM, Slawomir Kaplonski wrote: > Hi, > > Since some time we see that test > tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment > is failing sometimes. > Bug about that is reported for Tempest currently [1] but after small patch > [2] was merged I was today able to check what cause this issue. > > Test which is failing is in [3] and it looks that everything is going fine > with it up to last line of test. So volume and port are created, attached, > tags are set properly, both devices are detached properly also and at the end > test is failing as in http://169.254.169.254/openstack/latest/meta_data.json > still has some device inside. > And it looks now from [4] that it is volume which isn’t removed from this > meta_data.json. > So I think that it would be good if people from Nova and Cinder teams could > look at it and try to figure out what is going on there and how it can be > fixed. > > Thanks in advance for help. > > [1] https://bugs.launchpad.net/tempest/+bug/1775947 > [2] https://review.openstack.org/#/c/578765/ > [3] > https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L330 > [4] > http://logs.openstack.org/69/567369/15/check/tempest-full/528bc75/job-output.txt.gz#_2018-07-19_10_06_09_273919 > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Artom Lifshitz Software Engineer, OpenStack Compute DFG __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova][Cinder][Tempest] Help with tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment needed
Hi, Since some time we see that test tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment is failing sometimes. Bug about that is reported for Tempest currently [1] but after small patch [2] was merged I was today able to check what cause this issue. Test which is failing is in [3] and it looks that everything is going fine with it up to last line of test. So volume and port are created, attached, tags are set properly, both devices are detached properly also and at the end test is failing as in http://169.254.169.254/openstack/latest/meta_data.json still has some device inside. And it looks now from [4] that it is volume which isn’t removed from this meta_data.json. So I think that it would be good if people from Nova and Cinder teams could look at it and try to figure out what is going on there and how it can be fixed. Thanks in advance for help. [1] https://bugs.launchpad.net/tempest/+bug/1775947 [2] https://review.openstack.org/#/c/578765/ [3] https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_device_tagging.py#L330 [4] http://logs.openstack.org/69/567369/15/check/tempest-full/528bc75/job-output.txt.gz#_2018-07-19_10_06_09_273919 — Slawek Kaplonski Senior software engineer Red Hat __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
