[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Changed in: cloud-archive/rocky Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:19.0.0-0ubuntu2.3 --- nova (2:19.0.0-0ubuntu2.3) disco; urgency=medium * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). nova (2:19.0.0-0ubuntu2.2) disco; urgency=medium * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from upstream to ensure xenapi agent only raises a RuntimeError exception when openssl returns a non-zero exit code (LP: #1771506). nova (2:19.0.0-0ubuntu2.1) disco; urgency=medium * d/gbp.conf: Create stable/stein branch. * d/p/eventlet-monkey-patching-should-be-as-early-as-possible.patch: Cherry-picked from upstream stable/stein review to fix py3+wsgi+ssl crash (LP: #1808951). -- Sahid Orentino Ferdjaoui Thu, 16 May 2019 10:54:46 +0200 ** Changed in: nova (Ubuntu Disco) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:17.0.9-0ubuntu3~cloud0 --- nova (2:17.0.9-0ubuntu3~cloud0) xenial-queens; urgency=medium . * New update for the Ubuntu Cloud Archive. . nova (2:17.0.9-0ubuntu3) bionic; urgency=medium . * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). ** Changed in: cloud-archive/queens Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:19.0.0-0ubuntu2.3~cloud0 --- nova (2:19.0.0-0ubuntu2.3~cloud0) bionic-stein; urgency=medium . * New update for the Ubuntu Cloud Archive. . nova (2:19.0.0-0ubuntu2.3) disco; urgency=medium . * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). ** Changed in: cloud-archive/stein Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
OK, verified S/Q/R as commented on https://bugs.launchpad.net/bugs/1826523 The following exception is raised when the traffic to the ceph-mon is dropped by iptables. ** libvirt.libvirtError: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainAttachDeviceFlags) ** No attachment has been found at the vms after this operation fails, no volume added to volumes_attached property. (https://pastebin.canonical.com/p/kkmRcfrsjT/) ** Tags removed: verification-needed verification-queens-needed verification-rocky-needed verification-stein-needed ** Tags added: verification-done verification-queens-done verification-rocky-done verification-stein-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:17.0.9-0ubuntu3 --- nova (2:17.0.9-0ubuntu3) bionic; urgency=medium * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). nova (2:17.0.9-0ubuntu2) bionic; urgency=medium * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from upstream to ensure xenapi agent only raises a RuntimeError exception when openssl returns a non-zero exit code (LP: #1771506). * d/p/skip-double-word-hacking-test.patch: Cherry-picked from upstream to work-around test_hacking failures with new versions of python (LP: #1782786). -- Sahid Orentino Ferdjaoui Thu, 16 May 2019 11:06:11 +0200 ** Changed in: nova (Ubuntu Bionic) Status: Fix Committed => Fix Released ** Changed in: nova (Ubuntu Cosmic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:18.1.0-0ubuntu3 --- nova (2:18.1.0-0ubuntu3) cosmic; urgency=medium * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). nova (2:18.1.0-0ubuntu2) cosmic; urgency=medium * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from upstream to ensure xenapi agent only raises a RuntimeError exception when openssl returns a non-zero exit code (LP: #1771506). -- Sahid Orentino Ferdjaoui Thu, 16 May 2019 10:58:45 +0200 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Tags added: sts-sru-needed ** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
I have verified the following versions: * bionic: 17.0.9-0ubuntu3 * cosmic: 18.1.0-0ubuntu3 * disco: 19.0.0-0ubuntu2.3 The following exception is raised when the traffic to the ceph-mon is dropped by iptables. libvirt.libvirtError: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainAttachDeviceFlags) No attachment has been found at the vms after this operation fails, no volume added to volumes_attached property. ** Tags removed: sts sts-sru-needed verification-needed-bionic verification-needed-cosmic verification-needed-disco ** Tags added: verification-done-bionic verification-done-cosmic verification-done-disco -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
Hello Gustavo, or anyone else affected, Accepted nova into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:17.0.9-0ubuntu3 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: nova (Ubuntu Bionic) Status: Triaged => Fix Committed ** Tags added: verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
This bug was fixed in the package nova - 2:19.0.0-0ubuntu5 --- nova (2:19.0.0-0ubuntu5) eoan; urgency=medium * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure virsh disk attach does not fail silently (LP: #1825882). * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure always disconnect volumes after libvirt exceptions (LP: #1826523). -- Sahid Orentino Ferdjaoui Thu, 16 May 2019 10:41:27 +0200 ** Changed in: nova (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
Hello Gustavo, or anyone else affected, Accepted nova into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:18.1.0-0ubuntu3 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: nova (Ubuntu Cosmic) Status: Triaged => Fix Committed ** Tags added: verification-needed-cosmic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
Hello Gustavo, or anyone else affected, Accepted nova into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:19.0.0-0ubuntu2.3 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: nova (Ubuntu Disco) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-disco -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Description changed: [Impact] The following commit (1) is causing volume attachments which fail due to libvirt device attach erros to be silently ignored and Nova report the attachment as successful. It seems that the original intention of the commit was to log a condition and re-raise the exeption, but if the exception is of type libvirt.libvirtError and does not contain the searched pattern, the exception is ignored. If you unindent the raise statement, errors are reported again. In our case we had ceph/apparmor configuration problems in compute nodes which prevented virsh attaching the device; volumes appeared as successfully attached but the corresponding block device missing in guests VMs. Other libvirt attach error conditions are ignored also, as when you have already occuppied device names (i.e. 'Target vdb already exists', device is busy, etc.) (1) https://github.com/openstack/nova/commit/78891c2305bff6e16706339a9c5eca99a84e409c [Test Case] * Deploy any OpenStack version up to Pike , which includes ceph backed cinder * Create a guest VM (openstack server ...) * Create a test cinder volume $ openstack volume create test --size 10 * Force a drop on ceph traffic. Run the following command on the nova hypervisor on which the server runs. $ iptables -A OUTPUT -d ceph-mon-addr -p tcp --dport 6800 -j DROP * Attach the volume to a running instance. $ openstack server add volume 7151f507-a6b7-4f6d-a4cc-fd223d9feb5d 742ff117-21ae-4d1b-a52b-5b37955716ff * This should cause the volume attachment to fail $ virsh domblklist instance-x Target Source vda nova/7151f507-a6b7-4f6d-a4cc-fd223d9feb5d_disk No volume should attached after this step. * If the behavior is fixed: -* Check that openstack server show , doesn't displays the displays the volume as attached. -* Check that proper log entries states the libvirt exception and error. + * Check that openstack server show , doesn't displays the displays the volume as attached. + * Check that proper log entries states the libvirt exception and error. * If the behavior isn't fixed: -* openstack server show , will display the volume in the + * openstack server show , will display the volume in the volumes_attached property. [Expected result] * Volume attach fails and a proper exception is logged. [Actual result] * Volume attach fails but remains connected to the host and no further exception gets logged. [Regression Potential] - * We haven't identified any regression potential on this SRU. + * The patches have been cherry-picked from upstream which helps to + reduce the regression potential of these fixes. [Other Info] * N/A Description -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Also affects: cloud-archive/queens Importance: Undecided Status: New ** Also affects: cloud-archive/rocky Importance: Undecided Status: New ** Also affects: cloud-archive/stein Importance: Undecided Assignee: Sahid Orentino (sahid-ferdjaoui) Status: New ** Changed in: cloud-archive/queens Importance: Undecided => Medium ** Changed in: cloud-archive/queens Status: New => Triaged ** Changed in: cloud-archive/rocky Importance: Undecided => Medium ** Changed in: cloud-archive/rocky Status: New => Triaged ** Changed in: cloud-archive/stein Importance: Undecided => Medium ** Changed in: cloud-archive/stein Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Changed in: cloud-archive Assignee: (unassigned) => Sahid Orentino (sahid-ferdjaoui) ** Changed in: nova (Ubuntu) Assignee: (unassigned) => Sahid Orentino (sahid-ferdjaoui) ** Changed in: nova (Ubuntu Bionic) Assignee: (unassigned) => Sahid Orentino (sahid-ferdjaoui) ** Changed in: nova (Ubuntu Disco) Assignee: (unassigned) => Sahid Orentino (sahid-ferdjaoui) ** Changed in: nova (Ubuntu Cosmic) Assignee: (unassigned) => Sahid Orentino (sahid-ferdjaoui) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Changed in: nova (Ubuntu Disco) Status: New => Triaged ** Changed in: nova (Ubuntu Cosmic) Status: New => Triaged ** Changed in: nova (Ubuntu Bionic) Status: New => Triaged ** Changed in: nova (Ubuntu) Status: New => Triaged ** Changed in: nova (Ubuntu) Importance: Undecided => Medium ** Changed in: nova (Ubuntu Bionic) Importance: Undecided => Medium ** Changed in: nova (Ubuntu Cosmic) Importance: Undecided => Medium ** Changed in: nova (Ubuntu Disco) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Also affects: cloud-archive Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1825882] Re: [SRU] Virsh disk attach errors silently ignored
** Summary changed: - Virsh disk attach errors silently ignored + [SRU] Virsh disk attach errors silently ignored -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1825882 Title: [SRU] Virsh disk attach errors silently ignored To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1825882/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
