[Yahoo-eng-team] [Bug 2056276] [NEW] unmaintained/yoga periodic jobs broken
Public bug reported: Couple of jobs failing in unmaintained/yoga periodic pipeline. https://zuul.openstack.org/buildsets?project=openstack%2Fneutron=unmaintained%2Fyoga=stable%2Fyoga=periodic=0 Before switching to unmaintained/yoga jobs were green:- https://zuul.openstack.org/buildset/eab6ceae73d7437186cb7432e7ad8897 Currently following jobs broken:- - devstack-tobiko-neutron - neutron-ovn-tempest-slow - neutron-ovs-tempest-slow All fails with:- 2024-03-04 02:30:28.791273 | controller | + ./stack.sh:main:230 : SUPPORTED_DISTROS='bullseye|focal|f35|opensuse-15.2|opensuse-tumbleweed|rhel8|rhel9|openEuler-20.03' 2024-03-04 02:30:28.793551 | controller | + ./stack.sh:main:232 : [[ ! jammy =~ bullseye|focal|f35|opensuse-15.2|opensuse-tumbleweed|rhel8|rhel9|openEuler-20.03 ]] 2024-03-04 02:30:28.795963 | controller | + ./stack.sh:main:233 : echo 'WARNING: this script has not been tested on jammy' builds:- https://zuul.openstack.org/builds?job_name=devstack-tobiko-neutron_name=neutron-ovn-tempest-slow_name=neutron-ovs-tempest-slow=openstack%2Fneutron=%09unmaintained%2Fyoga=0 The bug is to track the fixes and also discuss what to do with similar issues going forward in unmaintained branches? ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2056276 Title: unmaintained/yoga periodic jobs broken Status in neutron: New Bug description: Couple of jobs failing in unmaintained/yoga periodic pipeline. https://zuul.openstack.org/buildsets?project=openstack%2Fneutron=unmaintained%2Fyoga=stable%2Fyoga=periodic=0 Before switching to unmaintained/yoga jobs were green:- https://zuul.openstack.org/buildset/eab6ceae73d7437186cb7432e7ad8897 Currently following jobs broken:- - devstack-tobiko-neutron - neutron-ovn-tempest-slow - neutron-ovs-tempest-slow All fails with:- 2024-03-04 02:30:28.791273 | controller | + ./stack.sh:main:230 : SUPPORTED_DISTROS='bullseye|focal|f35|opensuse-15.2|opensuse-tumbleweed|rhel8|rhel9|openEuler-20.03' 2024-03-04 02:30:28.793551 | controller | + ./stack.sh:main:232 : [[ ! jammy =~ bullseye|focal|f35|opensuse-15.2|opensuse-tumbleweed|rhel8|rhel9|openEuler-20.03 ]] 2024-03-04 02:30:28.795963 | controller | + ./stack.sh:main:233 : echo 'WARNING: this script has not been tested on jammy' builds:- https://zuul.openstack.org/builds?job_name=devstack-tobiko-neutron_name=neutron-ovn-tempest-slow_name=neutron-ovs-tempest-slow=openstack%2Fneutron=%09unmaintained%2Fyoga=0 The bug is to track the fixes and also discuss what to do with similar issues going forward in unmaintained branches? To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2056276/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1998789] Re: [SRU] PooledLDAPHandler.result3 does not release pool connection back when an exception is raised
This bug was fixed in the package keystone - 2:17.0.1-0ubuntu2 --- keystone (2:17.0.1-0ubuntu2) focal; urgency=medium * d/p/lp1998789-pooledldap-conn-release-on-exception.patch: Release pooled ldap connection back to pool when an exception is thrown by ldap methods (LP: #1998789). -- Mustafa Kemal GILOR Wed, 04 Oct 2023 12:29:32 +0300 ** Changed in: keystone (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1998789 Title: [SRU] PooledLDAPHandler.result3 does not release pool connection back when an exception is raised Status in Ubuntu Cloud Archive: Fix Released Status in Ubuntu Cloud Archive antelope series: Fix Released Status in Ubuntu Cloud Archive ussuri series: Fix Released Status in Ubuntu Cloud Archive victoria series: Fix Released Status in Ubuntu Cloud Archive wallaby series: Fix Released Status in Ubuntu Cloud Archive xena series: Fix Released Status in Ubuntu Cloud Archive yoga series: Fix Released Status in Ubuntu Cloud Archive zed series: Fix Released Status in OpenStack Identity (keystone): Fix Released Status in keystone package in Ubuntu: Fix Released Status in keystone source package in Focal: Fix Released Status in keystone source package in Jammy: Fix Released Status in keystone source package in Lunar: Fix Released Bug description: [Impact] This SRU is a backport of https://review.opendev.org/c/openstack/keystone/+/866723 to the respective Ubuntu and UCA releases. The patch is merged to the all respective upstream branches (master & stable/[u,v,w,x,y,z]). This SRU intends to fix a denial-of-service bug that happens when keystone uses pooled ldap connections. In pooled ldap connection mode, keystone borrows a connection from the pool, do the LDAP operation and release it back to the pool. But, if an exception or error happens while the LDAP connection is still borrowed, Keystone fails to release the connection back to the pool, hogging it forever. If this happens for all the pooled connections, the connection pool will be exhausted and Keystone will no longer be able to perform LDAP operations. The fix corrects this behavior by allowing the connection to release back to the pool even if an exception/error happens during the LDAP operation. [Test Case] - Deploy an LDAP server of your choice - Fill it with many data so the search takes more than `pool_connection_timeout` seconds - Define a keystone domain with the LDAP driver with following options: [ldap] use_pool = True page_size = 100 pool_connection_timeout = 3 pool_retry_max = 3 pool_size = 10 - Point the domain to the LDAP server - Try to login to the OpenStack dashboard, or try to do anything that uses the LDAP user - Observe the /var/log/apache2/keystone_error.log, it should contain ldap.TIMEOUT() stack traces followed by `ldappool.MaxConnectionReachedError` stack traces To confirm the fix, repeat the scenario and observe that the "/var/log/apache2/keystone_error.log" does not contain `ldappool.MaxConnectionReachedError` stack traces and LDAP operation in motion is successful (e.g. OpenStack Dashboard login) [Regression Potential] The patch is quite trivial and should not affect any deployment in a negative way. The LDAP pool functionality can be disabled by setting "use_pool=False" in case of any regression. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1998789/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2025056] Re: Router ports without IP addresses shouldn't be allowed to deletion using port's API directly
Patches merged, will close. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2025056 Title: Router ports without IP addresses shouldn't be allowed to deletion using port's API directly Status in neutron: Fix Released Bug description: Long time ago there was bug https://bugs.launchpad.net/neutron/+bug/1104337 and as fix for this bug there was patch https://review.opendev.org/c/openstack/neutron/+/20424 proposed. This patch allowed to remove router ports without fixed IPs directly using "port delete" command. But it may cause error 500 if port really belongs to an existing router. Steps to reproduce the issue: 1. Create network (external) and do NOT create subnet for it, 2. Create router, 3. Set network from p. 1 as external gateway for the router, 4. Try to delete external gateway's port using "openstack port delete" command - it will fail with error 500. Stacktrace in neutron server log is as below: 2023-06-22 05:41:06.672 16 DEBUG neutron.db.l3_db [req-a261d22f-9243-4b22-8d40-a5e7bcd63453 abd0fab2837040f383c986b6a723fbec 39e32a986a4d4f42bce967634a308f99 - default default] Port 9978f00d-4be2-474d-89a7-07d9b1e797df has owner network:router_gateway, but no IP address, so it can be deleted prevent_l3_port_deletion /usr/lib/python3.9/site-packages/neutron/db/l3_db.py:1675 2023-06-22 05:41:07.085 16 DEBUG neutron.plugins.ml2.plugin [req-a261d22f-9243-4b22-8d40-a5e7bcd63453 abd0fab2837040f383c986b6a723fbec 39e32a986a4d4f42bce967634a308f99 - default default] Calling delete_port for 9978f00d-4be2-474d-89a7-07d9b1e797df owned by network:router_gateway delete_port /usr/lib/python3.9/site-packages/neutron/plugins/ml2/plugin.py:2069 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation [req-a261d22f-9243-4b22-8d40-a5e7bcd63453 abd0fab2837040f383c986b6a723fbec 39e32a986a4d4f42bce967634a308f99 - default default] DELETE failed.: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`ovs_neutron`.`routers`, CONSTRAINT `routers_ibfk_1` FOREIGN KEY (`gw_port_id`) REFERENCES `ports` (`id`))') [SQL: DELETE FROM ports WHERE ports.id = %(id)s] [parameters: {'id': '9978f00d-4be2-474d-89a7-07d9b1e797df'}] (Background on this error at: http://sqlalche.me/e/13/gkpj) 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last): 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation self.dialect.do_execute( 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 609, in do_execute 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation cursor.execute(statement, parameters) 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation result = self._query(query) 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation conn.query(q) 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 505, in query 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 724, in _read_query_result 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation result.read() 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 1069, in read 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation first_packet = self.connection._read_packet() 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 676, in _read_packet 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation packet.raise_for_error() 2023-06-22 05:41:07.360 16 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.9/site-packages/pymysql/protocol.py", line 223, in raise_for_error 2023-06-22
[Yahoo-eng-team] [Bug 1999154] Re: ovs/ovn source deployment broken with ovs_branch=master
Seems fixed, closing. ** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1999154 Title: ovs/ovn source deployment broken with ovs_branch=master Status in neutron: Fix Released Bug description: Since [1] jobs running with OVS_BRANCH=master are broken, fails as below:- utilities/ovn-dbctl.c: In function ‘do_dbctl’: utilities/ovn-dbctl.c:724:9: error: too few arguments to function ‘ctl_context_init_command’ 724 | ctl_context_init_command(ctx, c); | ^~~~ In file included from utilities/ovn-dbctl.c:23: /opt/stack/ovs/lib/db-ctl-base.h:249:6: note: declared here 249 | void ctl_context_init_command(struct ctl_context *, struct ctl_command *, | ^~~~ make[1]: *** [Makefile:2352: utilities/ovn-dbctl.o] Error 1 make[1]: *** Waiting for unfinished jobs make[1]: Leaving directory '/opt/stack/ovn' make: *** [Makefile:1548: all] Error 2 + lib/neutron_plugins/ovn_agent:compile_ovn:1 : exit_trap Failure builds example:- - https://zuul.opendev.org/t/openstack/build/3a900a1cfe824746ac8ffc6a27fc8ec4 - https://zuul.opendev.org/t/openstack/build/7d862338d6194a4fb3a34e8c3c67f532 - https://zuul.opendev.org/t/openstack/build/ae092f4985af41908697240e3f64f522 Until OVN repo[2] get's updated to work with ovs master we have to pin ovs to working version to get these experimental jobs back to green. [1] https://github.com/openvswitch/ovs/commit/b8bf410a5c94173da02279b369d75875c4035959 [2] https://github.com/ovn-org/ovn To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1999154/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008912] Re: "_validate_create_network_callback" failing with 'NoneType' object has no attribute 'qos_policy_id'
Change merged, will close this. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008912 Title: "_validate_create_network_callback" failing with 'NoneType' object has no attribute 'qos_policy_id' Status in neutron: Fix Released Bug description: Logs: https://e138a887655b8fda005f-ea1d911c7c7db668a9aa6765a743313b.ssl.cf5.rackcdn.com/874133/2/check/neutron- tempest-plugin-openvswitch-enforce-scope-new- defaults/7e5cbf9/controller/logs/screen-q-svc.txt Error (snippet): https://paste.opendev.org/show/bYMju0ckz5GK5BYq0yhN/ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008912/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2022914] Re: [neutron-api] remove leader_only for maintenance worker
Patches have merged, will close. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2022914 Title: [neutron-api] remove leader_only for maintenance worker Status in neutron: Fix Released Bug description: Currently if you want to connect the neutron-api to the souhtbound database you cannot use relays, because the maintenance worker has a condition set that it requires a leader_only connection. This leader_only collection is not necessary since the maintenance tasks of the neutron-api are only getting information from the souhtbound and are not pushing information into the souhtbound database. If you adjust the neutron-api to use relays, it will log something like "relay database, cannot be leader" every time the maintenance task should run. I would expect to be able to set the southbound connection for the neutron-api to use the relays. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2022914/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1897928] Re: TestOvnDbNotifyHandler test cases failing due to missing attribute "_RowEventHandler__watched_events"
Seems to have been fixed with https://review.opendev.org/c/openstack/neutron/+/820911 will close this bug. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1897928 Title: TestOvnDbNotifyHandler test cases failing due to missing attribute "_RowEventHandler__watched_events" Status in neutron: Fix Released Bug description: Some neutron.tests.unit.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovsdb_monitor.TestOvnDbNotifyHandler test cases are failing: * test_shutdown * test_watch_and_unwatch_events The error [1] is caused because of a missing attribute: AttributeError: 'OvnDbNotifyHandler' object has no attribute '_RowEventHandler__watched_events' [1]https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bec/periodic/opendev.org/openstack/neutron/master/openstack-tox-py36-with-ovsdbapp-master/becf062/testr_results.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1897928/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1701410] Re: different behavior when deleting with request body (no BadRequest with core resources in case of pecan)
I am going to close this as it is over 6 years old and no one has stepped forward to fix it, so it's just not a priority. Please re-open if necessary. ** Changed in: neutron Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1701410 Title: different behavior when deleting with request body (no BadRequest with core resources in case of pecan) Status in neutron: Won't Fix Bug description: In master environment, it is different behavior when we try to delete with request body. I fixed it in [1] but CORE_RESOURCE(network/subnet/port/subnetpool) doesn't pass this code in case of web_framework = pecan in /etc/neutron/neutron.conf [1] https://github.com/openstack/neutron/blame/master/neutron/api/v2/base.py#L555 [FloatingIP, Router] $ source ~/devstack/openrc admin admin; export TOKEN=`openstack token issue | grep ' id ' | get_field 2` $ curl -i -X DELETE -d '{"floatingip":{"description": "aaa"}}' -H "content-type:application/json" -H 'accept:application/json' -H "x-auth-token:$TOKEN" 192.168.122.33:9696/v2.0/floatingips/f4e9b845-4472-4806-bd7a-bec8f7618af2 HTTP/1.1 400 Bad Request Content-Length: 113 Content-Type: application/json X-Openstack-Request-Id: req-deaffdb3-7c13-4604-89d0-78fbcc184ef5 Date: Fri, 30 Jun 2017 00:56:56 GMT {"NeutronError": {"message": "Request body is not supported in DELETE.", "type": "HTTPBadRequest", "detail": ""}} $ curl -i -X DELETE -d '{"router": {"name": "aaa"}}' -H "content-type:application/json" -H 'accept:application/json' -H "x-auth-token:$TOKEN" 192.168.122.33:9696/v2.0/routers/1d0ea30e-c481-4be3-a548-a659d9e3787c HTTP/1.1 400 Bad Request Content-Length: 113 Content-Type: application/json X-Openstack-Request-Id: req-a2f9babb-4eb3-471e-9b42-ccfe722c44f0 Date: Fri, 30 Jun 2017 01:44:40 GMT {"NeutronError": {"message": "Request body is not supported in DELETE.", "type": "HTTPBadRequest", "detail": ""}} [Core resources: Network/Subnet/Port/Subnetpool] $ source ~/devstack/openrc admin admin; export TOKEN=`openstack token issue | grep ' id ' | get_field 2` $ curl -i -X DELETE -d '{"network":{"name": ""}}' -H "content-type:application/json" -H 'accept:application/json' -H "x-auth-token:$TOKEN" 192.168.122.33:9696/v2.0/networks/1fb94931-dabe-49dc-bce4-68c8bafea8b0 HTTP/1.1 204 No Content Content-Length: 0 X-Openstack-Request-Id: req-7e838c38-e6cd-46c3-8703-c93f5bb4a503 Date: Fri, 30 Jun 2017 01:32:12 GMT $ curl -i -X DELETE -d '{"subnet": {"name": "aaa"}}' -H "content- type:application/json" -H 'accept:application/json' -H "x-auth- token:$TOKEN" 192.168.122.33:9696/v2.0/subnets/a18fb191-2a89-4193-80d1-5330a8052d64 HTTP/1.1 204 No Content Content-Length: 0 X-Openstack-Request-Id: req-901476cf-7e87-4b7c-ab20-209b81d2eb25 Date: Fri, 30 Jun 2017 01:37:01 GMT $ curl -i -X DELETE -d '{"port": {"name": "aaa"}}' -H "content- type:application/json" -H 'accept:application/json' -H "x-auth- token:$TOKEN" 192.168.122.33:9696/v2.0/ports/47f2c36a-7461-4c1a-a23e-931d5aee3f9c HTTP/1.1 204 No Content Content-Length: 0 X-Openstack-Request-Id: req-48452706-6309-42c2-ac80-f0f4e387060e Date: Fri, 30 Jun 2017 01:37:33 GMT $ curl -i -X DELETE -d '{"subnetpool": {"description": "aaa"}}' -H "content-type:application/json" -H 'accept:application/json' -H "x-auth-token:$TOKEN" 192.168.122.33:9696/v2.0/subnetpools/e0e09ffc-a4af-4cf0-ac2e-7a8b1475cef6 HTTP/1.1 204 No Content Content-Length: 0 X-Openstack-Request-Id: req-9601a3ae-74a0-49ca-9f99-02ad624ceacb Date: Fri, 30 Jun 2017 06:24:58 GMT To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1701410/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2056230] [NEW] clouds.yaml file documentation is misleading
Public bug reported: The cloud.yaml.template at https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/api_access/templates/api_access/clouds.yaml.template states that if I only have one cloud I don't need to set OS_CLOUD or --os_cloud=... I found this to be incorrect. I needed to set OS_CLOUD even with a single cloud listed in ~/.config/openstack/clouds.yaml. Its possible I'm using it wrong. Its possible I'm using an outdated client, but I wasted 30minutes banging my head against this and it would be nice if we could save someone else the same wasted 30 minutes. ** Affects: horizon Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/2056230 Title: clouds.yaml file documentation is misleading Status in OpenStack Dashboard (Horizon): New Bug description: The cloud.yaml.template at https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/api_access/templates/api_access/clouds.yaml.template states that if I only have one cloud I don't need to set OS_CLOUD or --os_cloud=... I found this to be incorrect. I needed to set OS_CLOUD even with a single cloud listed in ~/.config/openstack/clouds.yaml. Its possible I'm using it wrong. Its possible I'm using an outdated client, but I wasted 30minutes banging my head against this and it would be nice if we could save someone else the same wasted 30 minutes. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/2056230/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1865453] Re: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly
** Changed in: neutron Assignee: Adil Ishaq (iradvisor) => (unassigned) ** Changed in: identity-management Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1865453 Title: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before fails randomly Status in Identity Management: Invalid Status in neutron: Confirmed Bug description: Sometimes we see random failures of the test: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_before neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.test_mech_driver.TestVirtualPorts.test_virtual_port_created_beforetesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.6/site-packages/mock/mock.py", line 1330, in patched return func(*args, **keywargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func return f(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/plugins/ml2/drivers/ovn/mech_driver/test_mech_driver.py", line 280, in test_virtual_port_created_before ovn_vport.options[ovn_const.LSP_OPTIONS_VIRTUAL_PARENTS_KEY]) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.6/site-packages/testtools/testcase.py", line 417, in assertIn self.assertThat(haystack, Contains(needle), message) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.6/site-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: '88c0378b-71bd-454b-a0df-8c70b57d257a' not in '49043b88-554f-48d0-888d-eeaa749e752f' To manage notifications about this bug go to: https://bugs.launchpad.net/identity-management/+bug/1865453/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2012365] Re: Issues with nova-manage volume_attachment subcommand
Reviewed: https://review.opendev.org/c/openstack/nova/+/877446 Committed: https://opendev.org/openstack/nova/commit/a8f81d5f08692c16d538af3197460d9426cc00a1 Submitter: "Zuul (22348)" Branch:master commit a8f81d5f08692c16d538af3197460d9426cc00a1 Author: Amit Uniyal Date: Wed Mar 15 05:07:28 2023 + Disconnecting volume from the compute host cmd nova-manage volume_attachment refresh vm-id vol-id connetor There were cases where the instance said to live in compute#1 but the connection_info in the BDM record was for compute#2, and when the script called `remote_volume_connection` then nova would call os-brick on compute#1 (the wrong node) and try to detach it. In some case os-brick would mistakenly think that the volume was attached (because the target and lun matched an existing volume on the host) and would try to disconnect, resulting in errors on the compute logs. - Added HostConflict exception - Fixes dedent in cmd/manange.py - Updates nova-mange doc Closes-Bug: #2012365 Change-Id: I21109752ff1c56d3cefa58fcd36c68bf468e0a73 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2012365 Title: Issues with nova-manage volume_attachment subcommand Status in OpenStack Compute (nova): Fix Released Bug description: Downstream bug report from Red Hat Bugzilla against Train: https://bugzilla.redhat.com/show_bug.cgi?id=2161733 1 - fix handling of instance locking Add a context manager for locking and unlocking instance for volume attachement refresh cmd. 2 - disconnecting the volume from the correct host Verify if instance is attached to correct compute host before removing volume connection. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2012365/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1846820] Fix included in openstack/nova xena-eom
This issue was fixed in the openstack/nova xena-eom release. ** Changed in: nova/xena Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1846820 Title: nova-conductor may crash during deploy due to haproxy-keystone 504 Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) wallaby series: In Progress Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: keystone was busy (behind haproxy) nova-conductor: 2019-10-04 15:39:17.103 6 CRITICAL nova [-] Unhandled error: GatewayTimeout: Gateway Timeout (HTTP 504) 2019-10-04 15:39:17.103 6 ERROR nova Traceback (most recent call last): 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/bin/nova-conductor", line 10, in 2019-10-04 15:39:17.103 6 ERROR nova sys.exit(main()) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/cmd/conductor.py", line 44, in main 2019-10-04 15:39:17.103 6 ERROR nova topic=rpcapi.RPC_TOPIC) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/service.py", line 257, in create 2019-10-04 15:39:17.103 6 ERROR nova periodic_interval_max=periodic_interval_max) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/service.py", line 129, in __init__ 2019-10-04 15:39:17.103 6 ERROR nova self.manager = manager_class(host=self.host, *args, **kwargs) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/conductor/manager.py", line 117, in __init__ 2019-10-04 15:39:17.103 6 ERROR nova self.compute_task_mgr = ComputeTaskManager() 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/conductor/manager.py", line 243, in __init__ 2019-10-04 15:39:17.103 6 ERROR nova self.report_client = report.SchedulerReportClient() 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 186, in __init__ 2019-10-04 15:39:17.103 6 ERROR nova self._client = self._create_client() 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 229, in _create_client 2019-10-04 15:39:17.103 6 ERROR nova client = self._adapter or utils.get_sdk_adapter('placement') 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/utils.py", line 1039, in get_sdk_adapter 2019-10-04 15:39:17.103 6 ERROR nova return getattr(conn, service_type) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack/service_description.py", line 92, in __get__ 2019-10-04 15:39:17.103 6 ERROR nova endpoint = proxy_mod.Proxy.get_endpoint(proxy) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 282, in get_endpoint 2019-10-04 15:39:17.103 6 ERROR nova return self.session.get_endpoint(auth or self.auth, **kwargs) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/session.py", line 1200, in get_endpoint 2019-10-04 15:39:17.103 6 ERROR nova return auth.get_endpoint(self, **kwargs) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint 2019-10-04 15:39:17.103 6 ERROR nova allow_version_hack=allow_version_hack, **kwargs) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data 2019-10-04 15:39:17.103 6 ERROR nova service_catalog = self.get_access(session).service_catalog 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access 2019-10-04 15:39:17.103 6 ERROR nova self.auth_ref = self.get_auth_ref(session) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref 2019-10-04 15:39:17.103 6 ERROR nova return self._plugin.get_auth_ref(session, **kwargs) 2019-10-04 15:39:17.103 6 ERROR nova File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 184, in get_auth_ref 2019-10-04 15:39:17.103 6 ERROR nova authenticated=False, log=False, **rkwargs) 2019-10-04 15:39:17.103 6 ERROR nova File
[Yahoo-eng-team] [Bug 1970383] Fix included in openstack/nova xena-eom
This issue was fixed in the openstack/nova xena-eom release. ** Changed in: nova/xena Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1970383 Title: Segment-aware scheduling fails for non-admin users Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: This is a follow-up to https://bugs.launchpad.net/nova/+bug/1967314 Having deployed the Nova scheduler configuration for routed provider networks as follows (Xena deployment @ 7df9379d6661233174d49fb7be8eda0828a5e5ca), this was found to resolve issues around scheduling of instances to appropriate hypervisors, but it appears to have surfaced a side effect. [scheduler] query_placement_for_routed_network_aggregates = True When the above configuration is enabled, creation of new instances for admin users works correctly, but for non-admin users against the same networks results in the following error: 285768 ERROR oslo_messaging.rpc.server [req-79ca3cb3-eb52-4755-bba1-4c840c8ae5fc c35a1473225f422c90a6f75b25188bf2 d96f0cd70c6a4adbbbcf993502b264dc - default default] Exception during message handling: K> 285768 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 285768 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 285768 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 285768 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 241, in inner 285768 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/manager.py", line 154, in select_destinations 285768 ERROR oslo_messaging.rpc.server request_filter.process_reqspec(context, spec_obj) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 387, in process_reqspec 285768 ERROR oslo_messaging.rpc.server filter(ctxt, request_spec) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 41, in wrapper 285768 ERROR oslo_messaging.rpc.server ran = fn(ctxt, request_spec) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 348, in routed_networks_filter 285768 ERROR oslo_messaging.rpc.server aggregates = utils.get_aggregates_for_routed_network( 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/utils.py", line 1406, in get_aggregates_for_routed_network 285768 ERROR oslo_messaging.rpc.server segment_ids = network_api.get_segment_ids_for_network( 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/network/neutron.py", line 3721, in get_segment_ids_for_network 285768 ERROR oslo_messaging.rpc.server return [subnet['segment_id'] for subnet in subnets 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/network/neutron.py", line 3722, in 285768 ERROR oslo_messaging.rpc.server if subnet['segment_id'] is not None] 285768 ERROR oslo_messaging.rpc.server KeyError: 'segment_id' 285768 ERROR oslo_messaging.rpc.server It appears that the subnet dictionaries are returned empty from the Neutron client library in this case, causing the KeyError. As far as I can see, a matching command line request for 'openstack subnet show X' as the same requesting user correctly includes the 'segment_id', but I don't know how similar this code path and the permissions handling is. I'd be happy to test out other requests or obtain additional logs if useful. To manage notifications
[Yahoo-eng-team] [Bug 1970383] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1970383 Title: Segment-aware scheduling fails for non-admin users Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: This is a follow-up to https://bugs.launchpad.net/nova/+bug/1967314 Having deployed the Nova scheduler configuration for routed provider networks as follows (Xena deployment @ 7df9379d6661233174d49fb7be8eda0828a5e5ca), this was found to resolve issues around scheduling of instances to appropriate hypervisors, but it appears to have surfaced a side effect. [scheduler] query_placement_for_routed_network_aggregates = True When the above configuration is enabled, creation of new instances for admin users works correctly, but for non-admin users against the same networks results in the following error: 285768 ERROR oslo_messaging.rpc.server [req-79ca3cb3-eb52-4755-bba1-4c840c8ae5fc c35a1473225f422c90a6f75b25188bf2 d96f0cd70c6a4adbbbcf993502b264dc - default default] Exception during message handling: K> 285768 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 285768 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 285768 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 285768 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 241, in inner 285768 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/manager.py", line 154, in select_destinations 285768 ERROR oslo_messaging.rpc.server request_filter.process_reqspec(context, spec_obj) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 387, in process_reqspec 285768 ERROR oslo_messaging.rpc.server filter(ctxt, request_spec) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 41, in wrapper 285768 ERROR oslo_messaging.rpc.server ran = fn(ctxt, request_spec) 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/request_filter.py", line 348, in routed_networks_filter 285768 ERROR oslo_messaging.rpc.server aggregates = utils.get_aggregates_for_routed_network( 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/scheduler/utils.py", line 1406, in get_aggregates_for_routed_network 285768 ERROR oslo_messaging.rpc.server segment_ids = network_api.get_segment_ids_for_network( 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/network/neutron.py", line 3721, in get_segment_ids_for_network 285768 ERROR oslo_messaging.rpc.server return [subnet['segment_id'] for subnet in subnets 285768 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-24.0.0.0rc1/lib/python3.8/site-packages/nova/network/neutron.py", line 3722, in 285768 ERROR oslo_messaging.rpc.server if subnet['segment_id'] is not None] 285768 ERROR oslo_messaging.rpc.server KeyError: 'segment_id' 285768 ERROR oslo_messaging.rpc.server It appears that the subnet dictionaries are returned empty from the Neutron client library in this case, causing the KeyError. As far as I can see, a matching command line request for 'openstack subnet show X' as the same requesting user correctly includes the 'segment_id', but I don't know how similar this code path and the permissions handling is. I'd be happy to test out other requests or obtain additional logs if useful. To manage
[Yahoo-eng-team] [Bug 2004555] Fix included in openstack/nova xena-eom
This issue was fixed in the openstack/nova xena-eom release. ** Changed in: nova/xena Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2004555 Title: [OSSA-2023-003] Unauthorized volume access through deleted volume attachments (CVE-2023-2088) Status in Cinder: Fix Released Status in glance_store: Fix Released Status in kolla-ansible: Fix Released Status in kolla-ansible zed series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) antelope series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Status in OpenStack Compute (nova) zed series: Fix Released Status in os-brick: In Progress Status in OpenStack Security Advisory: Fix Released Status in OpenStack Security Notes: Fix Released Bug description: Hello OpenStack Security Team, I’m writing to you, as we faced a serious security breach in OpenStack functionality(correlated a bit with libvirt, iscsi and huawei driver). I was going through OSSA documents and correlated libvirt notes, but I couldn't find something similar. It is not related to https://security.openstack.org/ossa/OSSA-2020-006.html In short: we observed that newly created cinder volume(1GB size) was attached to compute node instance, but an instance recognized it as a 115GB volume, which(this 115GB volume) in fact was connected to another instance on the same compute node. [1. Test environment] Compute node: OpenStack Ussuri configured with Huawei dorado as a storage backend(configuration driver is available here: https://docs.openstack.org/cinder/rocky/configuration/block-storage/drivers/huawei-storage-driver.html) Packages: v# dpkg -l | grep libvirt ii libvirt-clients 6.0.0-0ubuntu8.16 amd64Programs for the libvirt library ii libvirt-daemon6.0.0-0ubuntu8.16 amd64Virtualization daemon ii libvirt-daemon-driver-qemu6.0.0-0ubuntu8.16 amd64Virtualization daemon QEMU connection driver ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.16 amd64Virtualization daemon RBD storage driver ii libvirt-daemon-system 6.0.0-0ubuntu8.16 amd64Libvirt daemon configuration files ii libvirt-daemon-system-systemd 6.0.0-0ubuntu8.16 amd64Libvirt daemon configuration files (systemd) ii libvirt0:amd646.0.0-0ubuntu8.16 amd64library for interfacing with different virtualization systems ii nova-compute-libvirt 2:21.2.4-0ubuntu1 all OpenStack Compute - compute node libvirt support ii python3-libvirt 6.1.0-1 amd64libvirt Python 3 bindings # dpkg -l | grep qemu ii ipxe-qemu 1.0.0+git-20190109.133f4c4-0ubuntu3.2all PXE boot firmware - ROM images for qemu ii ipxe-qemu-256k-compat-efi-roms1.0.0+git-20150424.a25a16d-0ubuntu4 all PXE boot firmware - Compat EFI ROM images for qemu ii libvirt-daemon-driver-qemu6.0.0-0ubuntu8.16 amd64Virtualization daemon QEMU connection driver ii qemu 1:4.2-3ubuntu6.23 amd64fast processor emulator, dummy package ii qemu-block-extra:amd641:4.2-3ubuntu6.23 amd64extra block backend modules for qemu-system and qemu-utils ii qemu-kvm 1:4.2-3ubuntu6.23 amd64QEMU Full virtualization on x86 hardware ii qemu-system-common1:4.2-3ubuntu6.23 amd64QEMU full system emulation binaries (common files) ii qemu-system-data 1:4.2-3ubuntu6.23 all QEMU full system emulation (data files) ii qemu-system-gui:amd64 1:4.2-3ubuntu6.23 amd64QEMU full system emulation binaries (user interface and audio support) ii qemu-system-x86 1:4.2-3ubuntu6.23 amd64QEMU full system emulation binaries
[Yahoo-eng-team] [Bug 2004555] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2004555 Title: [OSSA-2023-003] Unauthorized volume access through deleted volume attachments (CVE-2023-2088) Status in Cinder: Fix Released Status in glance_store: Fix Released Status in kolla-ansible: Fix Released Status in kolla-ansible zed series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) antelope series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Status in OpenStack Compute (nova) zed series: Fix Released Status in os-brick: In Progress Status in OpenStack Security Advisory: Fix Released Status in OpenStack Security Notes: Fix Released Bug description: Hello OpenStack Security Team, I’m writing to you, as we faced a serious security breach in OpenStack functionality(correlated a bit with libvirt, iscsi and huawei driver). I was going through OSSA documents and correlated libvirt notes, but I couldn't find something similar. It is not related to https://security.openstack.org/ossa/OSSA-2020-006.html In short: we observed that newly created cinder volume(1GB size) was attached to compute node instance, but an instance recognized it as a 115GB volume, which(this 115GB volume) in fact was connected to another instance on the same compute node. [1. Test environment] Compute node: OpenStack Ussuri configured with Huawei dorado as a storage backend(configuration driver is available here: https://docs.openstack.org/cinder/rocky/configuration/block-storage/drivers/huawei-storage-driver.html) Packages: v# dpkg -l | grep libvirt ii libvirt-clients 6.0.0-0ubuntu8.16 amd64Programs for the libvirt library ii libvirt-daemon6.0.0-0ubuntu8.16 amd64Virtualization daemon ii libvirt-daemon-driver-qemu6.0.0-0ubuntu8.16 amd64Virtualization daemon QEMU connection driver ii libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.16 amd64Virtualization daemon RBD storage driver ii libvirt-daemon-system 6.0.0-0ubuntu8.16 amd64Libvirt daemon configuration files ii libvirt-daemon-system-systemd 6.0.0-0ubuntu8.16 amd64Libvirt daemon configuration files (systemd) ii libvirt0:amd646.0.0-0ubuntu8.16 amd64library for interfacing with different virtualization systems ii nova-compute-libvirt 2:21.2.4-0ubuntu1 all OpenStack Compute - compute node libvirt support ii python3-libvirt 6.1.0-1 amd64libvirt Python 3 bindings # dpkg -l | grep qemu ii ipxe-qemu 1.0.0+git-20190109.133f4c4-0ubuntu3.2all PXE boot firmware - ROM images for qemu ii ipxe-qemu-256k-compat-efi-roms1.0.0+git-20150424.a25a16d-0ubuntu4 all PXE boot firmware - Compat EFI ROM images for qemu ii libvirt-daemon-driver-qemu6.0.0-0ubuntu8.16 amd64Virtualization daemon QEMU connection driver ii qemu 1:4.2-3ubuntu6.23 amd64fast processor emulator, dummy package ii qemu-block-extra:amd641:4.2-3ubuntu6.23 amd64extra block backend modules for qemu-system and qemu-utils ii qemu-kvm 1:4.2-3ubuntu6.23 amd64QEMU Full virtualization on x86 hardware ii qemu-system-common1:4.2-3ubuntu6.23 amd64QEMU full system emulation binaries (common files) ii qemu-system-data 1:4.2-3ubuntu6.23 all QEMU full system emulation (data files) ii qemu-system-gui:amd64 1:4.2-3ubuntu6.23 amd64QEMU full system emulation binaries (user interface and audio support) ii qemu-system-x86 1:4.2-3ubuntu6.23 amd64QEMU full system emulation
[Yahoo-eng-team] [Bug 1777608] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1777608 Title: Nova compute calls plug_vifs unnecessarily for ironic nodes in init_host Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: Fix Released Status in OpenStack Compute (nova) ussuri series: Fix Released Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Bug description: Originally reported there https://bugzilla.redhat.com/show_bug.cgi?id=1590297#c14 and tracked there https://bugzilla.redhat.com/show_bug.cgi?id=1592427 @owalsh: Looks like a race in the service startup. (undercloud) [stack@undercloud ~]$ openstack server list +--+-++++--+ | ID | Name | Status | Networks | Image | Flavor | +--+-++++--+ | fcda23f8-7b98-41bf-9e45-7a4579164874 | overcloud-controller-1 | ERROR | ctlplane=192.168.24.7 | overcloud-full_renamed | oooq_control | | ced0e0db-ad1f-4381-a523-0a79ae0303ff | overcloud-controller-2 | ERROR | ctlplane=192.168.24.21 | overcloud-full_renamed | oooq_control | | 0731685f-a8fc-4126-868a-bdf9879238f3 | overcloud-controller-0 | ERROR | ctlplane=192.168.24.12 | overcloud-full_renamed | oooq_control | | c4fd4e28-fe44-40d2-9b40-7196b7c3b6a2 | overcloud-novacompute-2 | ERROR | ctlplane=192.168.24.15 | overcloud-full_renamed | oooq_compute | | 5bef3afd-4485-4cbc-b76c-f126c85bd015 | overcloud-novacompute-1 | ERROR | ctlplane=192.168.24.8 | overcloud-full_renamed | oooq_compute | | e4c9da33-c452-446c-82c4-bd55a6b294d8 | overcloud-novacompute-0 | ERROR | ctlplane=192.168.24.9 | overcloud-full_renamed | oooq_compute | +--+-++++--+ Looking at controller-1... (undercloud) [stack@undercloud ~]$ openstack server show overcloud-controller-1 +-+---+ | Field | Value | +-+---+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | undercloud | | OS-EXT-SRV-ATTR:hypervisor_hostname | b6a32fba-b57a-4e7b-a6ce-99941b4d134d | | OS-EXT-SRV-ATTR:instance_name | instance-0005 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | error | | OS-SRV-USG:launched_at | 2018-06-11T14:56:34.00 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | ctlplane=192.168.24.7 | | config_drive | True | | created | 2018-06-11T14:53:09Z | | flavor | oooq_control (04f8ba26-e9bd-472f-b92a-919e7ec8bed1) | | hostId | da8848b6f3dc77a51235f70dcf44df197261eca0529173f670c94ce9 | | id | fcda23f8-7b98-41bf-9e45-7a4579164874 | | image | overcloud-full_renamed (4d965d80-61fc-45d0-88e1-e6365c4afd57) | | key_name | default | | name | overcloud-controller-1 | | project_id | 3b778414471a47e4b0760bbecc9d4070 | | properties | | | security_groups | name='default' | | status | ERROR | | updated | 2018-06-15T09:29:17Z | | user_id | 356b9a5451c643bb8162b9349bc9487b | | volumes_attached | | +-+---+ From /var/log/ironic/ironic-conductor.log I can see that ironic- conductor started loading extensions after the updated timestamp: 2018-06-15 09:29:18.576 1408 DEBUG oslo_concurrency.lockutils [req-6ad8cb26-b607-4f65-a8b7-30ac72a7a997 - - - - -] Lock "extension_manager" acquired by "ironic.common.driver_factory._init_extension_manager" :: waited 0.000s inner /usr/lib/python2.7/site- packages/oslo_concurrency/lockutils.py:273 The errors in /var/log/ironic/app.log occurred before this, because there wasn't a conductor registered that supports ipmi: 2018-06-15 09:29:14.419 1793 DEBUG wsme.api [req-81ce866e-1c15-4499-a452-044a44efa1c0 1a609f3c25c24c45ac30ee2fcc721eac 5909cc46dbad48058daa89d48f07ba71 - default default] Client-side error: No valid host was found. Reason: No conductor service registered which supports driver ipmi. format_exception /usr/lib/python2.7/site-packages/wsme/api.py:222 2018-06-15 09:29:15.304 1792 DEBUG
[Yahoo-eng-team] [Bug 1815989] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: neutron/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1815989 Title: OVS drops RARP packets by QEMU upon live-migration causes up to 40s ping pause in Rocky Status in neutron: In Progress Status in neutron train series: In Progress Status in neutron ussuri series: In Progress Status in neutron victoria series: In Progress Status in neutron wallaby series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: New Status in OpenStack Compute (nova) ussuri series: New Status in OpenStack Compute (nova) victoria series: New Status in OpenStack Compute (nova) wallaby series: New Status in os-vif: Invalid Bug description: This issue is well known, and there were previous attempts to fix it, like this one https://bugs.launchpad.net/neutron/+bug/1414559 This issue still exists in Rocky and gets worse. In Rocky, nova compute, nova libvirt and neutron ovs agent all run inside containers. So far the only simply fix I have is to increase the number of RARP packets QEMU sends after live-migration from 5 to 10. To be complete, the nova change (not merged) proposed in the above mentioned activity does not work. I am creating this ticket hoping to get an up-to-date (for Rockey and onwards) expert advise on how to fix in nova-neutron. For the record, below are the time stamps in my test between neutron ovs agent "activating" the VM port and rarp packets seen by tcpdump on the compute. 10 RARP packets are sent by (recompiled) QEMU, 7 are seen by tcpdump, the 2nd last packet barely made through. openvswitch-agent.log: 2019-02-14 19:00:13.568 73453 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-26129036-b514-4fa0-a39f-a6b21de17bb9 - - - - -] Port 57d0c265-d971-404d-922d-963c8263e6eb updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '1bf4b8e0-9299-485b-80b0-52e18e7b9b42', 'segmentation_id': 648, 'fixed_ips': [ {'subnet_id': 'b7c09e83-f16f-4d4e-a31a-e33a922c0bac', 'ip_address': '10.0.1.4'} ], 'device_owner': u'compute:nova', 'physical_network': u'physnet0', 'mac_address': 'fa:16:3e:de:af:47', 'device': u'57d0c265-d971-404d-922d-963c8263e6eb', 'port_security_enabled': True, 'port_id': '57d0c265-d971-404d-922d-963c8263e6eb', 'network_type': u'vlan', 'security_groups': [u'5f2175d7-c2c1-49fd-9d05-3a8de3846b9c']} 2019-02-14 19:00:13.568 73453 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-26129036-b514-4fa0-a39f-a6b21de17bb9 - - - - -] Assigning 4 as local vlan for net-id=1bf4b8e0-9299-485b-80b0-52e18e7b9b42 tcpdump for rarp packets: [root@overcloud-ovscompute-overcloud-0 nova]# tcpdump -i any rarp -nev tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 19:00:10.788220 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:11.138216 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:11.588216 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:12.138217 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:12.788216 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:13.538216 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 19:00:14.388320 B fa:16:3e:de:af:47 ethertype Reverse ARP (0x8035), length 62: Ethernet (len 6), IPv4 (len 4), Reverse Request who-is fa:16:3e:de:af:47 tell fa:16:3e:de:af:47, length 46 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1815989/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1944043] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1944043 Title: Wrong exception is expected to retry volume detachment API calls Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) queens series: New Status in OpenStack Compute (nova) rocky series: New Status in OpenStack Compute (nova) stein series: New Status in OpenStack Compute (nova) train series: In Progress Status in OpenStack Compute (nova) ussuri series: In Progress Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Description === The following change introduced the logic to retry cinder API calls to detach volumes. https://review.opendev.org/c/openstack/nova/+/669674 The logic detects the InternalServerError class from cindreclient.apiclient.exceptions. However this is wrong and these API calls raises the ClientException class from cinderclient.exceptions instead. Steps to reproduce == N/A Actual result = N/A Environment === N/A Logs & Configs == N/A To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1944043/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1944043] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1944043 Title: Wrong exception is expected to retry volume detachment API calls Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) queens series: New Status in OpenStack Compute (nova) rocky series: New Status in OpenStack Compute (nova) stein series: New Status in OpenStack Compute (nova) train series: In Progress Status in OpenStack Compute (nova) ussuri series: In Progress Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Description === The following change introduced the logic to retry cinder API calls to detach volumes. https://review.opendev.org/c/openstack/nova/+/669674 The logic detects the InternalServerError class from cindreclient.apiclient.exceptions. However this is wrong and these API calls raises the ClientException class from cinderclient.exceptions instead. Steps to reproduce == N/A Actual result = N/A Environment === N/A Logs & Configs == N/A To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1944043/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1949808] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1949808 Title: After nova live-migration-abort canceled "queued" live-migration, instance status remains "MIGRATING" Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Bug description: OpenStack supports cancelling "queued" live-migration by "nova live- migration-abort", but state of canceled instance remains "MIGRATING". To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1949808/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1960412] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1960412 Title: Live migration artifacts are not cleaned up properly when queued live migration is aborted Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Bug description: Bug #1949808 describes one of the problems affecting aborted queued live migrations: VM's status is not reverted back to ACTIVE and VM is left in MIGRATING state. However that's not the single problem with aborted queued live migrations: some changes (port bindings on destination host, resource allocations and possibly instance's pci_requests) are introduced by Nova control plane before live migration actually started by source compute host. Those left-overs should also be removed. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1960412/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1964149] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1964149 Title: nova dns lookups can block the nova api process leading to 503 errors. Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: In Progress Status in OpenStack Compute (nova) ussuri series: In Progress Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Bug description: we currently have 4 possibly related downstream bugs whereby DNS lookups can result in 503 errors as we do not monkey patch green DNS and that can result in blocking behavior. specifically we have seen callses to socket.getaddrinfo in py-amqp block the API when using ipv6. https://bugzilla.redhat.com/show_bug.cgi?id=2037690 https://bugzilla.redhat.com/show_bug.cgi?id=2050867 https://bugzilla.redhat.com/show_bug.cgi?id=2051631 https://bugzilla.redhat.com/show_bug.cgi?id=2056504 copying a summary of the rca from one of the bugs What happens: - A request comes in which requires rpc, so a new connection to rabbitmq is to be established - The hostname(s) from the transport_url setting are ultimately passed to py-amqp, which attempts to resolve the hostname to an ip address so it can set up the underlying socket and connect - py-amqp explicitly tries to resolve with AF_INET first and then only if that fails, then it tries with AF_INET6[1] - The customer environment is primarily IPv6. Attempting to resolve the hostname via AF_INET fails nss_hosts (the /etc/hosts file only have IPv6 addrs), and falls through to nss_dns - Something about the customer DNS infrastructure is slow, so it takes a long time (~10 seconds) for this IPv4-lookup to fail. - py-amqp finally tries with AF_INET6 and the hostname is resolved immediately via nss_hosts because the entry is in the /etc/hosts Critically, because nova explicitly disables greendns[2] with eventlet, the *entire* nova-api worker is blocked during the duration of the slow name resolution, because socket.getaddrinfo is a blocking call into glibc. [1] https://github.com/celery/py-amqp/blob/1f599c7213b097df07d0afd7868072ff9febf4da/amqp/transport.py#L155-L208 [2] https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L25-L36 nova currently disables greendns monkeypatch because of a very old bug on centos 6 on python 2.6 and the havana release of nova https://bugs.launchpad.net/nova/+bug/1164822 ipv6 support was added in v0.17 in the same release that added python 3 support back in 2015 https://github.com/eventlet/eventlet/issues/8#issuecomment-75490457 so we should not need to work around the lack of ipv6 support anymore. https://review.opendev.org/c/openstack/nova/+/830966 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1964149/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1978444] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1978444 Title: Volume can't be detached if attachment delete api call fails with 504 gateway timeout Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: In Progress Status in OpenStack Compute (nova) ussuri series: In Progress Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Description === When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout. When this timeout occurs while detaching a volume, this results in un-detachable volume. - nova-compute calls delete attachment api in cinder - haproxy detects server timeout and returns 504 - cinder continues processing the API and removes the attachment - nova-compute immediately aborts the volume detachment and leaves the bdm - when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova See for details https://bugzilla.redhat.com/show_bug.cgi?id=2002643 Steps to reproduce == * Stop cinder-volume * Detach a volume from an instance * Start cinder-volume * Detach the volume again Expected result === * Volume can be detached after cinder-volume is recovered Actual result === * Volume can't be detached Environment === * The issue was initially found in stable/train Logs & Configs == * See https://bugzilla.redhat.com/show_bug.cgi?id=2002643#c1 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1978444/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1978444] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1978444 Title: Volume can't be detached if attachment delete api call fails with 504 gateway timeout Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: In Progress Status in OpenStack Compute (nova) ussuri series: In Progress Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Description === When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout. When this timeout occurs while detaching a volume, this results in un-detachable volume. - nova-compute calls delete attachment api in cinder - haproxy detects server timeout and returns 504 - cinder continues processing the API and removes the attachment - nova-compute immediately aborts the volume detachment and leaves the bdm - when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova See for details https://bugzilla.redhat.com/show_bug.cgi?id=2002643 Steps to reproduce == * Stop cinder-volume * Detach a volume from an instance * Start cinder-volume * Detach the volume again Expected result === * Volume can be detached after cinder-volume is recovered Actual result === * Volume can't be detached Environment === * The issue was initially found in stable/train Logs & Configs == * See https://bugzilla.redhat.com/show_bug.cgi?id=2002643#c1 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1978444/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1978983] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1978983 Title: evacuate is not possible if the instance has task_state Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: Fix Released Status in OpenStack Compute (nova) ussuri series: Fix Released Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Description === A compute host dies but before anything notices it a VM that was running on that host is requested to be stopped by the user. The VM task_state is set to powering-off and the shutdown RPC is sent to the dead compute. A bit later the monitoring system detect that the compute is dead and fences the compute, set the compute to forced_down in nova and triggers the evacuation of the VM. However the evacuation is rejected by nova: Cannot 'evacuate' instance 81451eb2-4600-4036-a6f1-b99139f0d277 while it is in task_state powering-off (HTTP 409) (Request-ID: req-363ca0a3-0d68-42f6-95d2-122bd2a53463) Steps to reproduce == 0) deploy a multi node devstack 1) create a VM $openstack --os-compute-api-version 2.80 server create --image cirros-0.5.2-x86_64-disk --flavor c1 --nic net-id=public --use-config-drive vm1 --wait 2) stop the nova-compute service of the host the VM is scheduled to: $sudo systemctl stop devstack@n-cpu 3) stop the VM $openstack server stop vm1 4) fence the host and force the host down in nova 5) try to evacuate the VM $server evacuate vm1 See also [1] Expected result === The VM is evacuated successfully Actual result = Cannot 'evacuate' instance 81451eb2-4600-4036-a6f1-b99139f0d277 while it is in task_state powering-off (HTTP 409) (Request-ID: req-363ca0a3-0d68-42f6-95d2-122bd2a53463) Environment === devstack on recent master Workaround == The admin can reset the state of the VM with $nova reset-state --active vm1 then retry the evacuation. [1] https://paste.opendev.org/show/bQphEfOf8eLBnM6XmleQ/ [2] https://paste.opendev.org/show/bVI7D8H5g9Oqjjo4rKfk/ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1978983/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1986545] Fix included in openstack/nova wallaby-eom
This issue was fixed in the openstack/nova wallaby-eom release. ** Changed in: nova/wallaby Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1986545 Title: websockfiy open redirection unit test broken with Python >= 3.10.6 standard lib Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: Fix Released Status in OpenStack Compute (nova) ussuri series: Fix Released Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Lucas Nussbaum reported this Debian bug: https://bugs.debian.org/1017217 so I started investigating it. It took me a while to understand it was due to a change in the Python 3.10.6 standard http/server.py library. Running these 2 unit tests against Python 3.10.5 works: test_websocketproxy.NovaProxyRequestHandlerTestCase.test_reject_open_redirect console.test_websocketproxy.NovaProxyRequestHandlerTestCase.test_reject_open_redirect_3_slashes However, under Python 3.10.6, this fails. The reason isn't the interpreter itself, but the standard library, which has additional open redirection protection. Looking at the changelog here: https://docs.python.org/3/whatsnew/changelog.html we see this issue: https://github.com/python/cpython/issues/87389 which has been addressed by this commit: https://github.com/python/cpython/commit/defaa2b19a9a01c79c1d5641a8aa179bb10ead3f If I "fix" the Python 3.10.5 standard library using the 2 lines of code of the first hunk of this patch, then I can reproduce the issue. I guess that the unit testing should be skipped if using Python >= 3.10.6, probably, or adapted somehow. I leave this to the Nova maintainers: for the Debian package, I'll just skip these 2 unit tests. Cheers, Thomas Goirand (zigo) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1986545/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1986545] Fix included in openstack/nova victoria-eom
This issue was fixed in the openstack/nova victoria-eom release. ** Changed in: nova/victoria Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1986545 Title: websockfiy open redirection unit test broken with Python >= 3.10.6 standard lib Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: Fix Released Status in OpenStack Compute (nova) ussuri series: Fix Released Status in OpenStack Compute (nova) victoria series: Fix Released Status in OpenStack Compute (nova) wallaby series: Fix Released Status in OpenStack Compute (nova) xena series: Fix Released Status in OpenStack Compute (nova) yoga series: Fix Released Bug description: Lucas Nussbaum reported this Debian bug: https://bugs.debian.org/1017217 so I started investigating it. It took me a while to understand it was due to a change in the Python 3.10.6 standard http/server.py library. Running these 2 unit tests against Python 3.10.5 works: test_websocketproxy.NovaProxyRequestHandlerTestCase.test_reject_open_redirect console.test_websocketproxy.NovaProxyRequestHandlerTestCase.test_reject_open_redirect_3_slashes However, under Python 3.10.6, this fails. The reason isn't the interpreter itself, but the standard library, which has additional open redirection protection. Looking at the changelog here: https://docs.python.org/3/whatsnew/changelog.html we see this issue: https://github.com/python/cpython/issues/87389 which has been addressed by this commit: https://github.com/python/cpython/commit/defaa2b19a9a01c79c1d5641a8aa179bb10ead3f If I "fix" the Python 3.10.5 standard library using the 2 lines of code of the first hunk of this patch, then I can reproduce the issue. I guess that the unit testing should be skipped if using Python >= 3.10.6, probably, or adapted somehow. I leave this to the Nova maintainers: for the Debian package, I'll just skip these 2 unit tests. Cheers, Thomas Goirand (zigo) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1986545/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2056199] [NEW] ``DvrDriver`` and ``OvnDriver`` incorrectly define distributed flag
Public bug reported: The class ``L3ServiceProvider`` defines the distributed support with the class variable "distributed_support" [1]. The classes ``DvrDriver`` and ``OvnDriver`` are using the variable "dvr_support" instead. The method to ensure a driver "ha" and "distributed" support uses "distributed_support" too [2]. [1]https://github.com/openstack/neutron/blob/c4c14f9589b54cc518564f1e1679898d2729c9e2/neutron/services/l3_router/service_providers/base.py#L57 [2]https://github.com/openstack/neutron/blob/c4c14f9589b54cc518564f1e1679898d2729c9e2/neutron/services/l3_router/service_providers/driver_controller.py#L273 ** Affects: neutron Importance: Medium Assignee: Rodolfo Alonso (rodolfo-alonso-hernandez) Status: In Progress ** Changed in: neutron Importance: Undecided => Medium ** Changed in: neutron Assignee: (unassigned) => Rodolfo Alonso (rodolfo-alonso-hernandez) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2056199 Title: ``DvrDriver`` and ``OvnDriver`` incorrectly define distributed flag Status in neutron: In Progress Bug description: The class ``L3ServiceProvider`` defines the distributed support with the class variable "distributed_support" [1]. The classes ``DvrDriver`` and ``OvnDriver`` are using the variable "dvr_support" instead. The method to ensure a driver "ha" and "distributed" support uses "distributed_support" too [2]. [1]https://github.com/openstack/neutron/blob/c4c14f9589b54cc518564f1e1679898d2729c9e2/neutron/services/l3_router/service_providers/base.py#L57 [2]https://github.com/openstack/neutron/blob/c4c14f9589b54cc518564f1e1679898d2729c9e2/neutron/services/l3_router/service_providers/driver_controller.py#L273 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2056199/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2055295] Re: Unexpected API Error
** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2055295 Title: Unexpected API Error Status in OpenStack Compute (nova): Invalid Bug description: 024-02-28 20:28:16.731 2534 WARNING oslo_config.cfg [None req-edd6752a-8653-4f5e-bd96-11fbc924f60a 3e74f7ef625942f39484fd1b9bc7ca62 6861e0fb7b274ca397f6737d026e9a70 - - default default] Deprecated: Option "api_servers" from group "glance" is deprecated for removal ( Support for image service configuration via standard keystoneauth1 Adapter options was added in the 17.0.0 Queens release. The api_servers option was retained temporarily to allow consumers time to cut over to a real load balancing solution. ). Its value may be silently ignored in the future. 2024-02-28 20:28:17.235 2534 WARNING keystoneauth.identity.generic.base [None req-edd6752a-8653-4f5e-bd96-11fbc924f60a 3e74f7ef625942f39484fd1b9bc7ca62 6861e0fb7b274ca397f6737d026e9a70 - - default default] Failed to discover available identity versions when contacting https://controller/identity. Attempting to parse version from URL.: keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to https://controller/identity: HTTPSConnectionPool(host='controller', port=443): Max retries exceeded with url: /identity (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] ECONNREFUSED')) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi [None req-edd6752a-8653-4f5e-bd96-11fbc924f60a 3e74f7ef625942f39484fd1b9bc7ca62 6861e0fb7b274ca397f6737d026e9a70 - - default default] Unexpected exception in API method: keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. Unable to establish connection to https://controller/identity: HTTPSConnectionPool(host='controller', port=443): Max retries exceeded with url: /identity (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] ECONNREFUSED')) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi conn = connection.create_connection( 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi raise err 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi sock.connect(sa) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 256, in connect 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi socket_checkerr(fd) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 54, in socket_checkerr 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi raise socket.error(err, errno.errorcode[err]) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi ConnectionRefusedError: [Errno 111] ECONNREFUSED 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 700, in urlopen 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi httplib_response = self._make_request( 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi self._validate_conn(conn) 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1017, in _validate_conn 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi conn.connect() 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect 2024-02-28 20:28:17.244 2534 ERROR nova.api.openstack.wsgi conn = self._new_conn()
[Yahoo-eng-team] [Bug 2056195] [NEW] Return 409 at neutron-client conflict
Public bug reported: Description === When attaching a stateless and stateful security group to a VM, nova returns a 500 error but it's a user issue and a 409 conflict error should be returned. Steps to reproduce == 1. create network 2. create VM "test-vm" attached to the network 3. may create a statefull security group, but default group should already do 4. openstack securit group create --stateless stateless-group 5. openstack server add security group test-vm stateless-group Expected result === Nova forwards the 409 error from Neutron with the error description from Neutron. Actual result = Nova returns: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. (HTTP 500) (Request-ID: req-c6bbaf50-99b7-4108-98f0-808dfee84933) Environment === 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ # nova-api --version 26.2.2 (Zed) 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) Neutron with OVN Logs & Configs == Stacktrace: Traceback (most recent call last):, File "/usr/local/lib/python3.10/site-packages/nova/api/openstack/wsgi.py", line 658, in wrapped, return f(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/nova/api/openstack/compute/security_groups.py", line 437, in _addSecurityGroup, return security_group_api.add_to_instance(context, instance,, File "/usr/local/lib/python3.10/site-packages/nova/network/security_group_api.py", line 653, in add_to_instance, raise e, File "/usr/local/lib/python3.10/site-packages/nova/network/security_group_api.py", line 648, in add_to_instance, neutron.update_port(port['id'], {'port': updated_port}), File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 828, in update_port, return self._update_resource(self.port_path % (port), body=body,, File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 2548, in _update_resource, return self.put(path, **kwargs), File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 365, in put, return self.retry_request("PUT", action, body=body,, File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 333, in retry_request, return self.do_request(method, action, body=body,, File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 297, in do_request, self._handle_fault_response(status_code, replybody, resp), File "/usr/local/lib/python3.10/site-packages/nova/network/neutron.py", line 196, in wrapper, ret = obj(*args, **kwargs), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response, exception_handler_v20(status_code, error_body), File "/usr/local/lib/python3.10/site-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20, raise client_exc(message=error_message,, neutronclient.common.exceptions.Conflict: Error Cannot apply both stateful and stateless security groups on the same port at the same time while attempting the operation., Neutron server returns request_ids: ['req-1007ffaa-3501-4566-9ad9-c540931138f0'] ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2056195 Title: Return 409 at neutron-client conflict Status in OpenStack Compute (nova): New Bug description: Description === When attaching a stateless and stateful security group to a VM, nova returns a 500 error but it's a user issue and a 409 conflict error should be returned. Steps to reproduce == 1. create network 2. create VM "test-vm" attached to the network 3. may create a statefull security group, but default group should already do 4. openstack securit group create --stateless stateless-group 5. openstack server add security group test-vm stateless-group Expected result === Nova forwards the 409
[Yahoo-eng-team] [Bug 1936972] Re: MAAS deploys fail if host has NIC w/ random MAC
** Changed in: maas Milestone: 3.4.x => 3.5.x ** No longer affects: maas/3.3 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1936972 Title: MAAS deploys fail if host has NIC w/ random MAC Status in cloud-init: Expired Status in curtin: New Status in MAAS: Triaged Bug description: The Nvidia DGX A100 server includes a USB Redfish Host Interface NIC. This NIC apparently provides no MAC address of it's own, so the driver generates a random MAC for it: ./drivers/net/usb/cdc_ether.c: static int usbnet_cdc_zte_bind(struct usbnet *dev, struct usb_interface *intf) { int status = usbnet_cdc_bind(dev, intf); if (!status && (dev->net->dev_addr[0] & 0x02)) eth_hw_addr_random(dev->net); return status; } This causes a problem with MAAS because, during deployment, MAAS sees this as a normal NIC and records the MAC. The post-install reboot then fails: [ 43.652573] cloud-init[3761]: init.apply_network_config(bring_up=not args.local) [ 43.700516] cloud-init[3761]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 735, in apply_network_config [ 43.724496] cloud-init[3761]: self.distro.networking.wait_for_physdevs(netcfg) [ 43.740509] cloud-init[3761]: File "/usr/lib/python3/dist-packages/cloudinit/distros/networking.py", line 177, in wait_for_physdevs [ 43.764523] cloud-init[3761]: raise RuntimeError(msg) [ 43.780511] cloud-init[3761]: RuntimeError: Not all expected physical devices present: {'fe:b8:63:69:9f:71'} I'm not sure what the best answer for MAAS is here, but here's some thoughts: 1) Ignore all Redfish system interfaces. These are a connect between the host and the BMC, so they don't really have a use-case in the MAAS model AFAICT. These devices can be identified using the SMBIOS as described in the Redfish Host Interface Specification, section 8: https://www.dmtf.org/sites/default/files/standards/documents/DSP0270_1.3.0.pdf Which can be read from within Linux using dmidecode. 2) Ignore (or specially handle) all NICs with randomly generated MAC addresses. While this is the only time I've seen the random MAC with production server hardware, it is something I've seen on e.g. ARM development boards. Problem is, I don't know how to detect a generated MAC. I'd hoped the permanent MAC (ethtool -P) MAC would be NULL, but it seems to also be set to the generated MAC :( fyi, 2 workarounds for this that seem to work: 1) Delete the NIC from the MAAS model in the MAAS UI after every commissioning. 2) Use a tag's kernel_opts field to modprobe.blacklist the driver used for the Redfish NIC. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1936972/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1978489] Re: libvirt / cgroups v2: cannot boot instance with more than 16 CPUs
Re: > The same patch should also be available on cloud archive cloud:focal- yoga This will happen alongside the changes being made into 22.04 - the updates are in the yoga-proposed pocket at the moment. ** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/yoga Importance: Undecided Status: New ** Changed in: cloud-archive Status: New => Invalid ** Changed in: cloud-archive/yoga Status: New => Fix Committed ** Changed in: cloud-archive/yoga Importance: Undecided => High ** Changed in: nova (Ubuntu Jammy) Importance: Undecided => High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1978489 Title: libvirt / cgroups v2: cannot boot instance with more than 16 CPUs Status in Ubuntu Cloud Archive: Invalid Status in Ubuntu Cloud Archive yoga series: Fix Committed Status in OpenStack Compute (nova): In Progress Status in nova package in Ubuntu: Confirmed Status in nova source package in Jammy: Fix Committed Bug description: Description === Using the libvirt driver and a host OS that uses cgroups v2 (RHEL 9, Ubuntu Jammy), an instance with more than 16 CPUs cannot be booted. Steps to reproduce == 1. Boot an instance with 10 (or more) CPUs on RHEL 9 or Ubuntu Jammy using Nova with the libvirt driver. Expected result === Instance boots. Actual result = Instance fails to boot with a 'Value specified in CPUWeight is out of range' error. Environment === Originially report as a libvirt but in RHEL 9 [1] Additional information == This is happening because Nova defaults to 1024 * (# of CPUs) for the value of domain/cputune/shares in the libvirt XML. This is then passed directly by libvirt to the cgroups API, but cgroups v2 has a maximum value of 1. 1 / 1024 ~= 9.76 [1] https://bugzilla.redhat.com/show_bug.cgi?id=2035518 Ubuntu SRU Details: [Impact] See above. [Test Case] See above. [Regression Potential] We've had this change in other jammy-based versions of the nova package for a while now, including zed, antelope, bobcat. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1978489/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2053274] Re: [ovn] mtu for metadata veth interface is not set
Reviewed: https://review.opendev.org/c/openstack/neutron/+/909684 Committed: https://opendev.org/openstack/neutron/commit/47b4d14955451c320ce612e4e2e7d0a896e2fe36 Submitter: "Zuul (22348)" Branch:master commit 47b4d14955451c320ce612e4e2e7d0a896e2fe36 Author: Rodolfo Alonso Hernandez Date: Wed Feb 21 15:34:13 2024 + [OVN] Set MTU of the VETH interfaces between OVS and metadata The VETH pair between the metadata namespace and the local OVS now has the same MTU as the network associated to this metadata service. The "LSP.external_ids" field has a new key defined: "neutron:mtu". This is the value of the network MTU. This patch does not update the previous metadata datapaths nor the existing LSP. This change will affect only to new created ports and the corresponding metadata datapaths. Closes-Bug: #2053274 Change-Id: I7ff300e9634e5e3fc68d70540392109fd8b9babc ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2053274 Title: [ovn] mtu for metadata veth interface is not set Status in neutron: Fix Released Bug description: When using OVN, the `veth` interfaces which get created inside the network namespace (and the other half that goes into the OVS bridge) both do not get an MTU configured for them when they are provisioned. https://github.com/openstack/neutron/blob/stable/zed/neutron/agent/ovn/metadata/agent.py#L589-L594 This can cause some unknown/annoying errors with packets being dropped if a user is hitting large requests on the metadata service, the ideal solution would be to configure the correct MTU for the interface to avoid this issue. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2053274/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2056149] [NEW] Inconsistent volume naming when create instance (from volume)
Public bug reported: Description: When creating an instance from volume, there are inconsistent behaviours and users usability issue. Using Yoga and confirmed it with older versions as well. It likely will be present on new versions too. Cases: - using the CLI and the --boot-from-volume flag: Naming: The instance gets created, and the volume does not get any name, it is just blank "" Problems: By default when instance deletion, the volume does not get deleted. If a user wants to reuse the root volume, finding the right volume is just impossible. Suggestion: it would be nice to call the volume "volume-$INSTANCEUUID" to have direct correspondency between a VM and its root volume. - using Horizon and selecting the CREATE VOLUME button: Naming: The volume this time gets a name, that is equal to the volume UUID. Problems: The behaviour is different than the CLI and users (and admins) get confused. Suggestion: As above, to call the root volume when booting an instance from volume, to name it "volume-$INSTANCEUUID". Overall it would be nice to an option to template the volume naming when creating from volume, something like: boot_from_volume_naming_template: volume-%UUID The blank naming behaviour of case 1 should be fixed as a bug <--- ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2056149 Title: Inconsistent volume naming when create instance (from volume) Status in OpenStack Compute (nova): New Bug description: Description: When creating an instance from volume, there are inconsistent behaviours and users usability issue. Using Yoga and confirmed it with older versions as well. It likely will be present on new versions too. Cases: - using the CLI and the --boot-from-volume flag: Naming: The instance gets created, and the volume does not get any name, it is just blank "" Problems: By default when instance deletion, the volume does not get deleted. If a user wants to reuse the root volume, finding the right volume is just impossible. Suggestion: it would be nice to call the volume "volume-$INSTANCEUUID" to have direct correspondency between a VM and its root volume. - using Horizon and selecting the CREATE VOLUME button: Naming: The volume this time gets a name, that is equal to the volume UUID. Problems: The behaviour is different than the CLI and users (and admins) get confused. Suggestion: As above, to call the root volume when booting an instance from volume, to name it "volume-$INSTANCEUUID". Overall it would be nice to an option to template the volume naming when creating from volume, something like: boot_from_volume_naming_template: volume-%UUID The blank naming behaviour of case 1 should be fixed as a bug <--- To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2056149/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp