[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
** Changed in: cloud-archive Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
This bug was fixed in the package neutron - 2:12.1.1-0ubuntu7~cloud0 --- neutron (2:12.1.1-0ubuntu7~cloud0) xenial-queens; urgency=medium . * New update for the Ubuntu Cloud Archive. . neutron (2:12.1.1-0ubuntu7) bionic; urgency=medium . * Handle OVSFWPortNotFound and OVSFWTagNotFound in ovs firewall - d/p/0001-Handle-OVSFWPortNotFound-and-OVSFWTagNotFound-in-ovs.patch (LP: #1849098). . neutron (2:12.1.1-0ubuntu6) bionic; urgency=medium . * Do not initialize snat-ns twice (LP: #1850779) - d/p/0001-Do-not-initialize-snat-ns-twice.patch . neutron (2:12.1.1-0ubuntu5) bionic; urgency=medium . * Backport fix for dvr-snat missig rfp interfaces (LP: #1894843) - d/p/0001-Fix-deletion-of-rfp-interfaces-when-router-is-re-ena.patch ** Changed in: cloud-archive/queens Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
This bug was fixed in the package neutron - 2:12.1.1-0ubuntu7 --- neutron (2:12.1.1-0ubuntu7) bionic; urgency=medium * Handle OVSFWPortNotFound and OVSFWTagNotFound in ovs firewall - d/p/0001-Handle-OVSFWPortNotFound-and-OVSFWTagNotFound-in-ovs.patch (LP: #1849098). neutron (2:12.1.1-0ubuntu6) bionic; urgency=medium * Do not initialize snat-ns twice (LP: #1850779) - d/p/0001-Do-not-initialize-snat-ns-twice.patch neutron (2:12.1.1-0ubuntu5) bionic; urgency=medium * Backport fix for dvr-snat missig rfp interfaces (LP: #1894843) - d/p/0001-Fix-deletion-of-rfp-interfaces-when-router-is-re-ena.patch -- Seyeong Kim Mon, 03 May 2021 17:15:28 +0900 ** Changed in: neutron (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/queens Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
Verified the test case with the bionic-proposed (neutron 2:12.1.1-0ubuntu7) and test case is working fine $ openstack server list +--+-++---++---+ | ID | Name| Status | Networks | Image | Flavor| +--+-++---++---+ | c6ead240-8952-49ab-8f4f-7c3ed2007af9 | testvm-after-fix| ACTIVE | private=192.168.21.4 | cirros | m1.cirros | | 52688c3f-00b9-4ae8-bc08-0f87265e8bb3 | testvm-after-tagcomment | ERROR | | cirros | m1.cirros | | 5f23c575-94a5-48f3-b6ef-d0d9f6f2f7d4 | cirros-110548 | ACTIVE | private=192.168.21.10 | cirros | m1.cirros | +--+-++---++---+ cirros-110548: VM launched after deployment testvm-after-tagcomment: VM launched after changing the code as mentioned in test case and VM is in ERROR state and logs shows tag errrors testvm-after-fix: VM launched after the neutron package is upgraded with the one in bionic-proposed. ** Tags removed: verification-needed verification-needed-bionic ** Tags added: verification-done verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
Hello Oleg, or anyone else affected, Accepted neutron into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:12.1.1-0ubuntu7 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: neutron (Ubuntu Bionic) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
** Description changed:
[Impact]
- somehow port is unbounded, then neutron-openvswitch-agent raise
+ somehow port is unbounded, then neutron-openvswitch-agent raise
OVSFWTagNotFound, then creating new instance will be failed.
[Test Plan]
1. deploy bionic openstack env
2. launch one instance
3. modify neutron-openvswitch-agent code inside nova-compute
- https://pastebin.ubuntu.com/p/nBRKkXmjx8/
4. restart neutron-openvswitch-agent
5. check if there are a lot of cannot get tag for port ..
6. launch another instance.
7. It fails after vif_plugging_timeout, with "virtual interface creation
failed"
[Where problems could occur]
- You need to restart service. and as patch, Basically it will be ok as it adds
only exceptions. but getting or creating vif port part can have issue.
+ issues could occur getting or creating vif port
[Others]
Original description.
neutron-openvswitch-agent meets unbound port:
2019-10-17 11:32:21.868 135 WARNING
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-
aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Device
ef34215f-e099-4fd0-935f-c9a42951d166 not defined on plugin or binding
failed
Later when applying firewall rules:
2019-10-17 11:32:21.901 135 INFO neutron.agent.securitygroups_rpc
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Preparing filters for
devices {'ef34215f-e099-4fd0-935f-c9a42951d166',
'e9c97cf0-1a5e-4d77-b57b-0ba474d12e29', 'fff1bb24-6423-4486-87c4-1fe17c552cca',
'2e20f9ee-bcb5-445c-b31f-d70d276d45c9', '03a60047-cb07-42a4-8b49-619d5982a9bd',
'a452cea2-deaf-4411-bbae-ce83870cbad4', '79b03e5c-9be0-4808-9784-cb4878c3dbd5',
'9b971e75-3c1b-463d-88cf-3f298105fa6e'}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Error while processing VIF
ports: neutron.agent.linux.openvswitch_firewall.exceptions.OVSFWTagNotFound:
Cannot get tag for port o-hm0 from its other_config: {}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 530, in get_or_create_ofport
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port =
self.sg_port_map.ports[port_id]
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
'ef34215f-e099-4fd0-935f-c9a42951d166'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 81, in get_tag_from_other_config
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return
int(other_config['tag'])
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError: 'tag'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2280, in rpc_loop
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info,
provisioning_needed)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/osprofiler/profiler.py", line
160, in wrapper
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result =
f(*args, **kwargs)
2019-10-17
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
@Thanks Seyeong. I've picked the patch directly from the upstream stable/queens branch and uploaded to the bionic unapproved queue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1849098 Title: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1849098/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1849098] Re: ovs agent is stuck with OVSFWTagNotFound when dealing with unbound port
** Also affects: neutron (Ubuntu)
Importance: Undecided
Status: New
** Also affects: neutron (Ubuntu Bionic)
Importance: Undecided
Status: New
** Changed in: neutron (Ubuntu)
Status: New => Fix Released
** Changed in: neutron (Ubuntu Bionic)
Assignee: (unassigned) => Seyeong Kim (seyeongkim)
** Changed in: neutron (Ubuntu Bionic)
Status: New => In Progress
** Patch added: "lp1849098_bionic.debdiff"
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1849098/+attachment/5494377/+files/lp1849098_bionic.debdiff
** Description changed:
+ [Impact]
+
+ somehow port is unbounded, then neutron-openvswitch-agent raise
+ OVSFWTagNotFound, then creating new instance will be failed.
+
+ [Test Plan]
+ 1. deploy bionic openstack env
+ 2. launch one instance
+ 3. modify neutron-openvswitch-agent code inside nova-compute
+ - https://pastebin.ubuntu.com/p/nBRKkXmjx8/
+ 4. restart neutron-openvswitch-agent
+ 5. check if there are a lot of cannot get tag for port ..
+ 6. launch another instance.
+ 7. It fails after vif_plugging_timeout, with "virtual interface creation
failed"
+
+ [Where problems could occur]
+ You need to restart service. and as patch, Basically it will be ok as it adds
only exceptions. but getting or creating vif port part can have issue.
+
+ [Others]
+
+ Original description.
+
neutron-openvswitch-agent meets unbound port:
2019-10-17 11:32:21.868 135 WARNING
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-
aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Device
ef34215f-e099-4fd0-935f-c9a42951d166 not defined on plugin or binding
failed
Later when applying firewall rules:
2019-10-17 11:32:21.901 135 INFO neutron.agent.securitygroups_rpc
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Preparing filters for
devices {'ef34215f-e099-4fd0-935f-c9a42951d166',
'e9c97cf0-1a5e-4d77-b57b-0ba474d12e29', 'fff1bb24-6423-4486-87c4-1fe17c552cca',
'2e20f9ee-bcb5-445c-b31f-d70d276d45c9', '03a60047-cb07-42a4-8b49-619d5982a9bd',
'a452cea2-deaf-4411-bbae-ce83870cbad4', '79b03e5c-9be0-4808-9784-cb4878c3dbd5',
'9b971e75-3c1b-463d-88cf-3f298105fa6e'}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Error while processing VIF
ports: neutron.agent.linux.openvswitch_firewall.exceptions.OVSFWTagNotFound:
Cannot get tag for port o-hm0 from its other_config: {}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 530, in get_or_create_ofport
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port =
self.sg_port_map.ports[port_id]
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
'ef34215f-e099-4fd0-935f-c9a42951d166'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 81, in get_tag_from_other_config
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return
int(other_config['tag'])
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError: 'tag'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2280, in rpc_loop
2019-10-17 11:32:21.906 135 ERROR
