[Yahoo-eng-team] [Bug 1811352] [NEW] Include neutron CLI floatingip port-forwarding support
Public bug reported: The floating ip port-forwarding is supported since Rocky by neutron API: https://developer.openstack.org/api-ref/network/v2/index.html?expanded =create-port-forwarding-detail#floating-ips-port-forwarding but the neutron client does not include this option yet. Also floatingip-update method is missing. It should include an option similar to this for floatingip *-create/*-update $ neutron floatingip-create --port-forwarding protocol=tcp,internal_port_id=,internal_port=,external_port= You should be able to repeat --port-forwarding several times Also for floatingip-update with and extra option: --port-forwarding clean To remove the current port-forwarding rules list. * Version: OpenStack Rocky ** Affects: neutron Importance: Undecided Status: New ** Description changed: - - The floating ip port-forwarding is supported since Rocy by neutron API: + The floating ip port-forwarding is supported since Rocky by neutron API: https://developer.openstack.org/api-ref/network/v2/index.html?expanded =create-port-forwarding-detail#floating-ips-port-forwarding but the neutron client does not include this option yet. Also floatingip-update method is missing. - - It should include an option similar to this for floatingip *-create/*-update + It should include an option similar to this for floatingip + *-create/*-update $ neutron floatingip-create --port-forwarding protocol=tcp,internal_port_id=,internal_port=,external_port= You should be able to repeat --port-forwarding several times Also for floatingip-update with and extra option: --port-forwarding clean To remove the current port-forwarding rules list. * Version: OpenStack Rocky -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1811352 Title: Include neutron CLI floatingip port-forwarding support Status in neutron: New Bug description: The floating ip port-forwarding is supported since Rocky by neutron API: https://developer.openstack.org/api-ref/network/v2/index.html?expanded =create-port-forwarding-detail#floating-ips-port-forwarding but the neutron client does not include this option yet. Also floatingip-update method is missing. It should include an option similar to this for floatingip *-create/*-update $ neutron floatingip-create --port-forwarding protocol=tcp,internal_port_id=,internal_port=,external_port= You should be able to repeat --port-forwarding several times Also for floatingip-update with and extra option: --port-forwarding clean To remove the current port-forwarding rules list. * Version: OpenStack Rocky To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1811352/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1798796] Re: libvirt: Use VIR_MIGRATE_TLS to get QEMU's native TLS support for migration and NBD
Reviewed: https://review.openstack.org/625216 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9160fe50987131feda9429c4e95d573e176916b6 Submitter: Zuul Branch:master commit 9160fe50987131feda9429c4e95d573e176916b6 Author: Kashyap Chamarthy Date: Wed Dec 12 16:51:52 2018 +0100 libvirt: Support native TLS for migration and disks over NBD The encryption offered by Nova (via `live_migration_tunnelled`, i.e. "tunnelling via libvirtd") today secures only two migration streams: guest RAM and device state; but it does _not_ encrypt the NBD (Network Block Device) transport—which is used to migrate disks that are on non-shared storage setup (also called: "block migration"). Further, the "tunnelling via libvirtd" has a huge performance penalty and latency, because it burns more CPU and memory bandwidth due to increased number of data copies on both source and destination hosts. To solve this existing limitation, introduce a new config option `live_migration_with_native_tls`, which will take advantage of "native TLS" (i.e. TLS built into QEMU, and relevant support in libvirt). The native TLS transport will encrypt all migration streams, *including* disks that are not on shared storage — all of this without incurring the limitations of the "tunnelled via libvirtd" transport. Closes-Bug: #1798796 Blueprint: support-qemu-native-tls-for-live-migration Change-Id: I78f5fef41b6fbf118880cc8aa4036d904626b342 Signed-off-by: Kashyap Chamarthy ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1798796 Title: libvirt: Use VIR_MIGRATE_TLS to get QEMU's native TLS support for migration and NBD Status in OpenStack Compute (nova): Fix Released Bug description: Make Nova's libvirt driver use libvirt's VIR_MIGRATE_TLS, which will transport a Nova instance's migration and NBD data streams via QEMU's native TLS. Rationale - From a downstream bug description by Dan Berrangé: "The default QEMU migration transport runs a clear text TCP connection between the two QEMU servers. It is possible to tunnel the migration connection over libvirtd's secure connection but this imposes a significant performance penalty. It is also not possible to tunnel the NBD connection use for block migration at all. "As a step towards securing the management network we need to have Nova configure QEMU to use native TLS support on its migration and NBD data transports, without any tunnelling." Minimum version requirements for this feature to work: QEMU == 2.9 libvirt == v.4.4.0 * * * Broader context and background here: https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg00529.html RFC: Universal encryption on QEMU I/O channels To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1798796/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1811390] [NEW] L2/L3 Network components creation with wrong tenant-id should be restricted
Public bug reported:
I am trying to create vlan_transparent enabled network on NSXT + openstack
queens setup.
While creating network I am providing wrong tenant id, but instead of failing
it is creating that network with
provided tenant id.
nicira@utu1604template:~/devstack$ openstack project list
+--++
| ID | Name |
+--++
| 0906736f01d948318ad5c89e45a04076 | admin |
| 19d3974dadb04aeeac363086a7e6b5bf | alt_demo |
| ab633c528a7a40a089c02b27a7495038 | invisible_to_admin |
| dd8213720a5a4e5c85db304e8992d3c1 | service|
| f36b83bc02074eacacef7a15b48690b0 | demo |
+--++
nicira@utu1604template:~/devstack$ neutron net-create
--provider:network_type=vlan --vlan-transparent true --tenant-id
7838ggf2372d2139fgf922ff TestNet
neutron CLI is deprecated and will be removed in the future. Use openstack CLI
instead.
Created a new network:
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| availability_zone_hints | |
| availability_zones| defaultv3|
| created_at| 2018-06-12T09:23:09Z |
| description | |
| dns_domain| |
| id| 38c7eb87-6d83-479a-a56e-da7c27d36f15 |
| ipv4_address_scope| |
| ipv6_address_scope| |
| name | TestNet |
| port_security_enabled | True |
| project_id| 7838ggf2372d2139fgf922ff |
| provider:network_type | vlan |
| provider:physical_network | 060d4788-19ae-4ba5-a369-4bb2079f50eb |
| provider:segmentation_id | 0|
| qos_policy_id | |
| revision_number | 3|
| router:external | False|
| shared| False|
| status| ACTIVE |
| subnets | |
| tags | |
| tenant_id | 7838ggf2372d2139fgf922ff |
| updated_at| 2018-06-12T09:23:09Z |
| vlan_transparent | True |
+---+--+
I am facing same issue while creating subnet also.
nicira@utu1604template:~/devstack$ neutron subnet-create --tenant-id
89325t389256932532506329jsfhkjsfgwsjfbwejf --name testSubnet --disable-dhcp
dee859b4-fae6-429e-b290-9711ec205da2 20.0.0.0/24
neutron CLI is deprecated and will be removed in the future. Use openstack CLI
instead.
Created a new subnet:
+---++
| Field | Value |
+---++
| allocation_pools | {"start": "20.0.0.2", "end": "20.0.0.254"} |
| cidr | 20.0.0.0/24|
| created_at| 2018-06-12T09:38:41Z |
| description ||
| dns_nameservers ||
| enable_dhcp | False |
| gateway_ip| 20.0.0.1 |
| host_routes ||
| id| 51a77524-1b5a-4ab1-b98b-c170f656d6df |
| ip_version| 4 |
| ipv6_address_mode ||
| ipv6_ra_mode ||
| name | testSubnet |
| network_id| dee859b4-fae6-429e-b290-9711ec205da2 |
| project_id| 89325t389256932532506329jsfhkjsfgwsjfbwejf |
| revision_number | 0 |
| subnetpool_id ||
| tags ||
| tenant_id | 89325t389256932532506329jsfhkjsfgwsjfbwejf |
| updated_at| 2018-06-12T09:38:41Z |
[Yahoo-eng-team] [Bug 1811446] Re: chpasswd: is mangling certain password hashes
** Also affects: cloud-init (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1811446
Title:
chpasswd: is mangling certain password hashes
Status in cloud-init:
New
Status in cloud-init package in Ubuntu:
New
Bug description:
#cloud-config
# from 1 files
# part-001
---
chpasswd:
expire: false
list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va
'
From #cloud-init
Hey there, I'm not sure whether I'm running into a bug or not
I'm trying to set the password hash for the root user on a system
using the chpasswd module
It should match new hash at this line in the module but it doens't
seem to match
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
I can confirm this when running it through
https://regex101.com/r/Nj7VTZ/1
Then I was thinking, isn't [] for lists of characters rather than
lists of strings
Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
At least in regex101
smoser, you any idea, I saw you commited the change:
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
marlinc_: i'd think yes. that is a bug for the '2a' and '2y'
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1811446/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1811458] [NEW] Add per boot network configuration for OpenStack datasource
Public bug reported: The OpenStack datasource should support the capability of regenerating network configuration at boot. With Bionic, if a primary interface is removed from an instance and a new interface is added, the instance will fail to reconfigure the new interface unless it is assigned the same MAC address. ** Affects: cloud-init Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1811458 Title: Add per boot network configuration for OpenStack datasource Status in cloud-init: New Bug description: The OpenStack datasource should support the capability of regenerating network configuration at boot. With Bionic, if a primary interface is removed from an instance and a new interface is added, the instance will fail to reconfigure the new interface unless it is assigned the same MAC address. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1811458/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1791296] Re: Browsing /horizon/admin/info/ fails if Neutron not installed
Reviewed: https://review.openstack.org/627189 Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=26b838b7eb17189d1be49bd6d7fa0db0b1c5eed0 Submitter: Zuul Branch:master commit 26b838b7eb17189d1be49bd6d7fa0db0b1c5eed0 Author: Akihiro Motoki Date: Mon Dec 24 23:30:17 2018 +0900 Do not instantiate tables when a tab is not allowed This commit chnages TableTab class not to instantiate tables when the tab is not allowed. Previously table(s) in a tab is always instatiated regardless of that the tab is allowed or not. We usually hide a tab using "allowed" method or "permissions" attribute of the tab. Table(s) in the tab usually assumes that its dependencies are satisfied and they are already checked by allowed or permissions. Change-Id: I51d92221fdf02d836e1dc9343e85a69109159521 Closes-Bug: #1791296 ** Changed in: horizon Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1791296 Title: Browsing /horizon/admin/info/ fails if Neutron not installed Status in OpenStack Dashboard (Horizon): Fix Released Bug description: When browsing to Admin -> System -> System information, as an admin user, if there's no Neutron installed, then Horizon just crashes: [ ... snip ... ] File "/usr/lib/python3/dist-packages/horizon/utils/memoized.py" in wrapped 176. args.insert(request_index, request_func(request)) File "/usr/share/openstack-dashboard/openstack_dashboard/api/neutron.py" in get_auth_params_from_request 804. base.url_for(request, 'network'), File "/usr/share/openstack-dashboard/openstack_dashboard/api/base.py" in url_for 347. raise exceptions.ServiceCatalogException(service_type) Exception Type: ServiceCatalogException at /admin/info/ Exception Value: Invalid service catalog: network After installing Neutron, then things do display correctly. It'd be nice to fix this. Note, if this matters, I was using the Rocky package 3:14.0.0-2 under Debian Sid to do this test. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1791296/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1789402] Re: Security group rule - Other protocol
Reviewed: https://review.openstack.org/627045 Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=ccb21ca7a9906523a972e32df904744885183234 Submitter: Zuul Branch:master commit ccb21ca7a9906523a972e32df904744885183234 Author: Akihiro Motoki Date: Sun Dec 23 00:36:44 2018 +0900 python3: Fix handling of other protocol in SG rule In python3 we cannot compare None with an integer, while this works in python2. Change-Id: I1321ea68f08241db377a58ed6a22306c63aba204 Closes-Bug: #1789402 ** Changed in: horizon Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1789402 Title: Security group rule - Other protocol Status in OpenStack Dashboard (Horizon): Fix Released Bug description: Queens, horizon 13.0.1 If try to create rule with other ip protocol then exception rise. Cause of problem in openstack_dashboard/api/neutron.py https://github.com/openstack/horizon/blob/841bce062327d70fb834b99bc2f238323e400a51/openstack_dashboard/api/neutron.py#L418-L441 If we define ip_protocol it mean that we didn't define to_port and from_port. It means that comparison default values of to_port/from_port(None) with 0 will rise exception. It can be fixed with something like this: if not cidr: cidr = None +if from_port != None: if from_port < 0: from_port = None +if to_port != None: if to_port < 0: to_port = None To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1789402/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1811446] [NEW] chpasswd: is mangling certain password hashes
Public bug reported:
#cloud-config
# from 1 files
# part-001
---
chpasswd:
expire: false
list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va
'
>From #cloud-init
Hey there, I'm not sure whether I'm running into a bug or not
I'm trying to set the password hash for the root user on a system
using the chpasswd module
It should match new hash at this line in the module but it doens't
seem to match
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
I can confirm this when running it through
https://regex101.com/r/Nj7VTZ/1
Then I was thinking, isn't [] for lists of characters rather than
lists of strings
Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
At least in regex101
smoser, you any idea, I saw you commited the change:
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
marlinc_: i'd think yes. that is a bug for the '2a' and '2y'
** Affects: cloud-init
Importance: Undecided
Status: New
** Merge proposal linked:
https://code.launchpad.net/~marlinc/cloud-init/+git/cloud-init/+merge/361683
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1811446
Title:
chpasswd: is mangling certain password hashes
Status in cloud-init:
New
Bug description:
#cloud-config
# from 1 files
# part-001
---
chpasswd:
expire: false
list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va
'
From #cloud-init
Hey there, I'm not sure whether I'm running into a bug or not
I'm trying to set the password hash for the root user on a system
using the chpasswd module
It should match new hash at this line in the module but it doens't
seem to match
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
I can confirm this when running it through
https://regex101.com/r/Nj7VTZ/1
Then I was thinking, isn't [] for lists of characters rather than
lists of strings
Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
At least in regex101
smoser, you any idea, I saw you commited the change:
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
marlinc_: i'd think yes. that is a bug for the '2a' and '2y'
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1811446/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1811455] [NEW] QoS plugin fails if network is not found
Public bug reported: Master neutron (Stein): We are intermittently seeing gate failures with a q-svc exception: Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server [None req-9e4027ef-c0b5-4d46-99be-1a1da640c506 None None] Exception during message handling: AttributeError: 'NoneType' object has no attribute 'qos_policy_id' Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server Traceback (most recent call last): Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py", line 146, in get_active_networks_info Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server ports = plugin.get_ports(context, filters=filters) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 233, in wrapped Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server return method(*args, **kwargs) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 140, in wrapped Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server setattr(e, '_RETRY_EXCEEDED', True) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server self.force_reraise() Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/six.py", line 693, in reraise Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server raise value Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/neutron_lib/db/api.py", line 136, in wrapped Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server return f(*args, **kwargs) Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_db/api.py", line 154, in wrapper Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server ectxt.value = e.inner_exc Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server self.force_reraise() Jan 11 01:12:04.832940 ubuntu-bionic-ovh-bhs1-0001629903 neutron-server[15297]: ERROR oslo_messaging.rpc.server File
[Yahoo-eng-team] [Bug 1811405] [NEW] OVSFWTagNotFound raised many times in neutron-openvswitch-agent logs
Public bug reported:
I found out that recently exception like:
Jan 11 08:56:11.074296 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None
req-207e9202-dfc1-4b0d-b9d4-85eaec6174ee None None] Error while processing VIF
ports: neutron.agent.linux.openvswitch_firewall.exceptions.OVSFWTagNotFound:
Cannot get tag for port tap20acc597-23 from its other_config: {}
Jan 11 08:56:11.074665 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
Jan 11 08:56:11.074999 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 526, in get_or_create_ofport
Jan 11 08:56:11.075349 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port =
self.sg_port_map.ports[port_id]
Jan 11 08:56:11.075713 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
'20acc597-23f0-4c18-980e-061aa6494ccf'
Jan 11 08:56:11.076036 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
Jan 11 08:56:11.077854 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
Jan 11 08:56:11.078269 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
Jan 11 08:56:11.078611 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
Jan 11 08:56:11.079337 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 79, in get_tag_from_other_config
Jan 11 08:56:11.079670 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return
int(other_config['tag'])
Jan 11 08:56:11.08 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError: 'tag'
Jan 11 08:56:11.080351 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
Jan 11 08:56:11.080677 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
Jan 11 08:56:11.081027 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
Jan 11 08:56:11.081354 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
Jan 11 08:56:11.081681 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2196, in rpc_loop
Jan 11 08:56:11.082077 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info,
provisioning_needed)
Jan 11 08:56:11.082475 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/usr/local/lib/python3.5/dist-packages/osprofiler/profiler.py", line 160, in
wrapper
Jan 11 08:56:11.082907 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result =
f(*args, **kwargs)
Jan 11 08:56:11.083233 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1770, in process_network_ports
Jan 11 08:56:11.083560 ubuntu-xenial-rax-dfw-0001639322
neutron-openvswitch-agent[13907]: ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
port_info.get('updated', set()))
Jan 11 08:56:11.083903
[Yahoo-eng-team] [Bug 1809823] [NEW] Neutron_api (unhealthy) after few days
You have been subscribed to a public bug:
Description
===
on the undercloud ( pretty sure we also seen it on overcloud, i'll update when
sure )
Without any action, we notice that neutron_api service is in "unhealthy" state
and stop functioning.
Log shows -
2018-12-26 00:00:35.774 7 INFO oslo_service.service [-] Caught SIGHUP, stopping
children
2018-12-26 00:00:36.077 40997 ERROR oslo_service.service [-] Error starting
thread.: RuntimeError: A fixed interval looping call can only run one function
at a time
openstack commands that needs neutron fails ( e.g openstack server list
)
Restarting the docker ( neutron_api ) resolves the problem.
Steps to reproduce
==
Deploy.
Wait 4 days.
Expected result
===
Service should remain healthy..
Actual result
=
not healthy ..
Environment
===
Rocky , container based.
Logs & Configs
==
Logs : http://paste.openstack.org/show/738658/
More info:
==
Google showed this -
https://bugs.launchpad.net/oslo.service/+bug/1547029
follow by -
http://paste.openstack.org/show/487420/
It seems that if we'll add "eventlet.sleep(0)" in <<>> below, it
might resolve the issue. -
def run_service(service, done):
"""Service start wrapper.
:param service: service to run
:param done: event to wait on until a shutdown is triggered
:returns: None
"""
try:
< HERE
service.start()
except Exception:
LOG.exception('Error starting thread.')
raise SystemExit(1)
else:
done.wait()
The problem is that I didnt come up with an easy way to reproduce the issue in
order to confirm it.
Any suggestions ?
** Affects: oslo.service
Importance: High
Status: Confirmed
** Affects: neutron
Importance: Undecided
Assignee: Yossi Ovadia (jabadia)
Status: New
--
Neutron_api (unhealthy) after few days
https://bugs.launchpad.net/bugs/1809823
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1809823] Re: Neutron_api (unhealthy) after few days
UPDATE:
to reproduce the bug:
log into the neutron_api container :
docker exec -it --user root neutron_api bash
ps fax| grep neutron_api
()[root@undercloud /]# ps fax
PID TTY STAT TIME COMMAND
115 ?Ss 0:00 bash
140 ?S+ 0:00 \_ top
44 ?Ss 0:00 bash
241 ?R+ 0:00 \_ ps fax
1 ?Ss 0:00 /usr/local/bin/dumb-init /bin/bash
/usr/local/bin/kolla_sta
7 ?Ss 0:05 /usr/bin/python2 /usr/bin/neutron-server
--config-file /usr
27 ?S 0:08 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
28 ?S 0:00 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
29 ?S 0:03 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
30 ?S 0:03 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
31 ?S 0:03 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
32 ?R 5:27 \_ /usr/bin/python2 /usr/bin/neutron-server
--config-file
Kill the last on pid ( 32 ) with sigup-
kill -1 32
check the server.log after few seconds :
2018-12-26 00:00:36.077 40997 ERROR oslo_service.service [-] Error starting
thread.: RuntimeError: A fixed interval looping call can only run one function
at a time
in our environment, this occurs without someone issuing kill -1 , but just
after 4 days more or less there's a sigup and docker becomes unhealthy.
** Project changed: tripleo => neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1809823
Title:
Neutron_api (unhealthy) after few days
Status in neutron:
New
Status in oslo.service:
Confirmed
Bug description:
Description
===
on the undercloud ( pretty sure we also seen it on overcloud, i'll update
when sure )
Without any action, we notice that neutron_api service is in "unhealthy"
state and stop functioning.
Log shows -
2018-12-26 00:00:35.774 7 INFO oslo_service.service [-] Caught SIGHUP,
stopping children
2018-12-26 00:00:36.077 40997 ERROR oslo_service.service [-] Error starting
thread.: RuntimeError: A fixed interval looping call can only run one function
at a time
openstack commands that needs neutron fails ( e.g openstack server
list )
Restarting the docker ( neutron_api ) resolves the problem.
Steps to reproduce
==
Deploy.
Wait 4 days.
Expected result
===
Service should remain healthy..
Actual result
=
not healthy ..
Environment
===
Rocky , container based.
Logs & Configs
==
Logs : http://paste.openstack.org/show/738658/
More info:
==
Google showed this -
https://bugs.launchpad.net/oslo.service/+bug/1547029
follow by -
http://paste.openstack.org/show/487420/
It seems that if we'll add "eventlet.sleep(0)" in <<>> below, it
might resolve the issue. -
def run_service(service, done):
"""Service start wrapper.
:param service: service to run
:param done: event to wait on until a shutdown is triggered
:returns: None
"""
try:
< HERE
service.start()
except Exception:
LOG.exception('Error starting thread.')
raise SystemExit(1)
else:
done.wait()
The problem is that I didnt come up with an easy way to reproduce the issue
in order to confirm it.
Any suggestions ?
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1809823/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
