Public bug reported:
Test test_live_migration_with_trunk has been failing for the last two days.
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/testr_results.html
It's a test about live-migration, but it is important to notice that it
fails before any live migration happens.
The test creates a VM with a port and a subport.
The test waits until the VM status is ACTIVE -> this passes
The test waits until the subport status is ACTIVE -> this started failing two
days ago because the port status is DOWN
There was only one neutron patch merged that day[1], but I checked the
test failed during some jobs even before that patch was merged.
I compared some logs.
Neutron logs when the test passes: [2]
Neutron logs when the test fails: [3]
When it fails, I see this during the creation of the subport (and I don't see
this event when it passes):
Jun 14 18:13:43.052982 np0034303809 neutron-server[77531]: DEBUG
ovsdbapp.backend.ovs_idl.event [None req-929dd199-4247-46f5-9466-622c7d538547
None None] Matched DELETE: PortBindingUpdateVirtualPortsEvent(events=('update',
'delete'), table='Port_Binding', conditions=None, old_conditions=None),
priority=20 to row=Port_Binding(parent_port=[], mac=['fa:16:3e:93:9d:5a
19.80.0.42'], chassis=[], ha_chassis_group=[], options={'mcast_flood_reports':
'true', 'requested-chassis': ''}, type=, tag=[], requested_chassis=[],
tunnel_key=2, up=[False], logical_port=f8c707ec-ecd8-4f1e-99ba-6f8303b598b2,
gateway_chassis=[], encap=[], external_ids={'name':
'tempest-subport-2029248863', 'neutron:cidrs': '19.80.0.42/24',
'neutron:device_id': '', 'neutron:device_owner': '', 'neutron:network_name':
'neutron-5fd9faa7-ec1c-4f42-ab87-6ce19edda245', 'neutron:port_capabilities':
'', 'neutron:port_name': 'tempest-subport-2029248863', 'neutron:project_id':
'6f92a9f8e16144148026725b25711d3a', 'neutron:revision_num
ber': '1', 'neutron:security_group_ids':
'5eab41ef-c5c1-425c-a931-f5b6b4b330ad', 'neutron:subnet_pool_addr_scope4': '',
'neutron:subnet_pool_addr_scope6': '', 'neutron:vnic_type': 'normal'},
virtual_parent=[], nat_addresses=[],
datapath=3c472399-d6ee-4b7c-aa97-6777f2bc2772) old= {{(pid=77531) matches
/usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/event.py:43}}
...
Jun 14 18:13:49.597911 np0034303809 neutron-server[77531]: DEBUG
neutron.plugins.ml2.plugin [None req-3588521e-7878-408d-b1f8-15db562c69f8 None
None] Port f8c707ec-ecd8-4f1e-99ba-6f8303b598b2 cannot update to ACTIVE because
it is not bound. {{(pid=77531) _port_provisioned
/opt/stack/neutron/neutron/plugins/ml2/plugin.py:361}}
It seems the ovn version has changed between these jobs:
Passes [4]:
2023-06-14 10:01:46.358875 | controller | Preparing to unpack
.../ovn-common_22.03.0-0ubuntu1_amd64.deb ...
Fails [5]:
2023-06-14 17:55:07.077377 | controller | Preparing to unpack
.../ovn-common_22.03.2-0ubuntu0.22.04.1_amd64.deb ...
[1] https://review.opendev.org/c/openstack/neutron/+/883687
[2]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/controller/logs/screen-q-svc.txt
[3]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/controller/logs/screen-q-svc.txt
[4]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/job-output.txt
[5]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/job-output.txt
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2024160
Title:
[trunk ports] subport doesn't reach status ACTIVE
Status in neutron:
New
Bug description:
Test test_live_migration_with_trunk has been failing for the last two days.
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/testr_results.html
It's a test about live-migration, but it is important to notice that
it fails before any live migration happens.
The test creates a VM with a port and a subport.
The test waits until the VM status is ACTIVE -> this passes
The test waits until the subport status is ACTIVE -> this started failing two
days ago because the port status is DOWN
There was only one neutron patch merged that day[1], but I checked the
test failed during some jobs even before that patch was merged.
I compared some logs.
Neutron logs when the test passes: [2]
Neutron logs when the test fails: [3]
When it fails, I see this during the creation of the subport (and I don't see
this event when it passes):
Jun 14 18:13:43.052982 np0034303809 neutron-server[77531]: DEBUG
ovsdbapp.backend.ovs_idl.event [None req-929dd199-4247-46f5-9466-622c7d538547
None None] Matched DELETE: PortBindingUpdateVirtualPortsEvent(events=('update',
'delete'), table='Port_Binding', conditions=None, old_conditions=None),
priority=20 to row=Port_Binding(parent_port=[], mac=['fa:16:3e:93:9d:5a
19.80.0.42'], chassis=[], ha_chassis_group=[], options={'mcast_flood_reports':
'true', 'requested-chassis': ''}, type=, tag=[], requested_chassis=[],
tunnel_key=2, up=[False], logical_port=f8c707ec-ecd8-4f1e-99ba-6f8303b598b2,
gateway_chassis=[], encap=[], external_ids={'name':
'tempest-subport-2029248863', 'neutron:cidrs': '19.80.0.42/24',
'neutron:device_id': '', 'neutron:device_owner': '', 'neutron:network_name':
'neutron-5fd9faa7-ec1c-4f42-ab87-6ce19edda245', 'neutron:port_capabilities':
'', 'neutron:port_name': 'tempest-subport-2029248863', 'neutron:project_id':
'6f92a9f8e16144148026725b25711d3a', 'neutron:revision_n
umber': '1', 'neutron:security_group_ids':
'5eab41ef-c5c1-425c-a931-f5b6b4b330ad', 'neutron:subnet_pool_addr_scope4': '',
'neutron:subnet_pool_addr_scope6': '', 'neutron:vnic_type': 'normal'},
virtual_parent=[], nat_addresses=[],
datapath=3c472399-d6ee-4b7c-aa97-6777f2bc2772) old= {{(pid=77531) matches
/usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/event.py:43}}
...
Jun 14 18:13:49.597911 np0034303809 neutron-server[77531]: DEBUG
neutron.plugins.ml2.plugin [None req-3588521e-7878-408d-b1f8-15db562c69f8 None
None] Port f8c707ec-ecd8-4f1e-99ba-6f8303b598b2 cannot update to ACTIVE because
it is not bound. {{(pid=77531) _port_provisioned
/opt/stack/neutron/neutron/plugins/ml2/plugin.py:361}}
It seems the ovn version has changed between these jobs:
Passes [4]:
2023-06-14 10:01:46.358875 | controller | Preparing to unpack
.../ovn-common_22.03.0-0ubuntu1_amd64.deb ...
Fails [5]:
2023-06-14 17:55:07.077377 | controller | Preparing to unpack
.../ovn-common_22.03.2-0ubuntu0.22.04.1_amd64.deb ...
[1] https://review.opendev.org/c/openstack/neutron/+/883687
[2]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/controller/logs/screen-q-svc.txt
[3]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/controller/logs/screen-q-svc.txt
[4]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/job-output.txt
[5]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/job-output.txt
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2024160/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
