Public bug reported: Test tempest.api.network.test_networks.BulkNetworkOpsTestJSON.test_bulk_create_delete_port failed when used N1KV plugin. As I see from neutron log, Cisco plugin incorrectly process exceptions in method _get_policy_profile_by_name (file "/opt/stack/neutron/neutron/plugins/cisco/db/n1kv_db_v2.py") it's leads to return code 500 by Neutron service.
Problem reproduced when creating ports and n1kv profile_id not found in database. According Neutron API, for this request allowed only next error codes: - badRequest (400) - unauthorized (401) - forbidden (403) - itemNotFound (404) - macGenerationFailure (503) - serviceUnavailable (503) Test log: 2014-11-17 03:24:07,926 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 201 POST http://10.20.21.2:9696/v2.0/networks 0.406s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"network": {"name": "test-network--197486"}} Response - Headers: {'status': '201', 'content-length': '301', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:07 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-eca84ad6-9ba1-4f22-b668-018201e65660'} Body: {"network": {"status": "ACTIVE", "subnets": [], "name": "test-network--197486", "router:external": false, "tenant_id": "443d9ae88e094a5c9fa9daa36ac4d00f", "admin_state_up": true, "n1kv:profile_id": "2b8937d0-069c-461d-aedb-020ad0e2ed99", "shared": false, "id": "dc7b7044-d3ac-450d-9f73-d87b931170a1"}} 2014-11-17 03:24:08,335 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 201 POST http://10.20.21.2:9696/v2.0/networks 0.408s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"network": {"name": "test-network--380129587"}} Response - Headers: {'status': '201', 'content-length': '304', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:08 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-0f0ca9f6-82c0-4437-906e-005ae6b31377'} Body: {"network": {"status": "ACTIVE", "subnets": [], "name": "test-network--380129587", "router:external": false, "tenant_id": "443d9ae88e094a5c9fa9daa36ac4d00f", "admin_state_up": true, "n1kv:profile_id": "2b8937d0-069c-461d-aedb-020ad0e2ed99", "shared": false, "id": "381d122a-f55c-4c39-9eb9-17d5cd3ef818"}} 2014-11-17 03:24:08,390 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 500 POST http://10.20.21.2:9696/v2.0/ports 0.053s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"ports": [{"network_id": "dc7b7044-d3ac-450d-9f73-d87b931170a1", "name": "port--769863538", "admin_state_up": true}, {"network_id": "381d122a-f55c-4c39-9eb9-17d5cd3ef818", "admin_state_up": false}]} Response - Headers: {'status': '500', 'content-length': '150', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:08 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-7bfbe0d3-de6b-4450-b4ee-7ead1af3fa7f'} Body: {"NeutronError": {"message": "Request Failed: internal server error while processing your request.", "type": "HTTPInternalServerError", "detail": ""}} }}} Traceback (most recent call last): File "tempest/api/network/test_networks.py", line 463, in test_bulk_create_delete_port _, body = self.client.create_bulk_port(port_list) File "tempest/services/network/network_client_base.py", line 207, in create_bulk_port resp, body = self.post(uri, body) File "tempest/services/network/network_client_base.py", line 74, in post return self.rest_client.post(uri, body, headers) File "tempest/common/rest_client.py", line 234, in post return self.request('POST', url, extra_headers, headers, body) File "tempest/common/rest_client.py", line 454, in request resp, resp_body) File "tempest/common/rest_client.py", line 550, in _error_checker raise exceptions.ServerFault(message) ServerFault: Got server fault Details: Request Failed: internal server error while processing your request. Traceback (most recent call last): _StringException: Empty attachments: stderr stdout Neutron log: 2014-11-17 10:15:03.365 ERROR neutron.db.db_base_plugin_v2 [req-c8f914ec-00a8-4734-9557-dd14bc00b1d2 BulkNetworkOpsTestJSON-1775368720 77347f40bf50412eb636412a20c64aaf] An exception occurred while creating the port:{'port': {'binding:host_id': <object object at 0x7f0be269c110>, u'name': u'port--1581734127', u'admin_state_up': True, u'network_id': u'fee9d7af-edec-4d31-aa27-b94f5216ac7e', 'tenant_id': u'77347f40bf50412eb636412a20c64aaf', 'binding:vnic_type': 'normal', 'device_owner': '', 'n1kv:profile_id': <object object at 0x7f0be269c110>, 'mac_address': <object object at 0x7f0be269c110>, 'binding:profile': <object object at 0x7f0be269c110>, 'fixed_ips': <object object at 0x7f0be269c110>, 'device_id': ''}} 2014-11-17 10:15:03.366 ERROR neutron.api.v2.resource [req-c8f914ec-00a8-4734-9557-dd14bc00b1d2 BulkNetworkOpsTestJSON-1775368720 77347f40bf50412eb636412a20c64aaf] create failed 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource Traceback (most recent call last): 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 81, in resource 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource result = method(request=request, **args) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 431, in create 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource objs = obj_creator(request.context, body, **kwargs) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1310, in create_port_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource return self._create_bulk('port', context, ports) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 896, in _create_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource {'resource': resource, 'item': item}) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/openstack/common/excutils.py", line 82, in __exit__ 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 889, in _create_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource objects.append(obj_creator(context, item)) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/models/virt_phy_sw_v2.py", line 235, in create_port 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource args) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/models/virt_phy_sw_v2.py", line 124, in _invoke_plugin_per_device 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource return func(*args, **kwargs) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/n1kv/n1kv_neutron_plugin.py", line 1122, in create_port 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource p_profile = self._get_policy_profile_by_name(p_profile_name) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/db/n1kv_db_v2.py", line 1603, in _get_policy_profile_by_name 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource filter_by(name=name).one()) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2316, in one 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource raise orm_exc.NoResultFound("No row was found for one()") 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource NoResultFound: No row was found for one() 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource Devstack local.conf: [[local|localrc]] MYSQL_PASSWORD=nova RABBIT_PASSWORD=nova SERVICE_TOKEN=nova SERVICE_PASSWORD=nova ADMIN_PASSWORD=nova ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-cond,cinder,c-sch,c-api,c-vol,n-sch,n-novnc,n-xvnc,n-cauth,rabbit disable_service horizon disable_service n-net, q-agt, q-l3 enable_service mysql, q-svc, q-dhcp, q-meta, q-lbaas, neutron, tempest VOLUME_BACKING_FILE_SIZE=2052M Q_PLUGIN=cisco declare -a Q_CISCO_PLUGIN_SUBPLUGINS=(n1kv) Q_CISCO_PLUGIN_DEVSTACK_VSM=False Q_CISCO_PLUGIN_VSM_IP=192.168.1.10 Q_CISCO_PLUGIN_VSM_USERNAME=admin Q_CISCO_PLUGIN_VSM_PASSWORD=Admin12345 Q_CISCO_PLUGIN_UVEM_DEB_IMAGE=nexus_1000v_vem-12.04-5.2.1.SK1.2.1.29.S0-1.deb Q_CISCO_PLUGIN_INTEGRATION_BRIDGE=br-int Q_CISCO_PLUGIN_HOST_MGMT_INTF=eth0 PHYSICAL_NETWORK=test-physnet1 LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver API_RATE_LIMIT=False VERBOSE=True DEBUG=True ** Affects: neutron Importance: Undecided Status: New ** Tags: cisco n1kv ** Tags added: cisco ** Tags added: n1kv -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1393399 Title: Cisco N1KV plugin doesn't process correctly SQLAlchemy exceptions Status in OpenStack Neutron (virtual network service): New Bug description: Test tempest.api.network.test_networks.BulkNetworkOpsTestJSON.test_bulk_create_delete_port failed when used N1KV plugin. As I see from neutron log, Cisco plugin incorrectly process exceptions in method _get_policy_profile_by_name (file "/opt/stack/neutron/neutron/plugins/cisco/db/n1kv_db_v2.py") it's leads to return code 500 by Neutron service. Problem reproduced when creating ports and n1kv profile_id not found in database. According Neutron API, for this request allowed only next error codes: - badRequest (400) - unauthorized (401) - forbidden (403) - itemNotFound (404) - macGenerationFailure (503) - serviceUnavailable (503) Test log: 2014-11-17 03:24:07,926 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 201 POST http://10.20.21.2:9696/v2.0/networks 0.406s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"network": {"name": "test-network--197486"}} Response - Headers: {'status': '201', 'content-length': '301', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:07 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-eca84ad6-9ba1-4f22-b668-018201e65660'} Body: {"network": {"status": "ACTIVE", "subnets": [], "name": "test-network--197486", "router:external": false, "tenant_id": "443d9ae88e094a5c9fa9daa36ac4d00f", "admin_state_up": true, "n1kv:profile_id": "2b8937d0-069c-461d-aedb-020ad0e2ed99", "shared": false, "id": "dc7b7044-d3ac-450d-9f73-d87b931170a1"}} 2014-11-17 03:24:08,335 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 201 POST http://10.20.21.2:9696/v2.0/networks 0.408s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"network": {"name": "test-network--380129587"}} Response - Headers: {'status': '201', 'content-length': '304', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:08 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-0f0ca9f6-82c0-4437-906e-005ae6b31377'} Body: {"network": {"status": "ACTIVE", "subnets": [], "name": "test-network--380129587", "router:external": false, "tenant_id": "443d9ae88e094a5c9fa9daa36ac4d00f", "admin_state_up": true, "n1kv:profile_id": "2b8937d0-069c-461d-aedb-020ad0e2ed99", "shared": false, "id": "381d122a-f55c-4c39-9eb9-17d5cd3ef818"}} 2014-11-17 03:24:08,390 18176 DEBUG [tempest.common.rest_client] Request (BulkNetworkOpsTestJSON:test_bulk_create_delete_port): 500 POST http://10.20.21.2:9696/v2.0/ports 0.053s Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: {"ports": [{"network_id": "dc7b7044-d3ac-450d-9f73-d87b931170a1", "name": "port--769863538", "admin_state_up": true}, {"network_id": "381d122a-f55c-4c39-9eb9-17d5cd3ef818", "admin_state_up": false}]} Response - Headers: {'status': '500', 'content-length': '150', 'connection': 'close', 'date': 'Mon, 17 Nov 2014 03:24:08 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-7bfbe0d3-de6b-4450-b4ee-7ead1af3fa7f'} Body: {"NeutronError": {"message": "Request Failed: internal server error while processing your request.", "type": "HTTPInternalServerError", "detail": ""}} }}} Traceback (most recent call last): File "tempest/api/network/test_networks.py", line 463, in test_bulk_create_delete_port _, body = self.client.create_bulk_port(port_list) File "tempest/services/network/network_client_base.py", line 207, in create_bulk_port resp, body = self.post(uri, body) File "tempest/services/network/network_client_base.py", line 74, in post return self.rest_client.post(uri, body, headers) File "tempest/common/rest_client.py", line 234, in post return self.request('POST', url, extra_headers, headers, body) File "tempest/common/rest_client.py", line 454, in request resp, resp_body) File "tempest/common/rest_client.py", line 550, in _error_checker raise exceptions.ServerFault(message) ServerFault: Got server fault Details: Request Failed: internal server error while processing your request. Traceback (most recent call last): _StringException: Empty attachments: stderr stdout Neutron log: 2014-11-17 10:15:03.365 ERROR neutron.db.db_base_plugin_v2 [req-c8f914ec-00a8-4734-9557-dd14bc00b1d2 BulkNetworkOpsTestJSON-1775368720 77347f40bf50412eb636412a20c64aaf] An exception occurred while creating the port:{'port': {'binding:host_id': <object object at 0x7f0be269c110>, u'name': u'port--1581734127', u'admin_state_up': True, u'network_id': u'fee9d7af-edec-4d31-aa27-b94f5216ac7e', 'tenant_id': u'77347f40bf50412eb636412a20c64aaf', 'binding:vnic_type': 'normal', 'device_owner': '', 'n1kv:profile_id': <object object at 0x7f0be269c110>, 'mac_address': <object object at 0x7f0be269c110>, 'binding:profile': <object object at 0x7f0be269c110>, 'fixed_ips': <object object at 0x7f0be269c110>, 'device_id': ''}} 2014-11-17 10:15:03.366 ERROR neutron.api.v2.resource [req-c8f914ec-00a8-4734-9557-dd14bc00b1d2 BulkNetworkOpsTestJSON-1775368720 77347f40bf50412eb636412a20c64aaf] create failed 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource Traceback (most recent call last): 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/resource.py", line 81, in resource 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource result = method(request=request, **args) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/api/v2/base.py", line 431, in create 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource objs = obj_creator(request.context, body, **kwargs) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1310, in create_port_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource return self._create_bulk('port', context, ports) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 896, in _create_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource {'resource': resource, 'item': item}) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/openstack/common/excutils.py", line 82, in __exit__ 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 889, in _create_bulk 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource objects.append(obj_creator(context, item)) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/models/virt_phy_sw_v2.py", line 235, in create_port 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource args) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/models/virt_phy_sw_v2.py", line 124, in _invoke_plugin_per_device 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource return func(*args, **kwargs) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/n1kv/n1kv_neutron_plugin.py", line 1122, in create_port 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource p_profile = self._get_policy_profile_by_name(p_profile_name) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/opt/stack/neutron/neutron/plugins/cisco/db/n1kv_db_v2.py", line 1603, in _get_policy_profile_by_name 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource filter_by(name=name).one()) 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2316, in one 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource raise orm_exc.NoResultFound("No row was found for one()") 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource NoResultFound: No row was found for one() 2014-11-17 10:15:03.366 TRACE neutron.api.v2.resource Devstack local.conf: [[local|localrc]] MYSQL_PASSWORD=nova RABBIT_PASSWORD=nova SERVICE_TOKEN=nova SERVICE_PASSWORD=nova ADMIN_PASSWORD=nova ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-cond,cinder,c-sch,c-api,c-vol,n-sch,n-novnc,n-xvnc,n-cauth,rabbit disable_service horizon disable_service n-net, q-agt, q-l3 enable_service mysql, q-svc, q-dhcp, q-meta, q-lbaas, neutron, tempest VOLUME_BACKING_FILE_SIZE=2052M Q_PLUGIN=cisco declare -a Q_CISCO_PLUGIN_SUBPLUGINS=(n1kv) Q_CISCO_PLUGIN_DEVSTACK_VSM=False Q_CISCO_PLUGIN_VSM_IP=192.168.1.10 Q_CISCO_PLUGIN_VSM_USERNAME=admin Q_CISCO_PLUGIN_VSM_PASSWORD=Admin12345 Q_CISCO_PLUGIN_UVEM_DEB_IMAGE=nexus_1000v_vem-12.04-5.2.1.SK1.2.1.29.S0-1.deb Q_CISCO_PLUGIN_INTEGRATION_BRIDGE=br-int Q_CISCO_PLUGIN_HOST_MGMT_INTF=eth0 PHYSICAL_NETWORK=test-physnet1 LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver API_RATE_LIMIT=False VERBOSE=True DEBUG=True To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1393399/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
