[Yahoo-eng-team] [Bug 1488282] Re: Gate failures with 'the resource could not be found'
[Expired for neutron because there has been no activity for 60 days.] ** Changed in: neutron Status: Incomplete => Expired -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1488282 Title: Gate failures with 'the resource could not be found' Status in neutron: Expired Bug description: There have been spurious failures happening in the gate. The most prominent one is: ft1.186: tempest.api.compute.admin.test_servers.ServersAdminTestJSON.test_list_servers_by_admin_with_all_tenants[id-9f5579ae-19b4-4985-a091-2a5d56106580]_StringException: Empty attachments: stderr stdout pythonlogging:'': {{{ 2015-08-24 22:55:50,083 32355 INFO [tempest_lib.common.rest_client] Request (ServersAdminTestJSON:test_list_servers_by_admin_with_all_tenants): 404 GET http://127.0.0.1:8774/v2/fb99c79318b54e668713b25afc52f81a/servers/detail?all_tenants= 0.834s 2015-08-24 22:55:50,083 32355 DEBUG[tempest_lib.common.rest_client] Request - Headers: {'X-Auth-Token': '', 'Accept': 'application/json', 'Content-Type': 'application/json'} Body: None Response - Headers: {'content-length': '78', 'date': 'Mon, 24 Aug 2015 22:55:50 GMT', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-387b21a9-4ada-48ee-89ed-9acfe5274ef7', 'status': '404'} Body: {"itemNotFound": {"message": "The resource could not be found.", "code": 404}} }}} Traceback (most recent call last): File "tempest/api/compute/admin/test_servers.py", line 81, in test_list_servers_by_admin_with_all_tenants body = self.client.list_servers(detail=True, **params) File "tempest/services/compute/json/servers_client.py", line 159, in list_servers resp, body = self.get(url) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 271, in get return self.request('GET', url, extra_headers, headers) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 643, in request resp, resp_body) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 695, in _error_checker raise exceptions.NotFound(resp_body) tempest_lib.exceptions.NotFound: Object not found Details: {u'code': 404, u'message': u'The resource could not be found.'} but there are other similar failure modes. This seems to be related to bug #1269284 The logstash query: message:"tempest_lib.exceptions.NotFound: Object not found" AND build_name:"gate-tempest-dsvm-neutron-full" http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwidGVtcGVzdF9saWIuZXhjZXB0aW9ucy5Ob3RGb3VuZDogT2JqZWN0IG5vdCBmb3VuZFwiIEFORCBidWlsZF9uYW1lOlwiZ2F0ZS10ZW1wZXN0LWRzdm0tbmV1dHJvbi1mdWxsXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0NDA0NjIwNzcyMjksIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0= To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1488282/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1488282] Re: Gate failures with 'the resource could not be found'
This doesn't show up in the gate queue for a while now, other occurrences might be self inflicted. Marking Incomplete until this starts biting again. ** No longer affects: nova ** Changed in: neutron Assignee: Salvatore Orlando (salvatore-orlando) => (unassigned) ** Changed in: neutron Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1488282 Title: Gate failures with 'the resource could not be found' Status in neutron: Incomplete Bug description: There have been spurious failures happening in the gate. The most prominent one is: ft1.186: tempest.api.compute.admin.test_servers.ServersAdminTestJSON.test_list_servers_by_admin_with_all_tenants[id-9f5579ae-19b4-4985-a091-2a5d56106580]_StringException: Empty attachments: stderr stdout pythonlogging:'': {{{ 2015-08-24 22:55:50,083 32355 INFO [tempest_lib.common.rest_client] Request (ServersAdminTestJSON:test_list_servers_by_admin_with_all_tenants): 404 GET http://127.0.0.1:8774/v2/fb99c79318b54e668713b25afc52f81a/servers/detail?all_tenants= 0.834s 2015-08-24 22:55:50,083 32355 DEBUG[tempest_lib.common.rest_client] Request - Headers: {'X-Auth-Token': '', 'Accept': 'application/json', 'Content-Type': 'application/json'} Body: None Response - Headers: {'content-length': '78', 'date': 'Mon, 24 Aug 2015 22:55:50 GMT', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-387b21a9-4ada-48ee-89ed-9acfe5274ef7', 'status': '404'} Body: {"itemNotFound": {"message": "The resource could not be found.", "code": 404}} }}} Traceback (most recent call last): File "tempest/api/compute/admin/test_servers.py", line 81, in test_list_servers_by_admin_with_all_tenants body = self.client.list_servers(detail=True, **params) File "tempest/services/compute/json/servers_client.py", line 159, in list_servers resp, body = self.get(url) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 271, in get return self.request('GET', url, extra_headers, headers) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 643, in request resp, resp_body) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 695, in _error_checker raise exceptions.NotFound(resp_body) tempest_lib.exceptions.NotFound: Object not found Details: {u'code': 404, u'message': u'The resource could not be found.'} but there are other similar failure modes. This seems to be related to bug #1269284 The logstash query: message:"tempest_lib.exceptions.NotFound: Object not found" AND build_name:"gate-tempest-dsvm-neutron-full" http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwidGVtcGVzdF9saWIuZXhjZXB0aW9ucy5Ob3RGb3VuZDogT2JqZWN0IG5vdCBmb3VuZFwiIEFORCBidWlsZF9uYW1lOlwiZ2F0ZS10ZW1wZXN0LWRzdm0tbmV1dHJvbi1mdWxsXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0NDA0NjIwNzcyMjksIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0= To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1488282/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1488282] Re: Gate failures with 'the resource could not be found'
Actually the root cause for the failure I did observe was different. This appears to be a genuine nova error where a server is deleted by another test while the list operation is in progress. It is also interesting that nova fails with a 404 here - this appears to really be a bug. In support of my thesis I can provide examples where the same failure trace occurs with[1], [2] and without [3], [4] neutron Also, during a server list operation there is no interaction between nova and neutron. [1] http://logs.openstack.org/04/215604/15/gate/gate-tempest-dsvm-neutron-full/37eb7aa/console.html [2] http://logs.openstack.org/04/215604/15/gate/gate-tempest-dsvm-neutron-full/37eb7aa/logs/screen-n-api.txt.gz#_2015-08-26_15_32_57_698 [3] http://logs.openstack.org/67/214067/2/gate/gate-tempest-dsvm-full/1b348a3/console.html [4] http://logs.openstack.org/67/214067/2/gate/gate-tempest-dsvm-full/1b348a3/logs/screen-n-api.txt.gz#_2015-08-25_14_50_13_779 ** Also affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1488282 Title: Gate failures with 'the resource could not be found' Status in neutron: Confirmed Status in OpenStack Compute (nova): New Bug description: There have been spurious failures happening in the gate. The most prominent one is: ft1.186: tempest.api.compute.admin.test_servers.ServersAdminTestJSON.test_list_servers_by_admin_with_all_tenants[id-9f5579ae-19b4-4985-a091-2a5d56106580]_StringException: Empty attachments: stderr stdout pythonlogging:'': {{{ 2015-08-24 22:55:50,083 32355 INFO [tempest_lib.common.rest_client] Request (ServersAdminTestJSON:test_list_servers_by_admin_with_all_tenants): 404 GET http://127.0.0.1:8774/v2/fb99c79318b54e668713b25afc52f81a/servers/detail?all_tenants= 0.834s 2015-08-24 22:55:50,083 32355 DEBUG[tempest_lib.common.rest_client] Request - Headers: {'X-Auth-Token': '', 'Accept': 'application/json', 'Content-Type': 'application/json'} Body: None Response - Headers: {'content-length': '78', 'date': 'Mon, 24 Aug 2015 22:55:50 GMT', 'connection': 'close', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-387b21a9-4ada-48ee-89ed-9acfe5274ef7', 'status': '404'} Body: {"itemNotFound": {"message": "The resource could not be found.", "code": 404}} }}} Traceback (most recent call last): File "tempest/api/compute/admin/test_servers.py", line 81, in test_list_servers_by_admin_with_all_tenants body = self.client.list_servers(detail=True, **params) File "tempest/services/compute/json/servers_client.py", line 159, in list_servers resp, body = self.get(url) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 271, in get return self.request('GET', url, extra_headers, headers) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 643, in request resp, resp_body) File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 695, in _error_checker raise exceptions.NotFound(resp_body) tempest_lib.exceptions.NotFound: Object not found Details: {u'code': 404, u'message': u'The resource could not be found.'} but there are other similar failure modes. This seems to be related to bug #1269284 The logstash query: message:"tempest_lib.exceptions.NotFound: Object not found" AND build_name:"gate-tempest-dsvm-neutron-full" http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwidGVtcGVzdF9saWIuZXhjZXB0aW9ucy5Ob3RGb3VuZDogT2JqZWN0IG5vdCBmb3VuZFwiIEFORCBidWlsZF9uYW1lOlwiZ2F0ZS10ZW1wZXN0LWRzdm0tbmV1dHJvbi1mdWxsXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0NDA0NjIwNzcyMjksIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0= To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1488282/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp