Public bug reported: HTTP 500 error occurs when one tries to get metadata by path constructed from folder name with appended value.
Steps to repro: 1. Launch VM and access its terminal 2. curl http://169.254.169.254/latest/meta-data/instance-id -- this returns some string, i.e. i-00000001 3. curl http://169.254.169.254/latest/meta-data/instance-id/i-00000001 -- this returns HTTP 500 It's expected that the last call returns meaningful message and not produce trace backs in logs. Errors: ---------- In VM terminal: $ curl http://169.254.169.254/latest/meta-data/instance-id/i-00000001 <html> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> Remote metadata server experienced an internal server error.<br /><br /> </body> </html>$ In Neutron metadata agent: 2014-09-18 14:44:37.563 ERROR neutron.agent.metadata.agent [-] Unexpected error. 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent Traceback (most recent call last): 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/metadata/agent.py", line 130, in __call__ 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent return webob.exc.HTTPNotFound() 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/metadata/agent.py", line 248, in _proxy_request 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent def _sign_instance_id(self, instance_id): 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent Exception: Unexpected response code: 400 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent 2014-09-18 14:44:37.566 INFO eventlet.wsgi.server [-] 10.0.0.2,<local> - - [18/Sep/2014 14:44:37] "GET /latest/meta-data/instance-id/i-00000001 HTTP/1.1" 500 229 0.348877 In Nova API service: 2014-09-18 14:31:19.030 ERROR nova.api.ec2 [req-5c84e0ae-7d18-4113-a08b-ed068e5333ed None None] FaultWrapper: string indices must be integers, not unicode 2014-09-18 14:31:19.030 TRACE nova.api.ec2 Traceback (most recent call last): 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/ec2/__init__.py", line 87, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return req.get_response(self.application) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send 2014-09-18 14:31:19.030 TRACE nova.api.ec2 application, catch_exc_info=False) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application 2014-09-18 14:31:19.030 TRACE nova.api.ec2 app_iter = application(self.environ, start_response) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return self.func(req, *args, **kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/ec2/__init__.py", line 99, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 rv = req.get_response(self.application) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send 2014-09-18 14:31:19.030 TRACE nova.api.ec2 application, catch_exc_info=False) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application 2014-09-18 14:31:19.030 TRACE nova.api.ec2 app_iter = application(self.environ, start_response) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return self.func(req, *args, **kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/handler.py", line 128, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = meta_data.lookup(req.path_info) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 418, in lookup 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = self.get_ec2_item(path_tokens[1:]) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 300, in get_ec2_item 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return find_path_in_tree(data, path_tokens[1:]) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 565, in find_path_in_tree 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = data[path_tokens[i]] 2014-09-18 14:31:19.030 TRACE nova.api.ec2 TypeError: string indices must be integers, not unicode 2014-09-18 14:31:19.030 TRACE nova.api.ec2 2014-09-18 14:31:19.032 INFO nova.metadata.wsgi.server [req-5c84e0ae-7d18-4113-a08b-ed068e5333ed None None] 10.0.0.2,172.18.76.77 "GET /latest/meta-data/placement/availability-zone/nova HTTP/1.1" status: 400 len: 246 time: 0.5495760 ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1371160 Title: HTTP 500 while retrieving metadata by non-existent key Status in OpenStack Compute (Nova): New Bug description: HTTP 500 error occurs when one tries to get metadata by path constructed from folder name with appended value. Steps to repro: 1. Launch VM and access its terminal 2. curl http://169.254.169.254/latest/meta-data/instance-id -- this returns some string, i.e. i-00000001 3. curl http://169.254.169.254/latest/meta-data/instance-id/i-00000001 -- this returns HTTP 500 It's expected that the last call returns meaningful message and not produce trace backs in logs. Errors: ---------- In VM terminal: $ curl http://169.254.169.254/latest/meta-data/instance-id/i-00000001 <html> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> Remote metadata server experienced an internal server error.<br /><br /> </body> </html>$ In Neutron metadata agent: 2014-09-18 14:44:37.563 ERROR neutron.agent.metadata.agent [-] Unexpected error. 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent Traceback (most recent call last): 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/metadata/agent.py", line 130, in __call__ 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent return webob.exc.HTTPNotFound() 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent File "/opt/stack/neutron/neutron/agent/metadata/agent.py", line 248, in _proxy_request 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent def _sign_instance_id(self, instance_id): 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent Exception: Unexpected response code: 400 2014-09-18 14:44:37.563 TRACE neutron.agent.metadata.agent 2014-09-18 14:44:37.566 INFO eventlet.wsgi.server [-] 10.0.0.2,<local> - - [18/Sep/2014 14:44:37] "GET /latest/meta-data/instance-id/i-00000001 HTTP/1.1" 500 229 0.348877 In Nova API service: 2014-09-18 14:31:19.030 ERROR nova.api.ec2 [req-5c84e0ae-7d18-4113-a08b-ed068e5333ed None None] FaultWrapper: string indices must be integers, not unicode 2014-09-18 14:31:19.030 TRACE nova.api.ec2 Traceback (most recent call last): 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/ec2/__init__.py", line 87, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return req.get_response(self.application) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send 2014-09-18 14:31:19.030 TRACE nova.api.ec2 application, catch_exc_info=False) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application 2014-09-18 14:31:19.030 TRACE nova.api.ec2 app_iter = application(self.environ, start_response) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return self.func(req, *args, **kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/ec2/__init__.py", line 99, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 rv = req.get_response(self.application) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send 2014-09-18 14:31:19.030 TRACE nova.api.ec2 application, catch_exc_info=False) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application 2014-09-18 14:31:19.030 TRACE nova.api.ec2 app_iter = application(self.environ, start_response) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return self.func(req, *args, **kwargs) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/handler.py", line 128, in __call__ 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = meta_data.lookup(req.path_info) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 418, in lookup 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = self.get_ec2_item(path_tokens[1:]) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 300, in get_ec2_item 2014-09-18 14:31:19.030 TRACE nova.api.ec2 return find_path_in_tree(data, path_tokens[1:]) 2014-09-18 14:31:19.030 TRACE nova.api.ec2 File "/opt/stack/nova/nova/api/metadata/base.py", line 565, in find_path_in_tree 2014-09-18 14:31:19.030 TRACE nova.api.ec2 data = data[path_tokens[i]] 2014-09-18 14:31:19.030 TRACE nova.api.ec2 TypeError: string indices must be integers, not unicode 2014-09-18 14:31:19.030 TRACE nova.api.ec2 2014-09-18 14:31:19.032 INFO nova.metadata.wsgi.server [req-5c84e0ae-7d18-4113-a08b-ed068e5333ed None None] 10.0.0.2,172.18.76.77 "GET /latest/meta-data/placement/availability-zone/nova HTTP/1.1" status: 400 len: 246 time: 0.5495760 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1371160/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp