Hello Duncan, I can't see any logs of cinder-api service getting restarted. Although I can see some messages getting continuously logged not sure if that will help.
2016-07-28 16:13:56.934 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28 16:13:56.935 25399 INFO eventlet.wsgi.server [-] 192.168.2.24 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0026720 2016-07-28T16:13:56.935213+02:00 cic-1 cinder-api: 2016-07-28 16:13:56.934 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28T16:13:56.935974+02:00 cic-1 cinder-api: 2016-07-28 16:13:56.935 25399 INFO eventlet.wsgi.server [-] 192.168.2.24 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0026720 2016-07-28 16:13:57.099 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ 2016-07-28 16:13:57.100 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28 16:13:57.101 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0020089 2016-07-28T16:13:57.099882+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.099 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ 2016-07-28T16:13:57.100836+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.100 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28T16:13:57.101321+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.101 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0020089 2016-07-28 16:14:01.144 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ 2016-07-28T16:14:01.145297+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.144 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ 2016-07-28 16:14:01.146 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28T16:14:01.146887+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.146 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ returned with HTTP 300 2016-07-28 16:14:01.147 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0032442 2016-07-28T16:14:01.147704+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.147 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0032442 BR, Varun On Thu, Jul 28, 2016 at 3:44 PM, Duncan Thomas <[email protected]> wrote: > Looks like either you've got an intermittent network problem or the cinder > api service is restarting. Anything enlightening in the cinder-api log? > > On 28 Jul 2016 16:41, "varun bhatnagar" <[email protected]> wrote: > > Hello Steve, > > Thanks a lot for such a quick response. > > Yes the IP is reachable. > > ping 10.33.237.104 > PING 10.33.237.104 (10.33.237.104) 56(84) bytes of data. > 64 bytes from 10.33.237.104: icmp_seq=1 ttl=64 time=0.587 ms > 64 bytes from 10.33.237.104: icmp_seq=2 ttl=64 time=0.101 ms > 64 bytes from 10.33.237.104: icmp_seq=3 ttl=64 time=0.092 ms > 64 bytes from 10.33.237.104: icmp_seq=4 ttl=64 time=0.144 ms > ^C > --- 10.33.237.104 ping statistics --- > 4 packets transmitted, 4 received, 0% packet loss, time 3003ms > rtt min/avg/max/mdev = 0.092/0.231/0.587/0.206 ms > root@cic-1:~# > > > Endpoints for keystone and cinder are defined as below: > > | keystone | identity | RegionOne > | > | | | publicURL: > http://10.33.237.104:5000/v2.0 | > | | | internalURL: > http://192.168.2.28:5000/v2.0 | > | | | adminURL: > http://192.168.2.28:35357/v2.0 | > > > | cinderv2 | volumev2 | RegionOne > | > | | | publicURL: > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | > | | | internalURL: > http://192.168.2.28:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | > | | | adminURL: > http://192.168.2.28:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | > | | | > | > > > And adding debug gives the below details: > > openstack volume list --debug > START with options: ['volume', 'list', '--debug'] > options: Namespace(access_token_endpoint='', auth_type='', auth_url=' > http://192.168.2.28:5000/v2.0', cacert='', client_id='', > client_secret='***', cloud='', debug=True, default_domain='Default', > deferred_help=False, domain_id='', domain_name='', endpoint='', > identity_provider='', identity_provider_url='', insecure=None, > interface='', log_file=None, os_clustering_api_version='1', > os_compute_api_version='', os_data_processing_api_version='1.1', > os_data_processing_url='', os_dns_api_version='2', > os_identity_api_version='', os_image_api_version='', > os_key_manager_api_version='1', os_network_api_version='', > os_object_api_version='', os_orchestration_api_version='1', > os_project_id=None, os_project_name=None, os_queues_api_version='1.1', > os_volume_api_version='', os_workflow_api_version='2', password='***', > profile=None, project_domain_id='', project_domain_name='', project_id='', > project_name='admin', protocol='', region_name='RegionOne', scope='', > service_provider_endpoint='', timing=False, token='***', trust_id='', > url='', user_domain_id='', user_domain_name='', user_id='', > username='admin', verbose_level=3, verify=None) > defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': > None, u'database_api_version': u'1.0', 'api_timeout': None, > u'baremetal_api_version': u'1', u'image_api_version': u'2', 'cacert': None, > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', > u'orchestration_api_version': u'1', u'interface': None, > u'network_api_version': u'2', u'image_format': u'qcow2', > u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'verify': > True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': > None, u'secgroup_source': u'neutron', u'container_api_version': u'1', > u'dns_api_version': u'2', u'object_store_api_version': u'1', > u'disable_vendor_agent': {}} > cloud cfg: {'auth_type': 'password', u'compute_api_version': u'2', > u'orchestration_api_version': '1', u'database_api_version': u'1.0', > 'data_processing_api_version': '1.1', u'network_api_version': u'2', > u'image_format': u'qcow2', u'image_api_version': u'2', > 'clustering_api_version': '1', 'verify': True, u'dns_api_version': '2', > u'object_store_api_version': u'1', 'verbose_level': 3, 'region_name': > 'RegionOne', 'api_timeout': None, u'baremetal_api_version': u'1', > 'queues_api_version': '1.1', 'auth': {'username': 'admin', 'project_name': > 'admin', 'password': '***', 'auth_url': 'http://192.168.2.28:5000/v2.0'}, > 'default_domain': 'Default', u'container_api_version': u'1', > u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': > None, 'timing': False, 'cacert': None, u'key_manager_api_version': '1', > u'metering_api_version': u'2', 'deferred_help': False, > u'identity_api_version': u'2.0', 'workflow_api_version': '2', > u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', > 'debug': True, u'interface': None, u'disable_vendor_agent': {}} > compute API version 2, cmd group openstack.compute.v2 > network API version 2, cmd group openstack.network.v2 > image API version 2, cmd group openstack.image.v2 > volume API version 2, cmd group openstack.volume.v2 > identity API version 2.0, cmd group openstack.identity.v2 > object_store API version 1, cmd group openstack.object_store.v1 > orchestration API version 1, cmd group openstack.orchestration.v1 > clustering API version 1, cmd group openstack.clustering.v1 > workflow_engine API version 2, cmd group openstack.workflow_engine.v2 > key_manager API version 1, cmd group openstack.key_manager.v1 > dns API version 2, cmd group openstack.dns.v2 > data_processing API version 1.1, cmd group openstack.data_processing.v1 > messaging API version 1.1, cmd group openstack.messaging.v1 > command: volume list -> openstackclient.volume.v2.volume.ListVolume > Auth plugin password selected > auth_type: password > Using auth plugin: password > Using parameters {'username': 'admin', 'password': '***', 'project_name': > 'admin', 'auth_url': 'http://192.168.2.28:5000/v2.0'} > Get auth_ref > REQ: curl -g -i -X GET http://192.168.2.28:5000/v2.0 -H "Accept: > application/json" -H "User-Agent: python-openstackclient > keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.6" > Starting new HTTP connection (1): 192.168.2.28 > "GET /v2.0 HTTP/1.1" 200 None > RESP: [200] Transfer-Encoding: chunked Vary: > X-Keystone-Internal,X-Auth-Token Server: Apache Connection: close Date: > Thu, 28 Jul 2016 13:35:11 GMT Content-Type: application/json > x-openstack-request-id: req-27df9679-2ff5-4d6a-aecb-3499b5aa683b > RESP BODY: {"version": {"status": "stable", "updated": > "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", > "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", > "links": [{"href": "http://192.168.2.28:5000/v2.0/";, "rel": "self"}, > {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": > "describedby"}]}} > > Making authentication request to http://192.168.2.28:5000/v2.0/tokens > Resetting dropped connection: 192.168.2.28 > "POST /v2.0/tokens HTTP/1.1" 200 None > run(Namespace(all_projects=False, columns=[], formatter='table', > long=False, max_width=0, name=None, noindent=False, project=None, > project_domain=None, quote_mode='nonnumeric', status=None, user=None, > user_domain=None)) > Instantiating volume client: <class 'cinderclient.v2.client.Client'> > Instantiating compute client for VAPI Version Major: 2, Minor: 0 > Instantiating identity client: <class > 'openstackclient.identity.client.IdentityClientv2'> > Making authentication request to http://192.168.2.28:5000/v2.0/tokens > Resetting dropped connection: 192.168.2.28 > "POST /v2.0/tokens HTTP/1.1" 200 None > REQ: curl -g -i -X GET http://10.33.237.104:8774/v2.1/servers/detail -H > "User-Agent: python-novaclient" -H "Accept: application/json" -H > "X-Auth-Token: {SHA1}c2d3c0718b45d184c4a0b99cc0858a3f65d150b3" > Starting new HTTP connection (1): 10.33.237.104 > "GET /v2.1/servers/detail HTTP/1.1" 200 15 > RESP: [200] Content-Length: 15 X-Compute-Request-Id: > req-b78348d6-750a-44e6-b303-1e4d87ddb64c Vary: X-OpenStack-Nova-API-Version > Connection: close X-Openstack-Nova-Api-Version: 2.1 Date: Thu, 28 Jul 2016 > 13:35:11 GMT Content-Type: application/json > RESP BODY: {"servers": []} > > REQ: curl -g -i -X GET > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail > -H "User-Agent: python-cinderclient" -H "Accept: application/json" -H > "X-Auth-Token: {SHA1}c2d3c0718b45d184c4a0b99cc0858a3f65d150b3" > Starting new HTTP connection (1): 10.33.237.104 > Unable to establish connection to > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in > run_subcommand > result = cmd.run(parsed_args) > File > "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line > 38, in run > return super(Command, self).run(parsed_args) > File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in run > column_names, data = self.take_action(parsed_args) > File > "/usr/lib/python2.7/dist-packages/openstackclient/volume/v2/volume.py", > line 305, in take_action > data = volume_client.volumes.list(search_opts=search_opts) > File "/usr/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line > 282, in list > return self._list(url, resource_type, limit=limit) > File "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 73, > in _list > resp, body = self.api.client.get(url) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line > 111, in get > return self._cs_request(url, 'GET', **kwargs) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line > 108, in _cs_request > return self.request(url, method, **kwargs) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 99, > in request > **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line > 212, in request > resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line > 101, in request > return self.session.request(url, method, **kwargs) > File > "/usr/lib/python2.7/dist-packages/openstackclient/common/session.py", line > 40, in request > resp = super(TimingSession, self).request(url, method, **kwargs) > File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, > in inner > return func(*args, **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line > 452, in request > resp = send(**kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line > 496, in _send_request > raise exceptions.ConnectFailure(msg) > ConnectFailure: Unable to establish connection to > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail > clean_up ListVolume: Unable to establish connection to > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line > 118, in run > ret_val = super(OpenStackShell, self).run(argv) > File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 255, in run > result = self.run_subcommand(remainder) > File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line > 153, in run_subcommand > ret_value = super(OpenStackShell, self).run_subcommand(argv) > File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in > run_subcommand > result = cmd.run(parsed_args) > File > "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line > 38, in run > return super(Command, self).run(parsed_args) > File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in run > column_names, data = self.take_action(parsed_args) > File > "/usr/lib/python2.7/dist-packages/openstackclient/volume/v2/volume.py", > line 305, in take_action > data = volume_client.volumes.list(search_opts=search_opts) > File "/usr/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line > 282, in list > return self._list(url, resource_type, limit=limit) > File "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 73, > in _list > resp, body = self.api.client.get(url) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line > 111, in get > return self._cs_request(url, 'GET', **kwargs) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line > 108, in _cs_request > return self.request(url, method, **kwargs) > File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line 99, > in request > **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line > 212, in request > resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line > 101, in request > return self.session.request(url, method, **kwargs) > File > "/usr/lib/python2.7/dist-packages/openstackclient/common/session.py", line > 40, in request > resp = super(TimingSession, self).request(url, method, **kwargs) > File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, > in inner > return func(*args, **kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line > 452, in request > resp = send(**kwargs) > File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line > 496, in _send_request > raise exceptions.ConnectFailure(msg) > ConnectFailure: Unable to establish connection to > http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail > > > BR, > Varun > > On Thu, Jul 28, 2016 at 3:17 PM, Steve Martinelli <[email protected]> > wrote: > >> try running the command with --debug when it fails. normally that error >> happens when the client can't reach the host, does the IP address and port >> number look correct? >> >> On Thu, Jul 28, 2016 at 8:57 AM, varun bhatnagar <[email protected]> >> wrote: >> >>> Hi, >>> >>> I am using OpenStack Mitaka. >>> I am trying to list volumes using openstack client, the command works >>> sometimes but sometimes it fails: >>> >>> openstack volume list >>> Unable to establish connection to >>> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >>> >>> openstack volume list >>> >>> +--------------------------------------+--------------+-----------+------+-------------+ >>> | ID | Display Name | Status | Size >>> | Attached to | >>> >>> +--------------------------------------+--------------+-----------+------+-------------+ >>> | dfab04ce-5ad3-4787-8342-3e76afd877a0 | testvol | available | 1 >>> | | >>> >>> +--------------------------------------+--------------+-----------+------+-------------+ >>> >>> >>> Can anyone please tell me what is wrong here. >>> >>> BR, >>> Varun >>> >>> _______________________________________________ >>> Mailing list: >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>> Post to : [email protected] >>> Unsubscribe : >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>> >>> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
