Public bug reported: Description =========== After upgrading fresh OpenStack Liberty installation to Mitaka on Trusty with Juju, I am no longer able to create Nova instances.
Steps to reproduce ================== * Install OpenStack Liberty with Juju from the attached "bundle.yaml" file. * Upgrade OpenStack Liberty to Mitaka by executing the following commands: juju upgrade-charm keystone juju upgrade-charm ceilometer juju upgrade-charm ceilometer-agent juju upgrade-charm ceph juju upgrade-charm ceph-osd juju upgrade-charm cinder juju upgrade-charm glance juju upgrade-charm nova-cloud-controller juju upgrade-charm nova-compute juju upgrade-charm neutron-api juju upgrade-charm neutron-gateway juju upgrade-charm openstack-dashboard juju set-config ceph source="cloud:trusty-mitaka" juju set-config ceph-osd source="cloud:trusty-mitaka" juju set-config keystone openstack-origin="cloud:trusty-mitaka" juju ssh keystone/0 sudo keystone-manage db_sync juju set-config ceilometer openstack-origin="cloud:trusty-mitaka" juju set-config ceilometer-agent openstack-origin="cloud:trusty-mitaka" juju set-config cinder openstack-origin="cloud:trusty-mitaka" juju ssh cinder/0 sudo cinder-manage db sync juju set-config glance openstack-origin="cloud:trusty-mitaka" juju ssh glance/0 sudo glance-manage db_sync juju set-config nova-cloud-controller openstack-origin="cloud:trusty-mitaka" juju ssh nova-cloud-controller/0 sudo nova-manage db sync juju set-config nova-compute openstack-origin="cloud:trusty-mitaka" juju set-config neutron-api openstack-origin="cloud:trusty-mitaka" juju ssh neutron-api/0 sudo neutron-db-manage upgrade heads juju set-config neutron-gateway openstack-origin="cloud:trusty-mitaka" juju set-config openstack-dashboard openstack-origin="cloud:trusty-mitaka" juju ssh ceph/0 sudo reboot juju ssh ceph/1 sudo reboot juju ssh ceph/2 sudo reboot juju ssh ceph-osd/0 sudo service ceph restart juju ssh ceph-osd/1 sudo service ceph restart juju ssh ceph-osd/2 sudo service ceph restart juju ssh keystone/0 sudo reboot juju ssh keystone/1 sudo reboot juju ssh keystone/2 sudo reboot juju ssh ceilometer/0 sudo reboot juju ssh ceilometer/1 sudo reboot juju ssh ceilometer/2 sudo reboot juju ssh ceilometer-agent/0 sudo service ceilometer-agent-compute restart juju ssh ceilometer-agent/1 sudo service ceilometer-agent-compute restart juju ssh ceilometer-agent/2 sudo service ceilometer-agent-compute restart juju ssh cinder/0 sudo reboot juju ssh cinder/1 sudo reboot juju ssh cinder/2 sudo reboot juju ssh glance/0 sudo reboot juju ssh glance/1 sudo reboot juju ssh glance/2 sudo reboot juju ssh nova-cloud-controller/0 sudo reboot juju ssh nova-cloud-controller/1 sudo reboot juju ssh nova-cloud-controller/2 sudo reboot juju ssh nova-compute/0 sudo service nova-compute restart juju ssh nova-compute/1 sudo service nova-compute restart juju ssh nova-compute/2 sudo service nova-compute restart juju ssh neutron-api/0 sudo reboot juju ssh neutron-api/1 sudo reboot juju ssh neutron-api/2 sudo reboot juju ssh neutron-gateway/0 sudo service neutron-dhcp-agent restart juju ssh neutron-gateway/0 sudo service neutron-lbaas-agent restart juju ssh neutron-gateway/0 sudo service neutron-metadata-agent restart juju ssh neutron-gateway/0 sudo service neutron-metering-agent restart juju ssh neutron-gateway/0 sudo service neutron-openvswitch-agent restart juju ssh neutron-gateway/0 sudo service neutron-vpn-agent restart juju ssh nova-compute/0 sudo service neutron-openvswitch-agent restart juju ssh nova-compute/1 sudo service neutron-openvswitch-agent restart juju ssh nova-compute/2 sudo service neutron-openvswitch-agent restart juju ssh openstack-dashboard/0 sudo reboot juju ssh openstack-dashboard/1 sudo reboot juju ssh openstack-dashboard/2 sudo reboot * Attempt to create Nova instance. Expected result =============== Nova instance being created. Actual result ============= * Nova instance not being created. * The following error messages being displayed: ** from GUI: Error: Unable to create the server. ** from CLI: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.AttributeError'> (HTTP 500) (Request-ID: req-c035d518-50c0-4cab-913e-9a5263392a2a) Environment =========== 1. Exact version of OpenStack you are running. ubuntu@tkurek-maas:~$ juju ssh nova-cloud-controller/0 sudo dpkg-query -l | grep nova ii nova-api-os-compute 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - OpenStack Compute API frontend ii nova-cert 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - certificate management ii nova-common 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack Compute API - Python 2.7 Connection to 10.230.126.81 closed. ubuntu@tkurek-maas:~$ juju ssh nova-compute/0 sudo dpkg-query -l | grep nova ii nova-common 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-compute 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node base ii nova-compute-kvm 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node libvirt support ii python-nova 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack Compute API - Python 2.7 Connection to 10.230.126.54 closed. 2. Which hypervisor did you use? KVM What's the version of that? ubuntu@tkurek-maas:~$ juju ssh nova-compute/0 sudo dpkg-query -l | grep kvm ii nova-compute-kvm 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node (KVM) Connection to 10.230.126.54 closed. 3. Which storage type did you use? Ceph What's the version of that? ubuntu@tkurek-maas:~$ juju ssh ceph/0 sudo dpkg-query -l | grep ceph ii ceph 10.2.2-0ubuntu0.16.04.2~clo amd64 distributed storage and file system ii ceph-common 10.2.2-0ubuntu0.16.04.2~clo amd64 common utilities to mount and interact with a ceph storage cluster ii libcephfs1 10.2.2-0ubuntu0.16.04.2~clo amd64 Ceph distributed file system client library ii python-ceph 10.2.2-0ubuntu0.16.04.2~clo all Meta-package for python libraries for the Ceph libraries ii python-cephfs 10.2.2-0ubuntu0.16.04.2~clo amd64 Python libraries for the Ceph libcephfs library Connection to 10.230.126.65 closed. ubuntu@tkurek-maas:~$ juju ssh ceph-osd/0 sudo dpkg-query -l | grep ceph ii ceph 10.2.2-0ubuntu0.16.04.2~clo amd64 distributed storage and file system ii ceph-common 10.2.2-0ubuntu0.16.04.2~clo amd64 common utilities to mount and interact with a ceph storage cluster ii libcephfs1 10.2.2-0ubuntu0.16.04.2~clo amd64 Ceph distributed file system client library ii python-ceph 10.2.2-0ubuntu0.16.04.2~clo all Meta-package for python libraries for the Ceph libraries ii python-cephfs 10.2.2-0ubuntu0.16.04.2~clo amd64 Python libraries for the Ceph libcephfs library Connection to 10.230.126.54 closed. 4. Which networking type did you use? Neutron with Openvswitch Logs & Configs ============== Attached "/var/log/nova/nova-api-os-compute.log" file errors: 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions [req-b2f38657-7eae-4714-a1e1-0217950e7e29 d12595ae6b2d418eaf9ffd209fa25479 64c87895e12b4f4fb0bfd979b1ac70a4 - - -] Unexpected exception in API method 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions Traceback (most recent call last): 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return f(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 629, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions **create_kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 154, in inner 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions rv = f(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1562, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions check_server_group_quota=check_server_group_quota) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1175, in _create_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance_group, check_server_group_quota, filter_properties) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1033, in _provision_instances 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions quotas.rollback() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self.force_reraise() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions six.reraise(self.type_, self.value, self.tb) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 988, in _provision_instances 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions num_instances, i, shutdown_terminate) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1445, in create_db_entry_for_new_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance.create() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return fn(self, *args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 471, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self._load_ec2_ids() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 818, in _load_ec2_ids 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self.ec2_ids = objects.EC2Ids.get_by_instance(self._context, self) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 181, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions result = fn(cls, context, *args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/ec2.py", line 218, in get_by_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions ec2_ids = cls._get_ec2_ids(context, instance) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/ec2.py", line 204, in _get_ec2_ids 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance.image_ref) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/ec2/ec2utils.py", line 128, in glance_id_to_ec2_id 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions image_id = glance_id_to_id(context, glance_id) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/ec2/ec2utils.py", line 48, in memoizer 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions value = _CACHE.get(key) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/cache_utils.py", line 151, in get 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions value = self.region.get(key) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 456, in get 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions expiration_time, ignore_expiration)(value) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 475, in value_fn 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions current_time - value.metadata["ct"] > expiration_time: 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions AttributeError: 'int' object has no attribute 'metadata' 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions ** Affects: nova Importance: Undecided Status: New ** Attachment added: "OpenStack Liberty Juju bundle" https://bugs.launchpad.net/bugs/1615582/+attachment/4725592/+files/bundle.yaml -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1615582 Title: Error: Unable to create the server. Unexpected API Error. Status in OpenStack Compute (nova): New Bug description: Description =========== After upgrading fresh OpenStack Liberty installation to Mitaka on Trusty with Juju, I am no longer able to create Nova instances. Steps to reproduce ================== * Install OpenStack Liberty with Juju from the attached "bundle.yaml" file. * Upgrade OpenStack Liberty to Mitaka by executing the following commands: juju upgrade-charm keystone juju upgrade-charm ceilometer juju upgrade-charm ceilometer-agent juju upgrade-charm ceph juju upgrade-charm ceph-osd juju upgrade-charm cinder juju upgrade-charm glance juju upgrade-charm nova-cloud-controller juju upgrade-charm nova-compute juju upgrade-charm neutron-api juju upgrade-charm neutron-gateway juju upgrade-charm openstack-dashboard juju set-config ceph source="cloud:trusty-mitaka" juju set-config ceph-osd source="cloud:trusty-mitaka" juju set-config keystone openstack-origin="cloud:trusty-mitaka" juju ssh keystone/0 sudo keystone-manage db_sync juju set-config ceilometer openstack-origin="cloud:trusty-mitaka" juju set-config ceilometer-agent openstack-origin="cloud:trusty-mitaka" juju set-config cinder openstack-origin="cloud:trusty-mitaka" juju ssh cinder/0 sudo cinder-manage db sync juju set-config glance openstack-origin="cloud:trusty-mitaka" juju ssh glance/0 sudo glance-manage db_sync juju set-config nova-cloud-controller openstack-origin="cloud:trusty-mitaka" juju ssh nova-cloud-controller/0 sudo nova-manage db sync juju set-config nova-compute openstack-origin="cloud:trusty-mitaka" juju set-config neutron-api openstack-origin="cloud:trusty-mitaka" juju ssh neutron-api/0 sudo neutron-db-manage upgrade heads juju set-config neutron-gateway openstack-origin="cloud:trusty-mitaka" juju set-config openstack-dashboard openstack-origin="cloud:trusty-mitaka" juju ssh ceph/0 sudo reboot juju ssh ceph/1 sudo reboot juju ssh ceph/2 sudo reboot juju ssh ceph-osd/0 sudo service ceph restart juju ssh ceph-osd/1 sudo service ceph restart juju ssh ceph-osd/2 sudo service ceph restart juju ssh keystone/0 sudo reboot juju ssh keystone/1 sudo reboot juju ssh keystone/2 sudo reboot juju ssh ceilometer/0 sudo reboot juju ssh ceilometer/1 sudo reboot juju ssh ceilometer/2 sudo reboot juju ssh ceilometer-agent/0 sudo service ceilometer-agent-compute restart juju ssh ceilometer-agent/1 sudo service ceilometer-agent-compute restart juju ssh ceilometer-agent/2 sudo service ceilometer-agent-compute restart juju ssh cinder/0 sudo reboot juju ssh cinder/1 sudo reboot juju ssh cinder/2 sudo reboot juju ssh glance/0 sudo reboot juju ssh glance/1 sudo reboot juju ssh glance/2 sudo reboot juju ssh nova-cloud-controller/0 sudo reboot juju ssh nova-cloud-controller/1 sudo reboot juju ssh nova-cloud-controller/2 sudo reboot juju ssh nova-compute/0 sudo service nova-compute restart juju ssh nova-compute/1 sudo service nova-compute restart juju ssh nova-compute/2 sudo service nova-compute restart juju ssh neutron-api/0 sudo reboot juju ssh neutron-api/1 sudo reboot juju ssh neutron-api/2 sudo reboot juju ssh neutron-gateway/0 sudo service neutron-dhcp-agent restart juju ssh neutron-gateway/0 sudo service neutron-lbaas-agent restart juju ssh neutron-gateway/0 sudo service neutron-metadata-agent restart juju ssh neutron-gateway/0 sudo service neutron-metering-agent restart juju ssh neutron-gateway/0 sudo service neutron-openvswitch-agent restart juju ssh neutron-gateway/0 sudo service neutron-vpn-agent restart juju ssh nova-compute/0 sudo service neutron-openvswitch-agent restart juju ssh nova-compute/1 sudo service neutron-openvswitch-agent restart juju ssh nova-compute/2 sudo service neutron-openvswitch-agent restart juju ssh openstack-dashboard/0 sudo reboot juju ssh openstack-dashboard/1 sudo reboot juju ssh openstack-dashboard/2 sudo reboot * Attempt to create Nova instance. Expected result =============== Nova instance being created. Actual result ============= * Nova instance not being created. * The following error messages being displayed: ** from GUI: Error: Unable to create the server. ** from CLI: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.AttributeError'> (HTTP 500) (Request-ID: req-c035d518-50c0-4cab-913e-9a5263392a2a) Environment =========== 1. Exact version of OpenStack you are running. ubuntu@tkurek-maas:~$ juju ssh nova-cloud-controller/0 sudo dpkg-query -l | grep nova ii nova-api-os-compute 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - OpenStack Compute API frontend ii nova-cert 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - certificate management ii nova-common 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-conductor 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - conductor service ii nova-consoleauth 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - virtual machine scheduler ii python-nova 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack Compute API - Python 2.7 Connection to 10.230.126.81 closed. ubuntu@tkurek-maas:~$ juju ssh nova-compute/0 sudo dpkg-query -l | grep nova ii nova-common 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-compute 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node base ii nova-compute-kvm 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node libvirt support ii python-nova 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack Compute API - Python 2.7 Connection to 10.230.126.54 closed. 2. Which hypervisor did you use? KVM What's the version of that? ubuntu@tkurek-maas:~$ juju ssh nova-compute/0 sudo dpkg-query -l | grep kvm ii nova-compute-kvm 2:13.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node (KVM) Connection to 10.230.126.54 closed. 3. Which storage type did you use? Ceph What's the version of that? ubuntu@tkurek-maas:~$ juju ssh ceph/0 sudo dpkg-query -l | grep ceph ii ceph 10.2.2-0ubuntu0.16.04.2~clo amd64 distributed storage and file system ii ceph-common 10.2.2-0ubuntu0.16.04.2~clo amd64 common utilities to mount and interact with a ceph storage cluster ii libcephfs1 10.2.2-0ubuntu0.16.04.2~clo amd64 Ceph distributed file system client library ii python-ceph 10.2.2-0ubuntu0.16.04.2~clo all Meta-package for python libraries for the Ceph libraries ii python-cephfs 10.2.2-0ubuntu0.16.04.2~clo amd64 Python libraries for the Ceph libcephfs library Connection to 10.230.126.65 closed. ubuntu@tkurek-maas:~$ juju ssh ceph-osd/0 sudo dpkg-query -l | grep ceph ii ceph 10.2.2-0ubuntu0.16.04.2~clo amd64 distributed storage and file system ii ceph-common 10.2.2-0ubuntu0.16.04.2~clo amd64 common utilities to mount and interact with a ceph storage cluster ii libcephfs1 10.2.2-0ubuntu0.16.04.2~clo amd64 Ceph distributed file system client library ii python-ceph 10.2.2-0ubuntu0.16.04.2~clo all Meta-package for python libraries for the Ceph libraries ii python-cephfs 10.2.2-0ubuntu0.16.04.2~clo amd64 Python libraries for the Ceph libcephfs library Connection to 10.230.126.54 closed. 4. Which networking type did you use? Neutron with Openvswitch Logs & Configs ============== Attached "/var/log/nova/nova-api-os-compute.log" file errors: 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions [req-b2f38657-7eae-4714-a1e1-0217950e7e29 d12595ae6b2d418eaf9ffd209fa25479 64c87895e12b4f4fb0bfd979b1ac70a4 - - -] Unexpected exception in API method 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions Traceback (most recent call last): 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return f(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return func(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 629, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions **create_kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 154, in inner 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions rv = f(*args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1562, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions check_server_group_quota=check_server_group_quota) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1175, in _create_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance_group, check_server_group_quota, filter_properties) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1033, in _provision_instances 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions quotas.rollback() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self.force_reraise() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions six.reraise(self.type_, self.value, self.tb) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 988, in _provision_instances 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions num_instances, i, shutdown_terminate) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1445, in create_db_entry_for_new_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance.create() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions return fn(self, *args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 471, in create 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self._load_ec2_ids() 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 818, in _load_ec2_ids 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions self.ec2_ids = objects.EC2Ids.get_by_instance(self._context, self) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 181, in wrapper 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions result = fn(cls, context, *args, **kwargs) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/ec2.py", line 218, in get_by_instance 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions ec2_ids = cls._get_ec2_ids(context, instance) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/objects/ec2.py", line 204, in _get_ec2_ids 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions instance.image_ref) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/ec2/ec2utils.py", line 128, in glance_id_to_ec2_id 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions image_id = glance_id_to_id(context, glance_id) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/ec2/ec2utils.py", line 48, in memoizer 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions value = _CACHE.get(key) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/cache_utils.py", line 151, in get 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions value = self.region.get(key) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 456, in get 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions expiration_time, ignore_expiration)(value) 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 475, in value_fn 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions current_time - value.metadata["ct"] > expiration_time: 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions AttributeError: 'int' object has no attribute 'metadata' 2016-08-20 11:03:48.780 558458 ERROR nova.api.openstack.extensions To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1615582/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
