Public bug reported: we have an instance running on host (32G mem) and the VM itself is 24G mem when the host is filled with other VM consumption and we start the VM with 24G mem, it failed with memory not enough, however, the failed action lead to the instance is undefined which is unacceptable (we accept fail to start, but not the VM undefine)
checked the code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3059 will undefine the VM, then https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3101 will define the VM again, so when the VM creation at 3101 has some problem (just like the call stack below, it will lead to VM undefined...) note the following logic is from Stein but the logic seems same? 2020-02-20 13:14:29.534 8686 ERROR nova.virt.libvirt.driver [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] [instance: e087ba59-10e2-4930-978b-5c951b6cc996] Failed to start libvirt guest: libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:29.911 8686 INFO os_vif [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:de:8a:3f,bridge_name='qbr219907f5-f9',has_traffic_filtering=True,id=219907f5-f9e5-4bbf-bf2d-e8b3de4af072,network=Network(14e7db65-1297-4c8f-8928-a1764272c276),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap219907f5-f9') 2020-02-20 13:14:30.044 8686 INFO nova.compute.manager [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] [instance: e087ba59-10e2-4930-978b-5c951b6cc996] Successfully reverted task state from powering-on on failure for instance. 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] Exception during message handling: libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 79, in wrapped 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server function_name, call_dict, binary, tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 69, in wrapped 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 187, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 157, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1418, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 215, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 203, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2939, in start_instance 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._power_on(context, instance) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2909, in _power_on 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server block_device_info) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2975, in power_on 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._hard_reboot(context, instance, network_info, block_device_info) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2865, in _hard_reboot 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server vifs_already_plugged=True) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5717, in _create_domain_and_network 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server destroy_disks_on_failure) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5686, in _create_domain_and_network 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server post_xml_callback=post_xml_callback) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5621, in _create_domain 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server guest.launch(pause=pause) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._encoded_xml, errors='ignore') 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return self._domain.createWithFlags(flags) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1110, in createWithFlags 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1864347 Title: libvirt: start an instance failed lead to xml undefined Status in OpenStack Compute (nova): New Bug description: we have an instance running on host (32G mem) and the VM itself is 24G mem when the host is filled with other VM consumption and we start the VM with 24G mem, it failed with memory not enough, however, the failed action lead to the instance is undefined which is unacceptable (we accept fail to start, but not the VM undefine) checked the code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3059 will undefine the VM, then https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3101 will define the VM again, so when the VM creation at 3101 has some problem (just like the call stack below, it will lead to VM undefined...) note the following logic is from Stein but the logic seems same? 2020-02-20 13:14:29.534 8686 ERROR nova.virt.libvirt.driver [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] [instance: e087ba59-10e2-4930-978b-5c951b6cc996] Failed to start libvirt guest: libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:29.911 8686 INFO os_vif [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:de:8a:3f,bridge_name='qbr219907f5-f9',has_traffic_filtering=True,id=219907f5-f9e5-4bbf-bf2d-e8b3de4af072,network=Network(14e7db65-1297-4c8f-8928-a1764272c276),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap219907f5-f9') 2020-02-20 13:14:30.044 8686 INFO nova.compute.manager [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] [instance: e087ba59-10e2-4930-978b-5c951b6cc996] Successfully reverted task state from powering-on on failure for instance. 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server [req-0f8347ed-0191-4477-9cec-c5b37050e3e7 bfb0bf8fc2fe44908e638ab012266f0c 0dfede6c1a4b4bef8ce8c1936c5a418c - default default] Exception during message handling: libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 79, in wrapped 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server function_name, call_dict, binary, tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 69, in wrapped 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 187, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 157, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1418, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 215, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 203, in decorated_function 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2939, in start_instance 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._power_on(context, instance) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2909, in _power_on 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server block_device_info) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2975, in power_on 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._hard_reboot(context, instance, network_info, block_device_info) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2865, in _hard_reboot 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server vifs_already_plugged=True) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5717, in _create_domain_and_network 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server destroy_disks_on_failure) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5686, in _create_domain_and_network 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server post_xml_callback=post_xml_callback) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5621, in _create_domain 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server guest.launch(pause=pause) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self._encoded_xml, errors='ignore') 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server self.force_reraise() 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server return self._domain.createWithFlags(flags) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1110, in createWithFlags 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-02-20T05:14:29.061291Z qemu-kvm: cannot set up guest memory 's390.ram': Cannot allocate memory 2020-02-20 13:14:30.049 8686 ERROR oslo_messaging.rpc.server To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1864347/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
