ok, this is a nonissue; the issue was the deploy-helper opaque error.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1178568
Title:
possible race condition during bare metal deploys
Status in OpenStack Compute (Nova):
Invalid
Bug description:
There were no obvious failures during the deploy. This is what was
logged:
sudo grep 5f305000-ddf2-44b6-be74-059c88cf2109 /var/log/upstart/*
/var/log/upstart/nova-api.log:2013-05-10 07:52:11,743.743 14328 INFO
nova.osapi_compute.wsgi.server [req-ced5c8c5-f5b7-4158-9690-d9f59c590932
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] 10.10.16.134
"GET
/v2/c30643483cd849a6b61e98c6ccc03a66/servers/5f305000-ddf2-44b6-be74-059c88cf2109
HTTP/1.1" status: 200 len: 1986 time: 0.0977318
/var/log/upstart/nova-baremetal-deploy-helper.log:2013-05-10 07:51:31,210.210
14224 INFO nova.virt.baremetal.deploy_helper [-] request is queued: node 69,
params {'swap_mb': 1, 'iqn': 'iqn-5f305000-ddf2-44b6-be74-059c88cf2109',
'image_path': u'/var/lib/nova/instances/instance-00000096/disk', 'address':
'10.10.16.173', 'pxe_config_path':
u'/tftpboot/5f305000-ddf2-44b6-be74-059c88cf2109/config', 'port': '3260',
'lun': '1', 'root_mb': 10240}
/var/log/upstart/nova-baremetal-deploy-helper.log:2013-05-10 07:51:31,210.210
14224 INFO nova.virt.baremetal.deploy_helper [-] start deployment for node 69,
params {'swap_mb': 1, 'iqn': 'iqn-5f305000-ddf2-44b6-be74-059c88cf2109',
'image_path': u'/var/lib/nova/instances/instance-00000096/disk', 'address':
'10.10.16.173', 'pxe_config_path':
u'/tftpboot/5f305000-ddf2-44b6-be74-059c88cf2109/config', 'port': '3260',
'lun': '1', 'root_mb': 10240}
/var/log/upstart/nova-compute.log:2013-05-10 07:44:03,597.597 23016 AUDIT
nova.compute.manager [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Starting instance...
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,319.319 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Attempting claim: memory 512 MB, disk 10
GB, VCPUs 1
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,319.319 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Total Memory: 98304 MB, used: 0 MB
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,319.319 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Memory limit: 98304 MB, free: 98304 MB
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,320.320 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Total Disk: 2048 GB, used: 0 GB
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,320.320 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Disk limit not specified, defaulting to
unlimited
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,321.321 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Total CPU: 24 VCPUs, used: 0 VCPUs
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,321.321 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] CPU limit not specified, defaulting to
unlimited
/var/log/upstart/nova-compute.log:2013-05-10 07:44:07,322.322 23016 AUDIT
nova.compute.claims [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Claim successful
/var/log/upstart/nova-compute.log:2013-05-10 07:47:57,976.976 23016 INFO
nova.compute.manager [-] [instance: 5f305000-ddf2-44b6-be74-059c88cf2109]
During sync_power_state the instance has a pending task. Skip.
/var/log/upstart/nova-compute.log:2013-05-10 07:51:31,864.864 23016 ERROR
nova.virt.baremetal.driver [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] Error
deploying instance 5f305000-ddf2-44b6-be74-059c88cf2109 on baremetal node
1509a907-f0b6-471f-a0f5-7e2414cc896f.
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 ERROR
nova.compute.manager [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Instance failed to spawn
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109] Traceback
(most recent call last):
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109] File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py",
line 1200, in _spawn
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109]
block_device_info)
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109] File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py",
line 267, in spawn
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109]
self.power_off(instance, node)
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109] File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py",
line 330, in power_off
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109] "for
instance %r") % instance['uuid'])
/var/log/upstart/nova-compute.log:2013-05-10 07:51:36,528.528 23016 TRACE
nova.compute.manager [instance: 5f305000-ddf2-44b6-be74-059c88cf2109]
InstancePowerOffFailure: Baremetal power manager failed to stop node for
instance u'5f305000-ddf2-44b6-be74-059c88cf2109'
/var/log/upstart/nova-compute.log:2013-05-10
07:51:36,528./var/log/upstart/nova-compute.log:2013-05-10 07:51:37,307.307
23016 AUDIT nova.compute.manager [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Terminating instance
/var/log/upstart/nova-compute.log:2013-05-10 07:51:50,840.840 23016 ERROR
nova.virt.baremetal.driver [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] Error from
baremetal driver during destroy: Baremetal power manager failed to stop node
for instance u'5f305000-ddf2-44b6-be74-059c88cf2109'
/var/log/upstart/nova-compute.log:2013-05-10 07:51:50,922.922 23016 ERROR
nova.compute.manager [req-e10b52ef-c85b-446b-9fa8-fa7394c081dc
87b44714e906420a9e6a07f6835b5b61 c30643483cd849a6b61e98c6ccc03a66] [instance:
5f305000-ddf2-44b6-be74-059c88cf2109] Error: ['Traceback (most recent call
last):\n', ' File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py",
line 942, in _build_instance\n set_access_ip=set_access_ip)\n', ' File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py",
line 1204, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'),
instance=instance)\n', ' File "/usr/lib/python2.7/contextlib.py", line 24, in
__exit__\n self.gen.next()\n', ' File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py",
line 1200, in _spawn\n block_device_info)\n', ' File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py",
line 267, in spawn\n self.power_off(instance, node)\n', ' File
"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/virt/baremetal/driver.py",
line 330, in power_off\n "for instance %r") % instance[\'uuid\'])\n',
"InstancePowerOffFailure: Baremetal power manager failed to stop node for
instance u'5f305000-ddf2-44b6-be74-059c88cf2109'\n"]
/var/log/upstart/nova-compute.log:2013-05-10 07:51:51,547.547 23016 TRACE
nova.openstack.common.rpc.amqp InstancePowerOffFailure: Baremetal power manager
failed to stop node for instance u'5f305000-ddf2-44b6-be74-059c88cf2109'528
23016 TRACE nova.compute.manager [instance:
5f305000-ddf2-44b6-be74-059c88cf2109]
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1178568/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
