Public bug reported:
When instances are deployed on multiple compute nodes concurrently and
instance_path is set to shared storage (i.e. Virtuozzo Storage), that
doesn't support concurrent write operations on the same file,
_supports_direct_io() fails with OSError: [Errno 16] Device or resource
busy: '/var/lib/nova/instances/.directio.test'.
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] Traceback (most recent call last):
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in
_build_resources
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] yield resources
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2064, in
_build_and_run_instance
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] block_device_info=block_device_info)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2941, in
spawn
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] write_to_disk=True)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4909, in
_get_guest_xml
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] context)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4768, in
_get_guest_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] flavor, guest.os_type)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3806, in
_get_guest_storage_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] inst_type)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3748, in
_get_guest_disk_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] self.disk_cachemode,
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 739, in
disk_cachemode
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] if not
self._supports_direct_io(CONF.instances_path):
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3128, in
_supports_direct_io
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] {'path': dirpath, 'ex': e})
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] self.force_reraise()
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] six.reraise(self.type_, self.value,
self.tb)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3110, in
_supports_direct_io
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] f = os.open(testfile, os.O_CREAT |
os.O_WRONLY | os.O_DIRECT)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] OSError: [Errno 16] Device or resource
busy: '/var/lib/nova/instances/.directio.test'
** Affects: nova
Importance: Undecided
Assignee: Pavel Gluschak (scsnow)
Status: In Progress
** Changed in: nova
Assignee: (unassigned) => Pavel Gluschak (scsnow)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1727369
Title:
_supports_direct_io() check fails on shared storage
Status in OpenStack Compute (nova):
In Progress
Bug description:
When instances are deployed on multiple compute nodes concurrently and
instance_path is set to shared storage (i.e. Virtuozzo Storage), that
doesn't support concurrent write operations on the same file,
_supports_direct_io() fails with OSError: [Errno 16] Device or
resource busy: '/var/lib/nova/instances/.directio.test'.
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] Traceback (most recent call last):
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in
_build_resources
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] yield resources
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2064, in
_build_and_run_instance
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] block_device_info=block_device_info)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2941, in
spawn
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] write_to_disk=True)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4909, in
_get_guest_xml
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] context)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4768, in
_get_guest_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] flavor, guest.os_type)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3806, in
_get_guest_storage_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] inst_type)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3748, in
_get_guest_disk_config
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] self.disk_cachemode,
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 739, in
disk_cachemode
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] if not
self._supports_direct_io(CONF.instances_path):
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3128, in
_supports_direct_io
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] {'path': dirpath, 'ex': e})
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] self.force_reraise()
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] six.reraise(self.type_, self.value,
self.tb)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3110, in
_supports_direct_io
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] f = os.open(testfile, os.O_CREAT |
os.O_WRONLY | os.O_DIRECT)
2017-10-19 21:11:29.030 160580 ERROR nova.compute.manager [instance:
8ce8f6f4-a3a5-457e-bd18-72b453efe2e0] OSError: [Errno 16] Device or resource
busy: '/var/lib/nova/instances/.directio.test'
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1727369/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
