This may be happening because I changed cluster compatibility to 4.3 then
immediately after changed data center compatibility to 4.3 (before
restarting VMs after cluster compatibility change). If this is the case I
can't fix by downgrading the data center compatibility to 4.2 as it won't
allow me to do so. What can I do to fix this, any VM I restart will break
(I am leaving the others running for now, but there are some down that I
can't start).
Full error from VDSM:
2019-02-13 13:30:55,465-0400 ERROR (vm/d070ce80) [storage.TaskManager.Task]
(Task='d5c8e50a-0a6f-4fe7-be79-fd322b273a1e') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
in _run
return fn(*args, **kargs)
File "<string>", line 2, in prepareImage
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3198,
in prepareImage
legality = dom.produceVolume(imgUUID, volUUID).getLegality()
File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 818, in
produceVolume
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/glusterVolume.py",
line 45, in __init__
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 800,
in __init__
self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
71, in __init__
volUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86,
in __init__
self.validate()
File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 112,
in validate
self.validateVolumePath()
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
131, in validateVolumePath
raise se.VolumeDoesNotExist(self.volUUID)
VolumeDoesNotExist: Volume does not exist:
(u'2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',)
2019-02-13 13:30:55,468-0400 ERROR (vm/d070ce80) [storage.Dispatcher]
FINISH prepareImage error=Volume does not exist:
(u'2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',) (dispatcher:81)
2019-02-13 13:30:55,469-0400 ERROR (vm/d070ce80) [virt.vm]
(vmId='d070ce80-e0bc-489d-8ee0-47d5926d5ae2') The vm start process failed
(vm:937)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 866, in
_startUnderlyingVm
self._run()
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2749, in
_run
self._devices = self._make_devices()
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2589, in
_make_devices
disk_objs = self._perform_host_local_adjustment()
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2662, in
_perform_host_local_adjustment
self._preparePathsForDrives(disk_params)
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1011, in
_preparePathsForDrives
drive['path'] = self.cif.prepareVolumePath(drive, self.id)
File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 415, in
prepareVolumePath
raise vm.VolumeError(drive)
VolumeError: Bad volume specification {'address': {'function': '0x0',
'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'slot': '0x06'},
'serial': 'd81a6826-dc46-44db-8de7-405d30e44d57', 'index': 0, 'iface':
'virtio', 'apparentsize': '64293699584', 'specParams': {}, 'cache': 'none',
'imageID': 'd81a6826-dc46-44db-8de7-405d30e44d57', 'truesize':
'64293814272', 'type': 'disk', 'domainID':
'1f2e9989-9ab3-43d5-971d-568b8feca918', 'reqsize': '0', 'format': 'cow',
'poolID': 'a45e442e-9989-11e8-b0e4-00163e4bf18a', 'device': 'disk', 'path':
'/rhev/data-center/a45e442e-9989-11e8-b0e4-00163e4bf18a/1f2e9989-9ab3-43d5-971d-568b8feca918/images/d81a6826-dc46-44db-8de7-405d30e44d57/2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',
'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID':
'2d6d5f87-ccb0-48ce-b3ac-84495bd12d32', 'diskType': 'file', 'alias':
'ua-d81a6826-dc46-44db-8de7-405d30e44d57', 'discard': False}
On Wed, Feb 13, 2019 at 1:19 PM Jayme <[email protected]> wrote:
> I may have made matters worse. So I changed to 4.3 compatible cluster
> then 4.3 compatible data center. All VMs were marked as requiring a
> reboot. I restarted a couple of them and none of them will start up, they
> are saying "bad volume specification". The ones running that I did not yet
> restart are still running ok. I need to figure out why the VMs aren't
> restarting.
>
> Here is an example from vdsm.log
>
> olumeError: Bad volume specification {'address': {'function': '0x0',
> 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'slot': '0x06'},
> 'serial': 'd81a6826-dc46-44db-8de7-405d30e44d57', 'index': 0, 'iface':
> 'virtio', 'apparentsize': '64293699584', 'specParams': {}, 'cache': 'none',
> 'imageID': 'd81a6826-dc46-44db-8de7-405d30e44d57', 'truesize':
> '64293814272', 'type': 'disk', 'domainID':
> '1f2e9989-9ab3-43d5-971d-568b8feca918', 'reqsize': '0', 'format': 'cow',
> 'poolID': 'a45e442e-9989-11e8-b0e4-00163e4bf18a', 'device': 'disk', 'path':
> '/rhev/data-center/a45e442e-9989-11e8-b0e4-00163e4bf18a/1f2e9989-9ab3-43d5-971d-568b8feca918/images/d81a6826-dc46-44db-8de7-405d30e44d57/2d6d5f87-ccb0-48ce-b3ac-84495bd12d32',
> 'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID':
> '2d6d5f87-ccb0-48ce-b3ac-84495bd12d32', 'diskType': 'file', 'alias':
> 'ua-d81a6826-dc46-44db-8de7-405d30e44d57', 'discard': False}
>
> On Wed, Feb 13, 2019 at 1:01 PM Jayme <[email protected]> wrote:
>
>> I think I just figured out what I was doing wrong. On edit cluster
>> screen I was changing both the CPU type and cluster level 4.3. I tried it
>> again by switching to the new CPU type first (leaving cluster on 4.2) then
>> saving, then going back in and switching compat level to 4.3. It appears
>> that you need to do this in two steps for it to work.
>>
>>
>>
>> On Wed, Feb 13, 2019 at 12:57 PM Jayme <[email protected]> wrote:
>>
>>> Hmm interesting, I wonder how you were able to switch from SandyBridge
>>> IBRS to SandyBridge IBRS SSBD. I just attempted the same in both regular
>>> mode and in global maintenance mode and it won't allow me to, it says that
>>> all hosts have to be in maintenance mode (screenshots attached). Are you
>>> also running HCI/Gluster setup?
>>>
>>>
>>>
>>> On Wed, Feb 13, 2019 at 12:44 PM Ron Jerome <[email protected]> wrote:
>>>
>>>> > Environment setup:
>>>> >
>>>> > 3 Host HCI GlusterFS setup. Identical hosts, Dell R720s w/ Intel
>>>> E5-2690
>>>> > CPUs
>>>> >
>>>> > 1 default data center (4.2 compat)
>>>> > 1 default cluster (4.2 compat)
>>>> >
>>>> > Situation: I recently upgraded my three node HCI cluster from Ovirt
>>>> 4.2 to
>>>> > 4.3. I did so by first updating the engine to 4.3 then upgrading each
>>>> > ovirt-node host to 4.3 and rebooting.
>>>> >
>>>> > Currently engine and all hosts are running 4.3 and all is working
>>>> fine.
>>>> >
>>>> > To complete the upgrade I need to update cluster compatibility to 4.3
>>>> and
>>>> > then data centre to 4.3. This is where I am stuck.
>>>> >
>>>> > The CPU type on cluster is "Intel SandyBridge IBRS Family". This
>>>> option is
>>>> > no longer available if I select 4.3 compatibility. Any other option
>>>> chosen
>>>> > such as SandyBridge IBRS SSBD will not allow me to switch to 4.3 as
>>>> all
>>>> > hosts must be in maintenance mode (which is not possible w/ self
>>>> hosted
>>>> > engine).
>>>> >
>>>> > I saw another post about this where someone else followed steps to
>>>> create a
>>>> > second cluster on 4.3 with new CPU type then move one host to it,
>>>> start
>>>> > engine on it then perform other steps to eventually get to 4.3
>>>> > compatibility.
>>>> >
>>>>
>>>> I have the exact same hardware configuration and was able to change to
>>>> "SandyBridge IBRS SSBD" without creating a new cluster. How I made that
>>>> happen, I'm not so sure, but the cluster may have been in "Global
>>>> Maintenance" mode when I changed it.
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list -- [email protected]
>>>> To unsubscribe send an email to [email protected]
>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct:
>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>> List Archives:
>>>> https://lists.ovirt.org/archives/list/[email protected]/message/5B3TAXKO7IBTWRVNF2K4II472TDISO6P/
>>>>
>>>
_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/OHVD3LTXFJ26MRL3P65AXEQ5Y55A5EFK/
