Hi Tomas, [root@host1 vdsm]# tail -f vdsm.log StorageUnavailableError: Unable to get volume size for domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 volume 581f65d2-3db5-4fde-960f-4c4ffe27341e 2022-10-25 16:39:51,785+0800 INFO (periodic/3) [virt.vm] (vmId='a8d6fcaf-a673-46df-b00a-925a8b9791ef') Requesting extension for volume 581f65d2-3db5-4fde-960f-4c4ffe27341e on domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 (apparent: 91268055040, capacity: 322122547200, allocated: 90872676352, physical: 91268055040 threshold_state: exceeded) (vm:1289) 2022-10-25 16:39:51,787+0800 INFO (periodic/3) [vdsm.api] START sendExtendMsg(spUUID='81f9e45a-5879-11ea-a3bd-00163e27dfef', volDict={'newSize': 92341796864, 'domainID': '107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', 'name': 'vda', 'poolID': '81f9e45a-5879-11ea-a3bd-00163e27dfef', 'clock': <Clock(total=0.00*, extend-volume=0.00*)>, 'internal': False, 'volumeID': '581f65d2-3db5-4fde-960f-4c4ffe27341e', 'imageID': '2d1375ba-ce52-4da1-9873-d9181d9e775b'}, newSize=92341796864, callbackFunc=<bound method Vm.__afterVolumeExtension of <vdsm.virt.vm.Vm object at 0x7fea007336d0>>) from=internal, task_id=2b0c7a67-0f31-4f62-af70-a29e0b84dafb (api:48) 2022-10-25 16:39:51,787+0800 INFO (periodic/3) [vdsm.api] FINISH sendExtendMsg return=None from=internal, task_id=2b0c7a67-0f31-4f62-af70-a29e0b84dafb (api:54) 2022-10-25 16:39:52,635+0800 INFO (mailbox-hsm) [storage.MailBox.HsmMailMonitor] HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/81f9e45a-5879-11ea-a3bd-00163e27dfef/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=4096', 'count=1', 'seek=1'] (mailbox:380) 2022-10-25 16:39:52,710+0800 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2022-10-25 16:39:52,715+0800 INFO (jsonrpc/3) [api.virt] START getStats() from=::1,39184, vmId=6579a9c4-d55e-4d61-9c3a-c745cdcfa430 (api:48) 2022-10-25 16:39:52,715+0800 INFO (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'6579a9c4-d55e-4d61-9c3a-c745cdcfa430'} (api:129) 2022-10-25 16:39:52,715+0800 INFO (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'6579a9c4-d55e-4d61-9c3a-c745cdcfa430'}", 'code': 1}} from=::1,39184, vmId=6579a9c4-d55e-4d61-9c3a-c745cdcfa430 (api:54) 2022-10-25 16:39:52,715+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2022-10-25 16:39:53,786+0800 INFO (periodic/5) [virt.vm] (vmId='a8d6fcaf-a673-46df-b00a-925a8b9791ef') Requesting extension for volume 581f65d2-3db5-4fde-960f-4c4ffe27341e on domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 (apparent: 91268055040, capacity: 322122547200, allocated: 90872676352, physical: 91268055040 threshold_state: exceeded) (vm:1289) 2022-10-25 16:39:53,787+0800 INFO (periodic/5) [vdsm.api] START sendExtendMsg(spUUID='81f9e45a-5879-11ea-a3bd-00163e27dfef', volDict={'newSize': 92341796864, 'domainID': '107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', 'name': 'vda', 'poolID': '81f9e45a-5879-11ea-a3bd-00163e27dfef', 'clock': <Clock(total=0.00*, extend-volume=0.00*)>, 'internal': False, 'volumeID': '581f65d2-3db5-4fde-960f-4c4ffe27341e', 'imageID': '2d1375ba-ce52-4da1-9873-d9181d9e775b'}, newSize=92341796864, callbackFunc=<bound method Vm.__afterVolumeExtension of <vdsm.virt.vm.Vm object at 0x7fea007336d0>>) from=internal, task_id=197234ba-5cab-4098-86c8-7f2c979171d2 (api:48) 2022-10-25 16:39:53,787+0800 INFO (periodic/5) [vdsm.api] FINISH sendExtendMsg return=None from=internal, task_id=197234ba-5cab-4098-86c8-7f2c979171d2 (api:54) 2022-10-25 16:39:54,167+0800 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2022-10-25 16:39:54,658+0800 INFO (mailbox-hsm) [storage.MailBox.HsmMailMonitor] HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/81f9e45a-5879-11ea-a3bd-00163e27dfef/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=4096', 'count=1', 'seek=1'] (mailbox:380) 2022-10-25 16:39:54,661+0800 INFO (mailbox-hsm/3) [storage.ThreadPool.WorkerThread] START task b81cf2c1-c795-4f97-9c5a-6b06a6a3173c (cmd=<function runTask at 0x7fea630c8500>, args=(<bound method Vm.__afterVolumeExtension of <vdsm.virt.vm.Vm object at 0x7fea007336d0>>, {'newSize': 92341796864, 'domainID': '107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', 'name': 'vda', 'poolID': '81f9e45a-5879-11ea-a3bd-00163e27dfef', 'clock': <Clock(total=2.88*, extend-volume=2.88, refresh-volume=0.00*)>, 'internal': False, 'volumeID': '581f65d2-3db5-4fde-960f-4c4ffe27341e', 'imageID': '2d1375ba-ce52-4da1-9873-d9181d9e775b'})) (threadPool:208) 2022-10-25 16:39:54,663+0800 INFO (mailbox-hsm/3) [vdsm.api] START refreshVolume(sdUUID='107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', spUUID='81f9e45a-5879-11ea-a3bd-00163e27dfef', imgUUID='2d1375ba-ce52-4da1-9873-d9181d9e775b', volUUID='581f65d2-3db5-4fde-960f-4c4ffe27341e') from=internal, task_id=918b7e3f-d730-4226-a941-fc0e60ee36b0 (api:48) 2022-10-25 16:39:54,809+0800 WARN (mailbox-hsm/3) [storage.LVM] Command ['/sbin/lvm', 'vgs', '--config', 'devices { preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter=["a|^/dev/mapper/3600c0ff00051441d1e145f5e01000000$|^/dev/mapper/3600c0ff00051441de4f6555e01000000$|^/dev/mapper/364cd98f0bb99140025dd822d083dd183$|^/dev/mapper/364cd98f0bb99140025dd82520a7280ae$|", "r|.*|"] } global { locking_type=4 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min=50 retain_days=0 }', '--noheadings', '--units', 'b', '--nosuffix', '--separator', '|', '--ignoreskippedcluster', '-o', 'uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name', '--select', 'vg_name = 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19'] succeeded with warnings: [' /dev/mapper/3600c0ff00051441d1e145f5e01000000: Checksum error at offset 30099960109568', " Couldn't read volume group metadata from /dev/mapper/3600c0ff00051441d1e145f5e01000000.", ' Metadata location on /dev/mapper/3600c0ff00051441d1e145f5e01000000 at 30099960109568 has invalid summary for VG.', ' Failed to read metadata summary from /dev/mapper/3600c0ff00051441d1e145f5e01000000', ' Failed to scan VG from /dev/mapper/3600c0ff00051441d1e145f5e01000000'] (lvm:466) 2022-10-25 16:39:54,824+0800 INFO (mailbox-hsm/3) [vdsm.api] FINISH refreshVolume error=Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) from=internal, task_id=918b7e3f-d730-4226-a941-fc0e60ee36b0 (api:52) 2022-10-25 16:39:54,824+0800 ERROR (mailbox-hsm/3) [storage.TaskManager.Task] (Task='918b7e3f-d730-4226-a941-fc0e60ee36b0') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in refreshVolume File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3033, in refreshVolume sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/blockSD.py", line 1843, in findDomain return BlockStorageDomain(BlockStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python2.7/site-packages/vdsm/storage/blockSD.py", line 1701, in findDomainPath raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) 2022-10-25 16:39:54,825+0800 INFO (mailbox-hsm/3) [storage.TaskManager.Task] (Task='918b7e3f-d730-4226-a941-fc0e60ee36b0') aborting: Task is aborted: "Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',)" - code 358 (task:1181) 2022-10-25 16:39:54,825+0800 ERROR (mailbox-hsm/3) [storage.Dispatcher] FINISH refreshVolume error=Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) (dispatcher:83) 2022-10-25 16:39:54,826+0800 INFO (mailbox-hsm/3) [vdsm.api] START getVolumeSize(sdUUID='107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', spUUID='81f9e45a-5879-11ea-a3bd-00163e27dfef', imgUUID='2d1375ba-ce52-4da1-9873-d9181d9e775b', volUUID='581f65d2-3db5-4fde-960f-4c4ffe27341e', options=None) from=internal, task_id=c5a7364d-3b39-4e12-b75b-22d5c4487cad (api:48) 2022-10-25 16:39:54,970+0800 WARN (mailbox-hsm/3) [storage.LVM] Command ['/sbin/lvm', 'vgs', '--config', 'devices { preferred_names=["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter=["a|^/dev/mapper/3600c0ff00051441d1e145f5e01000000$|^/dev/mapper/3600c0ff00051441de4f6555e01000000$|^/dev/mapper/364cd98f0bb99140025dd822d083dd183$|^/dev/mapper/364cd98f0bb99140025dd82520a7280ae$|", "r|.*|"] } global { locking_type=4 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min=50 retain_days=0 }', '--noheadings', '--units', 'b', '--nosuffix', '--separator', '|', '--ignoreskippedcluster', '-o', 'uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name', '--select', 'vg_name = 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19'] succeeded with warnings: [' /dev/mapper/3600c0ff00051441d1e145f5e01000000: Checksum error at offset 30099960109568', " Couldn't read volume group metadata from /dev/mapper/3600c0ff00051441d1e145f5e01000000.", ' Metadata location on /dev/mapper/3600c0ff00051441d1e145f5e01000000 at 30099960109568 has invalid summary for VG.', ' Failed to read metadata summary from /dev/mapper/3600c0ff00051441d1e145f5e01000000', ' Failed to scan VG from /dev/mapper/3600c0ff00051441d1e145f5e01000000'] (lvm:466) 2022-10-25 16:39:54,983+0800 INFO (mailbox-hsm/3) [vdsm.api] FINISH getVolumeSize error=Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) from=internal, task_id=c5a7364d-3b39-4e12-b75b-22d5c4487cad (api:52) 2022-10-25 16:39:54,983+0800 ERROR (mailbox-hsm/3) [storage.TaskManager.Task] (Task='c5a7364d-3b39-4e12-b75b-22d5c4487cad') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in getVolumeSize File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3071, in getVolumeSize dom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/blockSD.py", line 1843, in findDomain return BlockStorageDomain(BlockStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python2.7/site-packages/vdsm/storage/blockSD.py", line 1701, in findDomainPath raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) 2022-10-25 16:39:54,983+0800 INFO (mailbox-hsm/3) [storage.TaskManager.Task] (Task='c5a7364d-3b39-4e12-b75b-22d5c4487cad') aborting: Task is aborted: "Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',)" - code 358 (task:1181) 2022-10-25 16:39:54,983+0800 ERROR (mailbox-hsm/3) [storage.Dispatcher] FINISH getVolumeSize error=Storage domain does not exist: ('107f73a2-8c3e-43a7-bf18-ce8ef4be2a19',) (dispatcher:83) 2022-10-25 16:39:54,983+0800 ERROR (mailbox-hsm/3) [storage.TaskManager.Task] (Task='033cec9a-5b65-4413-97c3-a273aafa6114') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1407, in __afterVolumeExtension volSize = self.__verifyVolumeExtension(volInfo) File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1325, in __verifyVolumeExtension volInfo['imageID'], volInfo['volumeID']) File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 6147, in _getVolumeSize (domainID, volumeID)) StorageUnavailableError: Unable to get volume size for domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 volume 581f65d2-3db5-4fde-960f-4c4ffe27341e 2022-10-25 16:39:54,983+0800 INFO (mailbox-hsm/3) [storage.TaskManager.Task] (Task='033cec9a-5b65-4413-97c3-a273aafa6114') aborting: Task is aborted: u'Unable to get volume size for domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 volume 581f65d2-3db5-4fde-960f-4c4ffe27341e' - code 100 (task:1181) 2022-10-25 16:39:54,983+0800 ERROR (mailbox-hsm/3) [storage.ThreadPool.WorkerThread] FINISH task b81cf2c1-c795-4f97-9c5a-6b06a6a3173c failed (callback=None, cmd=<function runTask at 0x7fea630c8500>, args=(<bound method Vm.__afterVolumeExtension of <vdsm.virt.vm.Vm object at 0x7fea007336d0>>, {'newSize': 92341796864, 'domainID': '107f73a2-8c3e-43a7-bf18-ce8ef4be2a19', 'name': 'vda', 'poolID': '81f9e45a-5879-11ea-a3bd-00163e27dfef', 'clock': <Clock(total=3.20*, extend-volume=2.88, refresh-volume=0.16)>, 'internal': False, 'volumeID': '581f65d2-3db5-4fde-960f-4c4ffe27341e', 'imageID': '2d1375ba-ce52-4da1-9873-d9181d9e775b'})) (threadPool:222) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/threadPool.py", line 209, in _processNextTask cmd(args) File "/usr/lib/python2.7/site-packages/vdsm/storage/mailbox.py", line 85, in runTask ctask.prepare(cmd, *args) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper return m(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1189, in prepare raise self.error StorageUnavailableError: Unable to get volume size for domain 107f73a2-8c3e-43a7-bf18-ce8ef4be2a19 volume 581f65d2-3db5-4fde-960f-4c4ffe27341e ^C
This is what I get from vdsm when I ran tail -f. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QRSLEBLJUE5NY5CTNB2IJ7MTFGEZCBBG/