Hi, You can try with the vdsm-client tool: https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html
Stopping a VM: 1) Get the vmId: # vdsm-client Host getVMList fullStatus=True 2) Destroy the VM # vdsm-client VM destroy vmID=<vmID> *Regards,* *Shani Leviim* On Sat, Apr 3, 2021 at 7:50 AM <[email protected]> wrote: > Hello, > > I've somehow gotten one of my VMs stuck in a state that ovirt seems to be > rather confused about its > existence of now. I'm running oVirt 4.3.10 and using oVirt Node on all > the hosts. > > My engine and host event logs are now filling up very rapidly with this > error: > > VDSM node217 command DestroyVDS failed: General Exception: ("'1048576'",) > > I was playing with hugetable support, and that error number or string > looks suspiciously > like the "hugetable size" custom property I set on the VM. > > This VM was migrated to another host at one point as well, and now that > host is also > generating the same error as well. > > When I try to move these hosts to maintenance mode, they get stuck in > "Preparing for > Maintenance" while it tries to migrate/deal with the VM that's not there > any more. > > Forcibly rebooting the hosts does not change anything. The VM state/host > seems to be > captured somewhere persistent in this case. > > The VM in question is not running, and I can start it up on another host > successfully, > but ovirt still thinks it exists on the other 2 hosts no matter what I do. > > Is there perhaps some way to delete it from the engine database directly > to straighten > things out? > > Here's a dump of the vdsm log on one of the hosts. I haven't been able to > pinpoint what > the exact issue is or how to fix it, but hopefully someone here will have > seen this before? > > 2021-04-03 04:40:35,515+0000 INFO (jsonrpc/1) [api.virt] START > destroy(gracefulAttempts=1) from=::ffff:10.100.0.210,58150, > vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:48) > 2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm] > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Release VM resources (vm:5186) > 2021-04-03 04:40:35,516+0000 WARN (jsonrpc/1) [virt.vm] > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') trying to set state to > Powering down when already Down (vm:626) > 2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm] > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection > (guestagent:455) > 2021-04-03 04:40:35,517+0000 INFO (jsonrpc/1) [vdsm.api] START > teardownImage(sdUUID='a08af6be-3802-4bb1-9fa5-4b6a10227290', > spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9', > imgUUID='9c896907-59b0-4983-9478-b36b2c2eb01e', volUUID=None) > from=::ffff:10.100.0.210,58150, task_id=fc946d20-126a-4fd0-9078-91 > 4b4a64b1d9 (api:48) > 2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) [storage.StorageDomain] > Removing image rundir link > u'/var/run/vdsm/storage/a08af6be-3802-4bb1-9fa5-4b6a10227290/9c896907-59b0-4983-9478-b36b2c2eb01e' > (fileSD:592) > 2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) [vdsm.api] FINISH > teardownImage return=None from=::ffff:10.100.0.210,58150, > task_id=fc946d20-126a-4fd0-9078-914b4a64b1d9 (api:54) > 2021-04-03 04:40:35,519+0000 INFO (jsonrpc/1) [vdsm.api] START > teardownImage(sdUUID='b891448d-dd92-4a7b-a51a-22abc3d7da67', > spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9', > imgUUID='c0e95483-35f1-4a61-958e-4e308b70d3f8', volUUID=None) > from=::ffff:10.100.0.210,58150, task_id=77c0fdca-e13a-44b5-9a00-29 > 0522b194b2 (api:48) > 2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) [storage.StorageDomain] > Removing image rundir link > u'/var/run/vdsm/storage/b891448d-dd92-4a7b-a51a-22abc3d7da67/c0e95483-35f1-4a61-958e-4e308b70d3f8' > (fileSD:592) > 2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) [vdsm.api] FINISH > teardownImage return=None from=::ffff:10.100.0.210,58150, > task_id=77c0fdca-e13a-44b5-9a00-290522b194b2 (api:54) > 2021-04-03 04:40:35,521+0000 INFO (jsonrpc/1) [virt.vm] > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection > (guestagent:455) > 2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] File: > /var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.ovirt-guest-agent.0 > already removed (fileutils:54) > 2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] Attempting to remove > a non existing network: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 > (libvirtnetwork:198) > 2021-04-03 04:40:35,522+0000 WARN (jsonrpc/1) [root] Attempting to remove > a non existing net user: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 > (libvirtnetwork:205) > 2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] Attempting to remove > a non existing network: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 > (libvirtnetwork:198) > 2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] Attempting to remove > a non existing net user: ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 > (libvirtnetwork:205) > 2021-04-03 04:40:35,527+0000 WARN (jsonrpc/1) [root] File: > /var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.org.qemu.guest_agent.0 > already removed (fileutils:54) > 2021-04-03 04:40:35,528+0000 WARN (jsonrpc/1) [virt.vm] > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') timestamp already removed > from stats cache (vm:2445) > 2021-04-03 04:40:35,531+0000 ERROR (jsonrpc/1) [api] FINISH destroy > error='1048576' (api:134) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in > method > ret = func(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 333, in destroy > res = self.vm.destroy(gracefulAttempts) > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5283, in > destroy > result = self.doDestroy(gracefulAttempts, reason) > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5302, in > doDestroy > return self.releaseVm(gracefulAttempts) > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5207, in > releaseVm > self._cleanup() > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2382, in > _cleanup > self._cleanup_hugepages() > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2393, in > _cleanup_hugepages > self.nr_hugepages, self.hugepagesz > File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 243, in > calculate_required_deallocation > _preallocated_hugepages(vm_hugepagesz)) > File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 262, in > _preallocated_hugepages > kernel_args['hugepagesz'] > File "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 291, in > _cmdline_hugepagesz_to_kb > }[cmdline] > KeyError: '1048576' > 2021-04-03 04:40:35,531+0000 INFO (jsonrpc/1) [api.virt] FINISH destroy > return={'status': {'message': 'General Exception: ("\'1048576\'",)', > 'code': 100}} from=::ffff:10.100.0.210,58150, > vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:54) > 2021-04-03 04:40:35,532+0000 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC > call VM.destroy failed (error 100) in 0.02 seconds (__init__:312) > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/KBABFNZIJ6T6QZXGYRSROHCO7LAS4VOM/ >
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/JALOOPH4LFONIVXVW4BYX6R5A33DYZHY/
