Hi, I have e-mailed vdsm log to Benny. Unfortunately, yesterday I deleted engine.log before restarting engine to have more shorter log.
Jirka, can you e-mail engine.log and vdsm logs (SPM at the time and the host running the VM) to Benny? > On 5 Aug 2022, at 11:42, Jirka Simon <[email protected]> wrote: > > Hi Benny, so we have the same problem last 2 days. > > there are two affected VMs (furtunately not more) and we cant start any > snapshot for these VMS. > > it looks that the snapshot haven't start yet , then to unlock objects > (snapshot and disk) shouldn't damage disk (i tried it on both VMS), then i > restarted them and VMS run again. I made clone from one of them ant the clone > works properly. > > > Jirka > > On 8/5/22 10:34, Benny Zlotnik wrote: >> So based on your logs the lock you are seeing is a memory lock, >> unlock_entity.sh can't really help with these. >> Also, the job table is used mainly for presentation so removing an >> entry from will not help. >> >> Do you have the logs from when this snapshot operation started, you >> can use the correlation id (28353fa0-5e36-4fe8-8609-e74cd1da6d36) to >> search? Also, do you have the vdsm logs (SPM at the time and the host >> running the VM), same correlation id can be used for this as well >> >> The table that's used to coordinate this is command_entities, so in >> theory removing the entries with this correlation id can help, but I'd >> like to see what led to this first >> >> On Fri, Aug 5, 2022 at 8:37 AM Andrei Verovski <[email protected]> wrote: >>> Hi, Benny, >>> >>> I have sent log on your mailbox, its too big to post here on mailing list. >>> >>> Looks like ghost task is still running, anything else need to be removed >>> from Postgres DB? >>> >>> BTW, frozen dead snapshot is in invalid state, is there any way to get rid >>> of it? I think it's actually exists, but due to invalid state its not >>> possible to do anything with it. >>> >>> >>> —— >>> >>> Log file still shows zombie task: >>> >>> 2022-08-04 20:57:30,145+03 INFO >>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-24) >>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id: >>> '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id: >>> 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to >>> complete >>> 2022-08-04 20:57:40,176+03 INFO >>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) >>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id: >>> '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id: >>> 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to >>> complete >>> 2022-08-04 20:57:50,252+03 INFO >>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) >>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id: >>> '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id: >>> 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to >>> complete >>> >>> >>>> On 4 Aug 2022, at 19:06, Benny Zlotnik <[email protected]> wrote: >>>> >>>> can you share the logs after restarting ovirt-engine? >>>> >>>> On Thu, Aug 4, 2022 at 4:58 PM Andrei Verovski <[email protected]> >>>> wrote: >>>>> Hi, >>>>> >>>>> >>>>> Creating snapshot of one of the VM vailed, and zombie tasks was killed >>>>> with: >>>>> >>>>> su postgres >>>>> psql -d engine -U postgres >>>>> select * from job order by start_time desc; >>>>> >>>>> select DeleteJob('UUID_FROZEN_TASK_ID’); >>>>> >>>>> >>>>> However, VM remains in locked state (with lock sign left-below red “DOWN” >>>>> arrow in status column of web interface. >>>>> >>>>> I run: >>>>> /usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t all >>>>> >>>>> then rebooted engine VM, still no luck. Can’t do anything with that VM. >>>>> >>>>> Please advise how to fix. >>>>> Thanks in advance. >>>>> _______________________________________________ >>>>> Users mailing list -- [email protected] >>>>> To unsubscribe send an email to [email protected] >>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/[email protected]/message/P2TVMLHC53JWCLDJNK6UXLZ7ZAUOSYFJ/ >> _______________________________________________ >> Users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/[email protected]/message/ILYSLIM7UNRU3B3G43IFM3VMSBGZNRDU/ _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/M66Z7JTEJHLBWPN6QDHIY4AI5AJK5MBB/
