GitHub user TheKunalSen edited a comment on the discussion: Cloudstack Migrate VM Query
i have turned off the host and the host went to down and maintainance but the vm is still on that host, the vm ha did not started and i am getting this message in the logs Unable to find next HA state for current HA state=[Checking] for event=[HealthCheckFailed] <img width="1725" height="848" alt="image" src="https://github.com/user-attachments/assets/bc1dd903-09a5-4fca-bcbf-87639cbd5ee8"; /> <img width="1467" height="49" alt="image" src="https://github.com/user-attachments/assets/c1e73cda-a20d-444c-8d0d-fd026473aa73"; /> the full log 2026-02-16 05:51:46,745 WARN [o.a.c.h.HAManagerImpl] (pool-2-thread-49:[]) (logid:) Unable to find next HA state for current HA state=[Checking] for event=[HealthCheckFailed] for host Host {"id":8,"name":"compute3","type":"Routing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"} with id 8. com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new state from Checking via HealthCheckFailed at com.cloud.utils.fsm.StateMachine2.getTransition(StateMachine2.java:108) at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:94) at org.apache.cloudstack.ha.HAManagerImpl.transitionHAState(HAManagerImpl.java:153) at jdk.internal.reflect.GeneratedMethodAccessor136.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework. aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) at jdk.proxy3/jdk.proxy3.$Proxy417.transitionHAState(Unknown Source) at org.apache.cloudstack.ha.task.HealthCheckTask.processResult(HealthCheckTask.java:57) at org.apache.cloudstack.ha.task.BaseHATask.call(BaseHATask.java:105) at org.apache.cloudstack.ha.task.BaseHATask.call(BaseHATask.java:37) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util. concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) outing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"} not being up 2026-02-16 05:53:39,242 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-27:[]) (logid:) Setting Host {"id":8,"name":"compute3","type":"Routing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"} to "Disconnected" status. 2026-02-16 05:53:39,244 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-27:[]) (logid:) Investigating Host {"id":8,"name":"compute3","type":"Routing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"} via neighbouring Host {"id":5,"name":"compute1","type":"Routing","uuid":"54056d1f-2ba5-4361-950d-63490eae20ba"}. 2026-02-16 05:53:39,245 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (BackgroundTaskPollManager-6:[ctx-046cf5e7]) (logid:807a4423) Retrieving all "Up" hosts from cluster [Cluster {id: "3", name: "IEM_Compute_Cluster_1", uuid: "ba7c3ced-1cad-4d86-b868-4163d5fba7dd"}]... 2026-02-16 05:53:39,246 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (pool-2-thread-27:[]) (logid:) Wait time setting on com.cloud.agent.api.CheckOnHostCommand is 20 seconds 2026-02-16 05:53:39,246 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-10:[]) (logid:) Checking Host {"id":5,"name":"compute1","type":"Routing","uuid":"54056d1f-2ba5-4361-950d-63490eae20ba"} status... 2026-02-16 05:53:39,247 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (pool-2-thread-10:[]) (logid:) Wait time setting on com.cloud.agent.api.CheckOnHostCommand is 20 seconds 2026-02-16 05:53:39,248 DEBUG [c.c.a.m.ClusteredAgentAttache] (pool-2-thread-27:[]) (logid:) Seq 6-6009209277795865144: Routed from 163763490546348 2026-02-16 05:53:39,248 DEBUG [c.c.a.t.Request] (pool-2-thread-27:[]) (logid:) Seq 5-6009209277795865144: Sending { Cmd , MgmtId: 163763490546348, via: 5(compute1), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"5cbb4794-681d-349e-8632-69348bdd67e2-LibvirtCom putingResource","privateNetwork":{"ip":"172.16.17.205","netmask":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"},"publicNetwork":{"ip":"172.16.17.205","netmask":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"},"storageNetwork1":{"ip":"172.16.17.205","netmask":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"}},"reportCheckFailureIfOneStorageIsDown":"true","wait":"20","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,249 DEBUG [c.c.a.m.ClusteredAgentAttache] (pool-2-thread-10:[]) (logid:) Seq 6-6009209277795865145: Routed from 163763490546348 2026-02-16 05:53:39,249 DEBUG [c.c.a.t.Request] (pool-2-thread-10:[]) (logid:) Seq 5-6009209277795865145: Sending { Cmd , MgmtId: 163763490546348, via: 5(compute1), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"acd4f02e-1b9f-39dc-bf36-9532c2463d62-LibvirtComputingResource","privateNetwork":{"ip":"172.16.17.203","netmask":"255.25 5.255.0","mac":"5a:41:d9:58:bd:d1","securityGroupEnabled":"false"},"publicNetwork":{"ip":"172.16.17.203","netmask":"255.255.255.0","mac":"5a:41:d9:58:bd:d1","securityGroupEnabled":"false"},"storageNetwork1":{"ip":"172.16.17.203","netmask":"255.255.255.0","mac":"5a:41:d9:58:bd:d1","securityGroupEnabled":"false"}},"reportCheckFailureIfOneStorageIsDown":"true","wait":"20","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,301 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:[]) (logid:) Seq 9-6636617000883919100: Processing: { Ans: , MgmtId: 163763490546348, via: 9, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,301 DEBUG [c.c.a.t.Request] (pool-2-thread-20:[]) (logid:) Seq 9-6636617000883919100: Received: { Ans: , MgmtId: 163763490546348, via: 9(compute2), Ver: v1, Flags: 10, { Answer } } 2026-02-16 05:53:39,301 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (pool -2-thread-20:[]) (logid:) Details from executing class com.cloud.agent.api.CheckOnHostCommand: Heart is beating... 2026-02-16 05:53:39,302 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-20:[]) (logid:) Host {"id":9,"name":"compute2","type":"Routing","uuid":"6a96b74f-b114-4b26-b810-018ab626e575"} has the status [Up]. 2026-02-16 05:53:39,302 DEBUG [o.a.c.h.HAManagerImpl] (pool-2-thread-40:[]) (logid:) HA state pre-transition:: new state=[Available], old state=[Available], for resource id=[9], status=[true], ha config state=[Available]. 2026-02-16 05:53:39,307 DEBUG [c.c.a.t.Request] (AgentManager-Handler-4:[]) (logid:) Seq 5-6009209277795865144: Processing: { Ans: , MgmtId: 163763490546348, via: 5, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,307 DEBUG [c.c.a.t.Request] (pool-2-thread-27:[]) (logid:) Seq 5-6009209277795865144: Received: { Ans: , MgmtId: 163763490546348, via: 5(compute1) , Ver: v1, Flags: 10, { Answer } } 2026-02-16 05:53:39,308 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-27:[]) (logid:) Neighbouring Host {"id":5,"name":"compute1","type":"Routing","uuid":"54056d1f-2ba5-4361-950d-63490eae20ba"} returned status [Down] for the investigated Host {"id":8,"name":"compute3","type":"Routing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"}. 2026-02-16 05:53:39,308 DEBUG [c.c.a.t.Request] (AgentManager-Handler-10:[]) (logid:) Seq 5-6009209277795865145: Processing: { Ans: , MgmtId: 163763490546348, via: 5, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Heart is beating...","wait":"0","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,308 DEBUG [c.c.a.t.Request] (pool-2-thread-10:[]) (logid:) Seq 5-6009209277795865145: Received: { Ans: , MgmtId: 163763490546348, via: 5(compute1), Ver: v1, Flags: 10, { Answer } } 2026-02-16 05:53:39,309 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (pool-2-thread-10:[]) (logid:) Deta ils from executing class com.cloud.agent.api.CheckOnHostCommand: Heart is beating... 2026-02-16 05:53:39,309 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-40:[]) (logid:) Retrieving all "Up" hosts from cluster [Cluster {id: "3", name: "IEM_Compute_Cluster_1", uuid: "ba7c3ced-1cad-4d86-b868-4163d5fba7dd"}]... 2026-02-16 05:53:39,309 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-27:[]) (logid:) Investigating Host {"id":8,"name":"compute3","type":"Routing","uuid":"c64a6f6b-3b43-43c7-b6b5-9b2510a056b0"} via neighbouring Host {"id":9,"name":"compute2","type":"Routing","uuid":"6a96b74f-b114-4b26-b810-018ab626e575"}. 2026-02-16 05:53:39,309 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-10:[]) (logid:) Host {"id":5,"name":"compute1","type":"Routing","uuid":"54056d1f-2ba5-4361-950d-63490eae20ba"} has the status [Up]. 2026-02-16 05:53:39,309 DEBUG [o.a.c.h.HAManagerImpl] (pool-2-thread-11:[]) (logid:) HA state pre-transition:: new state=[Available], old state=[Av ailable], for resource id=[5], status=[true], ha config state=[Available]. 2026-02-16 05:53:39,310 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (pool-2-thread-27:[]) (logid:) Wait time setting on com.cloud.agent.api.CheckOnHostCommand is 20 seconds 2026-02-16 05:53:39,311 DEBUG [c.c.a.m.ClusteredAgentAttache] (pool-2-thread-27:[]) (logid:) Seq 6-6636617000883919101: Routed from 163763490546348 2026-02-16 05:53:39,312 DEBUG [c.c.a.t.Request] (pool-2-thread-27:[]) (logid:) Seq 9-6636617000883919101: Sending { Cmd , MgmtId: 163763490546348, via: 9(compute2), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CheckOnHostCommand":{"host":{"guid":"5cbb4794-681d-349e-8632-69348bdd67e2-LibvirtComputingResource","privateNetwork":{"ip":"172.16.17.205","netmask":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"},"publicNetwork":{"ip":"172.16.17.205","netmask":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"},"storageNetwork1":{"ip":"172.16.17.205","netma sk":"255.255.255.0","mac":"56:f2:7a:a7:be:54","securityGroupEnabled":"false"}},"reportCheckFailureIfOneStorageIsDown":"true","wait":"20","bypassHostMaintenance":"false"}}] } 2026-02-16 05:53:39,316 DEBUG [o.a.c.k.h.KVMHostActivityChecker] (pool-2-thread-11:[]) (logid:) Retrieving all "Up" hosts from cluster [Cluster {id: "3", name: "IEM_Compute_Cluster_1", uuid: "ba7c3ced-1cad-4d86-b868-4163d5fba7dd"}]... 2026-02-16 05:53:39,345 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-13:[]) (logid:) Ping from Routing host AgentAttache {"_id":9,"_name":"compute2","_uuid":"6a96b74f-b114-4b26-b810-018ab626e575"} 2026-02-16 05:53:39,347 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-13:[]) (logid:) Process host VM state report from ping process. host: Host {"id":9,"name":"compute2","type":"Routing","uuid":"6a96b74f-b114-4b26-b810-018ab626e575"} 2026-02-16 05:53:39,348 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-13:[]) (logid:) Proce ss VM state report. Host {"id":9,"name":"compute2","type":"Routing","uuid":"6a96b74f-b114-4b26-b810-018ab626e575"}, number of records in report: 9. VMs: [321:PowerOn, 197:PowerOn, 248:PowerOn, 25:PowerOn, 314:PowerOn, 187:PowerOn, 316:PowerOn, 318:PowerOn, 319:PowerOn]] 2026-02-16 05:53:39,351 DEBUG GitHub link: https://github.com/apache/cloudstack/discussions/12629#discussioncomment-15819802 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
