Hello,
I had a server fail today, because of that I restored the Cloudstack
management on a fresh install and used the last db backup I had. Everything
looks fine in the UI, but not working, not sure why I see time out for all
hosts, this is not true, I can reach all the hosts via ip or hostname from
cloudstack management. ( all the hosts are connected in vCenter)
I see these events in the logs for all hosts.
2022-08-08 15:53:16,533 WARN [c.c.v.UserVmManagerImpl]
(StatsCollector-2:ctx-faa5a90c) (logid:0c0862da) Unable to obtain VM
statistics.
2022-08-08 15:53:16,548 DEBUG [c.c.a.m.ClusteredAgentAttache]
(StatsCollector-2:ctx-faa5a90c) (logid:0c0862da) Seq 42-8639592935156875294:
Forwarding null to 39908814623087
2022-08-08 15:53:16,548 DEBUG [c.c.a.m.ClusteredAgentAttache]
(AgentManager-Handler-8:null) (logid:) Seq 42-8639592935156875294: Routing
from 2130706433
2022-08-08 15:53:16,548 DEBUG [c.c.a.m.ClusteredAgentAttache]
(AgentManager-Handler-8:null) (logid:) Seq 42-8639592935156875294: Link is
closed
2022-08-08 15:53:16,548 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-8:null) (logid:) Seq 42-8639592935156875294: MgmtId
2130706433: Req: Resource [Host:42] is unreachable: Host 42: Link is closed
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-3:null) (logid:) Seq 37--1: MgmtId 2130706433: Req:
Routing to peer
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-4:null) (logid:) Seq 37--1: MgmtId 2130706433: Req:
Cancel request received
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-4:null) (logid:) Seq 37-3913346601208250383:
Cancelling.
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.AgentAttache]
(StatsCollector-6:ctx-353c6d50) (logid:7f9ce0ca) Seq 37-3913346601208250383:
Waiting some more time because this is the current command
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.AgentAttache]
(StatsCollector-6:ctx-353c6d50) (logid:7f9ce0ca) Seq 37-3913346601208250383:
Waiting some more time because this is the current command
2022-08-08 15:54:13,213 INFO [c.c.u.e.CSExceptionErrorCode]
(StatsCollector-6:ctx-353c6d50) (logid:7f9ce0ca) Could not find exception:
com.cloud.exception.OperationTimedoutException in error code list for
exceptions
2022-08-08 15:54:13,213 WARN [c.c.a.m.AgentAttache]
(StatsCollector-6:ctx-353c6d50) (logid:7f9ce0ca) Seq 37-3913346601208250383:
Timed out on null
2022-08-08 15:54:13,213 DEBUG [c.c.a.m.AgentAttache]
(StatsCollector-6:ctx-353c6d50) (logid:7f9ce0ca) Seq 37-3913346601208250383:
Cancelling.
The only difference right now, is the OS, I restored on rocky Linux 8,
previously I had ubuntu 20.04.
The cloudstack version is 4.16.1.0
Regards,
Cristian
