Damans227 commented on PR #13662:
URL: https://github.com/apache/cloudstack/pull/13662#issuecomment-5611850158

   one more from the lab. every time the msid changes, the cluster reads the 
old rows as management servers that died and runs the failover cleanup. single 
node lab, one machine:
   
   ```
   INFO [c.c.a.m.ClusteredAgentManagerImpl] Marking hosts as disconnected on 
Management server {"id":3,"msid":163047240293385,...}
   INFO [c.c.a.m.ClusteredAgentManagerImpl] Deleting entries from 
op_host_transfer table for Management server {"id":1,"msid":32987445986261,...}
   INFO [c.c.c.ClusterManagerImpl] Found 2 inactive management server node 
based on timestamp
   ```
   
   kvm host and both systemvms went Disconnected, `host.mgmt_server_id` 
emptied, and a vm deploy failed:
   
   ```
   DEBUG [o.a.c.a.c.a.v.DeployVMCmdByAdmin] No destination found for a 
deployment for VM instance ...
     com.cloud.exception.InsufficientServerCapacityException
   DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] Complete async job-38, jobStatus: 
FAILED, resultCode: 533
   ```
   
   agent said `Lost connection to host: 10.0.35.66. Attempting reconnection` 
then sat there 8 mins with no retry. came back straight away on `systemctl 
restart cloudstack-agent`.
   
   the one time switch into fqdn mode was clean, so this is only about the id 
changing when you didnt expect it, which is the dns thing in the other comment.
   
   worth documenting the switch as a procedure rather than just a restart?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to