It is CentOS Linux release 7.6.1810 (Core) Agent is misleading. virsh says no domain is in KVM.
[root@fr-kvm1 ~]# virsh list --all Id Name State ---------------------------------------------------- [root@fr-kvm1 ~]# On Wed, Aug 28, 2019 at 10:46 PM Andrija Panic <[email protected]> wrote: > That is brave :) I mean running unsupported CentOS8 with ACS. Seriously, if > not mistaken, we plan to support it in 4.14 only, and this might be causing > issues as well. > > Your KVM host reports its already running 3 VMs, and that there are pending > tasks on some of the VMs. > > Make sure that only VMs that exist in ACS database, also exist on KVM, and > virsh destroy all others (perhaps disable zone, destroy any running system > VMs and then make sure KVM vs DB vm list iz in correct sync.) > > Vlan 100 you were using for Public VLAN? Not sure how is your cloudbr1 > (used for public traffic) configured and what vlans etc you are using now. > How did you change vlan 100? > > Andrija > > > On Wed, Aug 28, 2019, 19:32 Fariborz Navidan <[email protected]> > wrote: > > > Any idea? I have also already managed devpts issue which causes issues on > > CentOS 8. I have copied devpts line from /etc/fstab from another host. > Also > > SELinux is already disabled > > > > On Wed, Aug 28, 2019 at 9:31 PM Fariborz Navidan <[email protected]> > > wrote: > > > > > This is agent log at debug log level: > > > > > > [root@fr-kvm1 primary1]# cat /var/log/cloudstack/agent/agent.log > > > > > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Received response: Seq 1-1: { Ans: , MgmtId: 157082893985413, > > > via: -1, Ver: v1, Flags: 100000, > > > > > > [{"com.cloud.agent.api.StartupAnswer":{"hostId":0,"pingInterval":60,"result":true,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,273 DEBUG [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Startup task cancelled > > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Proccess agent startup answer, agent id = 0 > > > 2019-08-28 18:59:16,273 INFO [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Set agent id 0 > > > 2019-08-28 18:59:16,276 DEBUG [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Adding a watch list > > > 2019-08-28 18:59:16,277 INFO [cloud.agent.Agent] > (Agent-Handler-2:null) > > > (logid:) Startup Response Received: agent id = 0 > > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-1:null) (logid:) Executing: > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py > > > get_rule_logs_for_vms > > > 2019-08-28 18:59:16,277 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-1:null) (logid:) Executing while with timeout : 1800000 > > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987393: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100011, > > > > > > [{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2473,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.CleanupNetworkRulesCmd > > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Adding a watch list > > > 2019-08-28 18:59:16,283 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq > 1-4941856166108987393: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10, > > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:16,284 DEBUG [org.reflections.Reflections] > > > (Agent-Handler-4:null) (logid:1f7ef831) going to scan these urls: > > > > > > > > > jar:file:/usr/share/cloudstack-agent/lib/cloud-plugin-hypervisor-kvm-4.12.0.0.jar!/ > > > 2019-08-28 18:59:16,295 INFO [org.reflections.Reflections] > > > (Agent-Handler-4:null) (logid:1f7ef831) Reflections took 9 ms to scan 1 > > > urls, producing 5 keys and 147 values > > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections] > > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype > > > java.lang.Comparable -> java.lang.Enum > > > 2019-08-28 18:59:16,296 DEBUG [org.reflections.Reflections] > > > (Agent-Handler-4:null) (logid:1f7ef831) expanded subtype > > > java.io.Serializable -> java.lang.Enum > > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987394: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100111, > > > > > > [{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":200,"privateNetworkName":"cloudbr0","guestNetworkName":"cloudbr1","storageNetworkName":"cloudbr0"}],"wait":0}}] > > > } > > > 2019-08-28 18:59:16,326 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.CheckNetworkCommand > > > 2019-08-28 18:59:16,327 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-1:null) (logid:) Execution is successful. > > > 2019-08-28 18:59:16,328 DEBUG [kvm.resource.LibvirtConnection] > > > (UgentTask-1:null) (logid:) Looking for libvirtd connection at: > > > qemu:///system > > > 2019-08-28 18:59:16,329 DEBUG [cloud.agent.Agent] (UgentTask-1:null) > > > (logid:) Sending ping: Seq 0-2: { Cmd , MgmtId: -1, via: 0, Ver: v1, > > > Flags: 11, > > > > > > [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":0,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,334 DEBUG [kvm.resource.LibvirtComputingResource] > > > (Agent-Handler-4:null) (logid:1f7ef831) Executing: > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py > > > cleanup_rules > > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory: > > > file name 'eth1.100' > > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource] > > > (Agent-Handler-4:null) (logid:1f7ef831) Executing while with timeout : > > > 1800000 > > > 2019-08-28 18:59:16,335 DEBUG [kvm.resource.LibvirtComputingResource] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) matchPifFileInDirectory: > > > file name 'eth1' > > > 2019-08-28 18:59:16,335 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq > 1-4941856166108987394: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110, > > > > > > [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":false,"result":true,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,396 DEBUG [kvm.resource.LibvirtComputingResource] > > > (Agent-Handler-4:null) (logid:1f7ef831) Execution is successful. > > > 2019-08-28 18:59:16,397 DEBUG [cloud.agent.Agent] > (Agent-Handler-4:null) > > > (logid:1f7ef831) Watch Sent: Seq 1-4941856166108987393: { Ans: , > MgmtId: > > > 157082893985413, via: 1, Ver: v1, Flags: 10, > > > [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] > } > > > 2019-08-28 18:59:16,398 DEBUG [cloud.agent.Agent] > (Agent-Handler-1:null) > > > (logid:) Received response: Seq 0-2: { Ans: , MgmtId: 157082893985413, > > > via: 1, Ver: v1, Flags: 100010, > > > > > > [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":0,"wait":0},"result":true,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987395: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100011, > > > > > > [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":3,"uuid":"5ea00099-5b91-3787-b948-8a18c40f3a88","host":"192.168.0.2","path":"/home/primary1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","wait":0}}] > > > } > > > 2019-08-28 18:59:16,399 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.ModifyStoragePoolCommand > > > 2019-08-28 18:59:16,399 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Attempting to create > > storage > > > pool 5ea00099-5b91-3787-b948-8a18c40f3a88 (NetworkFilesystem) in > libvirt > > > 2019-08-28 18:59:16,399 DEBUG [kvm.resource.LibvirtConnection] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd > > > connection at: qemu:///system > > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Found existing defined > > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88, using it. > > > 2019-08-28 18:59:16,400 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Trying to fetch storage > > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt > > > 2019-08-28 18:59:16,400 DEBUG [kvm.resource.LibvirtConnection] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Looking for libvirtd > > > connection at: qemu:///system > > > 2019-08-28 18:59:16,402 DEBUG [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Succesfully refreshed > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: > > 727711744 > > > Available: 984168267776 > > > 2019-08-28 18:59:16,403 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq > 1-4941856166108987395: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10, > > > > > > [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"192.168.0.2","localPath":"/mnt//5ea00099-5b91-3787-b948-8a18c40f3a88","hostPath":"/home/primary1","poolType":"NetworkFilesystem","capacityBytes":984895979520,"availableBytes":984168267776},"templateInfo":{},"result":true,"wait":0}}] > > > } > > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987397: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100111, > > > > > > [{"com.cloud.agent.api.SetHostParamsCommand":{"params":{"router.aggregation.command.each.timeout":"600"},"wait":0}}] > > > } > > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987396: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100011, > [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] > > } > > > 2019-08-28 18:59:16,524 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.ModifySshKeysCommand > > > 2019-08-28 18:59:16,523 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.SetHostParamsCommand > > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:1f7ef831) Seq > 1-4941856166108987397: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110, > > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987398: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100011, > [{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] > > } > > > 2019-08-28 18:59:16,525 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.ModifySshKeysCommand > > > 2019-08-28 18:59:16,529 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing: chmod 600 > > > /root/.ssh/id_rsa.cloud > > > 2019-08-28 18:59:16,529 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing: chmod 600 > > > /root/.ssh/id_rsa.cloud > > > 2019-08-28 18:59:16,531 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Executing while with > > timeout > > > : 1800000 > > > 2019-08-28 18:59:16,532 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Executing while with > > timeout > > > : 1800000 > > > 2019-08-28 18:59:16,534 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Execution is successful. > > > 2019-08-28 18:59:16,534 DEBUG > > > [resource.wrapper.LibvirtModifySshKeysCommandWrapper] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Execution is successful. > > > 2019-08-28 18:59:16,536 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:1f7ef831) Seq > 1-4941856166108987398: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10, > > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:16,537 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-5:null) (logid:1f7ef831) Seq > 1-4941856166108987396: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 10, > > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987399: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100111, > > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] } > > > 2019-08-28 18:59:16,542 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.ReadyCommand > > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Processing agent ready > > > command, agent id = 1 > > > 2019-08-28 18:59:16,542 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Set agent id 1 > > > 2019-08-28 18:59:16,551 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Ready command is > processed > > > for agent id = 1 > > > 2019-08-28 18:59:16,553 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:1f7ef831) Seq > 1-4941856166108987399: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110, > > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Request:Seq > > > 1-4941856166108987400: { Cmd , MgmtId: 157082893985413, via: 1, Ver: > v1, > > > Flags: 100111, > > > [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":1,"wait":0}}] } > > > 2019-08-28 18:59:16,641 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing command: > > > com.cloud.agent.api.ReadyCommand > > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Processing agent ready > > > command, agent id = 1 > > > 2019-08-28 18:59:16,641 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Set agent id 1 > > > 2019-08-28 18:59:16,642 INFO [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Ready command is > processed > > > for agent id = 1 > > > 2019-08-28 18:59:16,643 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-4:null) (logid:1f7ef831) Seq > 1-4941856166108987400: > > > { Ans: , MgmtId: 157082893985413, via: 1, Ver: v1, Flags: 110, > > > [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] } > > > 2019-08-28 18:59:25,626 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:9391b96b) Processing command: > > > com.cloud.agent.api.GetHostStatsCommand > > > 2019-08-28 18:59:27,984 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Processing command: > > > com.cloud.agent.api.GetStorageStatsCommand > > > 2019-08-28 18:59:27,985 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Trying to fetch storage > > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt > > > 2019-08-28 18:59:27,985 DEBUG [kvm.resource.LibvirtConnection] > > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Looking for libvirtd > > > connection at: qemu:///system > > > 2019-08-28 18:59:27,989 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Asking libvirt to > refresh > > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 > > > 2019-08-28 18:59:27,992 DEBUG [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-2:null) (logid:2fe46c5c) Succesfully refreshed > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: > > 727711744 > > > Available: 984168267776 > > > 2019-08-28 19:00:04,730 DEBUG [kvm.resource.LibvirtConnection] > > > (Thread-8:null) (logid:) Looking for libvirtd connection at: > > qemu:///system > > > 2019-08-28 19:00:04,733 DEBUG [kvm.resource.KVMHAMonitor] > (Thread-8:null) > > > (logid:) Found NFS storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 in > > > libvirt, continuing > > > 2019-08-28 19:00:04,734 DEBUG [kvm.resource.KVMHAMonitor] > (Thread-8:null) > > > (logid:) Executing: > > > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh > -i > > > 192.168.0.2 -p /home/primary1 -m > > /mnt/5ea00099-5b91-3787-b948-8a18c40f3a88 > > > -h 192.168.0.2 > > > 2019-08-28 19:00:04,735 DEBUG [kvm.resource.KVMHAMonitor] > (Thread-8:null) > > > (logid:) Executing while with timeout : 60000 > > > 2019-08-28 19:00:04,741 DEBUG [kvm.resource.KVMHAMonitor] > (Thread-8:null) > > > (logid:) Execution is successful. > > > 2019-08-28 19:00:16,278 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-2:null) (logid:) Executing: > > > /usr/share/cloudstack-common/scripts/vm/network/security_group.py > > > get_rule_logs_for_vms > > > 2019-08-28 19:00:16,281 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-2:null) (logid:) Executing while with timeout : 1800000 > > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtComputingResource] > > > (UgentTask-2:null) (logid:) Execution is successful. > > > 2019-08-28 19:00:16,331 DEBUG [kvm.resource.LibvirtConnection] > > > (UgentTask-2:null) (logid:) Looking for libvirtd connection at: > > > qemu:///system > > > 2019-08-28 19:00:16,332 DEBUG [cloud.agent.Agent] (UgentTask-2:null) > > > (logid:) Sending ping: Seq 1-3: { Cmd , MgmtId: -1, via: 1, Ver: v1, > > > Flags: 11, > > > > > > [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}] > > > } > > > 2019-08-28 19:00:16,377 DEBUG [cloud.agent.Agent] > (Agent-Handler-4:null) > > > (logid:1f7ef831) Received response: Seq 1-3: { Ans: , MgmtId: > > > 157082893985413, via: 1, Ver: v1, Flags: 100010, > > > > > > [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}] > > > } > > > 2019-08-28 19:00:25,663 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-5:null) (logid:919a0a37) Processing command: > > > com.cloud.agent.api.GetHostStatsCommand > > > 2019-08-28 19:00:28,053 DEBUG [cloud.agent.Agent] > > > (agentRequest-Handler-3:null) (logid:79632973) Processing command: > > > com.cloud.agent.api.GetStorageStatsCommand > > > 2019-08-28 19:00:28,053 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:79632973) Trying to fetch storage > > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 from libvirt > > > 2019-08-28 19:00:28,053 DEBUG [kvm.resource.LibvirtConnection] > > > (agentRequest-Handler-3:null) (logid:79632973) Looking for libvirtd > > > connection at: qemu:///system > > > 2019-08-28 19:00:28,058 INFO [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:79632973) Asking libvirt to > refresh > > > storage pool 5ea00099-5b91-3787-b948-8a18c40f3a88 > > > 2019-08-28 19:00:28,061 DEBUG [kvm.storage.LibvirtStorageAdaptor] > > > (agentRequest-Handler-3:null) (logid:79632973) Succesfully refreshed > pool > > > 5ea00099-5b91-3787-b948-8a18c40f3a88 Capacity: 984895979520 Used: > > 727711744 > > > Available: 984168267776 > > > [root@fr-kvm1 primary1]# > > > > > > On Wed, Aug 28, 2019 at 8:43 PM Andrija Panic <[email protected] > > > > > wrote: > > > > > >> I Don't see anything attached from agent.log? Make sure to put agent > in > > >> debug mode before posting logs (google for "cloudstack agent debug > > mode", > > >> there is a simple sed command, restart agent afterwards). > > >> > > >> Andrija > > >> > > >> On Wed, Aug 28, 2019, 17:44 Fariborz Navidan <[email protected]> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > Also this is agent log > > >> > > > >> > On Wed, Aug 28, 2019 at 6:41 PM Fariborz Navidan < > > [email protected] > > >> > > > >> > wrote: > > >> > > > >> > > Any idea? > > >> > > > > >> > > On Wed, Aug 28, 2019 at 5:09 PM Fariborz Navidan < > > >> [email protected]> > > >> > > wrote: > > >> > > > > >> > >> Thanks. I changed VLAN ID and now they are kept starting for > along > > >> time. > > >> > >> Latest logs: > > >> > >> > > >> > >> > > >> > >> 2019-08-28 14:34:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:04,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-e4b25970) (logid:4ca3d1a2) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor] > > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-318) has been > > >> pending > > >> > for > > >> > >> 959 seconds > > >> > >> 2019-08-28 14:34:10,577 WARN [o.a.c.f.j.i.AsyncJobMonitor] > > >> > >> (Timer-0:ctx-b00ab490) (logid:c9e4081e) Task (job-320) has been > > >> pending > > >> > for > > >> > >> 357 seconds > > >> > >> 2019-08-28 14:34:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:14,852 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 > > >> routers to > > >> > >> update status. > > >> > >> 2019-08-28 14:34:14,854 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 VPC > > >> > networks > > >> > >> to update Redundant State. > > >> > >> 2019-08-28 14:34:14,857 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-1d77361c) (logid:9bc2f8a4) Found 0 > > >> networks > > >> > to > > >> > >> update RvR status. > > >> > >> 2019-08-28 14:34:14,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-665ba8c1) (logid:07ce74f8) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:17,628 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-267:ctx-7f61f42c) (logid:69ceb90d) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598229 > > >> > >> 2019-08-28 14:34:17,637 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:17,659 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-267:ctx-7f61f42c ctx-c6adb54b) (logid:69ceb90d) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598229 > > >> > >> 2019-08-28 14:34:19,922 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Resetting > > >> > hosts > > >> > >> suitable for reconnect > > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Completed > > >> > >> resetting hosts suitable for reconnect > > >> > >> 2019-08-28 14:34:19,924 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Acquiring > > >> > hosts > > >> > >> for clusters already owned by this management server > > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Completed > > >> > >> acquiring hosts for clusters already owned by this management > > server > > >> > >> 2019-08-28 14:34:19,925 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Acquiring > > >> > hosts > > >> > >> for clusters not owned by any management server > > >> > >> 2019-08-28 14:34:19,926 DEBUG [c.c.h.d.HostDaoImpl] > > >> > >> (ClusteredAgentManager Timer:ctx-e1f47dc0) (logid:c0c93cd1) > > Completed > > >> > >> acquiring hosts for clusters not owned by any management server > > >> > >> 2019-08-28 14:34:20,494 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-51d12c34) (logid:f8b01d6e) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598230 > > >> > >> 2019-08-28 14:34:20,500 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:20,504 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-51d12c34 ctx-6814b565) (logid:f8b01d6e) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598230 > > >> > >> 2019-08-28 14:34:24,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:24,865 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-5b6871f5) (logid:01bd640f) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:24,974 DEBUG [c.c.a.m.AgentManagerImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Ping from 1( > > >> fr-kvm1.r9host.com) > > >> > >> 2019-08-28 14:34:24,974 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Process host VM state > > report > > >> > from > > >> > >> ping process. host: 1 > > >> > >> 2019-08-28 14:34:24,975 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Process VM state report. > > >> host: > > >> > 1, > > >> > >> number of records in report: 1 > > >> > >> 2019-08-28 14:34:24,975 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report. host: 1, > > vm > > >> id: > > >> > >> 103, power state: PowerOn > > >> > >> 2019-08-28 14:34:24,976 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is > updated. > > >> > host: > > >> > >> 1, vm id: 103, power state: PowerOn > > >> > >> 2019-08-28 14:34:24,977 INFO [c.c.v.VirtualMachineManagerImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or > HA > > >> tasks > > >> > >> working on the VM. vm id: 103, postpone power-change report by > > >> resetting > > >> > >> power-change counters > > >> > >> 2019-08-28 14:34:24,980 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Run missing VM report. > > >> current > > >> > >> time: 1566995664980 > > >> > >> 2019-08-28 14:34:24,980 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. > host: > > >> 1, vm > > >> > >> id: 101, power state: PowerReportMissing, last state update: > > >> > 1566995544000 > > >> > >> 2019-08-28 14:34:24,980 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 101 - time since > > last > > >> > state > > >> > >> update(120980ms) has passed graceful period > > >> > >> 2019-08-28 14:34:24,982 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is > updated. > > >> > host: > > >> > >> 1, vm id: 101, power state: PowerReportMissing > > >> > >> 2019-08-28 14:34:24,982 INFO [c.c.v.VirtualMachineManagerImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or > HA > > >> tasks > > >> > >> working on the VM. vm id: 101, postpone power-change report by > > >> resetting > > >> > >> power-change counters > > >> > >> 2019-08-28 14:34:24,984 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Detected missing VM. > host: > > >> 1, vm > > >> > >> id: 102, power state: PowerReportMissing, last state update: > > >> > 1566995544000 > > >> > >> 2019-08-28 14:34:24,984 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) vm id: 102 - time since > > last > > >> > state > > >> > >> update(120980ms) has passed graceful period > > >> > >> 2019-08-28 14:34:24,986 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) VM state report is > updated. > > >> > host: > > >> > >> 1, vm id: 102, power state: PowerReportMissing > > >> > >> 2019-08-28 14:34:24,986 INFO [c.c.v.VirtualMachineManagerImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) There is pending job or > HA > > >> tasks > > >> > >> working on the VM. vm id: 102, postpone power-change report by > > >> resetting > > >> > >> power-change counters > > >> > >> 2019-08-28 14:34:24,987 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-15:null) (logid:) Done with process of VM > > state > > >> > >> report. host: 1 > > >> > >> 2019-08-28 14:34:26,012 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-72cb5770) (logid:d2576b2d) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598231 > > >> > >> 2019-08-28 14:34:26,020 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:26,044 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-72cb5770 ctx-67029176) (logid:d2576b2d) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598231 > > >> > >> 2019-08-28 14:34:29,854 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-61e00b9c) (logid:6a21df96) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598232 > > >> > >> 2019-08-28 14:34:29,860 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:29,864 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-61e00b9c ctx-416ae397) (logid:6a21df96) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598232 > > >> > >> 2019-08-28 14:34:30,176 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-2:ctx-fb2716cb) (logid:0d4c72ca) AutoScaling > > Monitor > > >> is > > >> > >> running... > > >> > >> 2019-08-28 14:34:33,240 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-3062efb0) (logid:17ce85f1) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233 > > >> > >> 2019-08-28 14:34:33,249 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:33,267 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-3062efb0 ctx-2dfc105f) (logid:17ce85f1) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598233 > > >> > >> 2019-08-28 14:34:33,396 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-e40e8ed5) (logid:579437b8) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234 > > >> > >> 2019-08-28 14:34:33,404 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:33,415 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-e40e8ed5 ctx-1610f3df) (logid:579437b8) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598234 > > >> > >> 2019-08-28 14:34:33,532 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-4f090298) (logid:1929ff2c) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598235 > > >> > >> 2019-08-28 14:34:33,540 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:33,546 INFO [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) > Api > > >> > >> Discovery plugin was unable to find an api by that name or > process > > >> any > > >> > apis > > >> > >> 2019-08-28 14:34:33,546 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-4f090298 ctx-edba44a4) (logid:1929ff2c) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598235 > > >> > >> 2019-08-28 14:34:34,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-3db42e00) (logid:4d8db73d) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:35,481 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-e9a1d962) (logid:12e0b85c) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236 > > >> > >> 2019-08-28 14:34:35,487 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:35,494 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-e9a1d962 ctx-88b9d315) (logid:12e0b85c) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Disabled&response=json&_=1566995598236 > > >> > >> 2019-08-28 14:34:37,341 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) > HostStatsCollector > > >> is > > >> > >> running... > > >> > >> 2019-08-28 14:34:37,398 DEBUG [c.c.a.t.Request] > > >> > >> (StatsCollector-3:ctx-4c48013d) (logid:6aaadd07) Seq > > >> > 1-5778962746846479041: > > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1( > > >> fr-kvm1.r9host.com > > >> > ), > > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } } > > >> > >> 2019-08-28 14:34:38,526 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) StorageCollector > > is > > >> > >> running... > > >> > >> 2019-08-28 14:34:38,536 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) There is no > > >> secondary > > >> > >> storage VM for secondary storage host Secondary1 > > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] > > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) > > >> > getCommandHostDelegation: > > >> > >> class com.cloud.agent.api.GetStorageStatsCommand > > >> > >> 2019-08-28 14:34:38,539 DEBUG [c.c.h.XenServerGuru] > > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) We are returning > > the > > >> > >> default host to execute commands because the command is not of > Copy > > >> > type. > > >> > >> 2019-08-28 14:34:38,592 DEBUG [c.c.a.t.Request] > > >> > >> (StatsCollector-6:ctx-1caf1bfa) (logid:21e40ccc) Seq > > >> > 1-5778962746846479042: > > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1( > > >> fr-kvm1.r9host.com > > >> > ), > > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } } > > >> > >> 2019-08-28 14:34:38,732 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-b742baaf) (logid:fd475e73) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237 > > >> > >> 2019-08-28 14:34:38,740 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:38,759 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-b742baaf ctx-e306b339) (logid:fd475e73) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598237 > > >> > >> 2019-08-28 14:34:38,876 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-78a2ea0b) (logid:53245a8d) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238 > > >> > >> 2019-08-28 14:34:38,884 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:38,894 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-78a2ea0b ctx-54546774) (logid:53245a8d) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598238 > > >> > >> 2019-08-28 14:34:39,017 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-dc6ad71d) (logid:a29f4b7b) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598239 > > >> > >> 2019-08-28 14:34:39,025 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:39,031 INFO [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) > Api > > >> > >> Discovery plugin was unable to find an api by that name or > process > > >> any > > >> > apis > > >> > >> 2019-08-28 14:34:39,032 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-dc6ad71d ctx-d9671a26) (logid:a29f4b7b) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598239 > > >> > >> 2019-08-28 14:34:42,577 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-40d2ea6d) (logid:087bc8bb) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598240 > > >> > >> 2019-08-28 14:34:42,585 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:42,610 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-40d2ea6d ctx-82e1d1f7) (logid:087bc8bb) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598240 > > >> > >> 2019-08-28 14:34:44,850 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:44,852 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 > > >> routers to > > >> > >> update status. > > >> > >> 2019-08-28 14:34:44,855 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 VPC > > >> > networks > > >> > >> to update Redundant State. > > >> > >> 2019-08-28 14:34:44,857 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-45ec83d0) (logid:b2c00ab6) Found 0 > > >> networks > > >> > to > > >> > >> update RvR status. > > >> > >> 2019-08-28 14:34:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-61ce445a) (logid:21c23d72) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:48,198 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-4b52ed63) (logid:6e637323) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241 > > >> > >> 2019-08-28 14:34:48,207 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:48,215 WARN > > [c.c.a.d.ParamGenericValidationWorker] > > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) > > >> Received > > >> > >> unknown parameters for command listSystemVms. Unknown parameters > : > > >> > listall > > >> > >> 2019-08-28 14:34:48,268 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-4b52ed63 ctx-4d66e47d) (logid:6e637323) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598241 > > >> > >> 2019-08-28 14:34:48,462 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-4f0ef9bc) (logid:1065f9c5) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listHosts&response=json&details=min&_=1566995598242 > > >> > >> 2019-08-28 14:34:48,470 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:48,478 DEBUG [c.c.a.q.QueryManagerImpl] > > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) > > >> > >>>Searching > > >> > >> for hosts>>> > > >> > >> 2019-08-28 14:34:48,484 DEBUG [c.c.a.q.QueryManagerImpl] > > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) > > >> > >> >>>Generating Response>>> > > >> > >> 2019-08-28 14:34:48,485 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-4f0ef9bc ctx-ec0ccd73) (logid:1065f9c5) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listHosts&response=json&details=min&_=1566995598242 > > >> > >> 2019-08-28 14:34:54,843 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:34:54,845 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-51f8dd11) (logid:1ac9f638) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:34:55,209 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-d366af14) (logid:3489b330) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243 > > >> > >> 2019-08-28 14:34:55,217 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:55,235 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-d366af14 ctx-893fb42c) (logid:3489b330) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598243 > > >> > >> 2019-08-28 14:34:55,388 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-a73c5049) (logid:2b550fd0) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244 > > >> > >> 2019-08-28 14:34:55,395 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:34:55,448 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-a73c5049 ctx-1a77bfde) (logid:2b550fd0) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598244 > > >> > >> 2019-08-28 14:35:03,782 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-7e0bc117) (logid:50c19de5) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598245 > > >> > >> 2019-08-28 14:35:03,789 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:03,814 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-7e0bc117 ctx-575e2882) (logid:50c19de5) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598245 > > >> > >> 2019-08-28 14:35:04,849 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:35:04,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-21aa8d2e) (logid:1547d462) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:35:06,808 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-41662f1f) (logid:2682bbeb) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598246 > > >> > >> 2019-08-28 14:35:06,816 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:06,834 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-41662f1f ctx-38769150) (logid:2682bbeb) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listZones&page=1&pagesize=20&response=json&_=1566995598246 > > >> > >> 2019-08-28 14:35:08,838 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-77636d90) (logid:a415b9a3) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247 > > >> > >> 2019-08-28 14:35:08,845 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:08,857 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-77636d90 ctx-b8571a42) (logid:a415b9a3) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598247 > > >> > >> 2019-08-28 14:35:08,982 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-409e3da6) (logid:00800f6e) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248 > > >> > >> 2019-08-28 14:35:08,990 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:09,001 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-409e3da6 ctx-29f8f704) (logid:00800f6e) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598248 > > >> > >> 2019-08-28 14:35:09,126 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-67630fc0) (logid:eca6e20c) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598249 > > >> > >> 2019-08-28 14:35:09,134 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:09,142 INFO [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) > Api > > >> > >> Discovery plugin was unable to find an api by that name or > process > > >> any > > >> > apis > > >> > >> 2019-08-28 14:35:09,142 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-67630fc0 ctx-fcea72b5) (logid:eca6e20c) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598249 > > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor] > > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-318) has been > > >> pending > > >> > for > > >> > >> 1019 seconds > > >> > >> 2019-08-28 14:35:10,578 WARN [o.a.c.f.j.i.AsyncJobMonitor] > > >> > >> (Timer-0:ctx-dfc811d7) (logid:9c5404b2) Task (job-320) has been > > >> pending > > >> > for > > >> > >> 417 seconds > > >> > >> 2019-08-28 14:35:13,609 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-0bb9da2a) (logid:185d38c3) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250 > > >> > >> 2019-08-28 14:35:13,617 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:13,657 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-0bb9da2a ctx-7cf333f4) (logid:185d38c3) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=updateZone&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&allocationstate=Enabled&response=json&_=1566995598250 > > >> > >> 2019-08-28 14:35:14,844 INFO [c.c.r.ResourceLimitManagerImpl] > > >> > >> (ResourceCountChecker-1:ctx-2fb83b01) (logid:bd4a6195) Started > > >> resource > > >> > >> counters recalculation periodic task. > > >> > >> 2019-08-28 14:35:14,848 DEBUG > > [c.c.n.ExternalDeviceUsageManagerImpl] > > >> > >> (ExternalNetworkMonitor-1:ctx-c5fc84e6) (logid:094caf32) External > > >> > devices > > >> > >> are not used. Skipping external device usage collection > > >> > >> 2019-08-28 14:35:14,851 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:35:14,852 DEBUG [c.c.s.s.SnapshotSchedulerImpl] > > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Snapshot > > >> > scheduler.poll is > > >> > >> being called at 2019-08-28 12:35:14 GMT > > >> > >> 2019-08-28 14:35:14,857 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-43ce8c2a) (logid:82ac4d0c) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:35:14,857 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 > > >> routers to > > >> > >> update status. > > >> > >> 2019-08-28 14:35:14,857 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > (RouterMonitor-1:ctx-7cb475c8) > > >> > >> (logid:cc3e421d) Found 0 running routers. > > >> > >> 2019-08-28 14:35:14,858 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 VPC > > >> > networks > > >> > >> to update Redundant State. > > >> > >> 2019-08-28 14:35:14,858 DEBUG [c.c.s.s.SnapshotSchedulerImpl] > > >> > >> (SnapshotPollTask:ctx-30a5293f) (logid:17a7f1d7) Got 0 snapshots > to > > >> be > > >> > >> executed at 2019-08-28 12:35:14 GMT > > >> > >> 2019-08-28 14:35:14,858 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-50ce53ae) (logid:56e39189) Found 0 > > >> networks > > >> > to > > >> > >> update RvR status. > > >> > >> 2019-08-28 14:35:14,906 INFO [o.a.c.e.o.NetworkOrchestrator] > > >> > >> (Network-Scavenger-1:ctx-d8aa24b5) (logid:d725dea4) > > >> > NetworkGarbageCollector > > >> > >> uses '600' seconds for GC interval. > > >> > >> 2019-08-28 14:35:14,925 INFO [c.c.h.HighAvailabilityManagerImpl] > > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) checking health of usage > > server > > >> > >> 2019-08-28 14:35:14,928 DEBUG [c.c.h.HighAvailabilityManagerImpl] > > >> > >> (HA-2:ctx-0d95eac6) (logid:b56447b5) usage server running? true, > > >> > heartbeat: > > >> > >> Wed Aug 28 14:35:12 CEST 2019 > > >> > >> 2019-08-28 14:35:16,891 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-17ae8067) (logid:d2ff137a) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251 > > >> > >> 2019-08-28 14:35:16,899 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:16,917 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-17ae8067 ctx-97328468) (logid:d2ff137a) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598251 > > >> > >> 2019-08-28 14:35:17,380 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-c9973280) (logid:8c0fb627) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252 > > >> > >> 2019-08-28 14:35:17,388 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:17,398 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-c9973280 ctx-43947030) (logid:8c0fb627) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listDedicatedZones&response=json&zoneid=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598252 > > >> > >> 2019-08-28 14:35:17,899 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-d990bce3) (logid:b31fa0e8) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598253 > > >> > >> 2019-08-28 14:35:17,905 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:17,906 INFO [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) > Api > > >> > >> Discovery plugin was unable to find an api by that name or > process > > >> any > > >> > apis > > >> > >> 2019-08-28 14:35:17,906 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-d990bce3 ctx-43fdfc37) (logid:b31fa0e8) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > command=listApis&response=json&name=listVmwareDcs&_=1566995598253 > > >> > >> 2019-08-28 14:35:20,040 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-49453fa9) (logid:c0fd2ac6) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598254 > > >> > >> 2019-08-28 14:35:20,048 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:20,071 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-540:ctx-49453fa9 ctx-e7ed0e9f) (logid:c0fd2ac6) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listInfrastructure&response=json&_=1566995598254 > > >> > >> 2019-08-28 14:35:24,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:35:24,860 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-1dca0b93) (logid:6288c36f) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:35:24,889 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] > > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck > > >> Manager > > >> > is > > >> > >> running and getting the updates from LB providers and updating > > >> service > > >> > >> status > > >> > >> 2019-08-28 14:35:24,906 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] > > >> > >> (LBHealthCheck-1:ctx-9111e5e6) (logid:482454fb) LB HealthCheck > > >> Manager > > >> > is > > >> > >> running and getting the updates from LB providers and updating > > >> service > > >> > >> status > > >> > >> 2019-08-28 14:35:25,005 DEBUG [c.c.a.m.AgentManagerImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Ping from 1( > > >> fr-kvm1.r9host.com) > > >> > >> 2019-08-28 14:35:25,005 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Process host VM state > report > > >> from > > >> > >> ping process. host: 1 > > >> > >> 2019-08-28 14:35:25,006 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Process VM state report. > > >> host: 1, > > >> > >> number of records in report: 1 > > >> > >> 2019-08-28 14:35:25,006 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report. host: 1, > vm > > >> id: > > >> > >> 103, power state: PowerOn > > >> > >> 2019-08-28 14:35:25,007 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) VM state report is > updated. > > >> > host: 1, > > >> > >> vm id: 103, power state: PowerOn > > >> > >> 2019-08-28 14:35:25,008 INFO [c.c.v.VirtualMachineManagerImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) There is pending job or HA > > >> tasks > > >> > >> working on the VM. vm id: 103, postpone power-change report by > > >> resetting > > >> > >> power-change counters > > >> > >> 2019-08-28 14:35:25,011 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Run missing VM report. > > current > > >> > time: > > >> > >> 1566995725011 > > >> > >> 2019-08-28 14:35:25,012 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: > > 1, > > >> vm > > >> > id: > > >> > >> 101, power state: PowerReportMissing, last state update: > > >> 1566995664000 > > >> > >> 2019-08-28 14:35:25,012 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 101 - time since > last > > >> > state > > >> > >> update(61011ms) has not passed graceful period yet > > >> > >> 2019-08-28 14:35:25,012 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Detected missing VM. host: > > 1, > > >> vm > > >> > id: > > >> > >> 102, power state: PowerReportMissing, last state update: > > >> 1566995664000 > > >> > >> 2019-08-28 14:35:25,012 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) vm id: 102 - time since > last > > >> > state > > >> > >> update(61011ms) has not passed graceful period yet > > >> > >> 2019-08-28 14:35:25,012 DEBUG > > >> [c.c.v.VirtualMachinePowerStateSyncImpl] > > >> > >> (AgentManager-Handler-1:null) (logid:) Done with process of VM > > state > > >> > >> report. host: 1 > > >> > >> 2019-08-28 14:35:26,374 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-8a924b4d) (logid:411683a9) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255 > > >> > >> 2019-08-28 14:35:26,383 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:26,390 WARN > > [c.c.a.d.ParamGenericValidationWorker] > > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) > > >> Received > > >> > >> unknown parameters for command listSystemVms. Unknown parameters > : > > >> > listall > > >> > >> 2019-08-28 14:35:26,447 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-8a924b4d ctx-f05d2ed4) (logid:411683a9) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&response=json&listAll=true&page=1&pagesize=20&_=1566995598255 > > >> > >> 2019-08-28 14:35:26,639 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-9b77d72a) (logid:093d0fcb) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listHosts&response=json&details=min&_=1566995598256 > > >> > >> 2019-08-28 14:35:26,646 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:26,654 DEBUG [c.c.a.q.QueryManagerImpl] > > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) > > >> > >>>Searching > > >> > >> for hosts>>> > > >> > >> 2019-08-28 14:35:26,665 DEBUG [c.c.a.q.QueryManagerImpl] > > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) > > >> > >> >>>Generating Response>>> > > >> > >> 2019-08-28 14:35:26,672 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-9b77d72a ctx-b9a96d1b) (logid:093d0fcb) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> command=listHosts&response=json&details=min&_=1566995598256 > > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.t.Request] > > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Seq > > >> > 1-5778962746846479043: > > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1( > > >> fr-kvm1.r9host.com > > >> > ), > > >> > >> Ver: v1, Flags: 10, { GetVolumeStatsAnswer } } > > >> > >> 2019-08-28 14:35:29,918 DEBUG [c.c.a.m.AgentManagerImpl] > > >> > >> (StatsCollector-1:ctx-4918121d) (logid:33bb5737) Details from > > >> executing > > >> > >> class com.cloud.agent.api.GetVolumeStatsCommand: > > >> > >> 2019-08-28 14:35:30,179 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-4:ctx-319b4822) (logid:1b54b3f1) AutoScaling > > Monitor > > >> is > > >> > >> running... > > >> > >> 2019-08-28 14:35:32,307 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-cb8db901) (logid:522feec5) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257 > > >> > >> 2019-08-28 14:35:32,316 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:32,335 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-cb8db901 ctx-9388c3db) (logid:522feec5) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listZones&response=json&id=57a0f2fa-e111-486d-b8c7-e93d640d45d3&_=1566995598257 > > >> > >> 2019-08-28 14:35:32,519 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-db417a94) (logid:29f44323) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258 > > >> > >> 2019-08-28 14:35:32,527 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:32,575 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-db417a94 ctx-b7f6ada6) (logid:29f44323) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598258 > > >> > >> 2019-08-28 14:35:34,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:35:34,861 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-14a7b726) (logid:2d7cb685) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:35:37,400 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) > HostStatsCollector > > >> is > > >> > >> running... > > >> > >> 2019-08-28 14:35:37,456 DEBUG [c.c.a.t.Request] > > >> > >> (StatsCollector-5:ctx-cd7b3e15) (logid:764cc325) Seq > > >> > 1-5778962746846479044: > > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1( > > >> fr-kvm1.r9host.com > > >> > ), > > >> > >> Ver: v1, Flags: 10, { GetHostStatsAnswer } } > > >> > >> 2019-08-28 14:35:38,593 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) StorageCollector > > is > > >> > >> running... > > >> > >> 2019-08-28 14:35:38,604 DEBUG [c.c.s.StatsCollector] > > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) There is no > > >> secondary > > >> > >> storage VM for secondary storage host Secondary1 > > >> > >> 2019-08-28 14:35:38,607 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] > > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) > > >> > getCommandHostDelegation: > > >> > >> class com.cloud.agent.api.GetStorageStatsCommand > > >> > >> 2019-08-28 14:35:38,608 DEBUG [c.c.h.XenServerGuru] > > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) We are returning > > the > > >> > >> default host to execute commands because the command is not of > Copy > > >> > type. > > >> > >> 2019-08-28 14:35:38,660 DEBUG [c.c.a.t.Request] > > >> > >> (StatsCollector-3:ctx-c946fd6f) (logid:01b19d42) Seq > > >> > 1-5778962746846479045: > > >> > >> Received: { Ans: , MgmtId: 205425461278937, via: 1( > > >> fr-kvm1.r9host.com > > >> > ), > > >> > >> Ver: v1, Flags: 10, { GetStorageStatsAnswer } } > > >> > >> 2019-08-28 14:35:41,548 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-f65bf333) (logid:4ec07b47) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259 > > >> > >> 2019-08-28 14:35:41,556 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:41,611 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-f65bf333 ctx-10900b36) (logid:4ec07b47) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598259 > > >> > >> 2019-08-28 14:35:42,313 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-6b516c08) (logid:4673a9cc) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260 > > >> > >> 2019-08-28 14:35:42,321 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:42,377 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-549:ctx-6b516c08 ctx-1a2a1533) (logid:4673a9cc) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598260 > > >> > >> 2019-08-28 14:35:42,859 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-eaed2e53) (logid:c4337d08) ===START=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261 > > >> > >> 2019-08-28 14:35:42,867 DEBUG [c.c.a.ApiServer] > > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) > > CIDRs > > >> > from > > >> > >> which account 'Acct[193f9b2f-c974-11e9-b808-d05099d38681-admin]' > is > > >> > allowed > > >> > >> to perform API calls: 0.0.0.0/0,::/0 > > >> > >> 2019-08-28 14:35:42,920 DEBUG [c.c.a.ApiServlet] > > >> > >> (qtp2114650936-535:ctx-eaed2e53 ctx-99eba050) (logid:c4337d08) > > >> ===END=== > > >> > >> 137.74.35.102 -- GET > > >> > >> > > >> > > > >> > > > command=listSystemVms&id=ca6c6e69-2961-43a8-adbc-a03abd9e60d6&response=json&_=1566995598261 > > >> > >> 2019-08-28 14:35:44,845 DEBUG > [c.c.d.DeploymentPlanningManagerImpl] > > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) > > >> Checking > > >> > if > > >> > >> any host reservation can be released ... > > >> > >> 2019-08-28 14:35:44,852 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) Begin > > cleanup > > >> > >> expired async-jobs > > >> > >> 2019-08-28 14:35:44,852 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 > > >> routers to > > >> > >> update status. > > >> > >> 2019-08-28 14:35:44,854 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 VPC > > >> > networks > > >> > >> to update Redundant State. > > >> > >> 2019-08-28 14:35:44,856 DEBUG > > >> > >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] > > >> > >> (RouterStatusMonitor-1:ctx-a5613af1) (logid:61a8bf76) Found 0 > > >> networks > > >> > to > > >> > >> update RvR status. > > >> > >> 2019-08-28 14:35:44,856 DEBUG > [c.c.d.DeploymentPlanningManagerImpl] > > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) > > Cannot > > >> > >> release reservation, Found 3 VMs Running on host 1 > > >> > >> 2019-08-28 14:35:44,856 DEBUG > [c.c.d.DeploymentPlanningManagerImpl] > > >> > >> (HostReservationReleaseChecker:ctx-812a784d) (logid:ece06f8f) > Done > > >> > running > > >> > >> HostReservationReleaseChecker ... > > >> > >> 2019-08-28 14:35:44,859 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] > > >> > >> (AsyncJobMgr-Heartbeat-1:ctx-7c21bde8) (logid:86429daa) End > cleanup > > >> > expired > > >> > >> async-jobs > > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl] > > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Running Capacity > > >> Checker > > >> > >> ... > > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl] > > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) recalculating > > system > > >> > >> capacity > > >> > >> 2019-08-28 14:35:44,887 DEBUG [c.c.a.AlertManagerImpl] > > >> > >> (CapacityChecker:ctx-305972aa) (logid:95391fc9) Executing cpu/ram > > >> > capacity > > >> > >> update > > >> > > >
