abdelouahabb commented on issue #4659: URL: https://github.com/apache/cloudstack/issues/4659#issuecomment-777645868
> @abdelouahabb is there error log on management server ? The content of the folder belong to the user `cloud` ``` root@alien:/var/log/cloudstack/management# ls -l total 22536 -rw-r--r-- 1 cloud cloud 1858062 Feb 11 03:39 access.log -rw-r--r-- 1 cloud cloud 6751349 Feb 11 03:39 apilog.log -rw-r--r-- 1 cloud cloud 872990 Jan 20 00:06 apilog.log.2021-01-19.gz -rw-r--r-- 1 cloud cloud 6536434 Feb 9 00:01 apilog.log.2021-02-08.gz -rw-r--r-- 1 cloud cloud 3216363 Feb 11 00:03 apilog.log.2021-02-10.gz -rw-r--r-- 1 cloud cloud 2613436 Feb 11 16:26 management-server.log -rw-r--r-- 1 cloud cloud 1138719 Feb 11 00:00 management-server.log.2021-02-10.gz -rw-r--r-- 1 cloud cloud 54939 Feb 10 17:11 setupManagement.log ``` I made an operation so I can record the log : login, adding primary storage (error), secondary storage (success), adding new host (error) ` sudo tail -f /var/log/cloudstack/management/management-server.log > error.txt` ``` 2021-02-11 16:58:13,845 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34012. 2021-02-11 16:58:13,848 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-11:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:18,941 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34014. 2021-02-11 16:58:18,947 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:21,229 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-713e842d) (logid:459a0b20) AutoScaling Monitor is running... 2021-02-11 16:58:21,242 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-10955270) (logid:0c1d2c6f) HostStatsCollector is running... 2021-02-11 16:58:21,444 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-044de3ce) (logid:cdb25f6b) StorageCollector is running... 2021-02-11 16:58:22,102 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-e404b003) (logid:1ecba277) Skip capacity scan as there is no Primary Storage in 'Up' state 2021-02-11 16:58:24,064 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34016. 2021-02-11 16:58:24,069 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-15:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:29,180 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34018. 2021-02-11 16:58:29,184 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:34,232 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34020. 2021-02-11 16:58:34,236 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-4:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:35,146 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-85e017e4) (logid:4afb75f5) ===START=== 192.168.56.1 -- GET command=listIdps&response=json 2021-02-11 16:58:35,146 DEBUG [c.c.a.ApiServer] (qtp1766145591-17:ctx-85e017e4 ctx-9332f19e) (logid:4afb75f5) The given command listIdps either does not exist, is not available for user, or not available from ip address '192.168.56.1'. 2021-02-11 16:58:35,147 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-85e017e4 ctx-9332f19e) (logid:4afb75f5) ===END=== 192.168.56.1 -- GET command=listIdps&response=json 2021-02-11 16:58:36,102 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-7a3c3d1e) (logid:3fac441a) Found 0 routers to update status. 2021-02-11 16:58:36,106 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-7a3c3d1e) (logid:3fac441a) Found 0 VPC's to update Redundant State. 2021-02-11 16:58:36,111 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-7a3c3d1e) (logid:3fac441a) Found 0 networks to update RvR status. 2021-02-11 16:58:38,537 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) ===START=== 192.168.56.1 -- POST 2021-02-11 16:58:38,546 DEBUG [c.c.u.AccountManagerImpl] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) Attempting to log in user: admin in domain 1 2021-02-11 16:58:38,548 DEBUG [o.a.c.s.a.PBKDF2UserAuthenticator] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) Retrieving user: admin 2021-02-11 16:58:39,007 DEBUG [c.c.u.AccountManagerImpl] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,008 DEBUG [c.c.u.AccountManagerImpl] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) User: admin in domain 1 has successfully logged in 2021-02-11 16:58:39,018 INFO [c.c.a.ApiServer] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) Current user logged in under UTC timezone 2021-02-11 16:58:39,019 INFO [c.c.a.ApiServer] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) Timezone offset from UTC is: 0.0 2021-02-11 16:58:39,026 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-c1924809) (logid:6ac471ab) ===END=== 192.168.56.1 -- POST 2021-02-11 16:58:39,051 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-b459ae8c) (logid:80c15691) ===START=== 192.168.56.1 -- GET listall=true&command=listZones&response=json 2021-02-11 16:58:39,052 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-404e580f) (logid:f70657f3) ===START=== 192.168.56.1 -- GET command=listApis&response=json 2021-02-11 16:58:39,060 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-dbb776f8) (logid:4324c0f4) ===START=== 192.168.56.1 -- GET username=admin&command=listUsers&response=json 2021-02-11 16:58:39,061 DEBUG [c.c.a.ApiServlet] (qtp1766145591-21:ctx-6e25da50) (logid:2bcf881f) ===START=== 192.168.56.1 -- GET command=listCapabilities&response=json 2021-02-11 16:58:39,065 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-f49fae1c) (logid:5f53ea04) ===START=== 192.168.56.1 -- GET command=listLdapConfigurations&response=json 2021-02-11 16:58:39,072 DEBUG [c.c.a.ApiServer] (qtp1766145591-22:ctx-404e580f ctx-ac96fd95) (logid:f70657f3) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,077 DEBUG [c.c.a.ApiServer] (qtp1766145591-19:ctx-f49fae1c ctx-1b6ee335) (logid:5f53ea04) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,081 DEBUG [c.c.a.ApiServer] (qtp1766145591-17:ctx-b459ae8c ctx-472fadd9) (logid:80c15691) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,087 DEBUG [c.c.a.ApiServer] (qtp1766145591-21:ctx-6e25da50 ctx-c1cb6a8c) (logid:2bcf881f) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,082 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-a9b8d5fc) (logid:6acec26b) ===START=== 192.168.56.1 -- GET command=cloudianIsEnabled&response=json 2021-02-11 16:58:39,092 DEBUG [c.c.a.ApiServer] (qtp1766145591-15:ctx-dbb776f8 ctx-85fdd866) (logid:4324c0f4) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,099 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-f49fae1c ctx-1b6ee335) (logid:5f53ea04) ===END=== 192.168.56.1 -- GET command=listLdapConfigurations&response=json 2021-02-11 16:58:39,107 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-a9b8d5fc ctx-ea23ada2) (logid:6acec26b) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:39,111 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-17:ctx-b459ae8c ctx-472fadd9) (logid:80c15691) Received unknown parameters for command listZones. Unknown parameters : listall 2021-02-11 16:58:39,116 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-dbb776f8 ctx-85fdd866) (logid:4324c0f4) ===END=== 192.168.56.1 -- GET username=admin&command=listUsers&response=json 2021-02-11 16:58:39,133 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-b459ae8c ctx-472fadd9) (logid:80c15691) ===END=== 192.168.56.1 -- GET listall=true&command=listZones&response=json 2021-02-11 16:58:39,134 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-a9b8d5fc ctx-ea23ada2) (logid:6acec26b) ===END=== 192.168.56.1 -- GET command=cloudianIsEnabled&response=json 2021-02-11 16:58:39,136 DEBUG [c.c.a.ApiServlet] (qtp1766145591-21:ctx-6e25da50 ctx-c1cb6a8c) (logid:2bcf881f) ===END=== 192.168.56.1 -- GET command=listCapabilities&response=json 2021-02-11 16:58:39,357 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34022. 2021-02-11 16:58:39,364 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-6:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:40,808 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Resetting hosts suitable for reconnect 2021-02-11 16:58:40,809 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Completed resetting hosts suitable for reconnect 2021-02-11 16:58:40,809 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Acquiring hosts for clusters already owned by this management server 2021-02-11 16:58:40,810 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Completed acquiring hosts for clusters already owned by this management server 2021-02-11 16:58:40,810 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Acquiring hosts for clusters not owned by any management server 2021-02-11 16:58:40,811 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-077c3d23) (logid:7aeb1e26) Completed acquiring hosts for clusters not owned by any management server 2021-02-11 16:58:42,169 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-404e580f ctx-ac96fd95) (logid:f70657f3) ===END=== 192.168.56.1 -- GET command=listApis&response=json 2021-02-11 16:58:42,650 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-b1d1408c) (logid:253ee22f) ===START=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:42,653 DEBUG [c.c.a.ApiServlet] (qtp1766145591-18:ctx-281a2e2a) (logid:7e86594b) ===START=== 192.168.56.1 -- GET page=1&pagesize=6&listall=true&command=listEvents&response=json 2021-02-11 16:58:42,654 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-9dcfcbdb) (logid:aba30f6c) ===START=== 192.168.56.1 -- GET listAll=true&details=min&page=1&pageSize=500&command=listAndSwitchSamlAccount&response=json 2021-02-11 16:58:42,653 DEBUG [c.c.a.ApiServlet] (qtp1766145591-13:ctx-36942ae8) (logid:2d86e8c5) ===START=== 192.168.56.1 -- GET listAll=true&details=min&page=1&pageSize=500&command=listProjects&response=json 2021-02-11 16:58:42,677 DEBUG [c.c.a.ApiServer] (qtp1766145591-19:ctx-b1d1408c ctx-650d7135) (logid:253ee22f) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,679 DEBUG [c.c.a.ApiServer] (qtp1766145591-18:ctx-281a2e2a ctx-0d2835d6) (logid:7e86594b) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,680 DEBUG [c.c.a.ApiServer] (qtp1766145591-20:ctx-9dcfcbdb ctx-b3ced455) (logid:aba30f6c) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,680 DEBUG [c.c.a.ApiServer] (qtp1766145591-13:ctx-36942ae8 ctx-5337cb39) (logid:2d86e8c5) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,687 WARN [c.c.a.ApiServer] (qtp1766145591-20:ctx-9dcfcbdb ctx-b3ced455) (logid:aba30f6c) Unknown API command: listAndSwitchSamlAccount 2021-02-11 16:58:42,688 INFO [c.c.a.ApiServer] (qtp1766145591-20:ctx-9dcfcbdb ctx-b3ced455) (logid:aba30f6c) Unknown API command: listAndSwitchSamlAccount 2021-02-11 16:58:42,690 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-9dcfcbdb ctx-b3ced455) (logid:aba30f6c) ===END=== 192.168.56.1 -- GET listAll=true&details=min&page=1&pageSize=500&command=listAndSwitchSamlAccount&response=json 2021-02-11 16:58:42,724 DEBUG [c.c.a.ApiServlet] (qtp1766145591-18:ctx-281a2e2a ctx-0d2835d6) (logid:7e86594b) ===END=== 192.168.56.1 -- GET page=1&pagesize=6&listall=true&command=listEvents&response=json 2021-02-11 16:58:42,725 DEBUG [c.c.a.ApiServlet] (qtp1766145591-13:ctx-36942ae8 ctx-5337cb39) (logid:2d86e8c5) ===END=== 192.168.56.1 -- GET listAll=true&details=min&page=1&pageSize=500&command=listProjects&response=json 2021-02-11 16:58:42,725 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-b1d1408c ctx-650d7135) (logid:253ee22f) ===END=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:42,732 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-d3d8574a) (logid:fdbe0337) ===START=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:42,743 DEBUG [c.c.a.ApiServer] (qtp1766145591-15:ctx-d3d8574a ctx-eefd6224) (logid:fdbe0337) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,760 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-d3d8574a ctx-eefd6224) (logid:fdbe0337) ===END=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:42,797 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-96d15575) (logid:2df28513) ===START=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&fetchlatest=false&command=listCapacity&response=json 2021-02-11 16:58:42,808 DEBUG [c.c.a.ApiServer] (qtp1766145591-20:ctx-96d15575 ctx-e125eef4) (logid:2df28513) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:42,825 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-96d15575 ctx-e125eef4) (logid:2df28513) ===END=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&fetchlatest=false&command=listCapacity&response=json 2021-02-11 16:58:44,448 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34024. 2021-02-11 16:58:44,453 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-8:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:45,959 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-2db41c03) (logid:c89033b1) ===START=== 192.168.56.1 -- GET command=listInfrastructure&response=json 2021-02-11 16:58:45,969 DEBUG [c.c.a.ApiServer] (qtp1766145591-15:ctx-2db41c03 ctx-4e7e290d) (logid:c89033b1) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:46,007 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-2db41c03 ctx-4e7e290d) (logid:c89033b1) ===END=== 192.168.56.1 -- GET command=listInfrastructure&response=json 2021-02-11 16:58:48,736 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-802bbb6f) (logid:e44541a9) ===START=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listStoragePoolsMetrics&response=json 2021-02-11 16:58:48,744 DEBUG [c.c.a.ApiServer] (qtp1766145591-20:ctx-802bbb6f ctx-fd1020ff) (logid:e44541a9) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:48,748 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-20:ctx-802bbb6f ctx-fd1020ff) (logid:e44541a9) Received unknown parameters for command listStoragePoolsMetrics. Unknown parameters : listall 2021-02-11 16:58:48,757 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-802bbb6f ctx-fd1020ff) (logid:e44541a9) ===END=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listStoragePoolsMetrics&response=json 2021-02-11 16:58:49,542 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34026. 2021-02-11 16:58:49,547 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:50,768 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-544ebca9) (logid:ba59e5ae) ===START=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:50,769 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-9e5843fc) (logid:52456a77) ===START=== 192.168.56.1 -- GET type=primary&command=listStorageProviders&response=json 2021-02-11 16:58:50,770 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-fd7f54a8) (logid:b7fc8c93) ===START=== 192.168.56.1 -- GET command=listStorageTags&response=json 2021-02-11 16:58:50,781 DEBUG [c.c.a.ApiServer] (qtp1766145591-15:ctx-544ebca9 ctx-5a918db8) (logid:ba59e5ae) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:50,784 DEBUG [c.c.a.ApiServer] (qtp1766145591-17:ctx-9e5843fc ctx-6e1e57a2) (logid:52456a77) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:50,793 DEBUG [c.c.a.ApiServer] (qtp1766145591-20:ctx-fd7f54a8 ctx-8b64c32b) (logid:b7fc8c93) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:50,796 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-9e5843fc ctx-6e1e57a2) (logid:52456a77) ===END=== 192.168.56.1 -- GET type=primary&command=listStorageProviders&response=json 2021-02-11 16:58:50,803 DEBUG [c.c.a.ApiServlet] (qtp1766145591-15:ctx-544ebca9 ctx-5a918db8) (logid:ba59e5ae) ===END=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:58:50,805 DEBUG [c.c.a.ApiServlet] (qtp1766145591-20:ctx-fd7f54a8 ctx-8b64c32b) (logid:b7fc8c93) ===END=== 192.168.56.1 -- GET command=listStorageTags&response=json 2021-02-11 16:58:50,997 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-4cba34b6) (logid:b3282c3d) ===START=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=listPods&response=json 2021-02-11 16:58:51,005 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-4cba34b6 ctx-c1e1e3fd) (logid:b3282c3d) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:51,021 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-4cba34b6 ctx-c1e1e3fd) (logid:b3282c3d) ===END=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=listPods&response=json 2021-02-11 16:58:51,090 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-0a381b5d) (logid:daf10f2e) ===START=== 192.168.56.1 -- GET podid=d5274d8a-4878-44f3-89ab-ba4654299e1a&command=listClusters&response=json 2021-02-11 16:58:51,103 DEBUG [c.c.a.ApiServer] (qtp1766145591-17:ctx-0a381b5d ctx-42a8c25b) (logid:daf10f2e) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:51,123 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-0a381b5d ctx-42a8c25b) (logid:daf10f2e) ===END=== 192.168.56.1 -- GET podid=d5274d8a-4878-44f3-89ab-ba4654299e1a&command=listClusters&response=json 2021-02-11 16:58:51,176 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-de6a2ec6) (logid:baea16d4) ===START=== 192.168.56.1 -- GET clusterid=acd89d8e-d25b-4144-8b74-a8b08d544bf1&command=listHosts&response=json 2021-02-11 16:58:51,185 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-de6a2ec6 ctx-9db9874c) (logid:baea16d4) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:58:51,190 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1766145591-16:ctx-de6a2ec6 ctx-9db9874c) (logid:baea16d4) >>>Searching for hosts>>> 2021-02-11 16:58:51,195 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1766145591-16:ctx-de6a2ec6 ctx-9db9874c) (logid:baea16d4) >>>Generating Response>>> 2021-02-11 16:58:51,195 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-de6a2ec6 ctx-9db9874c) (logid:baea16d4) ===END=== 192.168.56.1 -- GET clusterid=acd89d8e-d25b-4144-8b74-a8b08d544bf1&command=listHosts&response=json 2021-02-11 16:58:52,112 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-3e125cd5) (logid:8004a04b) Skip capacity scan as there is no Primary Storage in 'Up' state 2021-02-11 16:58:54,613 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34028. 2021-02-11 16:58:54,619 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-12:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:58:59,702 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34030. 2021-02-11 16:58:59,708 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-14:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:04,801 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34032. 2021-02-11 16:59:04,812 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-1:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:06,092 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-124be51a) (logid:73ac7ab2) Found 0 routers to update status. 2021-02-11 16:59:06,097 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-124be51a) (logid:73ac7ab2) Found 0 VPC's to update Redundant State. 2021-02-11 16:59:06,102 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-124be51a) (logid:73ac7ab2) Found 0 networks to update RvR status. 2021-02-11 16:59:08,629 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-af578a13) (logid:e6a6d9a8) ===START=== 192.168.56.1 -- POST command=createStoragePool&response=json 2021-02-11 16:59:08,641 DEBUG [c.c.a.ApiServer] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:08,666 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) createPool Params @ scheme - nfs storageHost - 192.168.56.85 hostPath - /export/primary port - -1 2021-02-11 16:59:08,807 DEBUG [c.c.s.StorageManagerImpl] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) Failed to add data store: No host up to associate a storage pool with in cluster 4 com.cloud.utils.exception.CloudRuntimeException: No host up to associate a storage pool with in cluster 4 at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:430) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:741) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:210) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy96.createPool(Unknown Source) at org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) at com.cloud.api.ApiServlet.doPost(ApiServlet.java:98) at javax.servlet.http.HttpServlet.service(HttpServlet.java:665) at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Thread.java:834) 2021-02-11 16:59:08,815 DEBUG [c.c.s.StorageManagerImpl] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) Failed to clean up storage pool: null 2021-02-11 16:59:08,815 INFO [c.c.a.ApiServer] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) Failed to add data store: No host up to associate a storage pool with in cluster 4 2021-02-11 16:59:08,816 DEBUG [c.c.a.ApiServlet] (qtp1766145591-17:ctx-af578a13 ctx-c967a7cc) (logid:e6a6d9a8) ===END=== 192.168.56.1 -- POST command=createStoragePool&response=json 2021-02-11 16:59:09,885 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34034. 2021-02-11 16:59:09,888 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-3:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:11,388 DEBUG [c.c.a.ApiServlet] (qtp1766145591-21:ctx-5752d23f) (logid:679c0528) ===START=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listImageStores&response=json 2021-02-11 16:59:11,406 DEBUG [c.c.a.ApiServer] (qtp1766145591-21:ctx-5752d23f ctx-d0c09938) (logid:679c0528) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:11,416 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-21:ctx-5752d23f ctx-d0c09938) (logid:679c0528) Received unknown parameters for command listImageStores. Unknown parameters : listall 2021-02-11 16:59:11,423 DEBUG [c.c.a.ApiServlet] (qtp1766145591-21:ctx-5752d23f ctx-d0c09938) (logid:679c0528) ===END=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listImageStores&response=json 2021-02-11 16:59:12,956 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-6573050f) (logid:9b99717e) ===START=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:59:12,965 DEBUG [c.c.a.ApiServer] (qtp1766145591-22:ctx-6573050f ctx-740cb711) (logid:9b99717e) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:12,982 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-6573050f ctx-740cb711) (logid:9b99717e) ===END=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:59:14,981 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34036. 2021-02-11 16:59:14,995 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-5:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:20,080 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34038. 2021-02-11 16:59:20,083 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-7:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:21,240 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-7e0a37be) (logid:c7f23758) AutoScaling Monitor is running... 2021-02-11 16:59:21,252 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-26c1c289) (logid:2e30b588) HostStatsCollector is running... 2021-02-11 16:59:21,452 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-e78c0b57) (logid:718cfc59) StorageCollector is running... 2021-02-11 16:59:22,102 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-dd0caf32) (logid:83ad9e78) Skip capacity scan as there is no Primary Storage in 'Up' state 2021-02-11 16:59:25,146 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34040. 2021-02-11 16:59:25,154 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-9:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:27,263 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-5a199363) (logid:327b3b04) ===START=== 192.168.56.1 -- GET name=secondaire&url=nfs:%2F%2F192.168.56.85%2Fexport%2Fsecondary&provider=NFS&zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=addImageStore&response=json 2021-02-11 16:59:27,272 DEBUG [c.c.a.ApiServer] (qtp1766145591-22:ctx-5a199363 ctx-ecf15a4c) (logid:327b3b04) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:27,297 INFO [o.a.c.s.d.l.CloudStackImageStoreLifeCycleImpl] (qtp1766145591-22:ctx-5a199363 ctx-ecf15a4c) (logid:327b3b04) Trying to add a new data store at nfs://192.168.56.85/export/secondary to data center 1 2021-02-11 16:59:27,430 ERROR [o.a.c.s.i.TemplateServiceImpl] (qtp1766145591-22:ctx-5a199363 ctx-ecf15a4c) (logid:327b3b04) No remote endpoint to send command, check if host or ssvm is down? 2021-02-11 16:59:27,431 DEBUG [o.a.c.s.i.TemplateServiceImpl] (qtp1766145591-22:ctx-5a199363 ctx-ecf15a4c) (logid:327b3b04) can not list template for secondary storage host 3 2021-02-11 16:59:27,447 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-5a199363 ctx-ecf15a4c) (logid:327b3b04) ===END=== 192.168.56.1 -- GET name=secondaire&url=nfs:%2F%2F192.168.56.85%2Fexport%2Fsecondary&provider=NFS&zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=addImageStore&response=json 2021-02-11 16:59:27,504 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-0d943231) (logid:50c82862) ===START=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listImageStores&response=json 2021-02-11 16:59:27,512 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-0d943231 ctx-f87c5e98) (logid:50c82862) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:27,526 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-16:ctx-0d943231 ctx-f87c5e98) (logid:50c82862) Received unknown parameters for command listImageStores. Unknown parameters : listall 2021-02-11 16:59:27,534 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-0d943231 ctx-f87c5e98) (logid:50c82862) ===END=== 192.168.56.1 -- GET listall=true&page=1&pagesize=20&command=listImageStores&response=json 2021-02-11 16:59:30,239 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34042. 2021-02-11 16:59:30,244 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-11:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:30,355 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-68ebde37) (logid:339b3a73) ===START=== 192.168.56.1 -- GET listall=true&type=routing&page=1&pagesize=20&command=listHostsMetrics&response=json 2021-02-11 16:59:30,367 DEBUG [c.c.a.ApiServer] (qtp1766145591-22:ctx-68ebde37 ctx-f9fbe647) (logid:339b3a73) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:30,376 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-22:ctx-68ebde37 ctx-f9fbe647) (logid:339b3a73) Received unknown parameters for command listHostsMetrics. Unknown parameters : listall 2021-02-11 16:59:30,377 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1766145591-22:ctx-68ebde37 ctx-f9fbe647) (logid:339b3a73) >>>Searching for hosts>>> 2021-02-11 16:59:30,384 DEBUG [c.c.a.q.QueryManagerImpl] (qtp1766145591-22:ctx-68ebde37 ctx-f9fbe647) (logid:339b3a73) >>>Generating Response>>> 2021-02-11 16:59:30,385 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-68ebde37 ctx-f9fbe647) (logid:339b3a73) ===END=== 192.168.56.1 -- GET listall=true&type=routing&page=1&pagesize=20&command=listHostsMetrics&response=json 2021-02-11 16:59:31,942 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-4cb12cca) (logid:8a8d7253) ===START=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:59:31,944 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-bfe20a38) (logid:eb8f6f48) ===START=== 192.168.56.1 -- GET command=listHostTags&response=json 2021-02-11 16:59:31,958 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-4cb12cca ctx-ffcf85dc) (logid:8a8d7253) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:31,959 DEBUG [c.c.a.ApiServer] (qtp1766145591-22:ctx-bfe20a38 ctx-d8b0f446) (logid:eb8f6f48) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:31,971 DEBUG [c.c.a.ApiServlet] (qtp1766145591-22:ctx-bfe20a38 ctx-d8b0f446) (logid:eb8f6f48) ===END=== 192.168.56.1 -- GET command=listHostTags&response=json 2021-02-11 16:59:31,972 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-4cb12cca ctx-ffcf85dc) (logid:8a8d7253) ===END=== 192.168.56.1 -- GET command=listZones&response=json 2021-02-11 16:59:32,008 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-f8544c2a) (logid:b6903fab) ===START=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=listPods&response=json 2021-02-11 16:59:32,018 DEBUG [c.c.a.ApiServer] (qtp1766145591-19:ctx-f8544c2a ctx-9caaa3f8) (logid:b6903fab) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:32,031 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-f8544c2a ctx-9caaa3f8) (logid:b6903fab) ===END=== 192.168.56.1 -- GET zoneid=49101c55-454f-4131-acb8-3fa6e81671eb&command=listPods&response=json 2021-02-11 16:59:32,057 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-11d8bd37) (logid:d783b1a4) ===START=== 192.168.56.1 -- GET podid=d5274d8a-4878-44f3-89ab-ba4654299e1a&command=listClusters&response=json 2021-02-11 16:59:32,066 DEBUG [c.c.a.ApiServer] (qtp1766145591-16:ctx-11d8bd37 ctx-c53bbcde) (logid:d783b1a4) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:32,082 DEBUG [c.c.a.ApiServlet] (qtp1766145591-16:ctx-11d8bd37 ctx-c53bbcde) (logid:d783b1a4) ===END=== 192.168.56.1 -- GET podid=d5274d8a-4878-44f3-89ab-ba4654299e1a&command=listClusters&response=json 2021-02-11 16:59:35,340 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34044. 2021-02-11 16:59:35,343 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-13:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:36,090 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-900bd7de) (logid:fadcceb3) Found 0 routers to update status. 2021-02-11 16:59:36,094 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-900bd7de) (logid:fadcceb3) Found 0 VPC's to update Redundant State. 2021-02-11 16:59:36,098 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-900bd7de) (logid:fadcceb3) Found 0 networks to update RvR status. 2021-02-11 16:59:40,393 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34046. 2021-02-11 16:59:40,398 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-15:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. 2021-02-11 16:59:42,398 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-814b5d53) (logid:88b770f8) ===START=== 192.168.56.1 -- POST command=addHost&response=json 2021-02-11 16:59:42,411 DEBUG [c.c.a.ApiServer] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) CIDRs from which account 'Acct[e4b9dbf4-6a86-11eb-8c34-08002799a040-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-02-11 16:59:42,426 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Received unknown parameters for command addHost. Unknown parameters : clustertype 2021-02-11 16:59:42,433 INFO [c.c.r.ResourceManagerImpl] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Trying to add a new host at http://192.168.56.85 in data center 1 2021-02-11 16:59:42,512 WARN [c.c.h.k.d.LibvirtServerDiscoverer] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) can't setup agent, due to java.io.IOException: There was a problem while connecting to 192.168.56.85:22 - There was a problem while connecting to 192.168.56.85:22 2021-02-11 16:59:42,513 WARN [c.c.r.ResourceManagerImpl] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Unable to find the server resources at http://192.168.56.85 2021-02-11 16:59:42,513 INFO [c.c.u.e.CSExceptionErrorCode] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions 2021-02-11 16:59:42,513 WARN [o.a.c.a.c.a.h.AddHostCmd] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Exception: com.cloud.exception.DiscoveryException: Unable to add the host at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:826) at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:612) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy188.discoverHosts(Unknown Source) at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) at com.cloud.api.ApiServlet.doPost(ApiServlet.java:98) at javax.servlet.http.HttpServlet.service(HttpServlet.java:665) at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Thread.java:834) 2021-02-11 16:59:42,516 INFO [c.c.a.ApiServer] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) Unable to add the host 2021-02-11 16:59:42,517 DEBUG [c.c.a.ApiServlet] (qtp1766145591-19:ctx-814b5d53 ctx-64f6dac6) (logid:88b770f8) ===END=== 192.168.56.1 -- POST command=addHost&response=json 2021-02-11 16:59:45,464 ERROR [c.c.u.n.Link] (AgentManager-SSLHandshakeHandler-2:null) (logid:) SSL error caught during wrap data: Empty server certificate chain, for local address=/192.168.56.85:8250, remote address=/192.168.56.85:34050. 2021-02-11 16:59:45,466 INFO [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) Connection from /192.168.56.85 closed but no cleanup was done. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org