Hello,
I have cloudstack 4.15 with xcp-ng 8.0 and works fine. In last week
all cloudstack servers (hypervisors, storage,management) have power
outage. I restored all functionality of cloudstack but have problem
with consoleproxy VM. His running but no serve any services (don't see
console via web). Tried login to consoleproxy VM but not see any
services responsible to listen on port 80,8080 etc. only ssh and NTP.
root@v-105-VM:~# netstat -luntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 0 169.254.178.163:3922 0.0.0.0:*
LISTEN 1065/sshd
udp 0 0 0.0.0.0:3914 0.0.0.0:*
321/rsyslogd
udp 0 0 10.10.192.1:123 0.0.0.0:*
783/ntpd
udp 0 0 10.6.0.217:123 0.0.0.0:*
783/ntpd
udp 0 0 169.254.178.163:123 0.0.0.0:*
783/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:*
783/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:*
783/ntpd
udp6 0 0 :::3914 :::*
321/rsyslogd
udp6 0 0 :::123 :::*
783/ntpd
root@v-105-VM:~#
There is no information about errors in the logs
2021-02-16 18:20:25,755 INFO [cloud.agent.AgentShell] (main:null) Agent started
2021-02-16 18:20:25,774 INFO [cloud.agent.AgentShell] (main:null)
Implementation Version is 4.15.0.0
2021-02-16 18:20:25,778 INFO [cloud.agent.AgentShell] (main:null)
agent.properties found at
/usr/local/cloud/systemvm/conf/agent.properties
2021-02-16 18:20:25,818 INFO [cloud.agent.AgentShell] (main:null)
Defaulting to using properties file for storage
2021-02-16 18:20:25,820 INFO [cloud.agent.AgentShell] (main:null)
Defaulting to the constant time backoff algorithm
2021-02-16 18:20:25,832 INFO [cloud.utils.LogUtils] (main:null) log4j
configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml
2021-02-16 18:20:25,884 INFO [cloud.agent.AgentShell] (main:null)
Using default Java settings for IPv6 preference for agent connection
2021-02-16 18:20:26,107 INFO [cloud.agent.Agent] (main:null) id is 12
2021-02-16 18:20:26,189 INFO
[resource.consoleproxy.ConsoleProxyResource] (main:null) Receive
proxyVmId in ConsoleProxyResource configuration as 105
2021-02-16 18:20:26,258 INFO [cloud.agent.Agent] (main:null) Agent
[id = 12 : type = ConsoleProxyResource : zone = 1 : pod = 1 : workers
= 5 : host = 10.6.0.127 : port = 8250
2021-02-16 18:20:26,291 INFO [utils.nio.NioClient] (main:null)
Connecting to 10.6.0.127:8250
2021-02-16 18:20:26,299 INFO [utils.nio.Link] (main:null) Conf file
found: /usr/local/cloud/systemvm/conf/agent.properties
2021-02-16 18:20:27,630 INFO [utils.nio.NioClient] (main:null) SSL:
Handshake done
2021-02-16 18:20:27,632 INFO [utils.nio.NioClient] (main:null)
Connected to 10.6.0.127:8250
2021-02-16 18:20:27,761 INFO [cloud.serializer.GsonHelper]
(Agent-Handler-1:null) Default Builder inited.
2021-02-16 18:20:27,854 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Proccess agent startup answer, agent id = 0
2021-02-16 18:20:27,860 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Set agent id 0
2021-02-16 18:20:27,887 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Startup Response Received: agent id = 0
2021-02-16 18:20:27,950 INFO [cloud.agent.Agent]
(agentRequest-Handler-2:null) Processing agent ready command, agent id
= 12
2021-02-16 18:20:27,952 INFO [cloud.agent.Agent]
(agentRequest-Handler-2:null) Set agent id 12
2021-02-16 18:20:27,959 INFO [cloud.agent.Agent]
(agentRequest-Handler-2:null) Ready command is processed for agent id
= 12
2021-02-16 18:20:27,963 INFO
[resource.consoleproxy.ConsoleProxyResource]
(agentRequest-Handler-2:null) Receive ReadyCommand, response with
ReadyAnswer
2021-02-16 18:20:28,101 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) Processing agent ready command, agent id
= 12
2021-02-16 18:20:28,102 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) Set agent id 12
2021-02-16 18:20:28,111 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) Processed new management server list:
10.6.0.127@static
2021-02-16 18:20:28,112 INFO [cloud.agent.Agent]
(agentRequest-Handler-3:null) Ready command is processed for agent id
= 12
2021-02-16 18:20:28,113 INFO
[resource.consoleproxy.ConsoleProxyResource]
(agentRequest-Handler-3:null) Receive ReadyCommand, response with
ReadyAnswer
2021-02-16 18:29:15,776 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Lost connection to host: 10.6.0.127. Attempting
reconnection while we still have 0 commands in progress.
2021-02-16 18:29:15,786 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) NioClient connection closed
2021-02-16 18:29:15,788 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:15,789 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:15,794 WARN [utils.nio.NioConnection]
(Agent-Handler-4:null) Unable to connect to remote: is there a server
running on port 8250
2021-02-16 18:29:20,794 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:20,797 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:20,799 WARN [utils.nio.NioConnection]
(Agent-Handler-4:null) Unable to connect to remote: is there a server
running on port 8250
2021-02-16 18:29:25,800 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:25,803 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:25,806 WARN [utils.nio.NioConnection]
(Agent-Handler-4:null) Unable to connect to remote: is there a server
running on port 8250
2021-02-16 18:29:30,807 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:30,807 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:30,811 WARN [utils.nio.NioConnection]
(Agent-Handler-4:null) Unable to connect to remote: is there a server
running on port 8250
2021-02-16 18:29:35,814 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:35,815 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:35,817 WARN [utils.nio.NioConnection]
(Agent-Handler-4:null) Unable to connect to remote: is there a server
running on port 8250
2021-02-16 18:29:40,818 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Reconnecting to host:10.6.0.127
2021-02-16 18:29:40,818 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connecting to 10.6.0.127:8250
2021-02-16 18:29:40,821 INFO [utils.nio.Link] (Agent-Handler-4:null)
Conf file found: /usr/local/cloud/systemvm/conf/agent.properties
2021-02-16 18:29:41,219 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) SSL: Handshake done
2021-02-16 18:29:41,221 INFO [utils.nio.NioClient]
(Agent-Handler-4:null) Connected to 10.6.0.127:8250
2021-02-16 18:29:41,335 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Proccess agent startup answer, agent id = 0
2021-02-16 18:29:41,336 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Set agent id 0
2021-02-16 18:29:41,339 INFO [cloud.agent.Agent]
(Agent-Handler-2:null) Startup Response Received: agent id = 0
2021-02-16 18:29:41,550 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) Processing agent ready command, agent id
= 12
2021-02-16 18:29:41,551 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) Set agent id 12
2021-02-16 18:29:41,555 INFO [cloud.agent.Agent]
(agentRequest-Handler-4:null) Ready command is processed for agent id
= 12
2021-02-16 18:29:41,555 INFO
[resource.consoleproxy.ConsoleProxyResource]
(agentRequest-Handler-4:null) Receive ReadyCommand, response with
ReadyAnswer
2021-02-16 18:29:41,724 INFO [cloud.agent.Agent]
(agentRequest-Handler-5:null) Processing agent ready command, agent id
= 12
2021-02-16 18:29:41,725 INFO [cloud.agent.Agent]
(agentRequest-Handler-5:null) Set agent id 12
2021-02-16 18:29:41,730 INFO [cloud.agent.Agent]
(agentRequest-Handler-5:null) Processed new management server list:
10.6.0.127@static
2021-02-16 18:29:41,731 INFO [cloud.agent.Agent]
(agentRequest-Handler-5:null) Ready command is processed for agent id
= 12
2021-02-16 18:29:41,731 INFO
[resource.consoleproxy.ConsoleProxyResource]
(agentRequest-Handler-5:null) Receive ReadyCommand, response with
ReadyAnswer
2021-02-16 18:29:46,230 INFO [cloud.agent.Agent]
(Agent-Handler-4:null) Connected to the host: 10.6.0.127
