you can try destroying your cpvm from the cloudstack ui and wait till it is recreated. It should come up healthy again.
On Tue, Feb 16, 2021 at 8:18 PM Dominik Czerepiński <dominato...@gmail.com> wrote: > Oh, and i can reach management server from consoleproxy and see open port > 8250 > > [20:13][root@ACSM][/] # netstat -luntp > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address > State PID/Program name > [...] > tcp6 0 0 :::8080 :::* > LISTEN 12015/java > tcp6 0 0 :::39891 :::* > LISTEN 12015/java > tcp6 0 0 :::8250 :::* > LISTEN 12015/java > tcp6 0 0 :::8096 :::* > LISTEN 12015/java > tcp6 0 0 :::9090 :::* > LISTEN 12015/java > > > root@v-105-VM:~# ping 10.6.0.127 > PING 10.6.0.127 (10.6.0.127): 56 data bytes > 64 bytes from 10.6.0.127: icmp_seq=0 ttl=64 time=0.914 ms > ^C--- 10.6.0.127 ping statistics --- > 1 packets transmitted, 1 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.914/0.914/0.914/0.000 ms > root@v-105-VM:~# telnet 10.6.0.127 8250 > Trying 10.6.0.127... > Connected to 10.6.0.127. > Escape character is '^]'. > > ^CConnection closed by foreign host. > root@v-105-VM:~# > > wt., 16 lut 2021 o 20:12 Dominik Czerepiński <dominato...@gmail.com> > napisał(a): > > > > Hello, > > I have cloudstack 4.15 with xcp-ng 8.0 and works fine. In last week > > all cloudstack servers (hypervisors, storage,management) have power > > outage. I restored all functionality of cloudstack but have problem > > with consoleproxy VM. His running but no serve any services (don't see > > console via web). Tried login to consoleproxy VM but not see any > > services responsible to listen on port 80,8080 etc. only ssh and NTP. > > > > root@v-105-VM:~# netstat -luntp > > Active Internet connections (only servers) > > Proto Recv-Q Send-Q Local Address Foreign Address > > State PID/Program name > > tcp 0 0 169.254.178.163:3922 0.0.0.0:* > > LISTEN 1065/sshd > > udp 0 0 0.0.0.0:3914 0.0.0.0:* > > 321/rsyslogd > > udp 0 0 10.10.192.1:123 0.0.0.0:* > > 783/ntpd > > udp 0 0 10.6.0.217:123 0.0.0.0:* > > 783/ntpd > > udp 0 0 169.254.178.163:123 0.0.0.0:* > > 783/ntpd > > udp 0 0 127.0.0.1:123 0.0.0.0:* > > 783/ntpd > > udp 0 0 0.0.0.0:123 0.0.0.0:* > > 783/ntpd > > udp6 0 0 :::3914 :::* > > 321/rsyslogd > > udp6 0 0 :::123 :::* > > 783/ntpd > > root@v-105-VM:~# > > > > There is no information about errors in the logs > > 2021-02-16 18:20:25,755 INFO [cloud.agent.AgentShell] (main:null) Agent > started > > 2021-02-16 18:20:25,774 INFO [cloud.agent.AgentShell] (main:null) > > Implementation Version is 4.15.0.0 > > 2021-02-16 18:20:25,778 INFO [cloud.agent.AgentShell] (main:null) > > agent.properties found at > > /usr/local/cloud/systemvm/conf/agent.properties > > 2021-02-16 18:20:25,818 INFO [cloud.agent.AgentShell] (main:null) > > Defaulting to using properties file for storage > > 2021-02-16 18:20:25,820 INFO [cloud.agent.AgentShell] (main:null) > > Defaulting to the constant time backoff algorithm > > 2021-02-16 18:20:25,832 INFO [cloud.utils.LogUtils] (main:null) log4j > > configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml > > 2021-02-16 18:20:25,884 INFO [cloud.agent.AgentShell] (main:null) > > Using default Java settings for IPv6 preference for agent connection > > 2021-02-16 18:20:26,107 INFO [cloud.agent.Agent] (main:null) id is 12 > > 2021-02-16 18:20:26,189 INFO > > [resource.consoleproxy.ConsoleProxyResource] (main:null) Receive > > proxyVmId in ConsoleProxyResource configuration as 105 > > 2021-02-16 18:20:26,258 INFO [cloud.agent.Agent] (main:null) Agent > > [id = 12 : type = ConsoleProxyResource : zone = 1 : pod = 1 : workers > > = 5 : host = 10.6.0.127 : port = 8250 > > 2021-02-16 18:20:26,291 INFO [utils.nio.NioClient] (main:null) > > Connecting to 10.6.0.127:8250 > > 2021-02-16 18:20:26,299 INFO [utils.nio.Link] (main:null) Conf file > > found: /usr/local/cloud/systemvm/conf/agent.properties > > 2021-02-16 18:20:27,630 INFO [utils.nio.NioClient] (main:null) SSL: > > Handshake done > > 2021-02-16 18:20:27,632 INFO [utils.nio.NioClient] (main:null) > > Connected to 10.6.0.127:8250 > > 2021-02-16 18:20:27,761 INFO [cloud.serializer.GsonHelper] > > (Agent-Handler-1:null) Default Builder inited. > > 2021-02-16 18:20:27,854 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 > > 2021-02-16 18:20:27,860 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Set agent id 0 > > 2021-02-16 18:20:27,887 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Startup Response Received: agent id = 0 > > 2021-02-16 18:20:27,950 INFO [cloud.agent.Agent] > > (agentRequest-Handler-2:null) Processing agent ready command, agent id > > = 12 > > 2021-02-16 18:20:27,952 INFO [cloud.agent.Agent] > > (agentRequest-Handler-2:null) Set agent id 12 > > 2021-02-16 18:20:27,959 INFO [cloud.agent.Agent] > > (agentRequest-Handler-2:null) Ready command is processed for agent id > > = 12 > > 2021-02-16 18:20:27,963 INFO > > [resource.consoleproxy.ConsoleProxyResource] > > (agentRequest-Handler-2:null) Receive ReadyCommand, response with > > ReadyAnswer > > 2021-02-16 18:20:28,101 INFO [cloud.agent.Agent] > > (agentRequest-Handler-3:null) Processing agent ready command, agent id > > = 12 > > 2021-02-16 18:20:28,102 INFO [cloud.agent.Agent] > > (agentRequest-Handler-3:null) Set agent id 12 > > 2021-02-16 18:20:28,111 INFO [cloud.agent.Agent] > > (agentRequest-Handler-3:null) Processed new management server list: > > 10.6.0.127@static > > 2021-02-16 18:20:28,112 INFO [cloud.agent.Agent] > > (agentRequest-Handler-3:null) Ready command is processed for agent id > > = 12 > > 2021-02-16 18:20:28,113 INFO > > [resource.consoleproxy.ConsoleProxyResource] > > (agentRequest-Handler-3:null) Receive ReadyCommand, response with > > ReadyAnswer > > 2021-02-16 18:29:15,776 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Lost connection to host: 10.6.0.127. Attempting > > reconnection while we still have 0 commands in progress. > > 2021-02-16 18:29:15,786 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) NioClient connection closed > > 2021-02-16 18:29:15,788 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:15,789 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:15,794 WARN [utils.nio.NioConnection] > > (Agent-Handler-4:null) Unable to connect to remote: is there a server > > running on port 8250 > > 2021-02-16 18:29:20,794 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:20,797 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:20,799 WARN [utils.nio.NioConnection] > > (Agent-Handler-4:null) Unable to connect to remote: is there a server > > running on port 8250 > > 2021-02-16 18:29:25,800 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:25,803 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:25,806 WARN [utils.nio.NioConnection] > > (Agent-Handler-4:null) Unable to connect to remote: is there a server > > running on port 8250 > > 2021-02-16 18:29:30,807 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:30,807 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:30,811 WARN [utils.nio.NioConnection] > > (Agent-Handler-4:null) Unable to connect to remote: is there a server > > running on port 8250 > > 2021-02-16 18:29:35,814 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:35,815 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:35,817 WARN [utils.nio.NioConnection] > > (Agent-Handler-4:null) Unable to connect to remote: is there a server > > running on port 8250 > > 2021-02-16 18:29:40,818 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Reconnecting to host:10.6.0.127 > > 2021-02-16 18:29:40,818 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connecting to 10.6.0.127:8250 > > 2021-02-16 18:29:40,821 INFO [utils.nio.Link] (Agent-Handler-4:null) > > Conf file found: /usr/local/cloud/systemvm/conf/agent.properties > > 2021-02-16 18:29:41,219 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) SSL: Handshake done > > 2021-02-16 18:29:41,221 INFO [utils.nio.NioClient] > > (Agent-Handler-4:null) Connected to 10.6.0.127:8250 > > 2021-02-16 18:29:41,335 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Proccess agent startup answer, agent id = 0 > > 2021-02-16 18:29:41,336 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Set agent id 0 > > 2021-02-16 18:29:41,339 INFO [cloud.agent.Agent] > > (Agent-Handler-2:null) Startup Response Received: agent id = 0 > > 2021-02-16 18:29:41,550 INFO [cloud.agent.Agent] > > (agentRequest-Handler-4:null) Processing agent ready command, agent id > > = 12 > > 2021-02-16 18:29:41,551 INFO [cloud.agent.Agent] > > (agentRequest-Handler-4:null) Set agent id 12 > > 2021-02-16 18:29:41,555 INFO [cloud.agent.Agent] > > (agentRequest-Handler-4:null) Ready command is processed for agent id > > = 12 > > 2021-02-16 18:29:41,555 INFO > > [resource.consoleproxy.ConsoleProxyResource] > > (agentRequest-Handler-4:null) Receive ReadyCommand, response with > > ReadyAnswer > > 2021-02-16 18:29:41,724 INFO [cloud.agent.Agent] > > (agentRequest-Handler-5:null) Processing agent ready command, agent id > > = 12 > > 2021-02-16 18:29:41,725 INFO [cloud.agent.Agent] > > (agentRequest-Handler-5:null) Set agent id 12 > > 2021-02-16 18:29:41,730 INFO [cloud.agent.Agent] > > (agentRequest-Handler-5:null) Processed new management server list: > > 10.6.0.127@static > > 2021-02-16 18:29:41,731 INFO [cloud.agent.Agent] > > (agentRequest-Handler-5:null) Ready command is processed for agent id > > = 12 > > 2021-02-16 18:29:41,731 INFO > > [resource.consoleproxy.ConsoleProxyResource] > > (agentRequest-Handler-5:null) Receive ReadyCommand, response with > > ReadyAnswer > > 2021-02-16 18:29:46,230 INFO [cloud.agent.Agent] > > (Agent-Handler-4:null) Connected to the host: 10.6.0.127 > > > > -- > Pozdrawiam Dominik Czerepiński > -- Daan
