Hi Wei, Ive checked the systemvms and they were both set to UTC, I have run apt-get update and reinstalled tzdata on both and now both servers are reporting the correct time in BST.
However, in the cloud.log, the time that its reporting the VNC connection is 1 hour behind than the time set on the consolevm server, the times below should report 18 not 17 which is a bit strange. 2023-06-12 17:43:37,210 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-13:null) Get resource request for /resource/noVNC/app/images/error.svg 2023-06-12 17:43:37,212 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-13:null) file does not exist/resource/noVNC/app/images/error.svg 2023-06-12 17:50:04,823 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-14:null) Get resource request for /resource/noVNC/vnc.html 2023-06-12 17:50:04,833 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-14:null) Sent file /resource/noVNC/vnc.html with content type text/html 2023-06-12 17:50:04,950 INFO [cloud.consoleproxy.ConsoleProxyNoVNCHandler] (qtp1221518490-41:null) Get websocket connection request from remote IP : 192.168.0.97 2023-06-12 17:50:05,003 WARN [cloud.consoleproxy.ConsoleProxy] (qtp1221518490-41:null) External authenticator failed authentication request for vm 90faddb0-b512-4aff-9738-084b3eecc3af with sid HVaYsT4NZY2UTkt4V0v2Jw 2023-06-12 17:50:05,003 WARN [cloud.consoleproxy.ConsoleProxyNoVNCHandler] (qtp1221518490-41:null) Failed to create viewer due to External authenticator failed request for vm 90faddb0-b512-4aff-9738-084b3eecc3af with sid HVaYsT4NZY2UTkt4V0v2Jw com.cloud.consoleproxy.AuthenticationException: External authenticator failed request for vm 90faddb0-b512-4aff-9738-084b3eecc3af with sid HVaYsT4NZY2UTkt4V0v2Jw at com.cloud.consoleproxy.ConsoleProxy.authenticationExternally(ConsoleProxy.java:528) at com.cloud.consoleproxy.ConsoleProxy.getNoVncViewer(ConsoleProxy.java:562) at com.cloud.consoleproxy.ConsoleProxyNoVNCHandler.onConnect(ConsoleProxyNoVNCHandler.java:136) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:70) at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onConnect(JettyAnnotatedEventDriver.java:178) at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.openSession(AbstractEventDriver.java:217) at org.eclipse.jetty.websocket.common.WebSocketSession.open(WebSocketSession.java:493) at org.eclipse.jetty.websocket.common.WebSocketSession.onOpened(WebSocketSession.java:459) at org.eclipse.jetty.io.AbstractConnection.onOpened(AbstractConnection.java:213) at org.eclipse.jetty.io.AbstractConnection.onOpen(AbstractConnection.java:205) at org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:444) at org.eclipse.jetty.server.HttpConnection.onCompleted(HttpConnection.java:412) at org.eclipse.jetty.server.HttpChannel.onCompleted(HttpChannel.java:832) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:829) 2023-06-12 17:50:05,019 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-15:null) Get resource request for /resource/noVNC/app/images/error.svg 2023-06-12 17:50:05,021 INFO [cloud.consoleproxy.ConsoleProxyResourceHandler] (Thread-15:null) file does not exist/resource/noVNC/app/images/error.svg The time on the management server is correct. Do you think this is a time sync issue? Thanks Ian -----Original Message----- From: Wei ZHOU <ustcweiz...@gmail.com> Sent: 11 June 2023 19:10 To: users@cloudstack.apache.org Subject: Re: NoVNC failed to connect Yes, it could be relevant. Can you check if cpvm is time-synced with the management server ? If yes, can you check if there are any errors in the management-server.log ? -Wei On Sun, 11 Jun 2023 at 18:38, Ian Tobin <ito...@tidyhosts.com> wrote: > Hi Wei, > > I saw this in the log > > 2023-06-11 16:35:31,083 WARN > [cloud.consoleproxy.ConsoleProxyNoVNCHandler] (qtpp > 1670543353-39:null) Failed to create viewer due to External > authenticator failedd request for vm > 90faddb0-b512-4aff-9738-084b3eecc3af with sid > HVaYsT4NZY2UTkt4V00 > v2Jw > com.cloud.consoleproxy.AuthenticationException: External authenticator > failed ree quest for vm 90faddb0-b512-4aff-9738-084b3eecc3af with sid > HVaYsT4NZY2UTkt4V0v2JJ w > > not sure if that is relevant or not? > > Thanks > > Ian > > > > -----Original Message----- > From: Wei ZHOU <ustcweiz...@gmail.com> > Sent: 11 June 2023 17:13 > To: users@cloudstack.apache.org > Subject: Re: NoVNC failed to connect > > All looks good. > > Can you check /var/log/cloud.log in cpvm if there are some errors ? > > > > On Sunday, 11 June 2023, Ian Tobin <ito...@tidyhosts.com> wrote: > > > Hi Wei, > > > > This is the output from consoleproxy > > > > root@v-26-VM:/etc# telnet 192.168.0.34 5900 Trying 192.168.0.34... > > Connected to 192.168.0.34. > > Escape character is '^]'. > > RFB 003.008 > > > > > > Connection closed by foreign host. > > root@v-26-VM:/etc# telnet 192.168.0.34 5901 Trying 192.168.0.34... > > Connected to 192.168.0.34. > > Escape character is '^]'. > > RFB 003.008 > > > > Thanks > > > > Ian > > > > > > > > -----Original Message----- > > From: Wei ZHOU <ustcweiz...@gmail.com> > > Sent: 10 June 2023 16:19 > > To: users@cloudstack.apache.org > > Subject: Re: NoVNC failed to connect > > > > Hi, > > > > Can you check in the console proxy vm if the vnc ports are reachable ? > > > > telnet 192.168.0.34 5900 > > telnet 192.168.0.34 5901 > > > > -Wei > > > > > > On Sat, 10 Jun 2023 at 12:33, Ian Tobin <ito...@tidyhosts.com> wrote: > > > > > Hi Wei, > > > > > > Sorry I thought you wanted me to run inside consoleproxy. > > > > > > So on the hypervisor I can see the standard VNC ports open > > > > > > Active Internet connections (servers and established) > > > Proto Recv-Q Send-Q Local Address Foreign Address > State > > > tcp 0 0 127.0.0.53:53 0.0.0.0:* > > LISTEN > > > tcp 0 0 0.0.0.0:41757 0.0.0.0:* > > LISTEN > > > tcp 0 0 0.0.0.0:33523 0.0.0.0:* > > LISTEN > > > tcp 0 0 0.0.0.0:22 0.0.0.0:* > > LISTEN > > > tcp 0 0 0.0.0.0:111 0.0.0.0:* > > LISTEN > > > tcp 0 0 0.0.0.0:16514 0.0.0.0:* > > LISTEN > > > tcp 0 0 192.168.0.34:5900 0.0.0.0:* > > LISTEN > > > tcp 0 0 192.168.0.34:5901 0.0.0.0:* > > LISTEN > > > tcp 0 256 192.168.0.34:22 192.168.0.97:53409 > > > ESTABLISHED > > > tcp 0 0 192.168.0.34:670 192.168.0.37:2049 > > > ESTABLISHED > > > tcp 0 0 192.168.0.34:34234 185.125.190.36:80 > > > TIME_WAIT > > > tcp6 0 0 :::22 :::* > > LISTEN > > > tcp6 0 0 :::111 :::* > > LISTEN > > > tcp6 0 0 :::16514 :::* > > LISTEN > > > tcp6 0 0 :::43853 :::* > > LISTEN > > > tcp6 0 0 :::39889 :::* > > LISTEN > > > tcp6 0 0 192.168.0.34:47868 192.168.0.33:8250 > > > ESTABLISHED > > > > > > > > > > > > thanks > > > > > > Ian > > > > > > > > > -----Original Message----- > > > From: Wei ZHOU <ustcweiz...@gmail.com> > > > Sent: 10 June 2023 10:39 > > > To: users@cloudstack.apache.org > > > Subject: Re: NoVNC failed to connect > > > > > > Just noticed you run the commands in consoleproxy vm. > > > Can you please run same commands on ubuntu hosts ? > > > > > > -Wei > > > > > > On Saturday, 10 June 2023, Ian Tobin <ito...@tidyhosts.com> wrote: > > > > > > > Hi Wei, > > > > > > > > The Hypervisor is running Ubuntu 22.04.2 LTS > > > > > > > > The Console Proxy is running Debian 11.6, ive also just > > > > destroyed it and let it re create but the netstat shows there is > > > > no VNC ports > > running. > > > > > > > > Thanks > > > > > > > > Ian > > > > > > > > > > > > -----Original Message----- > > > > From: Wei ZHOU <ustcweiz...@gmail.com> > > > > Sent: 10 June 2023 06:50 > > > > To: users@cloudstack.apache.org > > > > Subject: Re: NoVNC failed to connect > > > > > > > > Hi, > > > > > > > > What OS do you use? > > > > > > > > Normally you can find the ports listened on 5900/5901/... which > > > > are vnc ports of vms. > > > > > > > > -Wei > > > > > > > > On Fri, 9 Jun 2023 at 23:40, Ian Tobin <ito...@tidyhosts.com> wrote: > > > > > > > > > Hi Wei, > > > > > > > > > > Sorry for the delay in coming back. > > > > > > > > > > I have got on to the console of the consoleproxy VM, when I > > > > > issue virsh domdisplay it say > > > > > > > > > > root@v-21-VM:~# virsh domdisplay > > > > > -bash: virsh: command not found > > > > > > > > > > Netstat -nat > > > > > > > > > > Active Internet connections (servers and established) > > > > > Proto Recv-Q Send-Q Local Address Foreign Address > > > State > > > > > tcp 0 0 0.0.0.0:8080 0.0.0.0:* > > > > LISTEN > > > > > tcp 0 0 0.0.0.0:80 0.0.0.0:* > > > > LISTEN > > > > > tcp 0 0 169.254.242.168:3922 0.0.0.0:* > > > > LISTEN > > > > > tcp 0 0 0.0.0.0:8001 0.0.0.0:* > > > > LISTEN > > > > > tcp 0 0 192.168.0.203:36154 192.168.0.33:8250 > > > > > ESTABLISHED > > > > > > > > > > thanks > > > > > > > > > > Ian > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Wei ZHOU <ustcweiz...@gmail.com> > > > > > Sent: 07 June 2023 16:33 > > > > > To: users@cloudstack.apache.org > > > > > Subject: Re: NoVNC failed to connect > > > > > > > > > > The default password for root user is "password". > > > > > If you use ssh, password is not required. > > > > > > > > > > -Wei > > > > > > > > > > On Wednesday, 7 June 2023, Ian Tobin <ito...@tidyhosts.com> wrote: > > > > > > > > > > > Hey Wei, > > > > > > > > > > > > I can console on to the proxy VM from the KVM host, but I > > > > > > cannot locate the system.vm.password parameter to get the > > > > > > password in the UI, unless im missing something? > > > > > > > > > > > > Thanks > > > > > > > > > > > > Ian > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Ian Tobin <ito...@tidyhosts.com> > > > > > > Sent: 07 June 2023 13:06 > > > > > > To: users@cloudstack.apache.org > > > > > > Subject: RE: NoVNC failed to connect > > > > > > > > > > > > Hi Wei, > > > > > > > > > > > > Thanks for the info, ill connect and report back. > > > > > > > > > > > > Thanks > > > > > > > > > > > > Ian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Wei ZHOU <ustcweiz...@gmail.com> > > > > > > Sent: 07 June 2023 13:04 > > > > > > To: users@cloudstack.apache.org > > > > > > Subject: Re: NoVNC failed to connect > > > > > > > > > > > > Hi Ian, > > > > > > > > > > > > You can access the system vm via "virsh console" on the kvm > > > > > > host or follow > > > > > > https://docs.cloudstack.apache.org/en/4.18.0.0/adminguide/sy > > > > > > stem > > > > > > vm > > > > > > .h > > > > > > tm > > > > > > l# > > > > > > accessing-system-vms > > > > > > > > > > > > -Wei > > > > > > > > > > > > On Wed, 7 Jun 2023 at 13:58, Ian Tobin > > > > > > <ito...@tidyhosts.com> > > wrote: > > > > > > > > > > > > > Hey Wei, > > > > > > > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > I think I didn’t write correctly, we can ping the systemvm > > > > > > > but also we cannot noVNC to the systemvms. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > Ian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Tel: +44 (0)560 367 4610 > > > > > > > Email: i...@tidyhosts.com > > > > > > > Visit us: www.tidyhosts.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Wei ZHOU <ustcweiz...@gmail.com> > > > > > > > Sent: 07 June 2023 12:21 > > > > > > > To: users@cloudstack.apache.org > > > > > > > Subject: Re: NoVNC failed to connect > > > > > > > > > > > > > > Hi Ian, > > > > > > > > > > > > > > Can you log into the console proxy vm, and check if the > > > > > > > vnc port of > > > > > > > (any) vm is reachable ? > > > > > > > You can get the vnc port by command "virsh domdisplay" or > > > > > > > "virsh vncdisplay" on the kvm host. > > > > > > > > > > > > > > -Wei > > > > > > > > > > > > > > > > > > > > > On Wed, 7 Jun 2023 at 10:19, Ian Tobin > > > > > > > <ito...@tidyhosts.com> > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > We are currently testing CloudStack. We have setup with > > > > > > > > 3 nodes > > > > > > > > (KVM) and the management server. > > > > > > > > > > > > > > > > We have deployed a new instance but we are unable to > > > > > > > > connect to noVNC, the error just says failed to connect to > > > > > > > > server. > > > > > > > > > > > > > > > > We can connect to the systemvm proxy VM so unsure how to > > > > > > > > troubleshoot the issue. > > > > > > > > > > > > > > > > At the moment there is no certificates installed so > > > > > > > > everything is over http. > > > > > > > > > > > > > > > > Any ideas how we can troubleshoot this? > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > Ian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >