Hi all! Luke and Mark, thank you for your helpful insights.
I haven't solved the problem yet, but I give you a little feedback anyway. To be clear, if you connect directly via SSH without Guacamole, the cat takes 4s. When using Guacamole with a defined SSH connection, it takes 1 minute 37 seconds. That's correct. The libssh2 library lead I have seen on the libssh2 project website that the latest version available is 1.11.0. My guacamole server is hosted on a Ubuntu 20.04 LTS server. The latest version of libssh2 package available is 1.8.0. On the latest LTS version of the Ubuntu server (22.04), the version of libssh2 available is 1.10.0. I have tried to upgrade my system like without proper preparation and all it did was break Guacamole, so I need to give it another try and more thoughts. I will either try to install Guacamole properly on a 22.04 server or try to install libssh2 from the sources on my new 20.04 server instead. By the way, which version of libssh2 are you using? The network lead My guacamole server is hosted in a data center, like most of our servers. In the data center, our servers are split into two rooms. We have a site-to-site VPN tunnel between our office and our data center. Also, we have some remote workers connected to our office network through remote VPN. I have run some basic network tests with "iperf". My guacamole server was the iperf server for those tests. To set up the iperf server, I run the command "iperf -s". To start the test, I run the command "iperf -c HOSTNAME" where HOSTNAME is the IP address or hostname of my Guacamole server. Measured network bandwidth : * When I ran the test locally (iperf client is the guacamole server): 12.3 Gbits/sec * When I ran the test from a client within the same room of the data center: 4.73 Gbits/sec * When I ran the test from a client within the other room of the data center: 930 Mbits/sec * When I ran the test from a client within the office: 63.1 Mbits/sec * When I ran the test from a client within the SSL VPN: 27.9 Mbits/sec To give you more insights, the server I tried to read the log file from was in the data center, but not in the same room as my Guacamole server while the user (me) was connected to Guacamole from the office or remote office (SSL VPN). I have run a similar test with a server from the same room and it is quite as bad (Direct connection: "real 0m13.209s user 0m0.005s sys 0m8.306s" vs Guacamole connection: "real 3m40.001s user 0m0.009s sys0m8.184s"). So I would be tempted to think that network is not the issue but it may be more subtle than that. I don't know yet. Out of curiosity, what is the smallest network bandwidth you made Guacamole work without causing noticeable latency for your users? Also, I have run the test Luke mentioned (with "dd" and "cat") to compare the running time. * Here is the command execution time when connected through Guacamole: real 27m59,303s user 0m0,008s sys 0m17,947s * Here is the command execution time when directly connected to the server: real 0m54,749s user 0m0,002s sys 0m24,072s I hope it all makes sense. English is not my native language. Best regards, Julien Lejeune IT administrator +32(0) 81 33 11 11 www.actia.be<http://www.actia.be/> [Linkedin] <https://www.linkedin.com/company/actia-telematics-services> This email message as well as all attachments are for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you. De : Mark Li <[email protected]> Envoyé : lundi 16 octobre 2023 16:11 À : [email protected]; Julien Lejeune <[email protected]> Objet : Re: Slow output - Linux connections Vous ne recevez pas souvent de courriers de la part de [email protected]<mailto:[email protected]>. Découvrez pourquoi cela est important<https://aka.ms/LearnAboutSenderIdentification> On a completely different system I had a similar issue. The problem turned out to be an old ssl library. Once upgraded all was good. The problem was the old library did an ssl setup per transfer vs one setup per connection Sent from my T-Mobile 5G Device Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Luke H <[email protected]<mailto:[email protected]>> Sent: Monday, October 16, 2023 9:52:56 AM To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>; Julien Lejeune <[email protected]<mailto:[email protected]>> Subject: RE: Slow output - Linux connections Hi Julien, dd if=/dev/urandom bs=1024k count=128 | base64 > BIGFILE.txt ls -al BIGFILE.txt -rw-r--r-- 1 user user 181311669 Oct 16 06:24 BIGFILE.txt # From direct SSH connection, sans Guacamole time cat BIGFILE.txt 0.008u 2.280s 2:18.43 1.6% 0+0k 0+0io 0pf+0w # From Guacamole SSH connection time cat BIGFILE.txt 0.006u 2.150s 4:39.53 0.7%0+0k 0+0io 0pf+0w While it takes the roughly same about of elapsed/CPU time, it takes double the amount of user time. But in both cases, I can Ctrl-C. Given my ratio is 1:2 for direct:guacamole user time, and yours is 1:24, it could be Ctrl-C is not received adequately, given bandwidth. This is not really an answer, and your question only raises more questions. The first being, what version of libssh2 are you using, or what version of Docker image? Beyond that, I'll have to defer to the folks on the list. Fascinating... Luke > On 10/16/2023 8:04 AM CDT Luke H > <[email protected]<mailto:[email protected]>> > wrote: > > > Even more fascinating. The reason I say so is that Apache Guacamole is one of > the most useful tools in a person's toolkit. > > While I have an idea why, let me see if I can replicate your issue. > > To be clear, if you connect directly via SSH without Guacamole, the cat takes > 4s. When using Guacamole with a defined SSH connection, it takes 1 minute 37 > seconds. > > People on this list can probably provide a quicker-to-the-point answer, but > it's my guess that the SSH (libssh2) library is mandating synchronous I/O to > the terminal (and therefore is limited to the bandwidth available to you), > where VNC/RDP can somewhat gloss over the terminal I/O. > > Put another way, SSH is more granular, thus requiring more time, based on > bandwidth. > > I'd love to know how many miles differentiate your computer and the VM, and > what type of Internet connection you have on your computer...hotspot, wired, > wifi, etc. > > It's my guess the answer lies somewhere in there. > > Luke > > > On 10/16/2023 6:34 AM CDT Julien Lejeune > > <[email protected]<mailto:[email protected]>> wrote: > > > > > > Hi Luke, > > > > This is an SSH connection. > > > > > > Julien Lejeune > > IT administrator > > > > +32(0) 81 33 11 11 > > www.actia.be<http://www.actia.be> > > > > > > > > > > > > > > > > This email message as well as all attachments are for the sole use of the > > intended recipient(s) and may contain confidential and privileged > > information. > > Any unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by reply > > e-mail and destroy all copies of the original message. Thank you. > > > > -----Message d'origine----- > > De : > > [email protected]<mailto:[email protected]> > > <[email protected]<mailto:[email protected]>> > > Envoyé : lundi 16 octobre 2023 13:10 > > À : [email protected]<mailto:[email protected]>; Julien > > Lejeune <[email protected]<mailto:[email protected]>> > > Objet : Re: Slow output - Linux connections > > > > [Vous ne recevez pas souvent de courriers de > > [email protected]<mailto:[email protected]>. > > Découvrez pourquoi ceci est important à > > https://aka.ms/LearnAboutSenderIdentification ] > > > > Hi Julien, this is a very interesting problem indeed, since you can confirm > > your VM has better timing sans Guacamole. > > > > A question: what type of connection to your VM is Guacamole using - VNC, > > RDP, SSH? > > > > Luke > > > > > On 10/16/2023 4:23 AM CDT Julien Lejeune > > > <[email protected]<mailto:[email protected]>> wrote: > > > > > > > > > Hi all ! > > > > > > I am currently experiencing a problem with my Guacamole instance. > > > When a user tries to show the content of a file within Guacamole with > > > something like « cat », the output takes a long time to come, and it is > > > not possible to interrupt the command with a CTRL + C for example. > > > I have done a comparison by using cat on a big log file within Guacamole > > > or directly connected through the Windows command line. > > > The command takes 1 minute and 37 seconds to execute with the Guacamole > > > session, but it only takes 4 seconds with the direct connection. > > > > > > I have monitored the Guacamole server's CPU and RAM. I see that the CPU > > > load average rises a little during the command execution (from 0.20 > > > before the test to 0.70 during the test on a 4CPU VM), but it doesn't > > > seem that bad to me. > > > > > > Has someone seen the same behavior? > > > If anybody has an idea of what could be the root cause, that would be > > > great. > > > > > > I'll send an update if I find what's causing the problem, in case it > > > helps someone. > > > > > > Best regards, > > > > > > [ACTIA]<http://www.actia.be/> > > > Julien Lejeune > > > IT administrator > > > > > > +32(0) 81 33 11 11 > > > > > > Contact me on > > > Teams<https://teams.microsoft.com/l/chat/0/[email protected]> > > > [cid:[email protected]] > > > > > > www.actia.be<http://www.actia.be/> [Linkedin] > > > <https://www.linkedin.com/company/actia-telematics-services> > > > Chaussée de Marche, 774 > > > B-5100 Naninne - BELGIUM > > > > > > > > > This email message as well as all attachments are for the sole use of the > > > intended recipient(s) and may contain confidential and privileged > > > information. > > > Any unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by reply > > > e-mail and destroy all copies of the original message. Thank you. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > > [email protected]<mailto:[email protected]> > > For additional commands, e-mail: > > [email protected]<mailto:[email protected]> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [email protected]<mailto:[email protected]> > For additional commands, e-mail: > [email protected]<mailto:[email protected]> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]>
