Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
I have to update that the container stuck has not happened again after i upgraded ubuntu 18 kernel from 4.5 to latest 5 kernel and apt-get upgrade all packages on machine. So if anyone is running into it this was the solution. Thanks. בתאריך יום ג׳, 12 באוק׳ 2021 ב-18:40 מאת Nick Couchman : > On Tue, Oct 12, 2021 at 5:48 AM Regev Batchen wrote: > >> Hey Again, i'm still not able to handle this error that happens once a >> day, all containers are up but api request to gqm gets 502 >> I did get the full postgres query when this usually happened: >> >> postgres_gqm_1| 2021-10-12T06:55:19.947186314Z 2021-10-12 >> 06:55:19.938 UTC [35] LOG: could not send data to client: Connection reset >> by peer >> > > I don't think your issues are specifically related to Guacamole - I think > there is something else driving network connectivity challenges between > your containers, which is very odd since networking on a local system > doesn't usually have much in the way of limits or instability. But, that's > really what it looks like, here - between this error, where the peer is > resetting the connection, and the previous gaucd one you posted about the > error reading select, it really seems like a network problem. > > >> postgres_gqm_1| 2021-10-12T06:55:19.947212467Z 2021-10-12 >> 06:55:19.938 UTC [35] STATEMENT: SELECT >> postgres_gqm_1| 2021-10-12T06:55:19.947215854Z >> guacamole_connection.connection_id, >> postgres_gqm_1| 2021-10-12T06:55:19.947218656Z >> guacamole_connection.connection_name, >> postgres_gqm_1| 2021-10-12T06:55:19.947221509Z >> parent_id, >> postgres_gqm_1| 2021-10-12T06:55:19.947224165Z >> protocol, >> postgres_gqm_1| 2021-10-12T06:55:19.947226788Z >> max_connections, >> postgres_gqm_1| 2021-10-12T06:55:19.947229387Z >> max_connections_per_user, >> postgres_gqm_1| 2021-10-12T06:55:19.947232302Z >> proxy_hostname, >> postgres_gqm_1| 2021-10-12T06:55:19.947234855Z >> proxy_port, >> postgres_gqm_1| 2021-10-12T06:55:19.947237399Z >> proxy_encryption_method, >> postgres_gqm_1| 2021-10-12T06:55:19.947239993Z >> connection_weight, >> postgres_gqm_1| 2021-10-12T06:55:19.947242599Z >> failover_only, >> postgres_gqm_1| 2021-10-12T06:55:19.947245130Z >> MAX(start_date) AS last_active >> postgres_gqm_1| 2021-10-12T06:55:19.947247726Z FROM >> guacamole_connection >> postgres_gqm_1| 2021-10-12T06:55:19.947250330Z LEFT JOIN >> guacamole_connection_history ON guacamole_connection_history.connection_id >> = guacamole_connection.connection_id >> postgres_gqm_1| 2021-10-12T06:55:19.947253528Z WHERE >> guacamole_connection.connection_id IN >> postgres_gqm_1| 2021-10-12T06:55:19.947256208Z ( >> postgres_gqm_1| 2021-10-12T06:55:19.947258731Z >> $1::integer >> postgres_gqm_1| 2021-10-12T06:55:19.947261284Z , >> postgres_gqm_1| 2021-10-12T06:55:19.947263751Z >> $2::integer >> postgres_gqm_1| 2021-10-12T06:55:21.230273860Z , >> postgres_gqm_1| 2021-10-12T06:55:21.230276158Z >> $25400::integer >> postgres_gqm_1| 2021-10-12T06:55:21.230278327Z ) >> postgres_gqm_1| 2021-10-12T06:55:21.230280533Z GROUP BY >> guacamole_connection.connection_id >> >> Is this helping in any way? My next step is gonna be upgrading the >> kernel and libs on the host and if that doesn't work to move from docker to >> kubernetes :( >> >> > I mean, you can try those things, but I would spend time figuring out why > network traffic is getting dropped to the point of causing these issues. Do > you see elevated error rates on any of the interfaces in your environment? > Any messages - not necessarily Guacamole-related - that point to traffic > getting dropped/stopped/blocked/lost? > > > ERRINFO_DISCONNECTED_BY_OTHER_CONNECTION (0x0005):Another user > connected to the server, forcing the disconnection of the current > connection. > > This is definitely not Guacamole-related, and is exactly what the error > message describes - some other user connected to the server and forced the > session to log off. This is a RDP server message, and Guacamole is just > forwarding the error information through. > > -Nick > >>
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
On Tue, Oct 12, 2021 at 5:48 AM Regev Batchen wrote: > Hey Again, i'm still not able to handle this error that happens once a > day, all containers are up but api request to gqm gets 502 > I did get the full postgres query when this usually happened: > > postgres_gqm_1| 2021-10-12T06:55:19.947186314Z 2021-10-12 06:55:19.938 > UTC [35] LOG: could not send data to client: Connection reset by peer > I don't think your issues are specifically related to Guacamole - I think there is something else driving network connectivity challenges between your containers, which is very odd since networking on a local system doesn't usually have much in the way of limits or instability. But, that's really what it looks like, here - between this error, where the peer is resetting the connection, and the previous gaucd one you posted about the error reading select, it really seems like a network problem. > postgres_gqm_1| 2021-10-12T06:55:19.947212467Z 2021-10-12 06:55:19.938 > UTC [35] STATEMENT: SELECT > postgres_gqm_1| 2021-10-12T06:55:19.947215854Z > guacamole_connection.connection_id, > postgres_gqm_1| 2021-10-12T06:55:19.947218656Z > guacamole_connection.connection_name, > postgres_gqm_1| 2021-10-12T06:55:19.947221509Z > parent_id, > postgres_gqm_1| 2021-10-12T06:55:19.947224165Z > protocol, > postgres_gqm_1| 2021-10-12T06:55:19.947226788Z > max_connections, > postgres_gqm_1| 2021-10-12T06:55:19.947229387Z > max_connections_per_user, > postgres_gqm_1| 2021-10-12T06:55:19.947232302Z > proxy_hostname, > postgres_gqm_1| 2021-10-12T06:55:19.947234855Z > proxy_port, > postgres_gqm_1| 2021-10-12T06:55:19.947237399Z > proxy_encryption_method, > postgres_gqm_1| 2021-10-12T06:55:19.947239993Z > connection_weight, > postgres_gqm_1| 2021-10-12T06:55:19.947242599Z > failover_only, > postgres_gqm_1| 2021-10-12T06:55:19.947245130Z > MAX(start_date) AS last_active > postgres_gqm_1| 2021-10-12T06:55:19.947247726Z FROM > guacamole_connection > postgres_gqm_1| 2021-10-12T06:55:19.947250330Z LEFT JOIN > guacamole_connection_history ON guacamole_connection_history.connection_id > = guacamole_connection.connection_id > postgres_gqm_1| 2021-10-12T06:55:19.947253528Z WHERE > guacamole_connection.connection_id IN > postgres_gqm_1| 2021-10-12T06:55:19.947256208Z ( > postgres_gqm_1| 2021-10-12T06:55:19.947258731Z > $1::integer > postgres_gqm_1| 2021-10-12T06:55:19.947261284Z , > postgres_gqm_1| 2021-10-12T06:55:19.947263751Z > $2::integer > postgres_gqm_1| 2021-10-12T06:55:21.230273860Z , > postgres_gqm_1| 2021-10-12T06:55:21.230276158Z > $25400::integer > postgres_gqm_1| 2021-10-12T06:55:21.230278327Z ) > postgres_gqm_1| 2021-10-12T06:55:21.230280533Z GROUP BY > guacamole_connection.connection_id > > Is this helping in any way? My next step is gonna be upgrading the kernel > and libs on the host and if that doesn't work to move from docker to > kubernetes :( > > I mean, you can try those things, but I would spend time figuring out why network traffic is getting dropped to the point of causing these issues. Do you see elevated error rates on any of the interfaces in your environment? Any messages - not necessarily Guacamole-related - that point to traffic getting dropped/stopped/blocked/lost? > ERRINFO_DISCONNECTED_BY_OTHER_CONNECTION (0x0005):Another user connected to the server, forcing the disconnection of the current connection. This is definitely not Guacamole-related, and is exactly what the error message describes - some other user connected to the server and forced the session to log off. This is a RDP server message, and Guacamole is just forwarding the error information through. -Nick >
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
And also this error on guacd : ERRINFO_DISCONNECTED_BY_OTHER_CONNECTION (0x0005):Another user connected to the server, forcing the disconnection of the current connection. בתאריך יום ג׳, 12 באוק׳ 2021 ב-12:47 מאת Regev Batchen < batch...@gmail.com>: > Hey Again, i'm still not able to handle this error that happens once a > day, all containers are up but api request to gqm gets 502 > I did get the full postgres query when this usually happened: > > postgres_gqm_1| 2021-10-12T06:55:19.947186314Z 2021-10-12 06:55:19.938 > UTC [35] LOG: could not send data to client: Connection reset by peer > postgres_gqm_1| 2021-10-12T06:55:19.947212467Z 2021-10-12 06:55:19.938 > UTC [35] STATEMENT: SELECT > postgres_gqm_1| 2021-10-12T06:55:19.947215854Z > guacamole_connection.connection_id, > postgres_gqm_1| 2021-10-12T06:55:19.947218656Z > guacamole_connection.connection_name, > postgres_gqm_1| 2021-10-12T06:55:19.947221509Z > parent_id, > postgres_gqm_1| 2021-10-12T06:55:19.947224165Z > protocol, > postgres_gqm_1| 2021-10-12T06:55:19.947226788Z > max_connections, > postgres_gqm_1| 2021-10-12T06:55:19.947229387Z > max_connections_per_user, > postgres_gqm_1| 2021-10-12T06:55:19.947232302Z > proxy_hostname, > postgres_gqm_1| 2021-10-12T06:55:19.947234855Z > proxy_port, > postgres_gqm_1| 2021-10-12T06:55:19.947237399Z > proxy_encryption_method, > postgres_gqm_1| 2021-10-12T06:55:19.947239993Z > connection_weight, > postgres_gqm_1| 2021-10-12T06:55:19.947242599Z > failover_only, > postgres_gqm_1| 2021-10-12T06:55:19.947245130Z > MAX(start_date) AS last_active > postgres_gqm_1| 2021-10-12T06:55:19.947247726Z FROM > guacamole_connection > postgres_gqm_1| 2021-10-12T06:55:19.947250330Z LEFT JOIN > guacamole_connection_history ON guacamole_connection_history.connection_id > = guacamole_connection.connection_id > postgres_gqm_1| 2021-10-12T06:55:19.947253528Z WHERE > guacamole_connection.connection_id IN > postgres_gqm_1| 2021-10-12T06:55:19.947256208Z ( > postgres_gqm_1| 2021-10-12T06:55:19.947258731Z > $1::integer > postgres_gqm_1| 2021-10-12T06:55:19.947261284Z , > postgres_gqm_1| 2021-10-12T06:55:19.947263751Z > $2::integer > postgres_gqm_1| 2021-10-12T06:55:21.230273860Z , > postgres_gqm_1| 2021-10-12T06:55:21.230276158Z > $25400::integer > postgres_gqm_1| 2021-10-12T06:55:21.230278327Z ) > postgres_gqm_1| 2021-10-12T06:55:21.230280533Z GROUP BY > guacamole_connection.connection_id > > Is this helping in any way? My next step is gonna be upgrading the kernel > and libs on the host and if that doesn't work to move from docker to > kubernetes :( > > בתאריך יום א׳, 26 בספט׳ 2021 ב-10:45 מאת Regev Batchen < > batch...@gmail.com>: > >> Hey, >> >> I dont have the full psql container query because i tried to do tail of >> 2 lines and it still shows >> >> postgres_gqm_1| 2021-09-13T14:59:26.208973521Z , >> postgres_gqm_1| 2021-09-13T14:59:26.208975652Z >> $22740::integer >> postgres_gqm_1| 2021-09-13T14:59:26.208977856Z , >> postgres_gqm_1| 2021-09-13T14:59:26.208980045Z >> $22741::integer >> >> docker ps shows containers are healthy even when service is stuck >> >> Is there a way to add guacamole debug logs? >> In the guacd container I have : GUACD_LOG_LEVEL: debug >> but for guacamole ? >> >> It looks like psql did a restart after the big quarry and guacd did too >> but something is not working correctly until i restart the guacd and >> guacamole and then it works for a day or two, >> what else can I check? >> I am going to create a bash script to restart containers if service is >> stuck because I really don't know what else to do :( >> >> >> >> >> בתאריך יום ד׳, 22 בספט׳ 2021 ב-22:40 מאת Mike Jumper < >> mike.jum...@glyptodon.com>: >> >>> On Wed, Sep 22, 2021 at 5:45 AM Regev Batchen >>> wrote: >>> Hello, the postgres error happened again. I dont understand why postgrece started to get stuck after i updated to guacamole 1.3 , befor i had an error that it was only get stuck with no sql error. i use psql 12.3 and here are some logs, PLEASE let me know what else i can check and debug? i have 10 more containers on this docker-compose and only guacamole gets stuck. It just stop receiving api request and no logs are written to the guac containers. what causing this SELECT error? >>> >>> Do you have the full message from the PostgreSQL container? >>> >>> The log message you see from guacd (Error reading "select") is not >>> actually related to the database (which is only used by the webapp, not >>> guacd). This is guacd letting you know that a connection to guacd was >>> established and then closed during the Guacamole protocol handshake, the >>> first
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
Hey Again, i'm still not able to handle this error that happens once a day, all containers are up but api request to gqm gets 502 I did get the full postgres query when this usually happened: postgres_gqm_1| 2021-10-12T06:55:19.947186314Z 2021-10-12 06:55:19.938 UTC [35] LOG: could not send data to client: Connection reset by peer postgres_gqm_1| 2021-10-12T06:55:19.947212467Z 2021-10-12 06:55:19.938 UTC [35] STATEMENT: SELECT postgres_gqm_1| 2021-10-12T06:55:19.947215854Z guacamole_connection.connection_id, postgres_gqm_1| 2021-10-12T06:55:19.947218656Z guacamole_connection.connection_name, postgres_gqm_1| 2021-10-12T06:55:19.947221509Z parent_id, postgres_gqm_1| 2021-10-12T06:55:19.947224165Z protocol, postgres_gqm_1| 2021-10-12T06:55:19.947226788Z max_connections, postgres_gqm_1| 2021-10-12T06:55:19.947229387Z max_connections_per_user, postgres_gqm_1| 2021-10-12T06:55:19.947232302Z proxy_hostname, postgres_gqm_1| 2021-10-12T06:55:19.947234855Z proxy_port, postgres_gqm_1| 2021-10-12T06:55:19.947237399Z proxy_encryption_method, postgres_gqm_1| 2021-10-12T06:55:19.947239993Z connection_weight, postgres_gqm_1| 2021-10-12T06:55:19.947242599Z failover_only, postgres_gqm_1| 2021-10-12T06:55:19.947245130Z MAX(start_date) AS last_active postgres_gqm_1| 2021-10-12T06:55:19.947247726Z FROM guacamole_connection postgres_gqm_1| 2021-10-12T06:55:19.947250330Z LEFT JOIN guacamole_connection_history ON guacamole_connection_history.connection_id = guacamole_connection.connection_id postgres_gqm_1| 2021-10-12T06:55:19.947253528Z WHERE guacamole_connection.connection_id IN postgres_gqm_1| 2021-10-12T06:55:19.947256208Z ( postgres_gqm_1| 2021-10-12T06:55:19.947258731Z $1::integer postgres_gqm_1| 2021-10-12T06:55:19.947261284Z , postgres_gqm_1| 2021-10-12T06:55:19.947263751Z $2::integer postgres_gqm_1| 2021-10-12T06:55:21.230273860Z , postgres_gqm_1| 2021-10-12T06:55:21.230276158Z $25400::integer postgres_gqm_1| 2021-10-12T06:55:21.230278327Z ) postgres_gqm_1| 2021-10-12T06:55:21.230280533Z GROUP BY guacamole_connection.connection_id Is this helping in any way? My next step is gonna be upgrading the kernel and libs on the host and if that doesn't work to move from docker to kubernetes :( בתאריך יום א׳, 26 בספט׳ 2021 ב-10:45 מאת Regev Batchen < batch...@gmail.com>: > Hey, > > I dont have the full psql container query because i tried to do tail of > 2 lines and it still shows > > postgres_gqm_1| 2021-09-13T14:59:26.208973521Z , > postgres_gqm_1| 2021-09-13T14:59:26.208975652Z > $22740::integer > postgres_gqm_1| 2021-09-13T14:59:26.208977856Z , > postgres_gqm_1| 2021-09-13T14:59:26.208980045Z > $22741::integer > > docker ps shows containers are healthy even when service is stuck > > Is there a way to add guacamole debug logs? > In the guacd container I have : GUACD_LOG_LEVEL: debug > but for guacamole ? > > It looks like psql did a restart after the big quarry and guacd did too > but something is not working correctly until i restart the guacd and > guacamole and then it works for a day or two, > what else can I check? > I am going to create a bash script to restart containers if service is > stuck because I really don't know what else to do :( > > > > > בתאריך יום ד׳, 22 בספט׳ 2021 ב-22:40 מאת Mike Jumper < > mike.jum...@glyptodon.com>: > >> On Wed, Sep 22, 2021 at 5:45 AM Regev Batchen wrote: >> >>> Hello, the postgres error happened again. >>> I dont understand why postgrece started to get stuck after i updated to >>> guacamole 1.3 , befor i had an error that it was only get stuck with no sql >>> error. >>> i use psql 12.3 and here are some logs, PLEASE let me know what else i >>> can check and debug? i have 10 more containers on this docker-compose and >>> only guacamole gets stuck. >>> >>> It just stop receiving api request and no logs are written to the guac >>> containers. >>> what causing this SELECT error? >>> >> >> Do you have the full message from the PostgreSQL container? >> >> The log message you see from guacd (Error reading "select") is not >> actually related to the database (which is only used by the webapp, not >> guacd). This is guacd letting you know that a connection to guacd was >> established and then closed during the Guacamole protocol handshake, the >> first message of which happens to be called "select". This is the result of >> a healthcheck ping within Docker and is not an error. >> >> - Mike >> >>
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
Hey, I dont have the full psql container query because i tried to do tail of 2 lines and it still shows postgres_gqm_1| 2021-09-13T14:59:26.208973521Z , postgres_gqm_1| 2021-09-13T14:59:26.208975652Z $22740::integer postgres_gqm_1| 2021-09-13T14:59:26.208977856Z , postgres_gqm_1| 2021-09-13T14:59:26.208980045Z $22741::integer docker ps shows containers are healthy even when service is stuck Is there a way to add guacamole debug logs? In the guacd container I have : GUACD_LOG_LEVEL: debug but for guacamole ? It looks like psql did a restart after the big quarry and guacd did too but something is not working correctly until i restart the guacd and guacamole and then it works for a day or two, what else can I check? I am going to create a bash script to restart containers if service is stuck because I really don't know what else to do :( בתאריך יום ד׳, 22 בספט׳ 2021 ב-22:40 מאת Mike Jumper < mike.jum...@glyptodon.com>: > On Wed, Sep 22, 2021 at 5:45 AM Regev Batchen wrote: > >> Hello, the postgres error happened again. >> I dont understand why postgrece started to get stuck after i updated to >> guacamole 1.3 , befor i had an error that it was only get stuck with no sql >> error. >> i use psql 12.3 and here are some logs, PLEASE let me know what else i >> can check and debug? i have 10 more containers on this docker-compose and >> only guacamole gets stuck. >> >> It just stop receiving api request and no logs are written to the guac >> containers. >> what causing this SELECT error? >> > > Do you have the full message from the PostgreSQL container? > > The log message you see from guacd (Error reading "select") is not > actually related to the database (which is only used by the webapp, not > guacd). This is guacd letting you know that a connection to guacd was > established and then closed during the Guacamole protocol handshake, the > first message of which happens to be called "select". This is the result of > a healthcheck ping within Docker and is not an error. > > - Mike > >
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
On Wed, Sep 22, 2021 at 5:45 AM Regev Batchen wrote: > Hello, the postgres error happened again. > I dont understand why postgrece started to get stuck after i updated to > guacamole 1.3 , befor i had an error that it was only get stuck with no sql > error. > i use psql 12.3 and here are some logs, PLEASE let me know what else i can > check and debug? i have 10 more containers on this docker-compose and only > guacamole gets stuck. > > It just stop receiving api request and no logs are written to the guac > containers. > what causing this SELECT error? > Do you have the full message from the PostgreSQL container? The log message you see from guacd (Error reading "select") is not actually related to the database (which is only used by the webapp, not guacd). This is guacd letting you know that a connection to guacd was established and then closed during the Guacamole protocol handshake, the first message of which happens to be called "select". This is the result of a healthcheck ping within Docker and is not an error. - Mike
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
On Mon, Sep 13, 2021 at 2:51 PM Regev Batchen wrote: > I did it when it got stuck , works fine for now but i have this random > guacamole stuck for 2 month so i feel like im gonna see this again this > week, i really hope not thats why i upgraded everything.. > > You think this current error was a 1 time thing? > > Not sure, but I don't think it's anything specific to Guacamole. -Nick >
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
I did it when it got stuck , works fine for now but i have this random guacamole stuck for 2 month so i feel like im gonna see this again this week, i really hope not thats why i upgraded everything.. You think this current error was a 1 time thing? I will post here again if it happens again .. 梁 בתאריך יום ב׳, 13 בספט׳ 2021 ב-21:47 מאת Nick Couchman : > On Mon, Sep 13, 2021 at 2:40 PM Regev Batchen wrote: > >> >> Hey thank you for your answer! >> I have upgraded it to 1.3.0 and it got stuck again with db error >> >> >> guacd[7]: DEBUG: Error reading "select": End of stream reached while >> reading instruction >> >> And db shows >> >> 3T14:59:26.208975652Z $22740::integer >> ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208977856Z >>, >> ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208980045Z >> $22741::integer >> ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208982374Z >>) >> ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208984696Z >> GROUP BY guacamole_connection.connection_id >> >> >> Looks like some quary from guacamole got it stuck this time not like >> other times before the upgrade ( have upgraded docker and compose and >> guacamole) >> >> I will see this days if this happened again i really think the last issue >> i had with a quary making psql stuck is not a docker issue.. >> >> My last resort now is to make a script that automatically restart >> guacamole containers when they get stuck or db gqm >> >> Thank you and if you have any ides about the last gqm db error i will >> look into whatever u suggest! >> >> > Have you tried restarting the DB container? > > -Nick > >>
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
On Mon, Sep 13, 2021 at 2:40 PM Regev Batchen wrote: > > Hey thank you for your answer! > I have upgraded it to 1.3.0 and it got stuck again with db error > > > guacd[7]: DEBUG: Error reading "select": End of stream reached while > reading instruction > > And db shows > > 3T14:59:26.208975652Z $22740::integer > ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208977856Z >, > ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208980045Z > $22741::integer > ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208982374Z >) > ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208984696Z > GROUP BY guacamole_connection.connection_id > > > Looks like some quary from guacamole got it stuck this time not like other > times before the upgrade ( have upgraded docker and compose and guacamole) > > I will see this days if this happened again i really think the last issue > i had with a quary making psql stuck is not a docker issue.. > > My last resort now is to make a script that automatically restart > guacamole containers when they get stuck or db gqm > > Thank you and if you have any ides about the last gqm db error i will look > into whatever u suggest! > > Have you tried restarting the DB container? -Nick >
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
Hey thank you for your answer! I have upgraded it to 1.3.0 and it got stuck again with db error guacd[7]: DEBUG: Error reading "select": End of stream reached while reading instruction And db shows 3T14:59:26.208975652Z $22740::integer ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208977856Z , ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208980045Z $22741::integer ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208982374Z ) ^[[36mpostgres_gqm_1|^[[0m 2021-09-13T14:59:26.208984696Z GROUP BY guacamole_connection.connection_id Looks like some quary from guacamole got it stuck this time not like other times before the upgrade ( have upgraded docker and compose and guacamole) I will see this days if this happened again i really think the last issue i had with a quary making psql stuck is not a docker issue.. My last resort now is to make a script that automatically restart guacamole containers when they get stuck or db gqm Thank you and if you have any ides about the last gqm db error i will look into whatever u suggest! בתאריך יום ב׳, 13 בספט׳ 2021 ב-19:20 מאת Nick Couchman : > On Mon, Sep 13, 2021 at 11:13 AM Regev Batchen wrote: > >> Hello team, >> can somone please help me ? >> >> I upgraded docker to version 20.10.8 and docker-compose version 1.29.2 >> and also guacd + guacamole version to latest version 1.3.0 >> >> Out of all my containers only guacamole gets stuck >> this is what i see on kern.log: >> >> Sep 13 14:28:44 ansible kernel: [1548303.041290] br-4260ec4ed3f1: port >> 19(veth842fd2a) entered disabled state >> Sep 13 14:28:44 ansible kernel: [1548303.041444] veth8c74c8b: renamed >> from eth0 >> Sep 13 14:28:44 ansible kernel: [1548303.103365] br-4260ec4ed3f1: port >> 19(veth842fd2a) entered disabled state >> Sep 13 14:28:44 ansible kernel: [1548303.114983] device veth842fd2a left >> promiscuous mode >> Sep 13 14:28:44 ansible kernel: [1548303.114987] br-4260ec4ed3f1: port >> 19(veth842fd2a) entered disabled state >> Sep 13 14:28:45 ansible kernel: [1548303.703639] br-4260ec4ed3f1: port >> 11(veth6370b5e) entered disabled state >> Sep 13 14:28:45 ansible kernel: [1548303.703767] veth3a75fd3: renamed >> from eth0 >> Sep 13 14:28:45 ansible kernel: [1548303.769807] br-4260ec4ed3f1: port >> 11(veth6370b5e) entered disabled state >> Sep 13 14:28:45 ansible kernel: [1548303.778901] device veth6370b5e left >> promiscuous mode >> Sep 13 14:28:45 ansible kernel: [1548303.778906] br-4260ec4ed3f1: port >> 11(veth6370b5e) entered disabled state >> Sep 13 14:28:45 ansible kernel: [1548304.023824] br-4260ec4ed3f1: port >> 11(veth0e30632) entered blocking state >> Sep 13 14:28:45 ansible kernel: [1548304.023826] br-4260ec4ed3f1: port >> 11(veth0e30632) entered disabled state >> Sep 13 14:28:45 ansible kernel: [1548304.023993] device veth0e30632 >> entered promiscuous mode >> Sep 13 14:28:45 ansible kernel: [1548304.024117] IPv6: >> ADDRCONF(NETDEV_UP): veth0e30632: link is not ready >> Sep 13 14:28:45 ansible kernel: [1548304.024125] br-4260ec4ed3f1: port >> 11(veth0e30632) entered blocking state >> Sep 13 14:28:45 ansible kernel: [1548304.024127] br-4260ec4ed3f1: port >> 11(veth0e30632) entered forwarding state >> Sep 13 14:28:45 ansible kernel: [1548304.066552] br-4260ec4ed3f1: port >> 11(veth0e30632) entered disabled state >> Sep 13 14:28:46 ansible kernel: [1548305.059156] eth0: renamed from >> veth0b98f17 >> Sep 13 14:28:46 ansible kernel: [1548305.091017] IPv6: >> ADDRCONF(NETDEV_CHANGE): veth0e30632: link becomes ready >> >> guacd : >> >> guacd_1 | 2021-09-13T14:15:42.167400545Z guacd[15338]: DEBUG: >> guac_rdpdr_fs_process_create: [file_id=-2] desired_access=0x80, >> file_attributes=0x0, create_disposition=0x1, create_options=0x1, path="" >> guacd_1 | 2021-09-13T14:15:42.167403278Z guacd[15338]: ERROR: >> File open refused (-2): "" >> guacd_1 | 2021-09-13T14:17:02.948295495Z guacd[15338]: DEBUG: >> Clipboard data received. Reporting availability of clipboard data to RDP >> server. >> guacd_1 | 2021-09-13T14:17:14.857231812Z guacd[15338]: DEBUG: >> Clipboard data received. Reporting availability of clipboard data to RDP >> server. >> guacd_1 | 2021-09-13T14:18:16.590645926Z guacd[15338]: INFO: >> Accepted format: 16-bit PCM with 2 channels at 44100 Hz >> guacd_1 | 2021-09-13T14:18:51.820263319Z guacd[7]: DEBUG: >> Guacamole connection closed during handshake >> guacd_1 | 2021-09-13T14:18:51.820418741Z guacd[7]: DEBUG: >> Error reading "select": End of stream reached while reading instruction >> guacd_1 | 2021-09-13T14:21:51.700790573Z guacd[14814]: DEBUG: >> Clipboard data received. Reporting availability of clipboard data to RDP >> server. >> guacd_1 | 2021-09-13T14:21:52.644317299Z guacd[14814]: INFO: >> User "@4e696ad7-a66f-4415-a84f-aa599d6239ee" disconnected (0 users remain) >> guacd_1 | 2021-09-13T14:21:52.644346809Z guacd[14814]: INFO:
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
On Mon, Sep 13, 2021 at 11:13 AM Regev Batchen wrote: > Hello team, > can somone please help me ? > > I upgraded docker to version 20.10.8 and docker-compose version 1.29.2 > and also guacd + guacamole version to latest version 1.3.0 > > Out of all my containers only guacamole gets stuck > this is what i see on kern.log: > > Sep 13 14:28:44 ansible kernel: [1548303.041290] br-4260ec4ed3f1: port > 19(veth842fd2a) entered disabled state > Sep 13 14:28:44 ansible kernel: [1548303.041444] veth8c74c8b: renamed from > eth0 > Sep 13 14:28:44 ansible kernel: [1548303.103365] br-4260ec4ed3f1: port > 19(veth842fd2a) entered disabled state > Sep 13 14:28:44 ansible kernel: [1548303.114983] device veth842fd2a left > promiscuous mode > Sep 13 14:28:44 ansible kernel: [1548303.114987] br-4260ec4ed3f1: port > 19(veth842fd2a) entered disabled state > Sep 13 14:28:45 ansible kernel: [1548303.703639] br-4260ec4ed3f1: port > 11(veth6370b5e) entered disabled state > Sep 13 14:28:45 ansible kernel: [1548303.703767] veth3a75fd3: renamed from > eth0 > Sep 13 14:28:45 ansible kernel: [1548303.769807] br-4260ec4ed3f1: port > 11(veth6370b5e) entered disabled state > Sep 13 14:28:45 ansible kernel: [1548303.778901] device veth6370b5e left > promiscuous mode > Sep 13 14:28:45 ansible kernel: [1548303.778906] br-4260ec4ed3f1: port > 11(veth6370b5e) entered disabled state > Sep 13 14:28:45 ansible kernel: [1548304.023824] br-4260ec4ed3f1: port > 11(veth0e30632) entered blocking state > Sep 13 14:28:45 ansible kernel: [1548304.023826] br-4260ec4ed3f1: port > 11(veth0e30632) entered disabled state > Sep 13 14:28:45 ansible kernel: [1548304.023993] device veth0e30632 > entered promiscuous mode > Sep 13 14:28:45 ansible kernel: [1548304.024117] IPv6: > ADDRCONF(NETDEV_UP): veth0e30632: link is not ready > Sep 13 14:28:45 ansible kernel: [1548304.024125] br-4260ec4ed3f1: port > 11(veth0e30632) entered blocking state > Sep 13 14:28:45 ansible kernel: [1548304.024127] br-4260ec4ed3f1: port > 11(veth0e30632) entered forwarding state > Sep 13 14:28:45 ansible kernel: [1548304.066552] br-4260ec4ed3f1: port > 11(veth0e30632) entered disabled state > Sep 13 14:28:46 ansible kernel: [1548305.059156] eth0: renamed from > veth0b98f17 > Sep 13 14:28:46 ansible kernel: [1548305.091017] IPv6: > ADDRCONF(NETDEV_CHANGE): veth0e30632: link becomes ready > > guacd : > > guacd_1 | 2021-09-13T14:15:42.167400545Z guacd[15338]: DEBUG: > guac_rdpdr_fs_process_create: [file_id=-2] desired_access=0x80, > file_attributes=0x0, create_disposition=0x1, create_options=0x1, path="" > guacd_1 | 2021-09-13T14:15:42.167403278Z guacd[15338]: ERROR: > File open refused (-2): "" > guacd_1 | 2021-09-13T14:17:02.948295495Z guacd[15338]: DEBUG: > Clipboard data received. Reporting availability of clipboard data to RDP > server. > guacd_1 | 2021-09-13T14:17:14.857231812Z guacd[15338]: DEBUG: > Clipboard data received. Reporting availability of clipboard data to RDP > server. > guacd_1 | 2021-09-13T14:18:16.590645926Z guacd[15338]: INFO: > Accepted format: 16-bit PCM with 2 channels at 44100 Hz > guacd_1 | 2021-09-13T14:18:51.820263319Z guacd[7]: DEBUG: > Guacamole connection closed during handshake > guacd_1 | 2021-09-13T14:18:51.820418741Z guacd[7]: DEBUG: > Error reading "select": End of stream reached while reading instruction > guacd_1 | 2021-09-13T14:21:51.700790573Z guacd[14814]: DEBUG: > Clipboard data received. Reporting availability of clipboard data to RDP > server. > guacd_1 | 2021-09-13T14:21:52.644317299Z guacd[14814]: INFO: > User "@4e696ad7-a66f-4415-a84f-aa599d6239ee" disconnected (0 users remain) > guacd_1 | 2021-09-13T14:21:52.644346809Z guacd[14814]: INFO: > Last user of connection "$ac241f0d-90de-487e-9bf5-d1fcab6463ed" > disconnected > guacd_1 | 2021-09-13T14:21:52.644350539Z guacd[14814]: DEBUG: > Requesting termination of client... > guacd_1 | 2021-09-13T14:21:52.712513048Z guacd[14814]: DEBUG: > Unloading device 0 (G) > guacd_1 | 2021-09-13T14:21:52.712527764Z guacd[14814]: DEBUG: > SVC "rdpdr" disconnected. > guacd_1 | 2021-09-13T14:21:52.712531208Z guacd[14814]: DEBUG: > SVC "rdpsnd" disconnected. > guacd_1 | 2021-09-13T14:21:52.713832474Z guacd[14814]: INFO: > Internal RDP client disconnected > guacd_1 | 2021-09-13T14:21:52.714336876Z guacd[14814]: DEBUG: > Client terminated successfully. > guacd_1 | 2021-09-13T14:21:52.719287612Z guacd[7]: INFO: > Connection "$ac241f0d-90de-487e-9bf5-d1fcab6463ed" removed. > guacd_1 | 2021-09-13T14:22:02.951448033Z guacd[15338]: DEBUG: > Clipboard data received. Reporting availability of clipboard data to RDP > server. > guacd_1 | 2021-09-13T14:23:52.041486801Z guacd[7]: DEBUG: > Guacamole connection closed during handshake > guacd_1 | 2021-09-13T14:23:52.041508185Z guacd[7]: DEBUG: > Error reading "select": End
Re: Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
Hello team, can somone please help me ? I upgraded docker to version 20.10.8 and docker-compose version 1.29.2 and also guacd + guacamole version to latest version 1.3.0 Out of all my containers only guacamole gets stuck this is what i see on kern.log: Sep 13 14:28:44 ansible kernel: [1548303.041290] br-4260ec4ed3f1: port 19(veth842fd2a) entered disabled state Sep 13 14:28:44 ansible kernel: [1548303.041444] veth8c74c8b: renamed from eth0 Sep 13 14:28:44 ansible kernel: [1548303.103365] br-4260ec4ed3f1: port 19(veth842fd2a) entered disabled state Sep 13 14:28:44 ansible kernel: [1548303.114983] device veth842fd2a left promiscuous mode Sep 13 14:28:44 ansible kernel: [1548303.114987] br-4260ec4ed3f1: port 19(veth842fd2a) entered disabled state Sep 13 14:28:45 ansible kernel: [1548303.703639] br-4260ec4ed3f1: port 11(veth6370b5e) entered disabled state Sep 13 14:28:45 ansible kernel: [1548303.703767] veth3a75fd3: renamed from eth0 Sep 13 14:28:45 ansible kernel: [1548303.769807] br-4260ec4ed3f1: port 11(veth6370b5e) entered disabled state Sep 13 14:28:45 ansible kernel: [1548303.778901] device veth6370b5e left promiscuous mode Sep 13 14:28:45 ansible kernel: [1548303.778906] br-4260ec4ed3f1: port 11(veth6370b5e) entered disabled state Sep 13 14:28:45 ansible kernel: [1548304.023824] br-4260ec4ed3f1: port 11(veth0e30632) entered blocking state Sep 13 14:28:45 ansible kernel: [1548304.023826] br-4260ec4ed3f1: port 11(veth0e30632) entered disabled state Sep 13 14:28:45 ansible kernel: [1548304.023993] device veth0e30632 entered promiscuous mode Sep 13 14:28:45 ansible kernel: [1548304.024117] IPv6: ADDRCONF(NETDEV_UP): veth0e30632: link is not ready Sep 13 14:28:45 ansible kernel: [1548304.024125] br-4260ec4ed3f1: port 11(veth0e30632) entered blocking state Sep 13 14:28:45 ansible kernel: [1548304.024127] br-4260ec4ed3f1: port 11(veth0e30632) entered forwarding state Sep 13 14:28:45 ansible kernel: [1548304.066552] br-4260ec4ed3f1: port 11(veth0e30632) entered disabled state Sep 13 14:28:46 ansible kernel: [1548305.059156] eth0: renamed from veth0b98f17 Sep 13 14:28:46 ansible kernel: [1548305.091017] IPv6: ADDRCONF(NETDEV_CHANGE): veth0e30632: link becomes ready guacd : guacd_1 | 2021-09-13T14:15:42.167400545Z guacd[15338]: DEBUG: guac_rdpdr_fs_process_create: [file_id=-2] desired_access=0x80, file_attributes=0x0, create_disposition=0x1, create_options=0x1, path="" guacd_1 | 2021-09-13T14:15:42.167403278Z guacd[15338]: ERROR: File open refused (-2): "" guacd_1 | 2021-09-13T14:17:02.948295495Z guacd[15338]: DEBUG: Clipboard data received. Reporting availability of clipboard data to RDP server. guacd_1 | 2021-09-13T14:17:14.857231812Z guacd[15338]: DEBUG: Clipboard data received. Reporting availability of clipboard data to RDP server. guacd_1 | 2021-09-13T14:18:16.590645926Z guacd[15338]: INFO: Accepted format: 16-bit PCM with 2 channels at 44100 Hz guacd_1 | 2021-09-13T14:18:51.820263319Z guacd[7]: DEBUG: Guacamole connection closed during handshake guacd_1 | 2021-09-13T14:18:51.820418741Z guacd[7]: DEBUG: Error reading "select": End of stream reached while reading instruction guacd_1 | 2021-09-13T14:21:51.700790573Z guacd[14814]: DEBUG: Clipboard data received. Reporting availability of clipboard data to RDP server. guacd_1 | 2021-09-13T14:21:52.644317299Z guacd[14814]: INFO: User "@4e696ad7-a66f-4415-a84f-aa599d6239ee" disconnected (0 users remain) guacd_1 | 2021-09-13T14:21:52.644346809Z guacd[14814]: INFO: Last user of connection "$ac241f0d-90de-487e-9bf5-d1fcab6463ed" disconnected guacd_1 | 2021-09-13T14:21:52.644350539Z guacd[14814]: DEBUG: Requesting termination of client... guacd_1 | 2021-09-13T14:21:52.712513048Z guacd[14814]: DEBUG: Unloading device 0 (G) guacd_1 | 2021-09-13T14:21:52.712527764Z guacd[14814]: DEBUG: SVC "rdpdr" disconnected. guacd_1 | 2021-09-13T14:21:52.712531208Z guacd[14814]: DEBUG: SVC "rdpsnd" disconnected. guacd_1 | 2021-09-13T14:21:52.713832474Z guacd[14814]: INFO: Internal RDP client disconnected guacd_1 | 2021-09-13T14:21:52.714336876Z guacd[14814]: DEBUG: Client terminated successfully. guacd_1 | 2021-09-13T14:21:52.719287612Z guacd[7]: INFO: Connection "$ac241f0d-90de-487e-9bf5-d1fcab6463ed" removed. guacd_1 | 2021-09-13T14:22:02.951448033Z guacd[15338]: DEBUG: Clipboard data received. Reporting availability of clipboard data to RDP server. guacd_1 | 2021-09-13T14:23:52.041486801Z guacd[7]: DEBUG: Guacamole connection closed during handshake guacd_1 | 2021-09-13T14:23:52.041508185Z guacd[7]: DEBUG: Error reading "select": End of stream reached while reading instruction guacd_1 | 2021-09-13T14:24:33.529359679Z guacd[15338]: DEBUG: Clipboard data received. Reporting availability of clipboard data to RDP server. guacd_1 |
Docker guacamole+guacd 1.2.0 gets stuck once a day randomly
Hello, I'm using guacamole on docker compose 1.25.0 with guacd and guacamole 1.2.0 on ubuntu 18.04.3 LTS (Bionic Beaver). Everything worked fine for more than 6 month and now it has started to get stuck and won't work until I restart both containers.. docker-compose: guacamole: image: guacamole/guacamole:1.2.0 depends_on: - guacd - postgres_gqm volumes: - /root/ansible/api/start.sh:/opt/guacamole/bin/start.sh - /root/ansible/api/program.jar:/opt/guacamole/program.jar environment: GUACD_HOSTNAME: guacd POSTGRES_DATABASE: guacd_db POSTGRES_HOSTNAME: postgres_gqm POSTGRES_PASSWORD: password POSTGRES_USER: guacd_user GUACD_LOG_LEVEL: debug links: - guacd ports: - 8080/tcp restart: always logging: driver: "json-file" options: max-size: "50m" guacd: image: guacamole/guacd:1.2.0 restart: always volumes: - ./drive:/drive:rw - ./record:/record:rw environment: GUACD_LOG_LEVEL: debug logging: driver: "json-file" options: max-size: "50m" containers logs: guacamole - guacamole_1 | 05:31:36.344 [http-nio-8080-exec-3] INFO o.a.g.r.auth.AuthenticationService - User "guacadmin" successfully authenticated from [172.23.0.1, 172.23.0.19]. guacamole_1 | 05:31:40.852 [http-nio-8080-exec-8] INFO o.a.g.r.auth.AuthenticationService - User "guacadmin" successfully authenticated from [172.23.0.1, 172.23.0.19]. guacamole_1 | 05:31:45.960 [http-nio-8080-exec-11] INFO o.a.g.r.auth.AuthenticationService - User "guacadmin" successfully authenticated from [172.23.0.1, 172.23.0.19]. guacamole_1 | 05:31:51.639 [http-nio-8080-exec-12] INFO o.a.g.tunnel.TunnelRequestService - User "guacadmin" disconnected from connection "22354". Duration: 1195913 milliseconds guacamole_1 | 05:31:51.641 [Thread-26] INFO o.a.g.tunnel.TunnelRequestService - User "" disconnected from connection "UqrJNFZyrUOvaC1vkfz2DP_Qm9-hJhTEB9yzAc9dpk-d". Duration: 1144359 milliseconds guacamole_1 | 05:31:53.033 [http-nio-8080-exec-11] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:32:00.223 [http-nio-8080-exec-16] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:32:14.894 [http-nio-8080-exec-21] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:33:15.593 [http-nio-8080-exec-23] ERROR o.a.g.w.GuacamoleWebSocketTunnelEndpoint - Creation of WebSocket tunnel to guacd failed: Requested tunnel destination does not exist. guacamole_1 | 05:33:15.643 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: Requested tunnel destination does not exist. guacamole_1 | 05:33:16.603 [http-nio-8080-exec-23] ERROR o.a.g.w.GuacamoleWebSocketTunnelEndpoint - Creation of WebSocket tunnel to guacd failed: Requested tunnel destination does not exist. guacamole_1 | 05:33:16.657 [http-nio-8080-exec-23] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: Requested tunnel destination does not exist. guacamole_1 | 05:34:35.958 [http-nio-8080-exec-24] ERROR o.a.g.w.GuacamoleWebSocketTunnelEndpoint - Creation of WebSocket tunnel to guacd failed: Requested tunnel destination does not exist. guacamole_1 | 05:34:36.009 [http-nio-8080-exec-24] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: Requested tunnel destination does not exist. guacamole_1 | 05:40:53.700 [http-nio-8080-exec-28] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:01.393 [http-nio-8080-exec-35] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:06.440 [http-nio-8080-exec-39] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:07.738 [http-nio-8080-exec-43] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:08.654 [http-nio-8080-exec-47] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:08.846 [http-nio-8080-exec-51] INFO o.a.g.environment.LocalEnvironment - GUACAMOLE_HOME is "/root/.guacamole". guacamole_1 | 05:42:09.022 [http-nio-8080-exec-55] INFO
