2023-02-15 09:57:21,782 INFO [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:null) Determined host 192.168.50.2 corresponds to IP 192.168.50.2 2023-02-15 09:57:21,783 WARN [cloud.agent.Agent] (agentRequest-Handler-2:null) Caught: com.cloud.utils.exception.CloudRuntimeException: GetRootDir for nfs://NFSServerIP/data/secondary failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to mount.nfs: Connection timed out
On 2/15/23 11:54, Granwille Strauss wrote:
UhhhhI have ABSOLUTELY no idea what just happened now, but the status has now magically changed to "Up" and I have not touched a single thing on the servers since I sent the cloud.log that Wei requested.On 2/15/23 11:46, Granwille Strauss wrote:Hi WeiHere is the cloud.log after restarting cloud service in SSVM: https://jpst.it/36KHUI believe these is the error:2023-02-15 09:35:35,956 INFO [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:null) Determined host NFSServerIP corresponds to IP NFSServerIP 2023-02-15 09:37:35,372 ERROR [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:null) Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to 2023-02-15 09:37:35,375 ERROR [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:null) GetRootDir for nfs://NFSServerIP/data/secondary failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to2023-02-15 09:35:35,936 ERROR [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:null) Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to mount.nfs: parsing error on 'vers=' option 2023-02-15 09:35:35,936 ERROR [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:null) GetRootDir for nfs://NFSServerIP/data/secondary failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to mount.nfs: parsing error on 'vers=' option com.cloud.utils.exception.CloudRuntimeException: Unable to mount NFSServerIP:/data/secondary at /mnt/SecStorage/91de6d1c-4c04-359c-82b5-fdcfe4a83da7 due to mount.nfs: parsing error on 'vers=' optionBut I have no idea what any of it means? On 2/15/23 11:34, Wei ZHOU wrote:Hi, Can you restart "cloud" service in SSVM and upload some logs in /var/log/cloud.log ? When the cloud service is started, it will check templates on secondary storage, which takes time if there are many templates. -Wei On Wednesday, 15 February 2023, Granwille Strauss<granwi...@namhost.com> wrote:Hi Wei I fully opened port 2049 now on the NFS server and ran telnet from SSVM, after rebooting it: root@s-141-VM:~# telnet NFSSERVERIP 2049 Trying NFSSERVERIP... Connected to NFSSERVERIP. Escape character is '^]'. ^] However, status remains "connecting". On 2/15/23 11:16, Wei ZHOU wrote: Right. I have seen this issue recently, which is because SSVM cannot connect to secondary storage. -Wei On Wednesday, 15 February 2023, Kiran Chavala<kiran.chav...@shapeblue.com> <kiran.chav...@shapeblue.com> wrote: HI Are you able to ssh to the ssvm, If yes can you please execute health check script /usr/local/cloud/systemvm/ssvm-check.sh Also are you able to connect to the management server on port 8250 from the ssvm telnet <Management server ip> 8250 Also could you please share the /var/log/cloud.out log it will be helpful Regards Kiran ________________________________ From: Granwille Strauss Sent: Wednesday, February 15, 2023 14:15 To:users@cloudstack.apache.org Cc: Harikrishna Patnala Subject: Re: SSVM Randomly Stopped Connecting Hi Harikrishna I have done so, status remains showing "connecting". And on the dashboard I can see my secondary storage usage, which means its supposed to be connected. However, if I reboot the cloudstack-management, it will show 0 again. See attached. May I add it most likely was not "random" as I mentioned previously. I did update my opensense gateway, which resulted in a reboot. However, connections and routing has been perfect again. And then I did also run yum update on the kvm host where this SSVM is tied to. And the only error I experienced was that mysql-connect-python required python 2.7, which I installed to resolve the dependency issue, but python3.6 is still active on the host as well. On 2/15/23 10:30, Harikrishna Patnala wrote: Hi, Have you tried destroying the SSVM, it will be recreated automatically. Please try that, it may solve the issue. Thanks, Harikrishna From: Granwille Strauss<granwi...@namhost.com.INVALID> <granwi...@namhost.com.INVALID><mailto: granwi...@namhost.com.INVALID> <granwi...@namhost.com.INVALID> Date: Wednesday, 15 February 2023 at 1:46 PM To:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> <users@cloudstack.apache.org> <users@cloudstack.apache.org><mailto:users@cloudstack.apache.org> <users@cloudstack.apache.org> Subject: SSVM Randomly Stopped Connecting Hi Guys As the error suggests, this morning my SSVM just randomly stopped connecting and agent status remains in "Connecting" mode. ConsolveVM seems perfectly fine. I did the regular checks in SSVM and it all seems fine, see attached screenshot. I tried force rebooting the SSVM but still nothing. The host kvm server is 100% connected as one typically gets the "host unreachable" error when something like this, but not this time. I SSH into the SSVM and ran the 'reboot' command and tailed the management log, this is what it throws:https://jpst.it/36K93 I do not see any errors there either. Any suggestions that can help with this, please? -- Regards / Groete [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/logo/621b3fa39fb210001f975298/ cd2904ba-304d-4a49-bf33-cbe9ac76d929_248x-.png]<https://www.namhost.com/> <https://www.namhost.com/>< https://www.namhost.com/> <https://www.namhost.com/> Granwille Strauss // Senior Systems Admin e:granwi...@namhost.com<mailto:granwi...@namhost.com> <granwi...@namhost.com><mailto:g ranwi...@namhost.com> <granwi...@namhost.com><mailto:granwi...@namhost.com> <granwi...@namhost.com> m: +264 81 323 1260<tel:+264813231260> <+264813231260><tel:+264813231260> <+264813231260> w:www.namhost.com<http://www.namhost.com> <http://www.namhost.com><https://www.namhost.com/> <https://www.namhost.com/>< https://www.namhost.com/> <https://www.namhost.com/> [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_01/621b3fa39fb210001f975298/ 9151954b-b298-41aa-89c8-1d68af075373_48x48.png]<https: //www.facebook.com/namhost> <https://www.facebook.com/namhost><https://www.facebook.com/namhost> <https://www.facebook.com/namhost>[https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_02/621b3fa39fb210001f975298/ 85a9dc7c-7bd1-4958-85a9-e6a25baeb028_48x48.png]<https: //twitter.com/namhost> <https://twitter.com/namhost><https://twitter.com/namhost> <https://twitter.com/namhost>[https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/ social_icon_03/621b3fa39fb210001f975298/c1c5386c-914c-43cf-9d37- 5b4aa8e317ab_48x48.png]<https://www.instagram.com/namhostinternetservices/ <https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/>[http s://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_04/621b3fa39fb210001f975298/ 3aaa7968-130e-48ec-821d-559a332cce47_48x48.png<http://www.adsigner.com/v1/s/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/social_icon_04/621b3fa39fb210001f975298/3aaa7968-130e-48ec-821d-559a332cce47_48x48.png>]<https: //www.linkedin.com/company/namhos> <https://www.linkedin.com/company/namhos><https://www.linkedin.com/company/namhos [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_05/621b3fa39fb210001f975298/3a8c09e6-588f-43a8-acfd-be4423fd3fb6_48x48.png]<https: //www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA><http s://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>[https://www.adsigner.com/v1/i/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner/940x300]<https://www.adsigner.com/v1/l/ 631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>< https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go tohttps://www.namhost.com/ privacy-policy[Powered by AdSigner]<https://www.adsigner.com/v1/c/ 631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818><http s://www.adsigner.com/v1/c/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>-- Regards / Groete [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/logo/621b3fa39fb210001f975298/ cd2904ba-304d-4a49-bf33-cbe9ac76d929_248x-.png]<https://www.namhost.com> <https://www.namhost.com> Granwille Strauss // Senior Systems Admin e:granwi...@namhost.com<mailto:granwi...@namhost.com> <granwi...@namhost.com> m: +264 81 323 1260<tel:+264813231260> <+264813231260> w:www.namhost.com<https://www.namhost.com/> <https://www.namhost.com/> [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_01/621b3fa39fb210001f975298/ 9151954b-b298-41aa-89c8-1d68af075373_48x48.png]<https: //www.facebook.com/namhost> <https://www.facebook.com/namhost> [https://www.adsigner.com/v1/ s/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/social_icon_02/ 621b3fa39fb210001f975298/85a9dc7c-7bd1-4958-85a9-e6a25baeb028_48x48.png]< https://twitter.com/namhost> <https://twitter.com/namhost> [https://www.adsigner.com/v1/ s/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/social_icon_03/ 621b3fa39fb210001f975298/c1c5386c-914c-43cf-9d37-5b4aa8e317ab_48x48.png]< https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/> [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_04/621b3fa39fb210001f975298/ 3aaa7968-130e-48ec-821d-559a332cce47_48x48.png]<https://www.linkedin.com/ company/namhos> <https://www.linkedin.com/company/namhos> [https://www.adsigner.com/v1/s/631091998d4670001fe43ec2/ 621c9b76c140bb001ed0f818/social_icon_05/621b3fa39fb210001f975298/ 3a8c09e6-588f-43a8-acfd-be4423fd3fb6_48x48.png]<https://www.youtube.com/ channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [https://www.adsigner.com/v1/i/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner/940x300]<https://www.adsigner.com/v1/l/ 631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go tohttps://www.namhost.com/ privacy-policy[Powered by AdSigner]<https://www.adsigner.com/v1/c/ 631091998d4670001fe43ec2/621c9b76c140bb001ed0f818> <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>-- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:*granwi...@namhost.com *m:* +264 81 323 1260 <+264813231260> *w:*www.namhost.com <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhos> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus<https://www.google.com/maps/search/24+Black+Eagle+Rd,+Hermanus?entry=gmail&source=g>, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go tohttps://www.namhost.com/ privacy-policy [image: Powered by AdSigner] <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>-- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:* granwi...@namhost.com *m:* +264 81 323 1260 <tel:+264813231260> *w:* www.namhost.com <https://www.namhost.com/> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus, 7210, RSAThe content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policyPowered by AdSigner <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>-- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:* granwi...@namhost.com *m:* +264 81 323 1260 <tel:+264813231260> *w:* www.namhost.com <https://www.namhost.com/> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus, 7210, RSAThe content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policyPowered by AdSigner <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
-- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:* granwi...@namhost.com *m:* +264 81 323 1260 <tel:+264813231260> *w:* www.namhost.com <https://www.namhost.com/> <https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner> Namhost Internet Services (Pty) Ltd, 24 Black Eagle Rd, Hermanus, 7210, RSAThe content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy
Powered by AdSigner <https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
smime.p7s
Description: S/MIME Cryptographic Signature
