Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm
It is strange. The cpu/ram should be good enough for a VR. anyway, good you have solved the problem. -Wei On Fri, 17 Sept 2021 at 11:46, wrote: > I didn't test with lower values, but with 256 RAM and 500Mhz CPU is not > working for me. (KVM ubuntu 20.04 ACS 4.15.1 ) > > Regards, > Cristian > > -Original Message- > From: Daan Hoogland > Sent: Friday, September 17, 2021 11:14 AM > To: users > Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm > > thanks for the update Christian, > sounds strange that the router would require that much to start but glad > it worked. > > On Wed, Sep 15, 2021 at 1:32 PM wrote: > > > Hello, > > > > I have found the issue, after I have changed the CPU and memory for > > this " System Offering For Software Router - Local Storage' system > > offering the router was deployed in few seconds without any issue. > > > > CPU mhz from 500 to 2000 and RAM from 256 to 2048. > > > > > > > > Regards, > > Cristian > > > > -Original Message- > > From: cristian.c@istream.today > > Sent: Wednesday, September 15, 2021 10:18 AM > > To: users@cloudstack.apache.org > > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with > > kvm > > > > Any suggestion on this issue? > > > > Thank you. > > > > -Original Message- > > From: cristian.c@istream.today > > Sent: Monday, September 13, 2021 5:40 PM > > To: users@cloudstack.apache.org > > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with > > kvm > > > > Hi, > > > > > >Yes, please see : > > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v > > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > > package: 0ubuntu8.13 (Frode Nordahl Fri, > > 16 Jul 2021 05:16:36 +) > > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : > > outputs=1:stderr > > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : > > output=1:stderr > > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > > package: 0ubuntu8.13 (Frode Nordahl Fri, > > 16 Jul 2021 05:16:36 +) > > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : > OBJECT_NEW: > > obj=0x55935a296b60 classname=virAccessManager > > 2021-09-13 14:33:40.944+: 75403: debug : > > virAccessManagerNewDriver:111 > > : Initialized with stack > > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : > OBJECT_NEW: > > obj=0x55935a297f90 classname=virAccessManager > > 2021-09-13 14:33:40.944+: 75403: debug : > > virAccessManagerNewDriver:111 > > : Initialized with none > > 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : > OBJECT_REF: > > obj=0x55935a296b60 > > 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : > > OBJECT_UNREF: obj=0x55935a296b60 > > 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on > > pid file path '/run/libvirtd.pid' > > 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on > > socket paths '/run/libvirt/libvirt-sock', > > '/run/libvirt/libvirt-sock-ro' and '/run/libvirt/libvirt-admin-sock' > > 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : > > Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily > > unavailable > > 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed > > fd 3 > > > > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# > > /usr/bin/qemu-system-x86_64 --version QEMU emulator version 4.2.1 > > (Debian 1:4.2-3ubuntu6.17) Copyright > > (c) 2003-2019 Fabrice Bellard and the QEMU Project developers > > > > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB > > modules are available. > > Distributor ID: Ubuntu > > Description:Ubuntu 20.04.2 LTS > > Release:20.04 > > Codename: focal > > > > > > CPU : > > vendor_id : AuthenticAMD > > cpu family : 23 > > model : 49 > > model name : AMD EPYC 7552 48-Core Processor > > > > > > BTW: the SSVM and Proxy console is running fine on this node.. > > > > > > > > Thank you! > > > > -Original Message- > > From: Andrija Pani
RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm
I didn't test with lower values, but with 256 RAM and 500Mhz CPU is not working for me. (KVM ubuntu 20.04 ACS 4.15.1 ) Regards, Cristian -Original Message- From: Daan Hoogland Sent: Friday, September 17, 2021 11:14 AM To: users Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm thanks for the update Christian, sounds strange that the router would require that much to start but glad it worked. On Wed, Sep 15, 2021 at 1:32 PM wrote: > Hello, > > I have found the issue, after I have changed the CPU and memory for > this " System Offering For Software Router - Local Storage' system > offering the router was deployed in few seconds without any issue. > > CPU mhz from 500 to 2000 and RAM from 256 to 2048. > > > > Regards, > Cristian > > -Original Message- > From: cristian.c@istream.today > Sent: Wednesday, September 15, 2021 10:18 AM > To: users@cloudstack.apache.org > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with > kvm > > Any suggestion on this issue? > > Thank you. > > -Original Message- > From: cristian.c@istream.today > Sent: Monday, September 13, 2021 5:40 PM > To: users@cloudstack.apache.org > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with > kvm > > Hi, > > >Yes, please see : > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > package: 0ubuntu8.13 (Frode Nordahl Fri, > 16 Jul 2021 05:16:36 +) > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : > outputs=1:stderr > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : > output=1:stderr > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > package: 0ubuntu8.13 (Frode Nordahl Fri, > 16 Jul 2021 05:16:36 +) > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: > obj=0x55935a296b60 classname=virAccessManager > 2021-09-13 14:33:40.944+: 75403: debug : > virAccessManagerNewDriver:111 > : Initialized with stack > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: > obj=0x55935a297f90 classname=virAccessManager > 2021-09-13 14:33:40.944+: 75403: debug : > virAccessManagerNewDriver:111 > : Initialized with none > 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : OBJECT_REF: > obj=0x55935a296b60 > 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : > OBJECT_UNREF: obj=0x55935a296b60 > 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on > pid file path '/run/libvirtd.pid' > 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on > socket paths '/run/libvirt/libvirt-sock', > '/run/libvirt/libvirt-sock-ro' and '/run/libvirt/libvirt-admin-sock' > 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : > Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily > unavailable > 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed > fd 3 > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# > /usr/bin/qemu-system-x86_64 --version QEMU emulator version 4.2.1 > (Debian 1:4.2-3ubuntu6.17) Copyright > (c) 2003-2019 Fabrice Bellard and the QEMU Project developers > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB > modules are available. > Distributor ID: Ubuntu > Description:Ubuntu 20.04.2 LTS > Release:20.04 > Codename: focal > > > CPU : > vendor_id : AuthenticAMD > cpu family : 23 > model : 49 > model name : AMD EPYC 7552 48-Core Processor > > > BTW: the SSVM and Proxy console is running fine on this node.. > > > > Thank you! > > -Original Message- > From: Andrija Panic > Sent: Monday, September 13, 2021 4:40 PM > To: users > Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with > kvm > > Can you share qemu/libvirt version/flavour? > > On Mon, 13 Sept 2021 at 14:18, wrote: > > > Hello, > > > > > > > > I'm trying to understand this error, I created new network > > with VPC in a new zone where I added KVM hypervisor, and I have this issue. > > > > > > > > > > > > 2021-09-13 11:36:18,621 DEBUG > > [resource.virtualnetwork.VirtualRoutingResource] > > (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to > > 169.254.52.234 > > > > 2021-09-13 11:36:21,684 D
Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm
thanks for the update Christian, sounds strange that the router would require that much to start but glad it worked. On Wed, Sep 15, 2021 at 1:32 PM wrote: > Hello, > > I have found the issue, after I have changed the CPU and memory for > this " System Offering For Software Router - Local Storage' system offering > the router was deployed in few seconds without any issue. > > CPU mhz from 500 to 2000 and RAM from 256 to 2048. > > > > Regards, > Cristian > > -Original Message- > From: cristian.c@istream.today > Sent: Wednesday, September 15, 2021 10:18 AM > To: users@cloudstack.apache.org > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm > > Any suggestion on this issue? > > Thank you. > > -Original Message- > From: cristian.c@istream.today > Sent: Monday, September 13, 2021 5:40 PM > To: users@cloudstack.apache.org > Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm > > Hi, > > >Yes, please see : > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > package: 0ubuntu8.13 (Frode Nordahl Fri, 16 > Jul 2021 05:16:36 +) > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : > outputs=1:stderr > 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : > output=1:stderr > 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, > package: 0ubuntu8.13 (Frode Nordahl Fri, 16 > Jul 2021 05:16:36 +) > 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: > obj=0x55935a296b60 classname=virAccessManager > 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 > : Initialized with stack > 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: > obj=0x55935a297f90 classname=virAccessManager > 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 > : Initialized with none > 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : OBJECT_REF: > obj=0x55935a296b60 > 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : > OBJECT_UNREF: obj=0x55935a296b60 > 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on pid > file path '/run/libvirtd.pid' > 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on socket > paths '/run/libvirt/libvirt-sock', '/run/libvirt/libvirt-sock-ro' and > '/run/libvirt/libvirt-admin-sock' > 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : > Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily > unavailable > 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed fd 3 > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# /usr/bin/qemu-system-x86_64 > --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright > (c) 2003-2019 Fabrice Bellard and the QEMU Project developers > > > root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB > modules are available. > Distributor ID: Ubuntu > Description:Ubuntu 20.04.2 LTS > Release:20.04 > Codename: focal > > > CPU : > vendor_id : AuthenticAMD > cpu family : 23 > model : 49 > model name : AMD EPYC 7552 48-Core Processor > > > BTW: the SSVM and Proxy console is running fine on this node.. > > > > Thank you! > > -Original Message- > From: Andrija Panic > Sent: Monday, September 13, 2021 4:40 PM > To: users > Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm > > Can you share qemu/libvirt version/flavour? > > On Mon, 13 Sept 2021 at 14:18, wrote: > > > Hello, > > > > > > > > I'm trying to understand this error, I created new network with > > VPC in a new zone where I added KVM hypervisor, and I have this issue. > > > > > > > > > > > > 2021-09-13 11:36:18,621 DEBUG > > [resource.virtualnetwork.VirtualRoutingResource] > > (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to > > 169.254.52.234 > > > > 2021-09-13 11:36:21,684 DEBUG > > [resource.virtualnetwork.VirtualRoutingResource] > > (agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to > > 169.254.52.234 > > > > > > > > Here is the full log : https://pastebin.com/0NwsccUc > > > > > > > > root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922 > > > >
RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm
Hello, I have found the issue, after I have changed the CPU and memory for this " System Offering For Software Router - Local Storage' system offering the router was deployed in few seconds without any issue. CPU mhz from 500 to 2000 and RAM from 256 to 2048. Regards, Cristian -Original Message- From: cristian.c@istream.today Sent: Wednesday, September 15, 2021 10:18 AM To: users@cloudstack.apache.org Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm Any suggestion on this issue? Thank you. -Original Message- From: cristian.c@istream.today Sent: Monday, September 13, 2021 5:40 PM To: users@cloudstack.apache.org Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm Hi, Yes, please see : root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : outputs=1:stderr 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : output=1:stderr 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a296b60 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with stack 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a297f90 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with none 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : OBJECT_REF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : OBJECT_UNREF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on pid file path '/run/libvirtd.pid' 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on socket paths '/run/libvirt/libvirt-sock', '/run/libvirt/libvirt-sock-ro' and '/run/libvirt/libvirt-admin-sock' 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily unavailable 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed fd 3 root@usa-nyc-kvm-001:/var/lib/libvirt/images# /usr/bin/qemu-system-x86_64 --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.2 LTS Release:20.04 Codename: focal CPU : vendor_id : AuthenticAMD cpu family : 23 model : 49 model name : AMD EPYC 7552 48-Core Processor BTW: the SSVM and Proxy console is running fine on this node.. Thank you! -Original Message- From: Andrija Panic Sent: Monday, September 13, 2021 4:40 PM To: users Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm Can you share qemu/libvirt version/flavour? On Mon, 13 Sept 2021 at 14:18, wrote: > Hello, > > > > I'm trying to understand this error, I created new network with > VPC in a new zone where I added KVM hypervisor, and I have this issue. > > > > > > 2021-09-13 11:36:18,621 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to > 169.254.52.234 > > 2021-09-13 11:36:21,684 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to > 169.254.52.234 > > > > Here is the full log : https://pastebin.com/0NwsccUc > > > > root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922 > > Trying 169.254.156.179... > > telnet: Unable to connect to remote host: Connection refused ( before > connection refused, I get no route to host, for sure the nic is not up > ) > > root@usa-nyc-kvm-001:~# virsh list > > Id NameState > > --- > > 2v-4048-VM running > > 4s-4050-VM running > > 7r-4092-VM running > > > > root@usa-nyc-kvm-001:~# > > > > > > If I check the log inside the router, I see this (the router is also > responding very slow, waiting a lot to connect on it via virsh console): > > > > r-4092-VM login: root > > Password: > > > > Login timed out after 60 seconds. > > > > > >
RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm
Any suggestion on this issue? Thank you. -Original Message- From: cristian.c@istream.today Sent: Monday, September 13, 2021 5:40 PM To: users@cloudstack.apache.org Subject: RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm Hi, Yes, please see : root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : outputs=1:stderr 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : output=1:stderr 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a296b60 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with stack 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a297f90 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with none 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : OBJECT_REF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : OBJECT_UNREF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on pid file path '/run/libvirtd.pid' 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on socket paths '/run/libvirt/libvirt-sock', '/run/libvirt/libvirt-sock-ro' and '/run/libvirt/libvirt-admin-sock' 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily unavailable 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed fd 3 root@usa-nyc-kvm-001:/var/lib/libvirt/images# /usr/bin/qemu-system-x86_64 --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.2 LTS Release:20.04 Codename: focal CPU : vendor_id : AuthenticAMD cpu family : 23 model : 49 model name : AMD EPYC 7552 48-Core Processor BTW: the SSVM and Proxy console is running fine on this node.. Thank you! -Original Message- From: Andrija Panic Sent: Monday, September 13, 2021 4:40 PM To: users Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm Can you share qemu/libvirt version/flavour? On Mon, 13 Sept 2021 at 14:18, wrote: > Hello, > > > > I'm trying to understand this error, I created new network with > VPC in a new zone where I added KVM hypervisor, and I have this issue. > > > > > > 2021-09-13 11:36:18,621 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to > 169.254.52.234 > > 2021-09-13 11:36:21,684 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to > 169.254.52.234 > > > > Here is the full log : https://pastebin.com/0NwsccUc > > > > root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922 > > Trying 169.254.156.179... > > telnet: Unable to connect to remote host: Connection refused ( before > connection refused, I get no route to host, for sure the nic is not up > ) > > root@usa-nyc-kvm-001:~# virsh list > > Id NameState > > --- > > 2v-4048-VM running > > 4s-4050-VM running > > 7r-4092-VM running > > > > root@usa-nyc-kvm-001:~# > > > > > > If I check the log inside the router, I see this (the router is also > responding very slow, waiting a lot to connect on it via virsh console): > > > > r-4092-VM login: root > > Password: > > > > Login timed out after 60 seconds. > > > > > >__?.o/ Apache CloudStack SystemVM 4.15.1 > > ( )#https://cloudstack.apache.org > > (___(_) Debian GNU/Linux 10 r-4093-VM ttyS0 > > > > r-4093-VM login: root > > Password: > > Linux r-4093-VM 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) > x86_64 > > > > > > > > root@r-4093-VM:~# cat /var/log/cloud.log > > Mon 13 Sep 2021 11:50:55 AM UTC Executing cloud-early-config > > Mon 13 Sep 2021 11:50:55 AM UTC Bootstrapping s
RE: virtual router stuck on starting - cloudstack 4.15.1 with kvm
Hi, Yes, please see : root@usa-nyc-kvm-001:/var/lib/libvirt/images# libvirtd -v 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutputs:1654 : outputs=1:stderr 2021-09-13 14:33:40.944+: 75403: debug : virLogParseOutput:1492 : output=1:stderr 2021-09-13 14:33:40.944+: 75403: info : libvirt version: 6.0.0, package: 0ubuntu8.13 (Frode Nordahl Fri, 16 Jul 2021 05:16:36 +) 2021-09-13 14:33:40.944+: 75403: info : hostname: usa-nyc-kvm-001 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a296b60 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with stack 2021-09-13 14:33:40.944+: 75403: info : virObjectNew:251 : OBJECT_NEW: obj=0x55935a297f90 classname=virAccessManager 2021-09-13 14:33:40.944+: 75403: debug : virAccessManagerNewDriver:111 : Initialized with none 2021-09-13 14:33:40.944+: 75403: info : virObjectRef:386 : OBJECT_REF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: info : virObjectUnref:348 : OBJECT_UNREF: obj=0x55935a296b60 2021-09-13 14:33:40.944+: 75403: debug : main:1177 : Decided on pid file path '/run/libvirtd.pid' 2021-09-13 14:33:40.944+: 75403: debug : main:1187 : Decided on socket paths '/run/libvirt/libvirt-sock', '/run/libvirt/libvirt-sock-ro' and '/run/libvirt/libvirt-admin-sock' 2021-09-13 14:33:40.944+: 75403: error : virPidFileAcquirePath:367 : Failed to acquire pid file '/run/libvirtd.pid': Resource temporarily unavailable 2021-09-13 14:33:40.944+: 75403: debug : virFileClose:110 : Closed fd 3 root@usa-nyc-kvm-001:/var/lib/libvirt/images# /usr/bin/qemu-system-x86_64 --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers root@usa-nyc-kvm-001:/var/lib/libvirt/images# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.2 LTS Release:20.04 Codename: focal CPU : vendor_id : AuthenticAMD cpu family : 23 model : 49 model name : AMD EPYC 7552 48-Core Processor BTW: the SSVM and Proxy console is running fine on this node.. Thank you! -Original Message- From: Andrija Panic Sent: Monday, September 13, 2021 4:40 PM To: users Subject: Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm Can you share qemu/libvirt version/flavour? On Mon, 13 Sept 2021 at 14:18, wrote: > Hello, > > > > I'm trying to understand this error, I created new network with > VPC in a new zone where I added KVM hypervisor, and I have this issue. > > > > > > 2021-09-13 11:36:18,621 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to > 169.254.52.234 > > 2021-09-13 11:36:21,684 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to > 169.254.52.234 > > > > Here is the full log : https://pastebin.com/0NwsccUc > > > > root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922 > > Trying 169.254.156.179... > > telnet: Unable to connect to remote host: Connection refused ( before > connection refused, I get no route to host, for sure the nic is not up > ) > > root@usa-nyc-kvm-001:~# virsh list > > Id NameState > > --- > > 2v-4048-VM running > > 4s-4050-VM running > > 7r-4092-VM running > > > > root@usa-nyc-kvm-001:~# > > > > > > If I check the log inside the router, I see this (the router is also > responding very slow, waiting a lot to connect on it via virsh console): > > > > r-4092-VM login: root > > Password: > > > > Login timed out after 60 seconds. > > > > > >__?.o/ Apache CloudStack SystemVM 4.15.1 > > ( )#https://cloudstack.apache.org > > (___(_) Debian GNU/Linux 10 r-4093-VM ttyS0 > > > > r-4093-VM login: root > > Password: > > Linux r-4093-VM 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) > x86_64 > > > > > > > > root@r-4093-VM:~# cat /var/log/cloud.log > > Mon 13 Sep 2021 11:50:55 AM UTC Executing cloud-early-config > > Mon 13 Sep 2021 11:50:55 AM UTC Bootstrapping systemvm appliance > > Mon 13 Sep 2021 11:50:57 AM UTC Starting guest services for kvm > > Mon 13 Sep 2021 11:51:04 AM UTC acpiphp and pci_hotplug module already > compiled in > > Mon 13 Sep 2021 11:52:59 AM UTC Received a new non-em
Re: virtual router stuck on starting - cloudstack 4.15.1 with kvm
Can you share qemu/libvirt version/flavour?
On Mon, 13 Sept 2021 at 14:18, wrote:
> Hello,
>
>
>
> I'm trying to understand this error, I created new network with VPC
> in
> a new zone where I added KVM hypervisor, and I have this issue.
>
>
>
>
>
> 2021-09-13 11:36:18,621 DEBUG
> [resource.virtualnetwork.VirtualRoutingResource]
> (agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to
> 169.254.52.234
>
> 2021-09-13 11:36:21,684 DEBUG
> [resource.virtualnetwork.VirtualRoutingResource]
> (agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to
> 169.254.52.234
>
>
>
> Here is the full log : https://pastebin.com/0NwsccUc
>
>
>
> root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922
>
> Trying 169.254.156.179...
>
> telnet: Unable to connect to remote host: Connection refused ( before
> connection refused, I get no route to host, for sure the nic is not up )
>
> root@usa-nyc-kvm-001:~# virsh list
>
> Id NameState
>
> ---
>
> 2v-4048-VM running
>
> 4s-4050-VM running
>
> 7r-4092-VM running
>
>
>
> root@usa-nyc-kvm-001:~#
>
>
>
>
>
> If I check the log inside the router, I see this (the router is also
> responding very slow, waiting a lot to connect on it via virsh console):
>
>
>
> r-4092-VM login: root
>
> Password:
>
>
>
> Login timed out after 60 seconds.
>
>
>
>
>
>__?.o/ Apache CloudStack SystemVM 4.15.1
>
> ( )#https://cloudstack.apache.org
>
> (___(_) Debian GNU/Linux 10 r-4093-VM ttyS0
>
>
>
> r-4093-VM login: root
>
> Password:
>
> Linux r-4093-VM 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30)
> x86_64
>
>
>
>
>
>
>
> root@r-4093-VM:~# cat /var/log/cloud.log
>
> Mon 13 Sep 2021 11:50:55 AM UTC Executing cloud-early-config
>
> Mon 13 Sep 2021 11:50:55 AM UTC Bootstrapping systemvm appliance
>
> Mon 13 Sep 2021 11:50:57 AM UTC Starting guest services for kvm
>
> Mon 13 Sep 2021 11:51:04 AM UTC acpiphp and pci_hotplug module already
> compiled in
>
> Mon 13 Sep 2021 11:52:59 AM UTC Received a new non-empty cmdline file from
> qemu-guest-agent
>
> Mon 13 Sep 2021 11:52:59 AM UTC Booting from cloudstack, remove old
> configuration files in /etc/cloudstack/
>
> Mon 13 Sep 2021 11:54:30 AM UTC Configuring systemvm type=vpcrouter
>
> Mon 13 Sep 2021 11:54:30 AM UTC Setting up VPC virtual router system vm
>
> Mon 13 Sep 2021 11:54:31 AM UTC Setting up apache web server for VPC
>
> Mon 13 Sep 2021 11:55:05 AM UTC Processors = 1 Enable service = 0
>
> Mon 13 Sep 2021 11:55:05 AM UTC cloud: disable rp_filter
>
> Mon 13 Sep 2021 11:55:05 AM UTC disable rpfilter
>
> Mon 13 Sep 2021 11:55:05 AM UTC cloud: enable_fwding = 1
>
> Mon 13 Sep 2021 11:55:05 AM UTC enable_fwding = 1
>
> Mon 13 Sep 2021 11:55:05 AM UTC cloud: enabling passive FTP for guest VMs
>
> Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
>
> Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
>
> 2021-09-13 11:55:13,823 INFO update_config.py :: Processing incoming
> file => cmd_line.json
>
> 2021-09-13 11:55:13,856 INFO Processing JSON file cmd_line.json
>
> 2021-09-13 11:55:13,858 INFO Continuing with the processing of file
> '/var/cache/cloud/cmd_line.json'
>
> 2021-09-13 11:55:13,877 INFO Command of type cmdline received
>
> 2021-09-13 11:55:13,897 INFO Command of type ips received
>
> 2021-09-13 11:55:13,923 INFO Executing: ip addr show dev eth0
>
> 2021-09-13 11:55:13,975 INFO Executing: ip addr show dev eth0
>
> 2021-09-13 11:55:13,979 INFO Address found in DataBag ==>
> {u'public_ip':
> u'169.254.41.129', u'one_to_one_nat': False, u'nic_dev_id': u'0',
> u'network': u'169.254.0.0/16', u'netmask': u'255.255.0.0', u'source_nat':
> False, u'broadcast': u'169.254.255.255', u'add': True, u'nw_type':
> u'control', u'device': u'eth0', u'cidr': u'169.254.41.129/16', u'gateway':
> u'', u'size': u'16'}
>
> 2021-09-13 11:55:13,979 INFO Address 169.254.41.129/16 on device eth0
> already configured
>
> 2021-09-13 11:55:13,984 INFO Executing: ip addr show |grep -v secondary
>
> 2021-09-13 11:55:14,010 INFO Wrote edited file
> /etc/dnsmasq.d/cloud.conf
>
> 2021-09-13 11:55:14,010 INFO Updated file in-cache configuration
>
> 2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcphosts.txt
> file did not change
>
> 2021-09-13 11:55:14,010 INFO Nothing to commit. The
> /var/lib/misc/dnsmasq.leases file did not change
>
> 2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcpopts.txt
> file did not change
>
> 2021-09-13 11:55:14,010 INFO Attempting to delete entries from
> dnsmasq.leases file for VMs which are not on dhcphosts file
>
> 2021-09-13 11:55:14,010 ERRORCaught error while trying to delete
> entries
> from dnsmasq.leases file: [Errno 2] No such file or directory:
> '/etc/dhcphosts.txt'
>
> 2021-09-13 11:55:14,034 INFO Wrote edited file /etc/hosts
>
> 2021-09-13 11:55:14,044 INFO Updated file in-cache configuration
>
> 2021-09-13
virtual router stuck on starting - cloudstack 4.15.1 with kvm
Hello,
I'm trying to understand this error, I created new network with VPC in
a new zone where I added KVM hypervisor, and I have this issue.
2021-09-13 11:36:18,621 DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to
169.254.52.234
2021-09-13 11:36:21,684 DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to
169.254.52.234
Here is the full log : https://pastebin.com/0NwsccUc
root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922
Trying 169.254.156.179...
telnet: Unable to connect to remote host: Connection refused ( before
connection refused, I get no route to host, for sure the nic is not up )
root@usa-nyc-kvm-001:~# virsh list
Id NameState
---
2v-4048-VM running
4s-4050-VM running
7r-4092-VM running
root@usa-nyc-kvm-001:~#
If I check the log inside the router, I see this (the router is also
responding very slow, waiting a lot to connect on it via virsh console):
r-4092-VM login: root
Password:
Login timed out after 60 seconds.
__?.o/ Apache CloudStack SystemVM 4.15.1
( )#https://cloudstack.apache.org
(___(_) Debian GNU/Linux 10 r-4093-VM ttyS0
r-4093-VM login: root
Password:
Linux r-4093-VM 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64
root@r-4093-VM:~# cat /var/log/cloud.log
Mon 13 Sep 2021 11:50:55 AM UTC Executing cloud-early-config
Mon 13 Sep 2021 11:50:55 AM UTC Bootstrapping systemvm appliance
Mon 13 Sep 2021 11:50:57 AM UTC Starting guest services for kvm
Mon 13 Sep 2021 11:51:04 AM UTC acpiphp and pci_hotplug module already
compiled in
Mon 13 Sep 2021 11:52:59 AM UTC Received a new non-empty cmdline file from
qemu-guest-agent
Mon 13 Sep 2021 11:52:59 AM UTC Booting from cloudstack, remove old
configuration files in /etc/cloudstack/
Mon 13 Sep 2021 11:54:30 AM UTC Configuring systemvm type=vpcrouter
Mon 13 Sep 2021 11:54:30 AM UTC Setting up VPC virtual router system vm
Mon 13 Sep 2021 11:54:31 AM UTC Setting up apache web server for VPC
Mon 13 Sep 2021 11:55:05 AM UTC Processors = 1 Enable service = 0
Mon 13 Sep 2021 11:55:05 AM UTC cloud: disable rp_filter
Mon 13 Sep 2021 11:55:05 AM UTC disable rpfilter
Mon 13 Sep 2021 11:55:05 AM UTC cloud: enable_fwding = 1
Mon 13 Sep 2021 11:55:05 AM UTC enable_fwding = 1
Mon 13 Sep 2021 11:55:05 AM UTC cloud: enabling passive FTP for guest VMs
Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
2021-09-13 11:55:13,823 INFO update_config.py :: Processing incoming
file => cmd_line.json
2021-09-13 11:55:13,856 INFO Processing JSON file cmd_line.json
2021-09-13 11:55:13,858 INFO Continuing with the processing of file
'/var/cache/cloud/cmd_line.json'
2021-09-13 11:55:13,877 INFO Command of type cmdline received
2021-09-13 11:55:13,897 INFO Command of type ips received
2021-09-13 11:55:13,923 INFO Executing: ip addr show dev eth0
2021-09-13 11:55:13,975 INFO Executing: ip addr show dev eth0
2021-09-13 11:55:13,979 INFO Address found in DataBag ==> {u'public_ip':
u'169.254.41.129', u'one_to_one_nat': False, u'nic_dev_id': u'0',
u'network': u'169.254.0.0/16', u'netmask': u'255.255.0.0', u'source_nat':
False, u'broadcast': u'169.254.255.255', u'add': True, u'nw_type':
u'control', u'device': u'eth0', u'cidr': u'169.254.41.129/16', u'gateway':
u'', u'size': u'16'}
2021-09-13 11:55:13,979 INFO Address 169.254.41.129/16 on device eth0
already configured
2021-09-13 11:55:13,984 INFO Executing: ip addr show |grep -v secondary
2021-09-13 11:55:14,010 INFO Wrote edited file /etc/dnsmasq.d/cloud.conf
2021-09-13 11:55:14,010 INFO Updated file in-cache configuration
2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcphosts.txt
file did not change
2021-09-13 11:55:14,010 INFO Nothing to commit. The
/var/lib/misc/dnsmasq.leases file did not change
2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcpopts.txt
file did not change
2021-09-13 11:55:14,010 INFO Attempting to delete entries from
dnsmasq.leases file for VMs which are not on dhcphosts file
2021-09-13 11:55:14,010 ERRORCaught error while trying to delete entries
from dnsmasq.leases file: [Errno 2] No such file or directory:
'/etc/dhcphosts.txt'
2021-09-13 11:55:14,034 INFO Wrote edited file /etc/hosts
2021-09-13 11:55:14,044 INFO Updated file in-cache configuration
2021-09-13 11:55:14,044 INFO Updated hosts file
2021-09-13 11:55:14,046 INFO Executing: systemctl restart dnsmasq
2021-09-13 11:55:15,409 INFO Service dnsmasq restart
2021-09-13 11:55:15,453 INFO Executing: ip addr show |grep -v secondary
2021-09-13 11:55:15,832 INFO Nothing to commit. The
/etc/dnsmasq.d/cloud.conf file did not change
