Probably try destroying them once? Any warn or error message in mgt logs or ssvm /var/log/messages?
Sent from my iPhone > On 26-Dec-2020, at 5:12 AM, Cloud List <cloud-l...@sg.or.id> wrote: > > Hi, > > Merry Christmas to all. > > We are using Cloudstack with KVM hypervisor. Since our console proxy SSL > certificate has expired, we updated our new SSL certificate using below > method: > > http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.9/systemvm.html#using-a-ssl-certificate-for-the-console-proxy > > We have done the above method in the past years without any issues, however > this time round, both the SSVM and CPVM agents are not able to start after > the update. > > The state for both VMs are up but agents are in "disconnected" state. We > are still able to login to the SSVM, and found out that the cloud service > is not running. > > root@s-4200-VM:~# service cloud status > CloudStack cloud service is not running > > Tried to start the service: > > root@s-4200-VM:~# service cloud start > Starting CloudStack cloud service (type=secstorage) Success > > But the service is not started: > > root@s-4200-VM:~# service cloud status > CloudStack cloud service is not running > > Below is the logs from /var/log/cloud.log: > > ===== > Sat Dec 26 03:45:04 UTC 2020 Executing cloud-early-config > Sat Dec 26 03:45:04 UTC 2020 Detected that we are running inside kvm guest > Sat Dec 26 03:45:04 UTC 2020 Found a non empty cmdline file. Will now exit > the loop and proceed with configuration. > Sat Dec 26 03:45:04 UTC 2020 Patching cloud service > Sat Dec 26 03:45:10 UTC 2020 Updating log4j-cloud.xml > Sat Dec 26 03:45:10 UTC 2020 Setting up secondary storage system vm > Sat Dec 26 03:45:10 UTC 2020 checking that eth0 has IP > Sat Dec 26 03:45:11 UTC 2020 waiting for eth0 interface setup with ip > timer=0 > Sat Dec 26 03:45:11 UTC 2020 checking that eth1 has IP > Sat Dec 26 03:45:11 UTC 2020 checking that eth2 has IP > Sat Dec 26 03:45:20 UTC 2020 checking that eth3 has IP > Sat Dec 26 03:45:20 UTC 2020 Successfully setup storage network with > STORAGE_IP:10.19.22.67, STORAGE_NETMASK:255.255.240.0, STORAGE_CIDR: > Sat Dec 26 03:45:20 UTC 2020 Setting up route of RFC1918 space to 10.19.16.1 > Sat Dec 26 03:45:20 UTC 2020 Setting up apache web server > Sat Dec 26 03:45:20 UTC 2020 setting up apache2 for post upload of > volume/template > Sat Dec 26 03:45:20 UTC 2020 rewrite rules already exist in file > /etc/apache2/sites-available/default-ssl > Sat Dec 26 03:45:20 UTC 2020 adding cors rules to file: > /etc/apache2/sites-available/default-ssl > Sat Dec 26 03:45:21 UTC 2020 cloud: disable rp_filter > Sat Dec 26 03:45:21 UTC 2020 disable rpfilter > Sat Dec 26 03:45:21 UTC 2020 cloud: enable_fwding = 0 > Sat Dec 26 03:45:21 UTC 2020 enable_fwding = 0 > Sat Dec 26 03:45:21 UTC 2020 Enable service haproxy = 0 > Sat Dec 26 03:45:21 UTC 2020 Processors = 1 Enable service = 0 > Sat Dec 26 03:45:21 UTC 2020 Enable service dnsmasq = 0 > Sat Dec 26 03:45:21 UTC 2020 Enable service cloud-passwd-srvr = 0 > Sat Dec 26 03:45:21 UTC 2020 Enable service cloud = 1 > ===== > > Result of /usr/local/cloud/systemvm/ssvm-check.sh: > > ===== > root@s-4200-VM:/var/log# /usr/local/cloud/systemvm/ssvm-check.sh > ================================================ > First DNS server is 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 48 data bytes > 56 bytes from 8.8.8.8: icmp_seq=0 ttl=122 time=0.531 ms > 56 bytes from 8.8.8.8: icmp_seq=1 ttl=122 time=0.676 ms > --- 8.8.8.8 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.531/0.604/0.676/0.073 ms > Good: Can ping DNS server > ================================================ > Good: DNS resolves download.cloud.com > ================================================ > ERROR: NFS is not currently mounted > Try manually mounting from inside the VM > NFS server is X.X.201.1 > PING X.X.201.1 (X.X.201.1): 48 data bytes > 56 bytes from X.X.201.1: icmp_seq=0 ttl=255 time=0.463 ms > 56 bytes from X.X.201.1: icmp_seq=1 ttl=255 time=0.482 ms > --- X.X.201.1 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.463/0.473/0.482/0.000 ms > Good: Can ping nfs server > ================================================ > Management server is 10.237.3.8. Checking connectivity. > Good: Can connect to management server port 8250 > ================================================ > ERROR: Java process not running. Try restarting the SSVM. > root@s-4200-VM:/var/log# > ===== > > The result is OK except the NFS test, but we checked the IP address is not > correct (X.X.201.1 which is the public IP address of the gateway rather > than the actual NFS server IP). We tested mounting to the actual NFS server > and it works fine. > > Have tried stopping and starting back the SSVM and the issue still persists. > > Anyone can help to advice how we can resolve the problem? > > Looking forward to your reply, thank you. > > -ip- > > <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> > Virus-free. > www.avg.com > <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
