Hi Pearl,
I did upgrades multiple times, I always add the template before upgrade. If was something related to this, i was expecting not to work for all zones, but right now is working for 2 zones from the same setup, environment. Regards, Cristian On Fri, Sep 24, 2021, 14:40 Pearl d'Silva <pearl.dsi...@shapeblue.com> wrote: > Hi Cristian, > > After going through your earlier mails again, and correct me if I am > wrong, but I noticed in one of your mails you mentioned that the env that's > causing issues was upgraded from a lower version, right? So, ideally when > doing upgrades, prior to initiating the upgrade we first take a backup of > the DB, register the template (new systemVM template) following this > https://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade-4.15.html#update-system-vm-templates > and then upgrade the environment. However, you mentioned that you used the > cloud-install-sys-tmplt script to seed and register the template. I am not > familiar if using that script during upgrades works (it may, but I haven't > ever used that approach). > If it is possible, it would be advisable to rollback your env to the > earlier version (disable your zone, destroy the systemVMs before that) and > then follow the upgrade steps as mentioned in the documentation: > https://docs.cloudstack.apache.org/en/latest/upgrading/index.html > > Thanks, > Pearl > > > ________________________________ > From: cristian.c@istream.today <cristian.c@istream.today> > Sent: Friday, September 24, 2021 3:08 PM > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: RE: Failed to authentication SSH user root on host - repetitive > error > > Hi Pearl, > > I just did a SSVM redeploy in the same environment, but different zone and > and I see the following differences : > > "isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB)" > > > Working zone - https://pastebin.com/raw/88N6Fg6z > > "isManaged":"false"}},"name":"ROOT-2055","size":"(2.44 GB) > > datastore-6588","managed":"false","storagePort":"0","volumeSize":"(2.44 > GB) > > 021-09-24 05:22:23,959 INFO [c.c.h.v.r.VmwareResource] > (DirectAgent-437:ctx-2f1519ba fra-de-001.test.host, job-25099/job-35371, > cmd: StartCommand) (logid:d442432d) Mapped disk datastore UUID is not the > same as the cloned VM datastore UUID: fra-de-001 - > 9d679070c7a6441592646bfbb82ef1b3 > > > > Not working zone - https://pastebin.com/raw/Um2uHgWM > > "isManaged":"false"}},"name":"ROOT-2056","size":"(0 bytes)" here is 0 > bytes... > > datastore-6694","managed":"false","storagePort":"0","volumeSize":"(0 > bytes) > here is 0 bytes... > > 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource] > (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host, job-25099/job-35365, > cmd: > StartCommand) (logid:d442432d) Mapped disk size is not the same as the > cloned VM disk size: 0 - 2621440000 ( here I see this line, with disk > size 0 ) > > 2021-09-24 05:00:55,594 INFO [c.c.h.v.r.VmwareResource] > (DirectAgent-30:ctx-5e49d288 lnd-uk-002.test.host, job-25099/job-35365, > cmd: > StartCommand) (logid:d442432d) Mapped disk datastore UUID is not the same > as > the cloned VM datastore UUID: lnd-uk-002 - fb7b551a584e4b5192b8f371af56b303 > > > > I do not understand why I have here the size "0" > > > Regards, > Cristian > > > > > -----Original Message----- > From: Pearl d'Silva <pearl.dsi...@shapeblue.com> > Sent: Friday, September 24, 2021 9:44 AM > To: users@cloudstack.apache.org > Subject: Re: Failed to authentication SSH user root on host - repetitive > error > > By any chance, are there any old systemvm.isos in the secondary store, if > yes, can you please delete them. Based on the information you've provided, > it seems like, the key has been injected into systemvm.iso, but during boot > up, a script (cloud-early-config) that sets up the VM before bootstrapping > / > patching isn't copying the auth key. > Can you try the steps mentioned under the VMWare section of the doc: > https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.iso > and see if it helps. > SystemVm.iso - Apache Cloudstack - Apache Software > Foundation< > https://cwiki.apache.org/confluence/display/CLOUDSTACK/SystemVm.i > so> > Mechanism of propagating systemvm.iso. Systemvm.iso gets propagated > differently for different hypervisors. It comes packaged as part of the > Cloudstack rpm or is built on dev environments with the command "mvn clean > install -P developer,systemvm" (More info on building cloudstack) .Once the > iso is at the appropriate location depending on the hypervisor it is > inserted as a cd drive and the ... > cwiki.apache.org > Thanks, > Pearl > > ________________________________ > From: cristian.c@istream.today <cristian.c@istream.today> > Sent: Friday, September 24, 2021 11:45 AM > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: RE: Failed to authentication SSH user root on host - repetitive > error > > Hi Pearl, > > I already did a check via console into the secondary storage vm, I > saw > that there are not keys in authorized_keys. > > If I try to inject, I see this : /bin/bash > /usr/share/cloudstack-common/scripts/vm/systemvm/injectkeys.sh > /var/cloudstack/management/.ssh/id_rsa.pub > /var/cloudstack/management/.ssh/id_rsa > /usr/share/cloudstack-common/vms/systemvm.iso > mount: /dev/loop0 is write-protected, mounting read-only New public key is > the same as the one in the systemvm.iso, not injecting it, not modifying > systemvm.iso > > > [root@cloud-emea systemvm]# cd /mnt/iso/ [root@cloud-emea iso]# ls > agent.zip > authorized_keys cloud-scripts.tgz [root@cloud-emea iso]# cat > authorized_keys ssh-rsa > > AAAAB3NzaC1yc2EAAAABIwAAAQEA0UEA0mRUzKbH1cHHY2GlsaQ18q1KvqfNSV/YqAScZhcPueIl > > BbqwPUNznzfSiz/K/+DH8u5QkDIz+fC0Sx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+Bd1Y2U1Tx > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+lYee > +zUh6vszEDwmiq5nTkuCJP1T3o8QL3gUekAFjW7CECsIzLkA41Q8lY0L3qaHBRUJntGGIMtZ > +26AP > IWMC7NHD0wFge3DEN5UhFODcB1f9U7oqa10XqgORjbd88JPfFv/0j92xaaerNpJKw== > cl...@cloud-emea.test.host[root@cloud-emea iso]# > > is the same key as I have in : /var/cloudstack/management/.ssh/id_rsa.pub > > (For security reason I have replaced characters from ssh pub key) > > > I'm stuck here, I do not understand the logic. > > > Regards, > Cristian > > > > > -----Original Message----- > From: Pearl d'Silva <pearl.dsi...@shapeblue.com> > Sent: Friday, September 24, 2021 8:05 AM > To: users@cloudstack.apache.org > Subject: Re: Failed to authentication SSH user root on host - repetitive > error > > Hi Cristian, > > The exception you are seeing is most likely to happen if the systemvm.iso > hasn't been injected with the Management server's key. One way to validate > it, would be to go to your secondary store - I presume you are working on a > VMware setup - mount the systemvm-4.15.2.iso and verify the authorized_keys > data with what's present on the MS at > /var/cloudstack/management/.ssh/id_rsa.pub. > > Thanks, > Pearl > > ________________________________ > From: cristian.c@istream.today <cristian.c@istream.today> > Sent: Thursday, September 23, 2021 9:50 PM > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: Failed to authentication SSH user root on host - repetitive error > > Hello, > > > > > > I was not able to fix this error "Failed to authentication SSH" and > looks like a bug for me, I will explain here why. > > > > My setup before adding new ZONE: > > > > 1. Cloudstack 4.15.2 + VMware 6.5 with 1 Basic Network Zone. ( This > is > an old setup, it was upgraded frequently, I think from 4.2 ) > 2. I add new VMware zone, but this one with Advanced Networking, I end > with this repetitive error and nothing else. > 3. I decide to add a new zone, the same as I have at (2) with Advanced > Networking, everything works perfect, no issue at systemvm deploy, routers, > instances. > 4. I delete the zone from point (2) > 5. I add again the same zone, end with the same error. > > > > I have tried multiple things, delete the template, add again, inject, > nothing works, I end with the same error, over and over, and I do not see > the why. > > > > > > 021-09-23 11:53:54,666 ERROR [c.c.h.v.r.VmwareResource] > (DirectAgent-228:ctx-083b9265 lnd-uk-002.test.host, job-35231/job-35263, > cmd: SetupKeyStoreCommand) (logid:df4131be) Command failed due to > Exception: > java.lang.Exception > > Message: Failed to authentication SSH user root on host 10.15.0.204 > > > > 2021-09-23 11:53:54,674 ERROR [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-19:ctx-808054ab job-35231/job-35263 ctx-eb7c37bb) > (logid:df4131be) Retrying after catching exception while trying to secure > agent for systemvm id=2025 > > com.cloud.utils.exception.CloudRuntimeException: Unable to read/process > CSR: > Command failed due to Exception: java.lang.Exception > > Message: Failed to authentication SSH user root on host 10.15.0.204 > > > > > > Here is the full log : https://pastebin.com/raw/fFfmquVB > > > > > > Maybe someone have a hint for me . > > > > Regards, > > Cristian > > > > > > > > > > >
