Cristian, I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I have heard this before but am not sure if it was exactly the same. It sounds like a bug to me. Can you enter one in jira? Anybody else? Was this fixed in 4.10 or master yet?
On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian <cristian.c@istream.today > wrote: > Hello, > > After I continued the investigation I found that router is not allowing > the VM to fetch the password (port 8080) not sure what is wrong with this > version but in order to fix I have added this rule "-A INPUT -p tcp -m tcp > --dport 8080 -j ACCEPT" on virtual router.. > > Any idea how to fix this, permanent? In case reinstall the router for > sure I will have to do this again. > > > Thank you. > Cristian > > -----Original Message----- > From: Ciobanu Cristian [mailto:cristian.c@istream.today] > Sent: Wednesday, October 11, 2017 9:45 PM > To: 'users@cloudstack.apache.org' <users@cloudstack.apache.org> > Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database > > I was able to update the virtual router only after I have restarted the > iptables-persistend form virtual router side, not sure what is wrong with > this router version but I don't get the right status if I don't restart the > iptables from virtual router, I have did multiple tests : > > Example : > - Reboot router > - getting this error : job-3247/job-3248, cmd: CheckSshCommand) > (logid:6b3de9fa) Trying to > > connect to 149.56.xxx.xx ( I see this error in management log ) > - restart iptables from router side > - works > > In case I destroy/restart the router I have to repeat the same thing, > restart iptables... > > I also tested a new deploy with a template used before on this setup, > looks like the password set for new deploy is not working... > > > Any idea why I have those issue? > > Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages ) > > > Regards, > Cristian > > > -----Original Message----- > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > Sent: Wednesday, October 11, 2017 6:10 PM > To: users@cloudstack.apache.org > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database > > that is a tricky one. You can destroy the vm from VmWare and set the state > to stopped in the DB. It should then start a new one. > check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore. > > the cause is probably some synchronisation issue. We've seen and solved it > before but are yet to find the root cause. > > > On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cristian.c@istream.today > > > wrote: > > > Hello, > > > > Was my mistake, forgot to add the system template from cloudstack > > interface before starting the upgrade, now I have other issue, looks > > like after I have enabled one of the zone and click on upgrade virtual > > router I still see the state "starting" if I check the router from > > vSphere side is up. > > > > > > This is what is see : > > > > tting > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk > > controller for vol ROOT -> osdefault, based on root disk controller > > settings at global configuration setting. > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on > > datastore catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1] > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on > > datastore catalina.out:INFO [c.c.s.r.VmwareStorageLayoutHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync > > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1] > > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1] > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not > > exist on datastore catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on > > datastore catalina.out:INFO [c.c.s.r.VmwareStorageLayoutHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync > > [datastore1] ROOT-4-000001.vmdk->[datastore1] > > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) > > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on > > datastore catalina.out:INFO [c.c.u.n.n.p.NiciraNvpApiVersion] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION: > > NOT PRESENT > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC > > device based on NicTO: {"deviceId":0,"networkRateMbps":200," > > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839- > > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5- > > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255. > > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00: > > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":" > > Native","type":"Guest","broadcastUri":"vlan://untagged"," > > isSecurityGroupEnabled":false,"name":"vSwitch1"} > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network > > on vmwaresvs vSwitch1 with name prefix: cloud.guest catalina.out:INFO > > [c.c.h.v.m.HypervisorHostHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC > > device on network cloud.guest.untagged.200.1-vSwitch1 > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC > > device based on NicTO: {"deviceId":1,"networkRateMbps":-1," > > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4- > > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987- > > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0"," > > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a"," > broadcastType":"LinkLocal"," > > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"} > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network > > on vmwaresvs vSwitch1 with name prefix: cloud.private > > catalina.out:WARN [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized > > broadcast type in VmwareResource, type: LinkLocal. Use vlan info from > > labeling: untagged > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network > > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC > > device on network cloud.private.untagged.0.1-vSwitch1 > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC > > port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing > > disk info from volume path: ROOT-4-000001 catalina.out:INFO > > [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to > > connect to 149.56.xxx.xx catalina.out:WARN > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180) > > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not > > connect to 149.56.xxx.xx due to java.net.ConnectException: Connection > > timed out catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to > > connect to 149.56.xxx.xx catalina.out:WARN > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48) > > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds > > catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not > > connect to 149.56.xxx.xx due to java.net.ConnectException: Connection > > timed out catalina.out:INFO [c.c.h.v.r.VmwareResource] > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to > > connect to 149.56.xxx.xx > > > > > > Any suggestion? Please. > > > > Regards, > > Cristian > > > > -----Original Message----- > > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > > Sent: Wednesday, October 11, 2017 3:41 PM > > To: users@cloudstack.apache.org > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the > > database > > > > Christian, > > > > The chances are I have not done the exact same upgrade as you are > > attempting. the generic upgrade of the database (assuming the right > > template and other things) is quite straight forward. You will need to > > give us at least the stacktrace and any related information. from the > > log above it, if this is a reproducible error. > > > > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian > > <cristian.c@istream.today > > > > > wrote: > > > > > I have restored back to 4.5.2, but if I upgrade again using the > > > steps provide in documentation I will end with the same error.. did > > > you tested this? With tha configuration (acs 4.5.2, vmware, local > storage) ? > > > > > > Regards, > > > Cristian > > > > > > -----Original Message----- > > > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > > > Sent: Wednesday, October 11, 2017 3:29 PM > > > To: users@cloudstack.apache.org > > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the > > > database > > > > > > Hi the management server on starting up should take care of the > upgrade. > > > if you have a clustered MS make sure to start only one. In your case > > > after the failure you should probably restore the latest backup > > > before > > continuing. > > > > > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian > > > <cristian.c@istream.today > > > > > > > wrote: > > > > > > > Hello, > > > > > > > > > > > > > > > > In order to upgrade cloudstack to 4.9 I have followed this > > > > doecumentation > > > > : > > > > http://docs.cloudstack.apache.org/projects/cloudstack- > > > > release-notes/en/4.9.2 > > > > .0/upgrade/upgrade-4.5.html and ended with this : > > > > > > > > > > > > > > > > management-server.log:2017-10-11 08:05:35,612 WARN > > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) > > > > Exception encountered during context initialization - cancelling > > > > refresh > > > attempt: > > > > org.springframework.context.ApplicationContextException: Failed to > > > > start bean 'cloudStackLifeCycle'; nested exception is > > > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade > > > > the database > > > > > > > > > > > > > > > > Is there a valid upgrade procedure or script for : Cloudstack > > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ? > > > > > > > > > > > > > > > > (cloudstack was installed from shapeblue repo) > > > > > > > > > > > > > > > > > > > > > > > > Thank you. > > > > > > > > Cristian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Daan > > > > > > > > > > > > -- > > Daan > > > > > > > -- > Daan > > -- Daan
