Adam, what are the specifications of the VR? can you upgrade the RAM and CPU and try to restart that VR? I had similar issue and giving the VR more RAM and CPU will start the VR.
Thanks, Motty On Mon, Nov 3, 2014 at 7:04 AM, Nux! <n...@li.nux.ro> wrote: > Adam, > > You can modify the qcow2 file that is in your secondary storage, i.e. you > should mount that storage somewhere and go to > /mount/point/template/tmpl/274 and modify the qcow2 file. > > Alternatively you could use my already modified copy to overwrite your > uuid.qcow2 with (at your own risk of course): > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2 > > HTH > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > ----- Original Message ----- > > From: "Adam" <adam.scarce...@gmail.com> > > To: users@cloudstack.apache.org > > Sent: Monday, 3 November, 2014 14:58:58 > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > > [root@cs-east-prod1 management]# netstat -a|grep 9090 > > unix 3 [ ] STREAM CONNECTED 29090 > > [root@cs-east-prod1 management]# nmap localhost > > > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST > > Nmap scan report for localhost (127.0.0.1) > > Host is up (0.000015s latency). > > Other addresses for localhost (not scanned): 127.0.0.1 > > Not shown: 990 closed ports > > PORT STATE SERVICE > > 22/tcp open ssh > > 25/tcp open smtp > > 80/tcp open http > > 111/tcp open rpcbind > > 199/tcp open smux > > 631/tcp open ipp > > 2049/tcp open nfs > > 3306/tcp open mysql > > 8080/tcp open http-proxy > > 9090/tcp open zeus-admin > > > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds > > [root@cs-east-prod1 management]# telnet localhost 9090 > > Trying ::1... > > Connected to localhost. > > Escape character is '^]'. > > Connection closed by foreign host. > > [root@cs-east-prod1 management]# nmap 10.97.38.10 > > > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10) > > Host is up (0.000015s latency). > > Not shown: 993 closed ports > > PORT STATE SERVICE > > 22/tcp open ssh > > 80/tcp open http > > 111/tcp open rpcbind > > 2049/tcp open nfs > > 3306/tcp open mysql > > 8080/tcp open http-proxy > > 9090/tcp open zeus-admin > > > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090 > > Trying 10.97.38.10... > > Connected to 10.97.38.10. > > Escape character is '^]'. > > Connection closed by foreign host. > > [root@cs-east-prod1 management]# > > > > Also, I checked for the /opt/cloud/bin/ directory on my Management Host, > > but it's not there: > > > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/ > > -bash: cd: /opt/cloud/bin/: No such file or directory > > [root@cs-east-prod1 management]# cd / > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh" > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh" > > [root@cs-east-prod1 /]# > > > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is inside > > the Virtual Router VM? If so, how could I possibly change that since it > > won't start? > > > > Best Regards, > > > > > > > > Adam Scarcella > > > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pdion...@apache.org> > wrote: > > > >> Could it be possible the firewall is blocking port 9090 from the > >> management-server? > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) > Unable > >> to ping management server at 10.97.38.10:9090 due to ConnectException > >> java.net.ConnectException: Connection refused > >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <n...@li.nux.ro> wrote: > >> > >> > BTW the agent logs should help clarify this. > >> > The aforementioned changes must be done either in the current > (broken) VR > >> > or in the systemvm (after which you should redeploy the VR to pick up > the > >> > change). > >> > > >> > HTH > >> > Lucian > >> > > >> > -- > >> > Sent from the Delta quadrant using Borg technology! > >> > > >> > Nux! > >> > www.nux.ro > >> > > >> > ----- Original Message ----- > >> > > From: "Nux!" <n...@li.nux.ro> > >> > > To: users@cloudstack.apache.org > >> > > Sent: Monday, 3 November, 2014 13:53:10 > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > >> > > >> > > Hello, > >> > > > >> > > I think you are hitting a problem I also had: > >> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl] > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2) > >> > Failed to > >> > > start instance VM[DomainRouter|r-4758-VM] > >> > > com.cloud.utils.exception.ExecutionException: Unable to start > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not > retrying" > >> > > > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781 > >> > > > >> > > Basically the systemvm template has some problems, you need to > somehow > >> > copy > >> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh > to > >> > get it > >> > > working. > >> > > > >> > > -- > >> > > Sent from the Delta quadrant using Borg technology! > >> > > > >> > > Nux! > >> > > www.nux.ro > >> > > > >> > > ----- Original Message ----- > >> > >> From: "Adam" <adam.scarce...@gmail.com> > >> > >> To: users@cloudstack.apache.org > >> > >> Sent: Monday, 3 November, 2014 13:32:12 > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > >> > > > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release > >> > >> CentOS release 6.6 (Final) > >> > >> [root@cs-east-prod1 ~]# uname -a > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 > 04:27:16 > >> UTC > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux > >> > >> [root@cs-east-prod1 ~]# virsh --version > >> > >> 0.10.2 > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g' > >> > >> /etc/cloudstack/agent/log4j-cloud.xml > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop > >> > >> Stopping cloudstack-management: [ OK ] > >> > >> [root@cs-east-prod1 ~]# mv > >> > >> /var/log/cloudstack/management/management-server.log > >> > >> /var/log/cloudstack/management/management-server.bak > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start > >> > >> Starting cloudstack-management: [ OK ] > >> > >> [root@cs-east-prod1 ~]# > >> > >> > >> > >> Please see attached ZIP of clean management-server.log in DEBUG > mode. > >> At > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached > >> > >> management-server.log*) I tried starting the Virtual Router from > the > >> > >> Management UI and received this "Status" message in the UI. > >> > >> > >> > >> [image: Inline image 1] > >> > >> > >> > >> I can see in the management-server.log that while the > >> > cloudstack-management > >> > >> service is initializing, it throws the following ERROR at exactly > >> > (*2014-11-03 > >> > >> 08:14:36,813 on line 342 of the attached management-server.log*): > >> > >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] > (main:null) > >> > Unable > >> > >> to ping management server at 10.97.38.10:9090 due to > ConnectException > >> > >> java.net.ConnectException: Connection refused > >> > >> at sun.nio.ch.Net.connect0(Native Method) > >> > >> at sun.nio.ch.Net.connect(Net.java:465) > >> > >> at sun.nio.ch.Net.connect(Net.java:457) > >> > >> at > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > >> > >> at > >> > >> > >> > > >> > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123) > >> > >> at > >> > >> > >> > > >> > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092) > >> > >> at > >> > >> > >> > > >> > com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170) > >> > >> at > >> > >> > >> > > >> > com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945) > >> > >> at > >> > >> > >> > > >> > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61) > >> > >> at > >> > >> > >> > > >> > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210) > >> > >> at > >> > >> > >> > > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > >> > >> at > >> > >> > >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > >> > >> at > >> > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964) > >> > >> at > >> > >> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502) > >> > >> at > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) > >> > >> at > >> > >> > >> > > >> > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) > >> > >> at > >> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) > >> > >> at > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722) > >> > >> at > >> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) > >> > >> at > >> > >> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) > >> > >> at > >> > >> > >> org.apache.catalina.core.StandardService.start(StandardService.java:516) > >> > >> at > >> > >> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710) > >> > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593) > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> > >> at > >> > >> > >> > > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >> > >> at > >> > >> > >> > > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> > >> at java.lang.reflect.Method.invoke(Method.java:606) > >> > >> at > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) > >> > >> at > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] > (main:null) > >> > >> Detected that another management node with the same IP 10.97.38.10 > is > >> > >> considered as running in DB, however it is not pingable, we will > >> > continue > >> > >> cluster initialization with this management server node > >> > >> > >> > >> I'm not sure what that error means, but more than likely that's the > >> > reason > >> > >> why the Virtual Router will not start. > >> > >> > >> > >> * PLEASE HELP ! * > >> > >> > >> > >> Best Regards, > >> > >> > >> > >> > >> > >> > >> > >> Adam Scarcella > >> > >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <n...@li.nux.ro> wrote: > >> > >> > >> > >>> Hi, > >> > >>> > >> > >>> More agent and management logs please, make sure they are in > DEBUG[1] > >> > mode. > >> > >>> > >> > >>> [1] - > >> > >>> > >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug > >> > >>> > >> > >>> -- > >> > >>> Sent from the Delta quadrant using Borg technology! > >> > >>> > >> > >>> Nux! > >> > >>> www.nux.ro > >> > >>> > >> > >>> ----- Original Message ----- > >> > >>> > From: "Adam" <adam.scarce...@gmail.com> > >> > >>> > To: users@cloudstack.apache.org > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22 > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1 > >> > >>> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed > >> the > >> > >>> > instructions perfectly, but my Virtual Router will not start, > nor > >> am > >> > I > >> > >>> able > >> > >>> > to "upgrade" it. > >> > >>> > > >> > >>> > I tried destroying the Virtual Router, starting any VM Instance, > >> and > >> > >>> > allowing CS to recreate the VR. However that didn't help. The VR > >> > still > >> > >>> did > >> > >>> > not start and it was still saying that it required to be > upgraded. > >> > >>> > > >> > >>> > [image: Inline image 1] > >> > >>> > [image: Inline image 2] > >> > >>> > [image: Inline image 3] > >> > >>> > > >> > >>> > I know there are manual steps to finalize the systemvm upgrade, > but > >> > I did > >> > >>> > those steps: > >> > >>> > > >> > >>> > > >> > >>> > >> > > >> > http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix > >> > >>> > > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p > >> > >>> > Enter password: > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g. > >> > >>> > Your MySQL connection id is 174 > >> > >>> > Server version: 5.1.73-log Source distribution > >> > >>> > > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All > rights > >> > >>> reserved. > >> > >>> > > >> > >>> > Oracle is a registered trademark of Oracle Corporation and/or > its > >> > >>> > affiliates. Other names may be trademarks of their respective > >> > >>> > owners. > >> > >>> > > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current > input > >> > >>> > statement. > >> > >>> > > >> > >>> > mysql> select id,name from cloud.vm_template where name = > >> > >>> > 'systemvm-kvm-4.4'; > >> > >>> > +-----+------------------+ > >> > >>> >| id | name | > >> > >>> > +-----+------------------+ > >> > >>> >| 274 | systemvm-kvm-4.4 | > >> > >>> > +-----+------------------+ > >> > >>> > 1 row in set (0.00 sec) > >> > >>> > > >> > >>> > mysql> select type from cloud.vm_template where id='274'; > >> > >>> > +--------+ > >> > >>> >| type | > >> > >>> > +--------+ > >> > >>> >| SYSTEM | > >> > >>> > +--------+ > >> > >>> > 1 row in set (0.00 sec) > >> > >>> > > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance > where > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM'; > >> > >>> > +----------------+ > >> > >>> >| vm_template_id | > >> > >>> > +----------------+ > >> > >>> >| 274 | > >> > >>> > +----------------+ > >> > >>> > 1 row in set (0.00 sec) > >> > >>> > > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance > where > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM'; > >> > >>> > +----------------+ > >> > >>> >| vm_template_id | > >> > >>> > +----------------+ > >> > >>> >| 274 | > >> > >>> > +----------------+ > >> > >>> > 1 row in set (0.03 sec) > >> > >>> > > >> > >>> > mysql> select value from cloud.configuration where name = > >> > >>> > 'router.template.kvm'; > >> > >>> > +------------------+ > >> > >>> >| value | > >> > >>> > +------------------+ > >> > >>> >| systemvm-kvm-4.4 | > >> > >>> > +------------------+ > >> > >>> > 1 row in set (0.00 sec) > >> > >>> > > >> > >>> > mysql> > >> > >>> > > >> > >>> > * PLEASE HELP ! * > >> > >>> > > >> > >>> > Best Regards, > >> > >>> > > >> > > >> > Adam Scarcella > >> > > -- Thanks for your support, Motty
