The address in Infrastructure > Hosts > (management server) is set to the correct IP address, not 127.0.0.1. Why are the logs referring to 127.0.0.1?
On Wed, Apr 30, 2014 at 3:00 PM, Ian Young <iyo...@ratespecial.com> wrote: > I notice my dashboard says "Management server node 127.0.0.1 is up." It > used to have an actual address, not localhost. Could this be causing > problems and if so, how can I set it back? > > > On Wed, Apr 30, 2014 at 12:40 PM, Ian Young <iyo...@ratespecial.com>wrote: > >> Yes, I replaced the new files with the rpmsave ones, which allowed the >> agent to start. However, most of the functions in the management console >> fail. >> >> >> On Wed, Apr 30, 2014 at 12:34 PM, stevenliang <stevenli...@yesup.com>wrote: >> >>> Do you have the file db.properties.rpmsave on management server and >>> agent.properties.rpmsave on agents? If so, and the date is correct, you can >>> use it rather than db.properties and agent.properties. >>> And then restart management and agent services. >>> >>> >>> On 30/04/14 03:26 PM, Ian Young wrote: >>> >>>> Yes, I restored the DB from the backup. When I try to start the router >>>> it >>>> says: >>>> >>>> Resource [Host:1] is unreachable: Host 1: Unable to start instance due >>>> to >>>> Unable to start VM[DomainRouter|r-63-VM] due to error in finalizeStart, >>>> not >>>> retrying >>>> >>>> The management server log says: >>>> >>>> 2014-04-30 12:20:52,485 ERROR [cloud.async.AsyncJobManagerImpl] >>>> (Job-Executor-7:job-520 = [ 9d7c898c-b5d0-4bd0-a711-563a91d7acc9 ]) >>>> Unexpected exception while executing >>>> org.apache.cloudstack.api.command.admin.router.StartRouterCmd >>>> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is >>>> unreachable: Host 1: Unable to start instance due to Unable to start >>>> VM[DomainRouter|r-63-VM] due to error in finalizeStart, not retrying >>>> >>>> >>>> On Wed, Apr 30, 2014 at 12:02 PM, stevenliang <stevenli...@yesup.com> >>>> wrote: >>>> >>>> I think you had backed up database, when you upgraded. >>>>> When you downgraded CS, you also need to restore DB. >>>>> >>>>> >>>>> On 30/04/14 02:58 PM, Ian Young wrote: >>>>> >>>>> I think my problem stems from a partially downloaded system VM >>>>>> template. >>>>>> I >>>>>> just noticed systemvm-kvm-4.3 is stuck at 27% downloaded. It must >>>>>> have >>>>>> been interrupted during the upgrade to 4.3. At the moment I've rolled >>>>>> back >>>>>> to 4.2.1 with a somewhat usable management interface, although the >>>>>> system >>>>>> VMs won't start. I suspect there is something in the database that is >>>>>> causing it to try to use the 4.3 template. How can I delete the >>>>>> template >>>>>> and make sure the management server is using the older one? >>>>>> >>>>>> >>>>>> On Tue, Apr 29, 2014 at 8:23 PM, Ian Young <iyo...@ratespecial.com> >>>>>> wrote: >>>>>> >>>>>> Ok, so I've figured out a way to identify volumes in the >>>>>> filesystem. For >>>>>> >>>>>>> instance, /var/storage/primary/4d324e1a-e3a6-4da8-9c4d-44ad723482ad >>>>>>> is >>>>>>> the >>>>>>> root volume for an instance I want to back up. Is this in qcow2 >>>>>>> format >>>>>>> or >>>>>>> something else? I'm using KVM. >>>>>>> >>>>>>> >>>>>>> On Tue, Apr 29, 2014 at 7:38 PM, Ian Young <iyo...@ratespecial.com> >>>>>>> wrote: >>>>>>> >>>>>>> Now I can't start cloudstack-agent. The agent.log says: >>>>>>> >>>>>>>> Unable to start agent: Failed to get private nic name >>>>>>>> >>>>>>>> I know this is because the network bridge is no longer set up >>>>>>>> correctly. >>>>>>>> I used to have a cloud0 and a cloudbr0 interface. Now I only >>>>>>>> have >>>>>>>> cloudbr0. I haven't changed my network configuration. Somehow it's >>>>>>>> been >>>>>>>> changed by CloudStack during the upgrade/downgrade. This is getting >>>>>>>> worse >>>>>>>> and worse the more I try to recover my data. Is there any way to >>>>>>>> back >>>>>>>> up >>>>>>>> the instances' volumes via the command line? I can't tell which is >>>>>>>> which >>>>>>>> because the filenames are all hashes. I really need to get these >>>>>>>> instances >>>>>>>> up and running--there are several months worth of work at stake >>>>>>>> here. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Apr 29, 2014 at 6:13 PM, ma y <breeze7...@gmail.com> wrote: >>>>>>>> >>>>>>>> I got the same problem, and how to downgrade CS 4.3.0 to 4.2.1 >>>>>>>> safely? >>>>>>>> >>>>>>>>> >>>>>>>>> 2014-04-30 8:45 GMT+08:00 Ian Young <iyo...@ratespecial.com>: >>>>>>>>> >>>>>>>>> Ok, my Cloudstack installation is now so broken that I think it's >>>>>>>>> probably >>>>>>>>> >>>>>>>>> best to backup all my instances and templates, wipe the >>>>>>>>>> databases, and >>>>>>>>>> start from scratch. However, I can't take snapshots or download >>>>>>>>>> >>>>>>>>>> volumes >>>>>>>>> >>>>>>>>> anymore. What's causing these errors? >>>>>>>>>> >>>>>>>>>> 2014-04-29 17:40:51,264 DEBUG [o.a.c.s.m. >>>>>>>>>> AncientDataMotionStrategy] >>>>>>>>>> (Job-Executor-11:ctx-0a3ead79 ctx-315eda05) copy object failed: >>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Failed to send >>>>>>>>>> >>>>>>>>>> command, >>>>>>>>> >>>>>>>>> due to Agent:1, com.cloud.exception.OperationTimedoutException: >>>>>>>>>> >>>>>>>>>> Commands >>>>>>>>> >>>>>>>>> 841744457 to Host 1 timed out after 21600 >>>>>>>>>> 2014-04-29 17:40:51,265 DEBUG [o.a.c.s.m. >>>>>>>>>> AncientDataMotionStrategy] >>>>>>>>>> (Job-Executor-11:ctx-0a3ead79 ctx-315eda05) copy failed >>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: >>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Failed to send >>>>>>>>>> >>>>>>>>>> command, >>>>>>>>> >>>>>>>>> due to Agent:1, com.cloud.exception.OperationTimedoutException: >>>>>>>>>> >>>>>>>>>> Commands >>>>>>>>> >>>>>>>>> 841744457 to Host 1 timed out after 21600 >>>>>>>>>> 2014-04-29 17:40:51,269 WARN [o.a.c.s.d. >>>>>>>>>> ObjectInDataStoreManagerImpl] >>>>>>>>>> (Job-Executor-11:ctx-0a3ead79 ctx-315eda05) Unsupported data >>>>>>>>>> object >>>>>>>>>> (VOLUME, >>>>>>>>>> org.apache.cloudstack.storage.datastore. >>>>>>>>>> PrimaryDataStoreImpl@7bbbd901 >>>>>>>>>> ), >>>>>>>>>> >>>>>>>>>> no >>>>>>>>> >>>>>>>>> need to delete from object in store ref table >>>>>>>>>> 2014-04-29 17:40:51,280 ERROR [c.c.a.ApiAsyncJobDispatcher] >>>>>>>>>> (Job-Executor-11:ctx-0a3ead79) Unexpected exception while >>>>>>>>>> executing >>>>>>>>>> org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd >>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Failed to copy >>>>>>>>>> the >>>>>>>>>> >>>>>>>>>> volume >>>>>>>>> >>>>>>>>> from the source primary storage pool to secondary storage. >>>>>>>>>> 2014-04-29 17:40:51,282 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>>>>> (Job-Executor-11:ctx-0a3ead79) Complete async job-501, jobStatus: >>>>>>>>>> >>>>>>>>>> FAILED, >>>>>>>>> >>>>>>>>> resultCode: 530, result: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/ >>>>>>>>>> >>>>>>>>> null/{"uuidList":[],"errorcode":530,"errortext":"Failed >>>>>>>>> >>>>>>>>> to copy the volume from the source primary storage pool to >>>>>>>>>> secondary >>>>>>>>>> storage."} >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Apr 29, 2014 at 4:15 PM, Ian Young < >>>>>>>>>> iyo...@ratespecial.com> >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I downgraded to 4.2.1 again but cloudstack-management won't start >>>>>>>>>> because >>>>>>>>>> the database is version 4.3. Is it safe to restore the database >>>>>>>>>> backup I >>>>>>>>>> made prior to this whole process? In the meantime I have destroyed >>>>>>>>>> and >>>>>>>>>> created system VMs, so I'm not sure it's a good idea. >>>>>>>>>> >>>>>>>>>>> On Apr 29, 2014 3:09 PM, "Ian Young" <iyo...@ratespecial.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> @stevenliang: I take it back--you can't set the VM size when >>>>>>>>>>> you >>>>>>>>>>> register >>>>>>>>>>> the template. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Apr 29, 2014 at 3:02 PM, motty cruz < >>>>>>>>>>>> motty.c...@gmail.com> >>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>> yes, you would have to shutdown the router, then click on "Change >>>>>>>>>>> >>>>>>>>>>>> Service >>>>>>>>>>>> >>>>>>>>>>> Offering" >>>>>>>>>>> >>>>>>>>>>>> restart the VR. >>>>>>>>>>>>> >>>>>>>>>>>>> To Ian, >>>>>>>>>>>>> >>>>>>>>>>>>> I suspect you forgot the last step: " >>>>>>>>>>>>> cloudstack-setup-management" >>>>>>>>>>>>> >>>>>>>>>>>>> that would fix your issue, I think, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> --- >>>>>>>>>>>>> I downgraded to 4.2.1 and then upgraded to 4.3. Now the >>>>>>>>>>>>> cloudstack-management service can't start because it can't >>>>>>>>>>>>> connect >>>>>>>>>>>>> >>>>>>>>>>>>> to >>>>>>>>>>>> >>>>>>>>>>> the >>>>>>>>>> >>>>>>>>>> database. >>>>>>>>>>> >>>>>>>>>>>> 2014-04-29 14:51:36,424 ERROR [c.c.u.d.Merovingian2] (main:null) >>>>>>>>>>>>> >>>>>>>>>>>>> Unable >>>>>>>>>>>> >>>>>>>>>>> to >>>>>>>>>> >>>>>>>>>>> get a new db connection >>>>>>>>>>>>> Caused by: java.sql.SQLException: Access denied for user >>>>>>>>>>>>> 'cloud'@ >>>>>>>>>>>>> 'localhost' >>>>>>>>>>>>> (using password: YES) >>>>>>>>>>>>> >>>>>>>>>>>>> Where are the credentials stored? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Apr 29, 2014 at 2:57 PM, stevenliang < >>>>>>>>>>>>> >>>>>>>>>>>>> stevenli...@yesup.com> >>>>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> oh, then change service offering for vr? >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 29/04/14 05:53 PM, motty cruz wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> for my VR, I created a new >>>>>>>>>>>>>> >>>>>>>>>>>>>>> "System Offering For Software Router" >>>>>>>>>>>>>>> CPU in (MHz) 1.00GHz >>>>>>>>>>>>>>> Memory (in MB) 1.00GB >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> this are my current offerings, I'm sure the more RAM and CPU >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> better >>>>>>>>>>>>>> >>>>>>>>>>>>> performance. >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Apr 29, 2014 at 2:44 PM, stevenliang < >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> stevenli...@yesup.com >>>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thank you again, motty. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I didn't notice this earlier. >>>>>>>>>>>>>>>> BTW, how did you make your vr had 1GB CPU and 512MB RAM? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 29/04/14 05:33 PM, motty cruz wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Stevellang, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I not sure if you saw this in the forums earlier : >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://mail-archives.apache. >>>>>>>>>>>>>>>>> org/mod_mbox/cloudstack-users/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 201404.mbox/% >>>>>>>>> >>>>>>>>> 3CCALoOYy6A10bz1zOQQs1VyFb9epqLfhf7mu6hc= >>>>>>>>>> >>>>>>>>>>> c2rfedroy...@mail.gmail.com%3E >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I don't know if the bug was fixed yet, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will try upgrade in the next couple of days on a testing >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> cluster, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> will >>>>>>>>>>> >>>>>>>>>>>> report back if the bug was fixed. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Apr 29, 2014 at 2:25 PM, stevenliang < >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> stevenli...@yesup.com >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thank you, motty. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I am also running kvm. Since that time I failed upgrade, >>>>>>>>>>>>>>>>> I am >>>>>>>>>>>>>>>>> still >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> using >>>>>>>>>>> >>>>>>>>>>>> 4.2.1. I'll try as your advice. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 29/04/14 05:19 PM, motty cruz wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Stevenllang, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I had the similar issue with VR, I notice it was >>>>>>>>>>>>>>>>>> because I >>>>>>>>>>>>>>>>>> leave >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> the >>>>>>>>>> >>>>>>>>>>> default system specs on the VR, for instance by default >>>>>>>>>>>>>> >>>>>>>>>>>>>>> 500MHz >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> on >>>>>>>>>> >>>>>>>>>> CPU >>>>>>>>>>> >>>>>>>>>>>> and >>>>>>>>>>>>>> >>>>>>>>>>>>>>> 128MB on RAM, if you upgrade to at least 1GB on CPU and >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 512MB of >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> RAM >>>>>>>>>> >>>>>>>>>>> your >>>>>>>>>>>>>> >>>>>>>>>>>>>>> VR will survive the upgrade from 4.2.1 to 4.3.1. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I am running KVM, when I upgrade from 4.2.1 to 4.3 my VMs >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> were >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> not >>>>>>>>>> >>>>>>>>>> able >>>>>>>>>>> >>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> access outside world, even if I created a new router. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wish you the best, >>>>>>>>>>>>>>>>>>> -motty >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tue, Apr 29, 2014 at 2:13 PM, stevenliang < >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> stevenli...@yesup.com> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, I had two zones(one is basic, another is advanced >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> mode). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> After I upgraded from 4.2.1 to 4.3, the vrouter lost. >>>>>>>>>> >>>>>>>>>>> So I rolled back to 4.2.1, the vrouter came back. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 29/04/14 04:54 PM, Ian Young wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Did rolling back to 4.2 fix the problem? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Tue, Apr 29, 2014 at 1:22 PM, stevenliang < >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> stevenli...@yesup.com >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I met your situation before. Finally I rolled back to >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> 4.2 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On 29/04/14 04:18 PM, Ian Young wrote: >>>>>>>>>> >>>>>>>>>>> I destroyed the old virtual router and was able to >>>>>>>>>>>>>>>>>>>>> create >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> a new >>>>>>>>>> >>>>>>>>>>> one >>>>>>>>>>>>>> >>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> adding a new instance. However, this new router >>>>>>>>>>>>>>>>>>>>>> also >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>> >>>>>>>>>>>> start, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> citing the same error. After that, the expungement >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> delay >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> elapsed >>>>>>>>>> >>>>>>>>>>> and >>>>>>>>>>>>>> >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> virtual router was expunged, so now I have none. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 28, 2014 at 8:52 PM, Ian Young < >>>>>>>>>>>>>>>>>>>>>>> iyo...@ratespecial.com> >>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I upgraded from 4.2.1 to 4.3.0 tonight, >>>>>>>>>>>>>>>>>>>>>>> following >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> instructions >>>>>>>>>> >>>>>>>>>>> here: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> http://docs.cloudstack.apache. >>>>>>>>>>>>>>>>>>>>>>> org/projects/cloudstack- >>>>>>>>>>>>>>>>>>>>>>> release-notes/en/latest/ >>>>>>>>>>>>>>>>>>>>>>> rnotes.html#upgrade-from-4-2- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> x-to-4-3 >>>>>>>>>> >>>>>>>>>> At the last step, I tried to restart the system VMs. >>>>>>>>>>> >>>>>>>>>>>> The >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> virtual >>>>>>>>>> >>>>>>>>>>> router >>>>>>>>>>>>>> >>>>>>>>>>>>>>> failed to start. Here is the message that was >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> displayed in >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>> >>>>>>>>>>> web >>>>>>>>>>>>>> >>>>>>>>>>>>>>> UI: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Resource [Host:1] is unreachable: Host 1: Unable to >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> start >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> instance >>>>>>>>>> >>>>>>>>>>> due >>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>> Unable to start VM[DomainRouter|r-4-VM] due to >>>>>>>>>>>>>>>>>>>>>>>> error in >>>>>>>>>>>>>>>>>>>>>>>> finalizeStart, >>>>>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>> retrying >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I tried running the script to restart the VMs but >>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> time >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> it >>>>>>>>>> >>>>>>>>>>> failed >>>>>>>>>>>>>> >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>> start the console proxy: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> [root@virthost1 ~]$ cloudstack-sysvmadm -d >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 192.168.100.6 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -u >>>>>>>>>> >>>>>>>>>> cloud >>>>>>>>>>> >>>>>>>>>>>> -p >>>>>>>>>>>>>>>>>>>>>>>> -a >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Stopping and starting 1 secondary storage vm(s)... >>>>>>>>>>>>>>>>>>>>>>>> Done stopping and starting secondary storage vm(s) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Stopping and starting 1 console proxy vm(s)... >>>>>>>>>>>>>>>>>>>>>>>> ERROR: Failed to start console proxy vm with id 2 >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Done stopping and starting console proxy vm(s) . >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Stopping and starting 0 running routing vm(s)... >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Is there a way to wipe the system VMs out and start >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> over? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>> >> >