Hi Simon, How can I find out which host the SSVM is on?
Thanks & Regards, Nezar. On 16 March 2016 at 12:28, Simon Weller <swel...@ena.com> wrote: > Can you ssh into the Secondary Storage VM? > > This can be done by getting the link-local-ip address from the > infrastructure menu in the UI and then sshing into the host the SSVM is on > and issuing this command as root: ssh -i /root/.ssh/id_rsa.cloud -p 3922 > root@link-local-ip (replace link-local-ip with the link local ip). > > Try running the SSVM diangnostic script: > usr/local/cloud/systemvm/ssvm-check.sh > > - Si > ________________________________________ > From: Nezar Madbouh <nezar.madb...@ipscape.com.au> > Sent: Tuesday, March 15, 2016 7:49 PM > To: users@cloudstack.apache.org > Subject: Re: Unable to create a New VM - Error message: Unable to start > instance due to Unable to get answer that is of class > com.cloud.agent.api.StartAnswer > > Hi Simon, > > Please see details below: > > > mysql> use cloud; select id, name, status, type from host where id = 5; > Reading table information for completion of table and column names > You can turn off this feature to get a quicker startup with -A > > Database changed > +----+------------------------------+--------+---------+ > | id | name | status | type | > +----+------------------------------+--------+---------+ > | 5 | eqx-cs-cmp-02.ipscape.com.au | up | routing | > +----+------------------------------+--------+---------+ > 1 row in set (0.00 sec) > > > > cat /var/log/cloudstack/agent/agent.log > com.cloud.utils.exception.CloudRuntimeException: > com.cloud.utils.exception.CloudRuntimeException: Can't find > volume:089ff3ca-9bff-402c-a14e-ea7a35aaad5c > at > > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:262) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVbd(LibvirtComputingResource.java:3921) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3789) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1333) > at com.cloud.agent.Agent.processRequest(Agent.java:501) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:808) > at com.cloud.utils.nio.Task.run(Task.java:84) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > agent.log.2016-03-03.gz (END) > > > > [root@eqx-cs-cmp-02 agent]# virsh pool-list > Name State Autostart > ----------------------------------------- > 2d0967f8-50d6-4d6d-87b7-4da0596f6924 active no > b3647480-5709-36eb-9ddd-e2c666f30b57 active no > e3831246-ff55-4af4-9964-a17565edaccc active no > > > > > [Root @ eqx-cs-cmp-02 agent] # df -h > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/vg_root-lv_root > 203G 215G 1.3G 1% / > tmpfs 48g 48g 0 0% / dev / shm > / Dev / sda1 477m 427m 25M 6% / boot > xx.xx.xx.xx:/Cloudstack/primary > 7.6T 4.1T 3.5T 54% > /mnt/b3647480-5709-36eb-9ddd-e2c666f30b57 > > > > > I also noticed the following in cloudstack-agent.err > > libvirt: QEMU Driver error : Domain not found: no domain with matching name > 'i-11-600-VM' > libvirt: Storage Driver error : Storage volume not found: no storage vol > with matching name '089ff3ca-9bff-402c-a14e-ea7a35aaad5c' > > Your help is much appreciated, > Nezar > > > On 16 March 2016 at 11:03, Simon Weller <swel...@ena.com> wrote: > > > Nezar, > > > > ok, I just took a look at a pastebin you posted. > > > > I see it complaining about the inability to find a volume prior to > > throwing the host exception. > > > > Firstly, let's find out who host 5 is. Can you log into your MySQL server > > and run this: > > > > use cloud;select id,name,status,type from host where id=5; > > > > Can you post the results? > > > > If it's a host, can you post the /var/log/cloudstack/agent/agent.log from > > that host? > > > > And also run virsh pool-list on the host in question? > > > > - Si > > > > > > > > > > > > ________________________________________ > > From: Nezar Madbouh <nezar.madb...@ipscape.com.au> > > Sent: Tuesday, March 15, 2016 6:11 PM > > To: users@cloudstack.apache.org > > Subject: Re: Unable to create a New VM - Error message: Unable to start > > instance due to Unable to get answer that is of class > > com.cloud.agent.api.StartAnswer > > > > Hi Simon, > > > > When you say "double check your routers tied to the network", do you mean > > the physical routers or the virtual routers? > > > > When I try to create a VM in Prod Environment it fails. I can't ping the > > Prod virtual router from management server or Host. > > > > But I can ping Dev virtual router from management server and Host and I > was > > able to create a VM in Dev Environment. > > > > How can I troubleshoot the Prod Virtual router. I launched the console > but > > couldn't login. > > > > > > Please note that we need to take into consideration that the affected > > network is the Production network and we need to minimise service outage > as > > much as possible. > > > > Your help is much appreciated, > > Nezar > > > > > > > > On 16 March 2016 at 08:37, Simon Weller <swel...@ena.com> wrote: > > > > > I've seen this before where a Virtual Router has crashed and the > > > management server can't communicate with it. > > > Are you using an advanced network setup? > > > > > > If so, double check your routers tied to the network where you are > > > attempting to start an instance. > > > > > > - Si > > > > > > > > > ________________________________________ > > > From: Nezar Madbouh <nezar.madb...@ipscape.com.au> > > > Sent: Tuesday, March 15, 2016 4:02 PM > > > To: users@cloudstack.apache.org > > > Subject: Re: Unable to create a New VM - Error message: Unable to start > > > instance due to Unable to get answer that is of class > > > com.cloud.agent.api.StartAnswer > > > > > > Hi Glenn, > > > > > > We have 16 hosts, all KVM. > > > > > > The error is not specific to Host 5. Every time I try to create a VM it > > > gives me the same error message but on different host. > > > > > > How do I restart the agent on the host. Is it service affecting? > > > > > > Thanks very much for your help. > > > Neza > > > > > > On 15 March 2016 at 17:37, Glenn Wagner <glenn.wag...@shapeblue.com> > > > wrote: > > > > > > > Hi, > > > > > > > > Can you explain your environment first , how many hosts , what is > your > > > > hypervisors (KVM,Xenserver,VMware) > > > > > > > > The error seems to be with your Host 5 Hypervisor > > > > Resource [Host:5] is unreachable: > > > > > > > > Go to the infrastructure section and see what host5 is reporting, > > > > Have you tried to restart the agent on the Host? > > > > > > > > Thanks > > > > Glenn > > > > > > > > > > > > > > > > > > > > [image: ShapeBlue] <http://www.shapeblue.com> > > > > Glenn Wagner > > > > Senior Consultant , ShapeBlue > > > > d: * | s: +27 21 527 0091* <%7C%20s:%20+27%2021%20527%200091> | > m: > > > > *+27 73 917 4111* <+27%2073%20917%204111> > > > > e: *glenn.wag...@shapeblue.com | t: * > > > > <glenn.wag...@shapeblue.com%20%7C%20t:> | w: *www.shapeblue.com* > > > > <http://www.shapeblue.com> > > > > a: > > > > 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130 > > > South Africa > > > > Shape Blue Ltd is a company incorporated in England & Wales. > ShapeBlue > > > > Services India LLP is a company incorporated in India and is operated > > > under > > > > license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a > > > > company incorporated in Brasil and is operated under license from > Shape > > > > Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The > Republic > > of > > > > South Africa and is traded under license from Shape Blue Ltd. > ShapeBlue > > > is > > > > a registered trademark. > > > > This email and any attachments to it may be confidential and are > > intended > > > > solely for the use of the individual to whom it is addressed. Any > views > > > or > > > > opinions expressed are solely those of the author and do not > > necessarily > > > > represent those of Shape Blue Ltd or related companies. If you are > not > > > the > > > > intended recipient of this email, you must neither take any action > > based > > > > upon its contents, nor copy or show it to anyone. Please contact the > > > sender > > > > if you believe you have received this email in error. > > > > > > > > > > > > -----Original Message----- > > > > From: Nezar Madbouh [mailto:nezar.madb...@ipscape.com.au] > > > > Sent: Tuesday, 15 March 2016 5:45 AM > > > > To: users@cloudstack.apache.org > > > > Subject: Re: Unable to create a New VM - Error message: Unable to > start > > > > instance due to Unable to get answer that is of class > > > > com.cloud.agent.api.StartAnswer > > > > > > > > Hi Jeremy, > > > > > > > > I have uploaded the contents to http://pastebin.com/1J3mmgTh > > > > > > > > your help is much appreciated, > > > > Nezar. > > > > > > > > On 15 March 2016 at 13:46, Jeremy Peterson <jpeter...@acentek.net> > > > wrote: > > > > > > > > > I don't see any attachments can you create a pastebin. That's > easier > > > > > to look through as I think attachments are removed before being > sent > > > > > to user group. > > > > > > > > > > Jeremy > > > > > > > > > > > > > > > -------- Original message -------- > > > > > From: Nezar Madbouh <nezar.madb...@ipscape.com.au> > > > > > Date: 3/14/2016 6:38 PM (GMT-06:00) > > > > > To: users@cloudstack.apache.org > > > > > Subject: Unable to create a New VM - Error message: Unable to start > > > > > instance due to Unable to get answer that is of class > > > > > com.cloud.agent.api.StartAnswer > > > > > > > > > > Hi Shweta and Sadhu, > > > > > > > > > > Please find attached managerment-server.log which contains the logs > > > > > since I started creating a new VM (VM id 600). > > > > > > > > > > Please note that we don't have agent.log files > > > > > > > > > > Your help is much appreciated. > > > > > > > > > > Regards, > > > > > Nezar. > > > > > > > > > Find out more about ShapeBlue and our range of CloudStack related > > > services: > > > > IaaS Cloud Design & Build > > > > <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – > rapid > > > > IaaS deployment framework <http://shapeblue.com/csforge/> > > > > CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> > | > > > CloudStack > > > > Software Engineering > > > > <http://shapeblue.com/cloudstack-software-engineering/> > > > > CloudStack Infrastructure Support > > > > <http://shapeblue.com/cloudstack-infrastructure-support/> | > CloudStack > > > > Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/ > > > > > > > > > > > >
