Can you ssh into the Secondary Storage VM?
This can be done by getting the link-local-ip address from the infrastructure
menu in the UI and then sshing into the host the SSVM is on and issuing this
command as root: ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@link-local-ip
(replace link-local-ip with the link local ip).
Try running the SSVM diangnostic script: usr/local/cloud/systemvm/ssvm-check.sh
- Si
________________________________________
From: Nezar Madbouh <nezar.madb...@ipscape.com.au>
Sent: Tuesday, March 15, 2016 7:49 PM
To: users@cloudstack.apache.org
Subject: Re: Unable to create a New VM - Error message: Unable to start
instance due to Unable to get answer that is of class
com.cloud.agent.api.StartAnswer
Hi Simon,
Please see details below:
mysql> use cloud; select id, name, status, type from host where id = 5;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
+----+------------------------------+--------+---------+
| id | name | status | type |
+----+------------------------------+--------+---------+
| 5 | eqx-cs-cmp-02.ipscape.com.au | up | routing |
+----+------------------------------+--------+---------+
1 row in set (0.00 sec)
cat /var/log/cloudstack/agent/agent.log
com.cloud.utils.exception.CloudRuntimeException:
com.cloud.utils.exception.CloudRuntimeException: Can't find
volume:089ff3ca-9bff-402c-a14e-ea7a35aaad5c
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:262)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVbd(LibvirtComputingResource.java:3921)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3789)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1333)
at com.cloud.agent.Agent.processRequest(Agent.java:501)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:808)
at com.cloud.utils.nio.Task.run(Task.java:84)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
agent.log.2016-03-03.gz (END)
[root@eqx-cs-cmp-02 agent]# virsh pool-list
Name State Autostart
-----------------------------------------
2d0967f8-50d6-4d6d-87b7-4da0596f6924 active no
b3647480-5709-36eb-9ddd-e2c666f30b57 active no
e3831246-ff55-4af4-9964-a17565edaccc active no
[Root @ eqx-cs-cmp-02 agent] # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_root-lv_root
203G 215G 1.3G 1% /
tmpfs 48g 48g 0 0% / dev / shm
/ Dev / sda1 477m 427m 25M 6% / boot
xx.xx.xx.xx:/Cloudstack/primary
7.6T 4.1T 3.5T 54%
/mnt/b3647480-5709-36eb-9ddd-e2c666f30b57
I also noticed the following in cloudstack-agent.err
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-11-600-VM'
libvirt: Storage Driver error : Storage volume not found: no storage vol
with matching name '089ff3ca-9bff-402c-a14e-ea7a35aaad5c'
Your help is much appreciated,
Nezar
On 16 March 2016 at 11:03, Simon Weller <swel...@ena.com> wrote:
> Nezar,
>
> ok, I just took a look at a pastebin you posted.
>
> I see it complaining about the inability to find a volume prior to
> throwing the host exception.
>
> Firstly, let's find out who host 5 is. Can you log into your MySQL server
> and run this:
>
> use cloud;select id,name,status,type from host where id=5;
>
> Can you post the results?
>
> If it's a host, can you post the /var/log/cloudstack/agent/agent.log from
> that host?
>
> And also run virsh pool-list on the host in question?
>
> - Si
>
>
>
>
>
> ________________________________________
> From: Nezar Madbouh <nezar.madb...@ipscape.com.au>
> Sent: Tuesday, March 15, 2016 6:11 PM
> To: users@cloudstack.apache.org
> Subject: Re: Unable to create a New VM - Error message: Unable to start
> instance due to Unable to get answer that is of class
> com.cloud.agent.api.StartAnswer
>
> Hi Simon,
>
> When you say "double check your routers tied to the network", do you mean
> the physical routers or the virtual routers?
>
> When I try to create a VM in Prod Environment it fails. I can't ping the
> Prod virtual router from management server or Host.
>
> But I can ping Dev virtual router from management server and Host and I was
> able to create a VM in Dev Environment.
>
> How can I troubleshoot the Prod Virtual router. I launched the console but
> couldn't login.
>
>
> Please note that we need to take into consideration that the affected
> network is the Production network and we need to minimise service outage as
> much as possible.
>
> Your help is much appreciated,
> Nezar
>
>
>
> On 16 March 2016 at 08:37, Simon Weller <swel...@ena.com> wrote:
>
> > I've seen this before where a Virtual Router has crashed and the
> > management server can't communicate with it.
> > Are you using an advanced network setup?
> >
> > If so, double check your routers tied to the network where you are
> > attempting to start an instance.
> >
> > - Si
> >
> >
> > ________________________________________
> > From: Nezar Madbouh <nezar.madb...@ipscape.com.au>
> > Sent: Tuesday, March 15, 2016 4:02 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Unable to create a New VM - Error message: Unable to start
> > instance due to Unable to get answer that is of class
> > com.cloud.agent.api.StartAnswer
> >
> > Hi Glenn,
> >
> > We have 16 hosts, all KVM.
> >
> > The error is not specific to Host 5. Every time I try to create a VM it
> > gives me the same error message but on different host.
> >
> > How do I restart the agent on the host. Is it service affecting?
> >
> > Thanks very much for your help.
> > Neza
> >
> > On 15 March 2016 at 17:37, Glenn Wagner <glenn.wag...@shapeblue.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Can you explain your environment first , how many hosts , what is your
> > > hypervisors (KVM,Xenserver,VMware)
> > >
> > > The error seems to be with your Host 5 Hypervisor
> > > Resource [Host:5] is unreachable:
> > >
> > > Go to the infrastructure section and see what host5 is reporting,
> > > Have you tried to restart the agent on the Host?
> > >
> > > Thanks
> > > Glenn
> > >
> > >
> > >
> > >
> > > [image: ShapeBlue] <http://www.shapeblue.com>
> > > Glenn Wagner
> > > Senior Consultant , ShapeBlue
> > > d: * | s: +27 21 527 0091* <%7C%20s:%20+27%2021%20527%200091> | m:
> > > *+27 73 917 4111* <+27%2073%20917%204111>
> > > e: *glenn.wag...@shapeblue.com | t: *
> > > <glenn.wag...@shapeblue.com%20%7C%20t:> | w: *www.shapeblue.com*
> > > <http://www.shapeblue.com>
> > > a:
> > > 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130
> > South Africa
> > > Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> > > Services India LLP is a company incorporated in India and is operated
> > under
> > > license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> > > company incorporated in Brasil and is operated under license from Shape
> > > Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic
> of
> > > South Africa and is traded under license from Shape Blue Ltd. ShapeBlue
> > is
> > > a registered trademark.
> > > This email and any attachments to it may be confidential and are
> intended
> > > solely for the use of the individual to whom it is addressed. Any views
> > or
> > > opinions expressed are solely those of the author and do not
> necessarily
> > > represent those of Shape Blue Ltd or related companies. If you are not
> > the
> > > intended recipient of this email, you must neither take any action
> based
> > > upon its contents, nor copy or show it to anyone. Please contact the
> > sender
> > > if you believe you have received this email in error.
> > >
> > >
> > > -----Original Message-----
> > > From: Nezar Madbouh [mailto:nezar.madb...@ipscape.com.au]
> > > Sent: Tuesday, 15 March 2016 5:45 AM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Unable to create a New VM - Error message: Unable to start
> > > instance due to Unable to get answer that is of class
> > > com.cloud.agent.api.StartAnswer
> > >
> > > Hi Jeremy,
> > >
> > > I have uploaded the contents to http://pastebin.com/1J3mmgTh
> > >
> > > your help is much appreciated,
> > > Nezar.
> > >
> > > On 15 March 2016 at 13:46, Jeremy Peterson <jpeter...@acentek.net>
> > wrote:
> > >
> > > > I don't see any attachments can you create a pastebin. That's easier
> > > > to look through as I think attachments are removed before being sent
> > > > to user group.
> > > >
> > > > Jeremy
> > > >
> > > >
> > > > -------- Original message --------
> > > > From: Nezar Madbouh <nezar.madb...@ipscape.com.au>
> > > > Date: 3/14/2016 6:38 PM (GMT-06:00)
> > > > To: users@cloudstack.apache.org
> > > > Subject: Unable to create a New VM - Error message: Unable to start
> > > > instance due to Unable to get answer that is of class
> > > > com.cloud.agent.api.StartAnswer
> > > >
> > > > Hi Shweta and Sadhu,
> > > >
> > > > Please find attached managerment-server.log which contains the logs
> > > > since I started creating a new VM (VM id 600).
> > > >
> > > > Please note that we don't have agent.log files
> > > >
> > > > Your help is much appreciated.
> > > >
> > > > Regards,
> > > > Nezar.
> > > >
> > > Find out more about ShapeBlue and our range of CloudStack related
> > services:
> > > IaaS Cloud Design & Build
> > > <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid
> > > IaaS deployment framework <http://shapeblue.com/csforge/>
> > > CloudStack Consulting <http://shapeblue.com/cloudstack-consultancy/> |
> > CloudStack
> > > Software Engineering
> > > <http://shapeblue.com/cloudstack-software-engineering/>
> > > CloudStack Infrastructure Support
> > > <http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack
> > > Bootcamp Training Courses <http://shapeblue.com/cloudstack-training/>
> > >
> >
>
