Hi Ilya, Here is the output of mount command:
[root@eqx-cs-cmp-01 ~]# mount /dev/mapper/vg_root-lv_root on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) nfsd on /proc/fs/nfsd type nfsd (rw) 10.61.154.10:/Cloudstack/primary on /mnt/b3647480-5709-36eb-9ddd-e2c666f30b57 type nfs (rw,noatime,nfsvers=3,proto=tcp,hard,nolock,intr,addr=10.61.154.10) [root@eqx-cs-cmp-01 ~]# On 15 April 2016 at 15:22, Nezar Madbouh <nezar.madb...@ipscape.com.au> wrote: > Hi Ilya, > > Thanks for the instructions. I followed them and pasted the information on > pastebin: > > management-server logs job-14938 > http://pastebin.com/GAiy8u5H > > Agent logs > http://pastebin.com/mfiADsJc > > > Your help is much appreciated. > > Nezar. > > On 15 April 2016 at 13:58, ilya <ilya.mailing.li...@gmail.com> wrote: > >> Here is how i would troubleshoot this: >> >> 1) ssh to MS host, run >> >> tail -f /var/cloudstack/management/management-server.log | grep -i >> StartVMCmdByAdmin >> >> 2) on cloudstack UI, press start on the VM and watch the tail log >> >> 3) on the tail log, you should see a StartVirtualMachine api call show >> up with job ID. For example: >> >> > 2016-04-15 03:41:57,193 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-24:ctx-7892387a job-291575) Executing AsyncJobVO >> {id:291575, userId: 157, accountId: 2, instanceType: VirtualMachine, >> instanceId: 7648, cmd: >> org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin, cmdInfo: >> {"id":"7648","response":"json","sessionkey":"mMJsduS9KLKApqH4dPpIlGZdt1Y\u003d","ctxDetails":"{\"com.cloud.vm.VirtualMachine\":7648,\"com.cloud.uservm.UserVm\":7648,\"com.cloud.network.router.VirtualRouter\":7648}","cmdEventType":"VM.START","expires":"2016-04-15T03:51:57+0000","ctxUserId":"157","signatureversion":"3","httpmethod":"GET","uuid":"7648","ctxAccountId":"2","ctxStartEventId":"314416"}, >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, >> result: null, initMsid: 345051337894, completeMsid: null, lastUpdated: >> null, lastPolled: null, created: null} >> >> Each task in cloudstack spins up its own job, in this case its >> *job-291575* >> >> 4) now execute >> > grep -A3 -B3 job-291575 >> /var/cloudstack/management/management-server.log | tee job-291575.log >> > grep job-291575 /var/cloudstack/management/management-server.log | grep >> via | head -1 >> >> you should see a response like so, >> >> > 2016-04-15 03:41:59,259 DEBUG [c.c.a.m.ClusteredAgentAttache] >> (Work-Job-Executor-42:ctx-b2c4af6b job-291575/job-291576 ctx-0988152e) Seq >> 441-4181310779036795138: Forwarding Seq 441-4181310779036795138: { Cmd , >> MgmtId: 345051337894, via: 441(my-kvm-host01.example.com), Ver: v1, >> Flags: 100011, >> [{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:fe:4e:00:12:ad","vmIpAddress":"10.10.27.178","vmName":"ol6-test-ilya1","defaultRouter":"10.10.27.1","defaultDns":"10.17.27.166","duid":"00:03:00:01:06:fe:4e:00:12:ad","isDefault":true,"executeInSequence":false,"accessDetails":{"zone.network.type":"Advanced","router.guest.ip":"10.10.27.166","router.ip":"169.254.0.238"," >> router.name":"r-7301-VM"},"wait":0}}] } to 345051337893 >> >> We are looking for this line "via: 441(my-kvm-host01.example.com)" >> >> SSH to my-kvm-host01.example.com, and copy the >> /var/log/cloudstack/agent.log to pastebin >> >> also copy job-291575.log to pastebin >> >> I'll try to have a look, though when i get chance. >> >> >> Also paste the output of mount command on kvm host in question. I'm >> suspecting one of your storage pools maybe bad. >> >> Lastly, what distro are you running for KVM? >> >> Regards >> ilya >> >> On 4/14/16 5:05 PM, Nezar Madbouh wrote: >> > Hi Ilya, >> > >> > I shutdown all the VMs in Pre-prod Environmnet expecting the Pre-prod >> > Virtual router to be deleted automatically because no VM is running on >> this >> > vlan but it wasn't deleted. >> > >> > Am I right to expect the VR to be deleted automatically if no VM is >> running >> > in that vlan? >> > >> > Thanks, >> > Nezar. >> > >> > On 15 April 2016 at 09:53, Nezar Madbouh <nezar.madb...@ipscape.com.au> >> > wrote: >> > >> >> Hi Ilya, >> >> >> >> I have run the script and it didn't return any results: >> >> >> >> [root@eqx-cs-mgt-01 ~]# ./async_jobs.sh >> >> [root@eqx-cs-mgt-01 ~]# >> >> >> >> >> >> I also ran the command inside mysql: >> >> >> >> mysql> SELECT id, user_id, account_id, instance_type, instance_id, >> >> job_cmd, job_cmd_info, job_dispatcher, job_pending_signals FROM >> >> cloud.async_job where job_status=0 and job_dispatcher not like >> >> "pseudoJobDispatcher"; >> >> Empty set (0.00 sec) >> >> >> >> your help is much appreciated, >> >> Nezar. >> >> >> >> >> >> On 14 April 2016 at 16:49, ilya <ilya.mailing.li...@gmail.com> wrote: >> >> >> >>> It appears you have pending jobs that are not letting you go through. >> >>> >> >>> Usually stop and starting management service should be enough. >> >>> >> >>> However, try running this script from your management server to show >> us >> >>> what you have running in async_job table. >> >>> >> >>> >> >>> >> >>>> #!/bin/bash >> >>>> DATESTAMP=$(date +%m%d%y-%H%M%S) >> >>>> DBPASS=$(java -classpath >> >>> /usr/share/cloudstack-common/lib/jasypt-1.9.0.jar >> >>> org.jasypt.intf.cli.JasyptPBEStringDecryptionCLI input="$(cat >> >>> /etc/cloudstack/management/db.properties | grep db.cloud.password | >> awk >> >>> -F'(' '{print $2}' | sed 's/)//g')" password="$(cat >> >>> /etc/cloudstack/management/key)" | grep -A2 OUTPUT | tail -1) >> >>>> DBHOST=$(cat /etc/cloudstack/management/db.properties | grep >> >>> db.cloud.host | awk -F'=' '{print $2}' | tail -1 ) >> >>>> DBUSER=$(cat /etc/cloudstack/management/db.properties | grep >> >>> db.cloud.username | awk -F'=' '{print $2}') >> >>>> DB=$(cat /etc/cloudstack/management/db.properties | grep >> db.cloud.name >> >>> | awk -F'=' '{print $2}') >> >>>> DBPORT=$(cat /etc/cloudstack/management/db.properties | grep >> >>> db.cloud.port | awk -F'=' '{print $2}') >> >>>> MYSQLCMD="mysql -h $DBHOST -u $DBUSER -P $DBPORT -p$DBPASS $DB" >> >>>> echo 'SELECT id, user_id, account_id, instance_type, instance_id, >> >>> job_cmd, job_cmd_info, job_dispatcher, job_pending_signals FROM >> >>> cloud.async_job where job_status=0 and job_dispatcher not like >> >>> "pseudoJobDispatcher"' | $MYSQLCMD >> >>> >> >>> >> >>> >> >>> >> >>> On 4/13/16 5:16 PM, Nezar Madbouh wrote: >> >>>> Hi Simon, >> >>>> >> >>>> The following work was done but unfortunately the problem is not >> fixed: >> >>>> >> >>>> >> >>>> on management server: >> >>>> >> >>>> service cloudstack-management stop >> >>>> >> >>>> >> >>>> on the host: >> >>>> >> >>>> service cloudstack-agent stop >> >>>> >> >>>> service libvirtd restart >> >>>> >> >>>> service cloudstack-agent start >> >>>> >> >>>> >> >>>> on management server: >> >>>> >> >>>> service cloudstack-management start >> >>>> >> >>>> Regards, >> >>>> Nezar. >> >>>> >> >>>> On 24 March 2016 at 05:06, Simon Weller <swel...@ena.com> wrote: >> >>>> >> >>>>> Yes, I think that order is fine. Make sure the processes are indeed >> >>>>> shutdown before moving on to the next step. >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> ________________________________________ >> >>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>> Sent: Tuesday, March 22, 2016 5:33 PM >> >>>>> To: users@cloudstack.apache.org >> >>>>> Subject: Re: Unable to create a New VM - Error message: Unable to >> start >> >>>>> instance due to Unable to get answer that is of class >> >>>>> com.cloud.agent.api.StartAnswer >> >>>>> >> >>>>> Hi Simon, >> >>>>> >> >>>>> Can you please check if the following sequence of action is correct: >> >>>>> >> >>>>> on management server: >> >>>>> >> >>>>> service cloudstack-management stop >> >>>>> >> >>>>> >> >>>>> on the host: >> >>>>> >> >>>>> service cloudstack-agent stop >> >>>>> >> >>>>> service libvirtd restart >> >>>>> >> >>>>> service cloudstack-agent start >> >>>>> >> >>>>> >> >>>>> on management server: >> >>>>> >> >>>>> service cloudstack-management start >> >>>>> >> >>>>> Thanks very much for your help. >> >>>>> Nezar. >> >>>>> >> >>>>> On 23 March 2016 at 08:39, Simon Weller <swel...@ena.com> wrote: >> >>>>> >> >>>>>> Shutting down all the management services should clear the jobs. >> >>>>>> >> >>>>>> >> >>>>>> ________________________________________ >> >>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>> Sent: Tuesday, March 22, 2016 4:28 PM >> >>>>>> To: users@cloudstack.apache.org >> >>>>>> Subject: Re: Unable to create a New VM - Error message: Unable to >> >>> start >> >>>>>> instance due to Unable to get answer that is of class >> >>>>>> com.cloud.agent.api.StartAnswer >> >>>>>> >> >>>>>> Hi Simon, >> >>>>>> >> >>>>>> I will plan to restart agents, libvirtd and management services. >> >>>>>> >> >>>>>> How do I clear hung jobs. >> >>>>>> >> >>>>>> Your help is much appreciated, >> >>>>>> Nezar. >> >>>>>> >> >>>>>> On 22 March 2016 at 23:03, Simon Weller <swel...@ena.com> wrote: >> >>>>>> >> >>>>>>> Nezar, >> >>>>>>> >> >>>>>>> I've looked through the code related to your release and it seems >> >>> that >> >>>>> in >> >>>>>>> order to throw this exception, 10 attempts have been made to >> retrieve >> >>>>> the >> >>>>>>> disk information prior to the exception being thrown. >> >>>>>>> >> >>>>>>> It appears there is something really odd going on here. I'm >> starting >> >>>>> to >> >>>>>>> wonder whether it might be worth while for you to stop all of your >> >>>>>> agents, >> >>>>>>> plus management server processes, restart libvirtd on each of the >> >>> hosts >> >>>>>> and >> >>>>>>> then bring the agents back up prior to starting the management >> >>> servers. >> >>>>>>> >> >>>>>>> There are a bunch of logs the pastebin you posted that seems to >> >>>>> indicate >> >>>>>>> you have some jobs hung, for example: >> >>>>>>> >> >>>>>>> 2016-03-14 21:45:31,828 WARN [o.a.c.f.j.i.AsyncJobMonitor] >> >>>>>>> (Timer-1:ctx-0c090bd7) Task (job-9066) has been pending for >> 1660851 >> >>>>>> seconds >> >>>>>>> 2016-03-14 21:45:31,828 WARN [o.a.c.f.j.i.AsyncJobMonitor] >> >>>>>>> (Timer-1:ctx-0c090bd7) Task (job-8133) has been pending for >> 4253003 >> >>>>>> seconds >> >>>>>>> >> >>>>>>> Restarting the agents, libvirtd and management will not cause an >> >>>>> outage >> >>>>>>> to your cloud, but I would suggest, if you decided to do this, you >> >>>>> should >> >>>>>>> complete it in a maintenance window. >> >>>>>>> >> >>>>>>> - Si >> >>>>>>> >> >>>>>>> ________________________________________ >> >>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>> Sent: Sunday, March 20, 2016 10:27 PM >> >>>>>>> To: users@cloudstack.apache.org >> >>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable to >> >>> start >> >>>>>>> instance due to Unable to get answer that is of class >> >>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>> >> >>>>>>> Hi Simon, >> >>>>>>> >> >>>>>>> Please see answers to your questions below. I am not sure if I >> >>> provided >> >>>>>> the >> >>>>>>> correct answer for the primary and secondary storage >> configuration. >> >>>>>> Please >> >>>>>>> let me know if this is not what you are after: >> >>>>>>> >> >>>>>>> >> >>>>>>> Can you screenshot and post somewhere your secondary storage >> >>>>>> configuration >> >>>>>>> and your primary storage configuration from the UI with the >> details? >> >>>>>>> >> >>>>>>> mysql> select * from configuration where name like '%secondary%'\G >> >>>>>>> *************************** 1. row *************************** >> >>>>>>> category: Account Defaults >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: max.account.secondary.storage >> >>>>>>> value: 400 >> >>>>>>> description: The default maximum secondary storage space (in >> GiB) >> >>>>> that >> >>>>>>> can be used for an account >> >>>>>>> default_value: 400 >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 2. row *************************** >> >>>>>>> category: Project Defaults >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: max.project.secondary.storage >> >>>>>>> value: 400 >> >>>>>>> description: The default maximum secondary storage space (in >> GiB) >> >>>>> that >> >>>>>>> can be used for a project >> >>>>>>> default_value: 400 >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 3. row *************************** >> >>>>>>> category: Hidden >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: secondary.storage.vm >> >>>>>>> value: +rwRhJOXPymbQOuAMLnhiw== >> >>>>>>> description: Deploys a VM per zone to manage secondary storage >> if >> >>>>> true, >> >>>>>>> otherwise secondary storage is mounted on management server >> >>>>>>> default_value: NULL >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 4. row *************************** >> >>>>>>> category: Advanced >> >>>>>>> instance: DEFAULT >> >>>>>>> component: ExtensionRegistry >> >>>>>>> name: secondary.storage.vm.allocators.exclude >> >>>>>>> value: NULL >> >>>>>>> description: Extensions to exclude from being registered >> >>>>>>> default_value: NULL >> >>>>>>> updated: 2014-12-04 00:18:15 >> >>>>>>> scope: Global >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 5. row *************************** >> >>>>>>> category: NetworkManager >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: vm.network.nic.max.secondary.ipaddresses >> >>>>>>> value: NULL >> >>>>>>> description: Specify the number of secondary ip addresses per >> nic >> >>> per >> >>>>>> vm >> >>>>>>> default_value: 256 >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> 5 rows in set (0.00 sec) >> >>>>>>> >> >>>>>>> >> >>>>>>> mysql> select * from configuration where name like '%primary%'\G >> >>>>>>> *************************** 1. row *************************** >> >>>>>>> category: Account Defaults >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: max.account.primary.storage >> >>>>>>> value: 200 >> >>>>>>> description: The default maximum primary storage space (in GiB) >> >>> that >> >>>>>> can >> >>>>>>> be used for an account >> >>>>>>> default_value: 200 >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 2. row *************************** >> >>>>>>> category: Project Defaults >> >>>>>>> instance: DEFAULT >> >>>>>>> component: management-server >> >>>>>>> name: max.project.primary.storage >> >>>>>>> value: 200 >> >>>>>>> description: The default maximum primary storage space (in GiB) >> >>> that >> >>>>>> can >> >>>>>>> be used for a project >> >>>>>>> default_value: 200 >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> *************************** 3. row *************************** >> >>>>>>> category: Storage >> >>>>>>> instance: DEFAULT >> >>>>>>> component: TemplateManager >> >>>>>>> name: primary.storage.download.wait >> >>>>>>> value: 10800 >> >>>>>>> description: In second, timeout for download template to primary >> >>>>>> storage >> >>>>>>> default_value: NULL >> >>>>>>> updated: NULL >> >>>>>>> scope: NULL >> >>>>>>> is_dynamic: 0 >> >>>>>>> 3 rows in set (0.00 sec) >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> Also, what is the Cloudstack version you are running and what is >> the >> >>> OS >> >>>>>>> you're running it on? >> >>>>>>> >> >>>>>>> Cloudstack 4.4.1 >> >>>>>>> >> >>>>>>> Centos 6.6 >> >>>>>>> >> >>>>>>> Are you using actimeo=0 in your NFS mount line? >> >>>>>>> >> >>>>>>> Primary storage on Hypervisors has the following parameters >> >>>>>>> 10.61.154.10:/Cloudstack/primary on >> >>>>>>> /mnt/b3647480-5709-36eb-9ddd-e2c666f30b57 type nfs >> >>>>>>> >> (rw,noatime,nfsvers=3,proto=tcp,hard,nolock,intr,addr=10.61.154.10) >> >>>>>>> >> >>>>>>> >> >>>>>>> Secondary storage on SSVM has the following parameters >> >>>>>>> 10.61.154.10:/Cloudstack/secondary on >> >>>>>>> /mnt/SecStorage/d11d5c92-6950-30e8-a0b5-3cd46a17ec14 type nfs4 >> >>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> (rw,relatime,vers=4,rsize=262144,wsize=262144,namlen=255,acdirmin=0,acdirmax=0,soft,proto=tcp,port=0,timeo=133,retrans= >> >>> 2147483647 >> >>> >> ,sec=sys,clientaddr=10.61.154.70,minorversion=0,local_lock=none,addr=10.61.154.10) >> >>>>>>> >> >>>>>>> >> >>>>>>> Thanks & Regards, >> >>>>>>> Nezar. >> >>>>>>> >> >>>>>>> On 18 March 2016 at 23:50, Simon Weller <swel...@ena.com> wrote: >> >>>>>>> >> >>>>>>>> Also, what is the Cloudstack version you are running and what is >> the >> >>>>> OS >> >>>>>>>> you're running it on? >> >>>>>>>> >> >>>>>>>> Are you using actimeo=0 in your NFS mount line? >> >>>>>>>> >> >>>>>>>> ________________________________________ >> >>>>>>>> From: Simon Weller <swel...@ena.com> >> >>>>>>>> Sent: Friday, March 18, 2016 7:40 AM >> >>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable to >> >>>>> start >> >>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>> >> >>>>>>>> Nexar, >> >>>>>>>> >> >>>>>>>> That's normal behaviour. You can only ssh to the link local on >> the >> >>>>> host >> >>>>>>>> where the VM resides. >> >>>>>>>> >> >>>>>>>> Can you screenshot and post somewhere your secondary storage >> >>>>>>> configuration >> >>>>>>>> and your primary storage configuration from the UI with the >> details? >> >>>>>>>> >> >>>>>>>> - Si >> >>>>>>>> >> >>>>>>>> ________________________________________ >> >>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>> Sent: Thursday, March 17, 2016 8:19 PM >> >>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable to >> >>>>> start >> >>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>> >> >>>>>>>> Hi Simon, >> >>>>>>>> >> >>>>>>>> I just noticed that from other Hosts I can't ssh into the SSVM: >> >>>>>>>> >> >>>>>>>> [root@eqx-cs-cmp-01 ~]# ssh -i /root/.ssh/id_rsa.cloud -p 3922 >> >>>>>>>> root@169.254.3.161 >> >>>>>>>> ssh: connect to host 169.254.3.161 port 3922: No route to host >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> I can only ssh from one Host: >> >>>>>>>> [root@eqx-cs-cmp-06 ~]# ssh -i /root/.ssh/id_rsa.cloud -p 3922 >> >>>>>>>> root@169.254.3.161 >> >>>>>>>> Linux s-11-VM 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3 x86_64 >> >>>>>>>> >> >>>>>>>> Thanks & Regards, >> >>>>>>>> Nezar. >> >>>>>>>> >> >>>>>>>> On 18 March 2016 at 10:36, Nezar Madbouh < >> >>>>> nezar.madb...@ipscape.com.au >> >>>>>>> >> >>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> Hi Simon, >> >>>>>>>>> >> >>>>>>>>> Yes existing VMs are successfully running. Yesterday I was able >> to >> >>>>>>> create >> >>>>>>>>> a VM in Dev Environment but I can't create VMs in Prod >> environment. >> >>>>>>>>> >> >>>>>>>>> Thanks, >> >>>>>>>>> Nezar. >> >>>>>>>>> >> >>>>>>>>> On 18 March 2016 at 09:12, Simon Weller <swel...@ena.com> >> wrote: >> >>>>>>>>> >> >>>>>>>>>> I'm assuming you have other VMs successfully running, correct? >> So >> >>>>>> you >> >>>>>>>> can >> >>>>>>>>>> write to your primary storage. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> ________________________________________ >> >>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>> Sent: Thursday, March 17, 2016 4:43 PM >> >>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable >> to >> >>>>>>> start >> >>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>> >> >>>>>>>>>> Hi Simon and Shweta, >> >>>>>>>>>> >> >>>>>>>>>> Please see the result below: >> >>>>>>>>>> >> >>>>>>>>>> root@s-11-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh >> >>>>>>>>>> ================================================ >> >>>>>>>>>> First DNS server is 8.8.8.8 >> >>>>>>>>>> PING 8.8.8.8 (8.8.8.8): 48 data bytes >> >>>>>>>>>> 56 bytes from 8.8.8.8: icmp_seq=0 ttl=57 time=5.209 ms >> >>>>>>>>>> 56 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=0.936 ms >> >>>>>>>>>> --- 8.8.8.8 ping statistics --- >> >>>>>>>>>> 2 packets transmitted, 2 packets received, 0% packet loss >> >>>>>>>>>> round-trip min/avg/max/stddev = 0.936/3.072/5.209/2.137 ms >> >>>>>>>>>> Good: Can ping DNS server >> >>>>>>>>>> ================================================ >> >>>>>>>>>> Good: DNS resolves download.cloud.com >> >>>>>>>>>> ================================================ >> >>>>>>>>>> nfs is currently mounted >> >>>>>>>>>> Mount point is >> >>>>> /mnt/SecStorage/d11d5c92-6950-30e8-a0b5-3cd46a17ec14 >> >>>>>>>>>> Good: Can write to mount point >> >>>>>>>>>> ================================================ >> >>>>>>>>>> Management server is 10.61.153.20. Checking connectivity. >> >>>>>>>>>> Good: Can connect to management server port 8250 >> >>>>>>>>>> ================================================ >> >>>>>>>>>> Good: Java process is running >> >>>>>>>>>> ================================================ >> >>>>>>>>>> Tests Complete. Look for ERROR or WARNING above. >> >>>>>>>>>> root@s-11-VM:~# >> >>>>>>>>>> >> >>>>>>>>>> Thanks & Regards, >> >>>>>>>>>> Nezar. >> >>>>>>>>>> >> >>>>>>>>>> On 17 March 2016 at 09:42, Simon Weller <swel...@ena.com> >> wrote: >> >>>>>>>>>> >> >>>>>>>>>>> Nezar, >> >>>>>>>>>>> >> >>>>>>>>>>> Log into the Cloudstack UI. >> >>>>>>>>>>> >> >>>>>>>>>>> Click on "Infrastructure" on the left menu. Click on View All >> >>>>>> below >> >>>>>>>>>>> "System VMs". find the item marked "Secondary Storage VM" and >> >>>>>> then >> >>>>>>>>>> click >> >>>>>>>>>>> on the Name of that VM on the left (it will be s-XXXX-VM). >> >>>>> You'll >> >>>>>>> see >> >>>>>>>> a >> >>>>>>>>>>> bunch of information about the VM. Find the Host line and also >> >>>>> the >> >>>>>>>> Link >> >>>>>>>>>>> Local IP Address line and then follow the original >> instructions >> >>>>> in >> >>>>>>> my >> >>>>>>>>>>> previous email. >> >>>>>>>>>>> >> >>>>>>>>>>> - Si >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> ________________________________________ >> >>>>>>>>>>> From: Shweta Agarwal <shweta.agarw...@accelerite.com> >> >>>>>>>>>>> Sent: Wednesday, March 16, 2016 12:16 AM >> >>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable >> >>>>> to >> >>>>>>>> start >> >>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>> >> >>>>>>>>>>> Hi Nezar, >> >>>>>>>>>>> >> >>>>>>>>>>> You can find host name on detail view of SSVM in UI . >> >>>>>>>>>>> You can also use list api command listSystemVms to get the >> host >> >>>>>>>> details >> >>>>>>>>>> of >> >>>>>>>>>>> system vms. >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks >> >>>>>>>>>>> Shweta >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> ________________________________________ >> >>>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>>> Sent: Wednesday, March 16, 2016 9:58 AM >> >>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: Unable >> >>>>> to >> >>>>>>>> start >> >>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>> >> >>>>>>>>>>> Hi Simon, >> >>>>>>>>>>> >> >>>>>>>>>>> How can I find out which host the SSVM is on? >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks & Regards, >> >>>>>>>>>>> Nezar. >> >>>>>>>>>>> >> >>>>>>>>>>> On 16 March 2016 at 12:28, Simon Weller <swel...@ena.com> >> >>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> Can you ssh into the Secondary Storage VM? >> >>>>>>>>>>>> >> >>>>>>>>>>>> This can be done by getting the link-local-ip address from >> the >> >>>>>>>>>>>> infrastructure menu in the UI and then sshing into the host >> >>>>> the >> >>>>>>> SSVM >> >>>>>>>>>> is >> >>>>>>>>>>> on >> >>>>>>>>>>>> and issuing this command as root: ssh -i >> >>>>> /root/.ssh/id_rsa.cloud >> >>>>>>> -p >> >>>>>>>>>> 3922 >> >>>>>>>>>>>> root@link-local-ip (replace link-local-ip with the link >> >>>>> local >> >>>>>>> ip). >> >>>>>>>>>>>> >> >>>>>>>>>>>> Try running the SSVM diangnostic script: >> >>>>>>>>>>>> usr/local/cloud/systemvm/ssvm-check.sh >> >>>>>>>>>>>> >> >>>>>>>>>>>> - Si >> >>>>>>>>>>>> ________________________________________ >> >>>>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>>>> Sent: Tuesday, March 15, 2016 7:49 PM >> >>>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: >> Unable >> >>>>>> to >> >>>>>>>>>> start >> >>>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>>> >> >>>>>>>>>>>> Hi Simon, >> >>>>>>>>>>>> >> >>>>>>>>>>>> Please see details below: >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> mysql> use cloud; select id, name, status, type from host >> >>>>> where >> >>>>>>> id = >> >>>>>>>>>> 5; >> >>>>>>>>>>>> Reading table information for completion of table and column >> >>>>>> names >> >>>>>>>>>>>> You can turn off this feature to get a quicker startup with >> -A >> >>>>>>>>>>>> >> >>>>>>>>>>>> Database changed >> >>>>>>>>>>>> +----+------------------------------+--------+---------+ >> >>>>>>>>>>>> | id | name | status | type | >> >>>>>>>>>>>> +----+------------------------------+--------+---------+ >> >>>>>>>>>>>> | 5 | eqx-cs-cmp-02.ipscape.com.au | up | routing | >> >>>>>>>>>>>> +----+------------------------------+--------+---------+ >> >>>>>>>>>>>> 1 row in set (0.00 sec) >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> cat /var/log/cloudstack/agent/agent.log >> >>>>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: >> >>>>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Can't find >> >>>>>>>>>>>> volume:089ff3ca-9bff-402c-a14e-ea7a35aaad5c >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:262) >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVbd(LibvirtComputingResource.java:3921) >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3789) >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1333) >> >>>>>>>>>>>> at >> >>>>> com.cloud.agent.Agent.processRequest(Agent.java:501) >> >>>>>>>>>>>> at >> >>>>>>>>>>> >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:808) >> >>>>>>>>>>>> at com.cloud.utils.nio.Task.run(Task.java:84) >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >>>>>>>>>>>> at >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >> >>>>>>>>>>>> agent.log.2016-03-03.gz (END) >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> [root@eqx-cs-cmp-02 agent]# virsh pool-list >> >>>>>>>>>>>> Name State Autostart >> >>>>>>>>>>>> ----------------------------------------- >> >>>>>>>>>>>> 2d0967f8-50d6-4d6d-87b7-4da0596f6924 active no >> >>>>>>>>>>>> b3647480-5709-36eb-9ddd-e2c666f30b57 active no >> >>>>>>>>>>>> e3831246-ff55-4af4-9964-a17565edaccc active no >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> [Root @ eqx-cs-cmp-02 agent] # df -h >> >>>>>>>>>>>> Filesystem Size Used Avail Use% Mounted on >> >>>>>>>>>>>> /dev/mapper/vg_root-lv_root >> >>>>>>>>>>>> 203G 215G 1.3G 1% / >> >>>>>>>>>>>> tmpfs 48g 48g 0 0% / dev / shm >> >>>>>>>>>>>> / Dev / sda1 477m 427m 25M 6% / boot >> >>>>>>>>>>>> xx.xx.xx.xx:/Cloudstack/primary >> >>>>>>>>>>>> 7.6T 4.1T 3.5T 54% >> >>>>>>>>>>>> /mnt/b3647480-5709-36eb-9ddd-e2c666f30b57 >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> I also noticed the following in cloudstack-agent.err >> >>>>>>>>>>>> >> >>>>>>>>>>>> libvirt: QEMU Driver error : Domain not found: no domain with >> >>>>>>>> matching >> >>>>>>>>>>> name >> >>>>>>>>>>>> 'i-11-600-VM' >> >>>>>>>>>>>> libvirt: Storage Driver error : Storage volume not found: no >> >>>>>>> storage >> >>>>>>>>>> vol >> >>>>>>>>>>>> with matching name '089ff3ca-9bff-402c-a14e-ea7a35aaad5c' >> >>>>>>>>>>>> >> >>>>>>>>>>>> Your help is much appreciated, >> >>>>>>>>>>>> Nezar >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> On 16 March 2016 at 11:03, Simon Weller <swel...@ena.com> >> >>>>>> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> Nezar, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> ok, I just took a look at a pastebin you posted. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I see it complaining about the inability to find a volume >> >>>>>> prior >> >>>>>>> to >> >>>>>>>>>>>>> throwing the host exception. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Firstly, let's find out who host 5 is. Can you log into your >> >>>>>>> MySQL >> >>>>>>>>>>> server >> >>>>>>>>>>>>> and run this: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> use cloud;select id,name,status,type from host where id=5; >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Can you post the results? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> If it's a host, can you post the >> >>>>>>>> /var/log/cloudstack/agent/agent.log >> >>>>>>>>>>> from >> >>>>>>>>>>>>> that host? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> And also run virsh pool-list on the host in question? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> - Si >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> ________________________________________ >> >>>>>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>>>>> Sent: Tuesday, March 15, 2016 6:11 PM >> >>>>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: >> >>>>> Unable >> >>>>>>> to >> >>>>>>>>>> start >> >>>>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Hi Simon, >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> When you say "double check your routers tied to the >> >>>>> network", >> >>>>>> do >> >>>>>>>> you >> >>>>>>>>>>> mean >> >>>>>>>>>>>>> the physical routers or the virtual routers? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> When I try to create a VM in Prod Environment it fails. I >> >>>>>> can't >> >>>>>>>> ping >> >>>>>>>>>>> the >> >>>>>>>>>>>>> Prod virtual router from management server or Host. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> But I can ping Dev virtual router from management server and >> >>>>>>> Host >> >>>>>>>>>> and I >> >>>>>>>>>>>> was >> >>>>>>>>>>>>> able to create a VM in Dev Environment. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> How can I troubleshoot the Prod Virtual router. I launched >> >>>>> the >> >>>>>>>>>> console >> >>>>>>>>>>>> but >> >>>>>>>>>>>>> couldn't login. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Please note that we need to take into consideration that the >> >>>>>>>>>> affected >> >>>>>>>>>>>>> network is the Production network and we need to minimise >> >>>>>>> service >> >>>>>>>>>>> outage >> >>>>>>>>>>>> as >> >>>>>>>>>>>>> much as possible. >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Your help is much appreciated, >> >>>>>>>>>>>>> Nezar >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On 16 March 2016 at 08:37, Simon Weller <swel...@ena.com> >> >>>>>>> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> I've seen this before where a Virtual Router has crashed >> >>>>> and >> >>>>>>> the >> >>>>>>>>>>>>>> management server can't communicate with it. >> >>>>>>>>>>>>>> Are you using an advanced network setup? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> If so, double check your routers tied to the network where >> >>>>>> you >> >>>>>>>> are >> >>>>>>>>>>>>>> attempting to start an instance. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> - Si >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> ________________________________________ >> >>>>>>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>>>>>> Sent: Tuesday, March 15, 2016 4:02 PM >> >>>>>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: >> >>>>>> Unable >> >>>>>>>> to >> >>>>>>>>>>> start >> >>>>>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Hi Glenn, >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> We have 16 hosts, all KVM. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> The error is not specific to Host 5. Every time I try to >> >>>>>>> create >> >>>>>>>> a >> >>>>>>>>>> VM >> >>>>>>>>>>> it >> >>>>>>>>>>>>>> gives me the same error message but on different host. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> How do I restart the agent on the host. Is it service >> >>>>>>> affecting? >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Thanks very much for your help. >> >>>>>>>>>>>>>> Neza >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On 15 March 2016 at 17:37, Glenn Wagner < >> >>>>>>>>>> glenn.wag...@shapeblue.com> >> >>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Hi, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Can you explain your environment first , how many hosts >> >>>>> , >> >>>>>>> what >> >>>>>>>>>> is >> >>>>>>>>>>>> your >> >>>>>>>>>>>>>>> hypervisors (KVM,Xenserver,VMware) >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> The error seems to be with your Host 5 Hypervisor >> >>>>>>>>>>>>>>> Resource [Host:5] is unreachable: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Go to the infrastructure section and see what host5 is >> >>>>>>>>>> reporting, >> >>>>>>>>>>>>>>> Have you tried to restart the agent on the Host? >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Thanks >> >>>>>>>>>>>>>>> Glenn >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> [image: ShapeBlue] <http://www.shapeblue.com> >> >>>>>>>>>>>>>>> Glenn Wagner >> >>>>>>>>>>>>>>> Senior Consultant , ShapeBlue >> >>>>>>>>>>>>>>> d: * | s: +27 21 527 0091* >> >>>>>>>>>> <%7C%20s:%20+27%2021%20527%200091> | >> >>>>>>>>>>>> m: >> >>>>>>>>>>>>>>> *+27 73 917 4111* <+27%2073%20917%204111> >> >>>>>>>>>>>>>>> e: *glenn.wag...@shapeblue.com | t: * >> >>>>>>>>>>>>>>> <glenn.wag...@shapeblue.com%20%7C%20t:> | w: * >> >>>>>>>>>> www.shapeblue.com >> >>>>>>>>>>> * >> >>>>>>>>>>>>>>> <http://www.shapeblue.com> >> >>>>>>>>>>>>>>> a: >> >>>>>>>>>>>>>>> 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West >> >>>>>> Cape >> >>>>>>>> Town >> >>>>>>>>>>> 7130 >> >>>>>>>>>>>>>> South Africa >> >>>>>>>>>>>>>>> Shape Blue Ltd is a company incorporated in England & >> >>>>>> Wales. >> >>>>>>>>>>>> ShapeBlue >> >>>>>>>>>>>>>>> Services India LLP is a company incorporated in India >> >>>>> and >> >>>>>> is >> >>>>>>>>>>> operated >> >>>>>>>>>>>>>> under >> >>>>>>>>>>>>>>> license from Shape Blue Ltd. Shape Blue Brasil >> >>>>> Consultoria >> >>>>>>>> Ltda >> >>>>>>>>>> is >> >>>>>>>>>>> a >> >>>>>>>>>>>>>>> company incorporated in Brasil and is operated under >> >>>>>> license >> >>>>>>>>>> from >> >>>>>>>>>>>> Shape >> >>>>>>>>>>>>>>> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered >> >>>>> by >> >>>>>>> The >> >>>>>>>>>>>> Republic >> >>>>>>>>>>>>> of >> >>>>>>>>>>>>>>> South Africa and is traded under license from Shape Blue >> >>>>>>> Ltd. >> >>>>>>>>>>>> ShapeBlue >> >>>>>>>>>>>>>> is >> >>>>>>>>>>>>>>> a registered trademark. >> >>>>>>>>>>>>>>> This email and any attachments to it may be confidential >> >>>>>> and >> >>>>>>>> are >> >>>>>>>>>>>>> intended >> >>>>>>>>>>>>>>> solely for the use of the individual to whom it is >> >>>>>>> addressed. >> >>>>>>>>>> Any >> >>>>>>>>>>>> views >> >>>>>>>>>>>>>> or >> >>>>>>>>>>>>>>> opinions expressed are solely those of the author and do >> >>>>>> not >> >>>>>>>>>>>>> necessarily >> >>>>>>>>>>>>>>> represent those of Shape Blue Ltd or related companies. >> >>>>> If >> >>>>>>> you >> >>>>>>>>>> are >> >>>>>>>>>>>> not >> >>>>>>>>>>>>>> the >> >>>>>>>>>>>>>>> intended recipient of this email, you must neither take >> >>>>>> any >> >>>>>>>>>> action >> >>>>>>>>>>>>> based >> >>>>>>>>>>>>>>> upon its contents, nor copy or show it to anyone. Please >> >>>>>>>> contact >> >>>>>>>>>>> the >> >>>>>>>>>>>>>> sender >> >>>>>>>>>>>>>>> if you believe you have received this email in error. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> -----Original Message----- >> >>>>>>>>>>>>>>> From: Nezar Madbouh [mailto: >> >>>>> nezar.madb...@ipscape.com.au] >> >>>>>>>>>>>>>>> Sent: Tuesday, 15 March 2016 5:45 AM >> >>>>>>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>>>>>> Subject: Re: Unable to create a New VM - Error message: >> >>>>>>> Unable >> >>>>>>>>>> to >> >>>>>>>>>>>> start >> >>>>>>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Hi Jeremy, >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I have uploaded the contents to >> >>>>>>> http://pastebin.com/1J3mmgTh >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> your help is much appreciated, >> >>>>>>>>>>>>>>> Nezar. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On 15 March 2016 at 13:46, Jeremy Peterson < >> >>>>>>>>>> jpeter...@acentek.net> >> >>>>>>>>>>>>>> wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> I don't see any attachments can you create a pastebin. >> >>>>>>>> That's >> >>>>>>>>>>>> easier >> >>>>>>>>>>>>>>>> to look through as I think attachments are removed >> >>>>>> before >> >>>>>>>>>> being >> >>>>>>>>>>>> sent >> >>>>>>>>>>>>>>>> to user group. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Jeremy >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> -------- Original message -------- >> >>>>>>>>>>>>>>>> From: Nezar Madbouh <nezar.madb...@ipscape.com.au> >> >>>>>>>>>>>>>>>> Date: 3/14/2016 6:38 PM (GMT-06:00) >> >>>>>>>>>>>>>>>> To: users@cloudstack.apache.org >> >>>>>>>>>>>>>>>> Subject: Unable to create a New VM - Error message: >> >>>>>> Unable >> >>>>>>>> to >> >>>>>>>>>>> start >> >>>>>>>>>>>>>>>> instance due to Unable to get answer that is of class >> >>>>>>>>>>>>>>>> com.cloud.agent.api.StartAnswer >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Hi Shweta and Sadhu, >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Please find attached managerment-server.log which >> >>>>>> contains >> >>>>>>>> the >> >>>>>>>>>>> logs >> >>>>>>>>>>>>>>>> since I started creating a new VM (VM id 600). >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Please note that we don't have agent.log files >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Your help is much appreciated. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Regards, >> >>>>>>>>>>>>>>>> Nezar. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Find out more about ShapeBlue and our range of >> >>>>> CloudStack >> >>>>>>>>>> related >> >>>>>>>>>>>>>> services: >> >>>>>>>>>>>>>>> IaaS Cloud Design & Build >> >>>>>>>>>>>>>>> <http://shapeblue.com/iaas-cloud-design-and-build//> | >> >>>>>>>> CSForge >> >>>>>>>>>> - >> >>>>>>>>>>>> rapid >> >>>>>>>>>>>>>>> IaaS deployment framework < >> >>>>> http://shapeblue.com/csforge/> >> >>>>>>>>>>>>>>> CloudStack Consulting < >> >>>>>>>>>>> http://shapeblue.com/cloudstack-consultancy/> >> >>>>>>>>>>>> | >> >>>>>>>>>>>>>> CloudStack >> >>>>>>>>>>>>>>> Software Engineering >> >>>>>>>>>>>>>>> <http://shapeblue.com/cloudstack-software-engineering/> >> >>>>>>>>>>>>>>> CloudStack Infrastructure Support >> >>>>>>>>>>>>>>> < >> >>>>> http://shapeblue.com/cloudstack-infrastructure-support/> >> >>>>>> | >> >>>>>>>>>>>> CloudStack >> >>>>>>>>>>>>>>> Bootcamp Training Courses < >> >>>>>>>>>>> http://shapeblue.com/cloudstack-training/ >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> DISCLAIMER >> >>>>>>>>>>> ========== >> >>>>>>>>>>> This e-mail may contain privileged and confidential >> information >> >>>>>>> which >> >>>>>>>> is >> >>>>>>>>>>> the property of Accelerite, a Persistent Systems business. It >> is >> >>>>>>>>>> intended >> >>>>>>>>>>> only for the use of the individual or entity to which it is >> >>>>>>> addressed. >> >>>>>>>>>> If >> >>>>>>>>>>> you are not the intended recipient, you are not authorized to >> >>>>>> read, >> >>>>>>>>>> retain, >> >>>>>>>>>>> copy, print, distribute or use this message. If you have >> >>>>> received >> >>>>>>> this >> >>>>>>>>>>> communication in error, please notify the sender and delete >> all >> >>>>>>> copies >> >>>>>>>>>> of >> >>>>>>>>>>> this message. Accelerite, a Persistent Systems business does >> not >> >>>>>>>> accept >> >>>>>>>>>> any >> >>>>>>>>>>> liability for virus infected mails. >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> >> >> > >> >> >
