Hi Kirk, I did sign into the ssvm and got the connection refused as expected when wget to management server. However when trying to go to the other server outside of my basic network, wget re-tried several times until it was successful. The ssvm has 4 interfaces on it 3 in the basic network (1 or 2 of those was in the guest network range the other(s) in the system vm range) the 4th one was the link local.
>From looking at netstat while the retries where going on it looked like it was trying to go out the 3 different interfaces on my basic network until it found one that worked. My setup is currently down so can't check now. It is very possible that my host network setup is not correct. I currently have 2 network bonds (with 2 nic each) and both bridged to xenbr0 and 1. xenbr1 is dedicated to my primary storage (different network switch). Xen does also show xenbr2 and 3. Do I need to delete those? I have done all the network setup from /etc/network/interfaces and have not touched anything through brctl or any xe command. Is that my problem? thanks, Carlos On Fri, Mar 8, 2013 at 3:55 PM, Kirk Kosinski <kirkkosin...@gmail.com>wrote: > Reinstalling on CentOS is unlikely to help. The SSVM downloads > templates, not the management server, and the SSVM is the same > regardless of the management server OS. As Ahmad said, if the web > server with the template is on an internal server, you need to add its > CIDR to secstorage.allowed.internal.sites in Global Settings. Otherwise > iptables on the SSVM will block access, hence the connection refused > error. I think this setting only needs a management server restart, but > if you still get connection refused, also try a stop/start of the SSVM. > > For templates on external servers, or if you still get errors with > internal servers after fixing the configuration, you should SSH to the > SSVM and try to wget the template. It will probably fail, just like > when CloudStack tries, so troubleshoot why it fails. There may be a > problem with the hypervisor networking, or there may be something wrong > with a proxy, router, or firewall on the network. > > Best regards, > Kirk > > On 03/08/2013 02:10 PM, Carlos Reategui wrote: > > That did not seem to make a difference. > > > > If the www is on the MS on the basic network cidr I get Connection > refused > > > > If the www is in a network outside of my basic network it times out. > > > > I am going to try compiling 4.1 and if that fails I'll give up on ubuntu > > 12.04 + xcp-xapi and try XS 6.0.2 > > > > > > On Fri, Mar 8, 2013 at 9:34 AM, Ahmad Emneina <aemne...@gmail.com> > wrote: > > > >> Is the www you are trying to download ISOs from internal/private? If so > >> there's another global setting allowed.sec.storage ( or something > similar ) > >> to add an exception for your www servers address. > >> > >> Ahmad > >> > >> On Mar 8, 2013, at 2:31 AM, Carlos Reategui <car...@reategui.com> > wrote: > >> > >>> Host parameter is correct. > >>> > >>> I wiped everything out and re-installed host and management server. I > >> made > >>> sure follow installation guide and these are my findings: > >>> 1) I tried again with 4.0.1 from CS repo and it System VMs would not > >> start > >>> so I can confirm that CLOUDSTACK-587 is required for Ubuntu 12.04 + > >>> xcp-xapi hosts > >>> 2) Went back to my 4.0.1+CS587 and I am more or less back to where I > was > >>> before. Except this time I forgot to set the > system.vm.use.local.storage > >>> to true, but the System VMs did start from NFS primary storage so > >> something > >>> must have been screwed up in my previous install. > >>> > >>> Current status: > >>> 1) centos template says 2% downloaded for last 30-45mins. with no > >> apparent > >>> progress. Where is it trying to get this from? > >>> 2) Still unable to register an iso. I logged into the ssvm and tried > to > >> do > >>> a wget from there and that failed with connection refused when trying > to > >>> wget from a www on the management server. I then tried from a different > >>> server and that succeeded after a few retries until it hit an interface > >>> that worked. However all this retrying take a while and when doing it > >> from > >>> the UI it times out. How do I increase this timeout? Why is outbound > >>> traffic not working on all the interfaces with IPs on my basic network? > >>> Should it be able to fetch from a www in the same bsaic network? > >>> > >>> > >>> > >>> On Fri, Mar 8, 2013 at 1:23 AM, Sebastien Goasguen <run...@gmail.com> > >> wrote: > >>> > >>>> just a hunch, but in global settings there is a 'host' variable. > >>>> > >>>> make sure it points to the ip of your mgt server. > >>>> > >>>> restart mgt server and see what happens > >>>> > >>>> Ps: I swear it's not black magic :) > >>>> > >>>> On Mar 8, 2013, at 2:44 AM, Carlos Reategui <car...@reategui.com> > >> wrote: > >>>> > >>>>> Unfortunately my woes are not over. The ssvm is up but I seem to > have > >> a > >>>>> problem with secondary storage. > >>>>> > >>>>> I keep seeing these messages even though ssvm is up: > >>>>> 2013-03-07 19:10:10,635 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) > >> Zone > >>>> 1 > >>>>> is ready to launch secondary storage VM > >>>>> > >>>>> Also in the templates I see the centos template but it says it is not > >>>> ready > >>>>> so I cant start a VM in the logs I see (greping for [storage.): > >>>>> 2013-03-07 19:39:10,634 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) > >> Zone > >>>> 1 > >>>>> is ready to launch secondary storage VM > >>>>> 2013-03-07 19:39:16,880 INFO > >>>> [storage.secondary.SecondaryStorageListener] > >>>>> (AgentManager-Handler-14:null) Received a host startup notification > >>>>> com.cloud.agent.api.StartupSecondaryStorageCommand > >>>>> 2013-03-07 19:39:18,319 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] > >>>>> (AgentManager-Handler-14:null) Successfully programmed secondary > >> storage > >>>>> nfs://172.30.45.2/store/secondary in secondary storage VM s-1-VM > >>>>> 2013-03-07 19:39:18,405 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] > >>>>> (AgentManager-Handler-14:null) Successfully programmed firewall rules > >>>> into > >>>>> s-1-VM > >>>>> 2013-03-07 19:39:18,513 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] > >>>>> (AgentManager-Handler-14:null) Successfully programmed http auth into > >>>> s-1-VM > >>>>> 2013-03-07 19:39:18,722 INFO [storage.download.DownloadMonitorImpl] > >>>>> (AgentManager-Handler-14:null) Template Sync found routing-1 already > in > >>>> the > >>>>> template host table > >>>>> 2013-03-07 19:39:18,728 INFO [storage.download.DownloadMonitorImpl] > >>>>> (AgentManager-Handler-14:null) Template Sync did not find > >> centos55-x86_64 > >>>>> ready on server 2, will request download to start/resume shortly > >>>>> 2013-03-07 19:39:18,729 INFO [storage.download.DownloadMonitorImpl] > >>>>> (AgentManager-Handler-14:null) Template Sync did not find > >>>>> centos56-x86_64-xen ready on server 2, will request download to > >>>>> start/resume shortly > >>>>> 2013-03-07 19:39:18,731 INFO [storage.download.DownloadMonitorImpl] > >>>>> (AgentManager-Handler-14:null) Template Sync did not find > centos53-x64 > >>>>> ready on server 2, will request download to start/resume shortly > >>>>> 2013-03-07 19:39:18,741 DEBUG [storage.download.DownloadMonitorImpl] > >>>>> (AgentManager-Handler-14:null) Template CentOS 5.6(64-bit) no GUI > >>>>> (XenServer) needs to be downloaded to nfs:// > >> 172.30.45.2/store/secondary > >>>>> 2013-03-07 19:39:40,636 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) > >> Zone > >>>> 1 > >>>>> is ready to launch secondary storage VM > >>>>> 2013-03-07 19:39:48,751 WARN [storage.download.DownloadListener] > >>>>> (Timer-5:null) Entering download error state: timeout waiting for > >>>> response > >>>>> from storage host, template=CentOS 5.6(64-bit) no GUI (XenServer) at > >> host > >>>>> nfs://172.30.45.2/store/secondary > >>>>> 2013-03-07 19:40:10,635 DEBUG > >>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) > >> Zone > >>>> 1 > >>>>> is ready to launch secondary storage VM > >>>>> 2 > >>>>> > >>>>> When trying to register an iso I see connection refused error in the > >> logs > >>>>> even though I don't see any entries in my web logs of attempting to > >>>>> download the file (wget from the management server and the host > works > >>>>> fine): > >>>>> 2013-03-07 19:15:13,772 DEBUG [cloud.api.ApiServlet] > >>>>> (catalina-exec-23:null) ===START=== 172.30.61.63 -- GET > >>>> > >> > command=registerIso&name=Ubuntu%2012.04%20server%2064bit&displayText=Ubuntu%2012.04%20LTS&url=http%3A%2F%2F172.30.45.2%2Fisos%2Fubuntu-12.04.1-server-amd64.iso&zoneid=-1&isextractable=false&bootable=true&osTypeId=8609a566-0fa8-41a0-9d8f-3bf6d70faefa&ispublic=false&isfeatured=false&response=json&sessionkey=RuCxKbUIJNJYVENXDQ28A9oCIgE%3D&_=1362712512719 > >>>>> 2013-03-07 19:15:13,831 DEBUG [agent.transport.Request] > >>>>> (catalina-exec-23:null) Seq 4-17236017: Sending { Cmd , MgmtId: > >>>>> 233845174730255, via: 4, Ver: v1, Flags: 100011, > >>>>> [{"storage.DownloadCommand":{"hvm":true,"description":"Ubuntu 12.04 > >>>> > >> > LTS","maxDownloadSizeInBytes":53687091200,"id":201,"resourceType":"TEMPLATE","url":" > >>>>> http://172.30.45.2/isos/ubuntu-12.04.1-server-amd64.iso > >>>> > >> > ","format":"ISO","accountId":2,"name":"201-2-597c145a-1f38-355e-9225-943a184a1f8d","secUrl":"nfs:// > >>>>> 172.30.45.2/store/secondary","wait":0}}] } > >>>>> 2013-03-07 19:15:13,867 DEBUG [cloud.api.ApiServlet] > >>>>> (catalina-exec-23:null) ===END=== 172.30.61.63 -- GET > >>>> > >> > command=registerIso&name=Ubuntu%2012.04%20server%2064bit&displayText=Ubuntu%2012.04%20LTS&url=http%3A%2F%2F172.30.45.2%2Fisos%2Fubuntu-12.04.1-server-amd64.iso&zoneid=-1&isextractable=false&bootable=true&osTypeId=8609a566-0fa8-41a0-9d8f-3bf6d70faefa&ispublic=false&isfeatured=false&response=json&sessionkey=RuCxKbUIJNJYVENXDQ28A9oCIgE%3D&_=1362712512719 > >>>>> 2013-03-07 19:15:16,118 DEBUG [agent.transport.Request] > (Timer-3:null) > >>>> Seq > >>>>> 4-17236018: Sending { Cmd , MgmtId: 233845174730255, via: 4, Ver: > v1, > >>>>> Flags: 100011, > >>>> > >> > [{"storage.DownloadProgressCommand":{"jobId":"c1749be1-5c30-4a59-8014-bfeec0dea76a","request":"GET_STATUS","hvm":false,"description":"CentOS > >>>>> 5.6(64-bit) no GUI > >>>> > >> > (XenServer)","checksum":"905cec879afd9c9d22ecc8036131a180","maxDownloadSizeInBytes":53687091200,"id":5,"resourceType":"TEMPLATE","url":" > >>>>> http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 > >>>> > >> > ","format":"VHD","accountId":1,"name":"centos56-x86_64-xen","secUrl":"nfs:// > >>>>> 172.30.45.2/store/secondary","wait":0}}] } > >>>>> 2013-03-07 19:15:16,124 DEBUG [agent.transport.Request] > >>>>> (AgentManager-Handler-14:null) Seq 4-17236018: Processing: { Ans: , > >>>>> MgmtId: 233845174730255, via: 4, Ver: v1, Flags: 10, > >>>> > >> > [{"storage.DownloadAnswer":{"jobId":"c1749be1-5c30-4a59-8014-bfeec0dea76a","downloadPct":2,"errorString":" > >>>> > >> > ","downloadStatus":"DOWNLOAD_IN_PROGRESS","downloadPath":"/mnt/SecStorage/c536fc4b-e34d-34f5-8a5a-fc87ad852fd2/template/tmpl/1/5/dnld8635841906619140940tmp_","templateSize":0,"templatePhySicalSize":0,"checkSum":"905cec879afd9c9d22ecc8036131a180","result":false,"details":" > >>>>> ","wait":0}}] } > >>>>> 2013-03-07 19:15:16,925 DEBUG [agent.transport.Request] > >>>>> (AgentManager-Handler-15:null) Seq 4-17236017: Processing: { Ans: , > >>>>> MgmtId: 233845174730255, via: 4, Ver: v1, Flags: 10, > >>>> > >> > [{"storage.DownloadAnswer":{"jobId":"0c9055a8-b23f-49c5-82dd-5c7c00de1629","downloadPct":0,"errorString":" > >>>> > >> > ","downloadStatus":"NOT_DOWNLOADED","downloadPath":"/mnt/SecStorage/c536fc4b-e34d-34f5-8a5a-fc87ad852fd2/template/tmpl/2/201/dnld7244558075029424555tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"details":" > >>>>> ","wait":0}}] } > >>>>> 2013-03-07 19:15:18,460 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-98:null) Ping from 1 > >>>>> 2013-03-07 19:15:19,170 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-138:null) Seq 1-81068034: Executing request > >>>>> 2013-03-07 19:15:19,496 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-138:null) Seq 1-81068034: Response Received: > >>>>> 2013-03-07 19:15:19,496 DEBUG [agent.transport.Request] > >>>>> (DirectAgent-138:null) Seq 1-81068034: Processing: { Ans: , MgmtId: > >>>>> 233845174730255, via: 1, Ver: v1, Flags: 10, > >>>> > >> > [{"ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false,"result":true,"wait":0}}] > >>>>> } > >>>>> 2013-03-07 19:15:21,231 DEBUG [cloud.server.StatsCollector] > >>>>> (StatsCollector-1:null) StorageCollector is running... > >>>>> 2013-03-07 19:15:21,291 DEBUG [agent.transport.Request] > >>>>> (StatsCollector-1:null) Seq 4-17236019: Received: { Ans: , MgmtId: > >>>>> 233845174730255, via: 4, Ver: v1, Flags: 10, { GetStorageStatsAnswer > } > >> } > >>>>> 2013-03-07 19:15:21,298 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-30:null) Seq 1-81068109: Executing request > >>>>> 2013-03-07 19:15:21,825 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-30:null) Seq 1-81068109: Response Received: > >>>>> 2013-03-07 19:15:21,826 DEBUG [agent.transport.Request] > >>>>> (StatsCollector-1:null) Seq 1-81068109: Received: { Ans: , MgmtId: > >>>>> 233845174730255, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer > } > >> } > >>>>> 2013-03-07 19:15:21,832 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-19:null) Seq 1-81068110: Executing request > >>>>> 2013-03-07 19:15:22,343 DEBUG [agent.manager.DirectAgentAttache] > >>>>> (DirectAgent-19:null) Seq 1-81068110: Response Received: > >>>>> 2013-03-07 19:15:22,343 DEBUG [agent.transport.Request] > >>>>> (StatsCollector-1:null) Seq 1-81068110: Received: { Ans: , MgmtId: > >>>>> 233845174730255, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer > } > >> } > >>>>> 2013-03-07 19:15:25,595 DEBUG [cloud.server.StatsCollector] > >>>>> (StatsCollector-3:null) VmStatsCollector is running... > >>>>> 2013-03-07 19:15:26,127 DEBUG [agent.transport.Request] > (Timer-3:null) > >>>> Seq > >>>>> 4-17236020: Sending { Cmd , MgmtId: 233845174730255, via: 4, Ver: > v1, > >>>>> Flags: 100011, > >>>> > >> > [{"storage.DownloadProgressCommand":{"jobId":"c1749be1-5c30-4a59-8014-bfeec0dea76a","request":"GET_STATUS","hvm":false,"description":"CentOS > >>>>> 5.6(64-bit) no GUI > >>>> > >> > (XenServer)","checksum":"905cec879afd9c9d22ecc8036131a180","maxDownloadSizeInBytes":53687091200,"id":5,"resourceType":"TEMPLATE","url":" > >>>>> http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 > >>>> > >> > ","format":"VHD","accountId":1,"name":"centos56-x86_64-xen","secUrl":"nfs:// > >>>>> 172.30.45.2/store/secondary","wait":0}}] } > >>>>> 2013-03-07 19:15:26,169 DEBUG [agent.transport.Request] > >>>>> (AgentManager-Handler-4:null) Seq 4-17236020: Processing: { Ans: , > >>>> MgmtId: > >>>>> 233845174730255, via: 4, Ver: v1, Flags: 10, > >>>> > >> > [{"storage.DownloadAnswer":{"jobId":"c1749be1-5c30-4a59-8014-bfeec0dea76a","downloadPct":2,"errorString":" > >>>> > >> > ","downloadStatus":"DOWNLOAD_IN_PROGRESS","downloadPath":"/mnt/SecStorage/c536fc4b-e34d-34f5-8a5a-fc87ad852fd2/template/tmpl/1/5/dnld8635841906619140940tmp_","templateSize":0,"templatePhySicalSize":0,"checkSum":"905cec879afd9c9d22ecc8036131a180","result":false,"details":" > >>>>> ","wait":0}}] } > >>>>> 2013-03-07 19:15:26,933 DEBUG [agent.transport.Request] > (Timer-4:null) > >>>> Seq > >>>>> 4-17236021: Sending { Cmd , MgmtId: 233845174730255, via: 4, Ver: > v1, > >>>>> Flags: 100011, > >>>> > >> > [{"storage.DownloadProgressCommand":{"jobId":"0c9055a8-b23f-49c5-82dd-5c7c00de1629","request":"GET_STATUS","hvm":true,"description":"Ubuntu > >>>>> 12.04 > >>>> > >> > LTS","maxDownloadSizeInBytes":53687091200,"id":201,"resourceType":"TEMPLATE","url":" > >>>>> http://172.30.45.2/isos/ubuntu-12.04.1-server-amd64.iso > >>>> > >> > ","format":"ISO","accountId":2,"name":"201-2-597c145a-1f38-355e-9225-943a184a1f8d","secUrl":"nfs:// > >>>>> 172.30.45.2/store/secondary","wait":0}}] } > >>>>> 2013-03-07 19:15:26,978 DEBUG [agent.transport.Request] > >>>>> (AgentManager-Handler-5:null) Seq 4-17236021: Processing: { Ans: , > >>>> MgmtId: > >>>>> 233845174730255, via: 4, Ver: v1, Flags: 10, > >>>> > >> > [{"storage.DownloadAnswer":{"jobId":"0c9055a8-b23f-49c5-82dd-5c7c00de1629","downloadPct":0,"errorString":"Connection > >>>> > >> > refused","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/SecStorage/c536fc4b-e34d-34f5-8a5a-fc87ad852fd2/template/tmpl/2/201/dnld7244558075029424555tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"details":"Connection > >>>>> refused","wait":0}}] } > >>>>> > >>>>> > >>>>> I am going to re-install the management server and the hosts as well > as > >>>>> clean up the secondary and primary stores and try again, but if > anyone > >>>> has > >>>>> any ideas let me know. > >>>>> > >>>>> thanks, > >>>>> -Carlos > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Mar 7, 2013 at 9:13 PM, Ahmad Emneina <aemne...@gmail.com> > >>>> wrote: > >>>>> > >>>>>> yeehaw! :D awesome to hear! > >>>>>> > >>>>>> > >>>>>> On Thu, Mar 7, 2013 at 6:58 PM, Carlos Reategui < > car...@reategui.com> > >>>>>> wrote: > >>>>>> > >>>>>>> woohoo success.... > >>>>>>> > >>>>>>> Thanks to Ahmad's email in the other thread regarding enabling > local > >>>>>>> storage, I decided to see if that would make a difference in my > setup > >>>>>> too. > >>>>>>> Now my system VMs are up in local storage. I have also added nfs > >>>>>> primary > >>>>>>> storage and will now see if I am able to get my own instances up > and > >>>>>>> running there. > >>>>>>> > >>>>>>> > >>>>>>> On Wed, Mar 6, 2013 at 4:53 PM, Carlos Reategui < > create...@gmail.com > >>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> I have been able to add my host but the system VM's won't start. > I > >>>>>>>> thought it had to do with the avoid state problem mentioned in > >> another > >>>>>>>> thread (hence the CS-587) but once I looked further up in the > logs, > >> it > >>>>>> is > >>>>>>>> not. > >>>>>>>> > >>>>>>>> My setup: > >>>>>>>> Management Server: ubuntu 12.04, CS 4.0.1 + CLOUDSTACK-587 (from > >> local > >>>>>>> apt > >>>>>>>> repo built from source), Basic networking > >>>>>>>> Hosts: ubuntu 12.04, default xen and xcp-xapi > >>>>>>>> Primary storage: NFS, presetup > >>>>>>>> > >>>>>>>> Excption claims it is unable to find local SR. This is from my > >> host: > >>>>>>>> root@ubuntu:~# df -h > >>>>>>>> Filesystem Size Used Avail Use% Mounted on > >>>>>>>> /dev/mapper/vg00-lv_root 9.2G 1.8G 7.0G 21% / > >>>>>>>> udev 431M 4.0K 431M 1% /dev > >>>>>>>> tmpfs 181M 396K 181M 1% /run > >>>>>>>> none 5.0M 0 5.0M 0% /run/lock > >>>>>>>> none 452M 0 452M 0% /run/shm > >>>>>>>> /dev/sda1 223M 48M 163M 23% /boot > >>>>>>>> 172.30.19.1:/export/primary 5.0T 2.6G 5.0T 1% > >>>>>>>> /run/sr-mount/d558fc56-d287-523e-8a9f-0fe061cb36c5 > >>>>>>>> root@ubuntu:~# xe sr-list > >>>>>>>> uuid ( RO) : d558fc56-d287-523e-8a9f-0fe061cb36c5 > >>>>>>>> name-label ( RW): nfs > >>>>>>>> name-description ( RW): > >>>>>>>> host ( RO): ubuntu > >>>>>>>> type ( RO): nfs > >>>>>>>> content-type ( RO): > >>>>>>>> > >>>>>>>> > >>>>>>>> uuid ( RO) : 7648436c-2fdd-e8b4-d1e5-372ac9eeec0d > >>>>>>>> name-label ( RW): XenServer Tools > >>>>>>>> name-description ( RW): XenServer Tools ISOs > >>>>>>>> host ( RO): <shared> > >>>>>>>> type ( RO): iso > >>>>>>>> content-type ( RO): iso > >>>>>>>> > >>>>>>>> > >>>>>>>> Management Logs: > >>>>>>>> 2013-03-06 16:21:48,858 DEBUG [cloud.storage.StorageManagerImpl] > >>>>>>>> (secstorage-1:null) Checking if we need to prepare 1 volumes for > >>>>>>>> VM[SecondaryStorageVm|s-1-VM] > >>>>>>>> 2013-03-06 16:21:48,861 DEBUG [cloud.storage.StorageManagerImpl] > >>>>>>>> (secstorage-1:null) Creating volume: Vol[1|vm=1|ROOT] > >>>>>>>> 2013-03-06 16:21:48,861 DEBUG [cloud.storage.StorageManagerImpl] > >>>>>>>> (secstorage-1:null) Trying to create in Pool[201|PreSetup] > >>>>>>>> 2013-03-06 16:22:00,399 DEBUG [agent.manager.DirectAgentAttache] > >>>>>>>> (DirectAgent-17:null) Seq 1-1910374413: Response Received: > >>>>>>>> 2013-03-06 16:22:00,400 DEBUG [agent.transport.Request] > >>>>>>>> (DirectAgent-17:null) Seq 1-1910374413: Processing: { Ans: , > >> MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 110, > >>>> > >> > [{"storage.PrimaryStorageDownloadAnswer":{"installPath":"75e4eaaf-fc67-408f-9208-e54352def313","templateSize":565240320,"result":true,"wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:00,400 DEBUG [agent.manager.AgentAttache] > >>>>>>>> (DirectAgent-17:null) Seq 1-1910374413: No more commands found > >>>>>>>> 2013-03-06 16:22:00,401 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374413: Received: { Ans: , > MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 110, { > >>>>>>>> PrimaryStorageDownloadAnswer } } > >>>>>>>> 2013-03-06 16:22:00,405 DEBUG [cloud.template.TemplateManagerImpl] > >>>>>>>> (consoleproxy-1:null) Template 1 is downloaded via 1 > >>>>>>>> 2013-03-06 16:22:00,415 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374414: Sending { Cmd , MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 100111, > >>>> > >> > [{"storage.CreateCommand":{"volId":2,"pool":{"id":201,"uuid":"nfs","host":"localhost","path":"/nfs","port":0,"type":"PreSetup"},"diskCharacteristics":{"size":0,"tags":[],"type":"ROOT","name":"ROOT-2","useLocalStorage":false,"recreatable":true,"diskOfferingId":8,"volumeId":2,"hyperType":"XenServer"},"templateUrl":"75e4eaaf-fc67-408f-9208-e54352def313","wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:00,417 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374414: Executing: { Cmd , > MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 100111, > >>>> > >> > [{"storage.CreateCommand":{"volId":2,"pool":{"id":201,"uuid":"nfs","host":"localhost","path":"/nfs","port":0,"type":"PreSetup"},"diskCharacteristics":{"size":0,"tags":[],"type":"ROOT","name":"ROOT-2","useLocalStorage":false,"recreatable":true,"diskOfferingId":8,"volumeId":2,"hyperType":"XenServer"},"templateUrl":"75e4eaaf-fc67-408f-9208-e54352def313","wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:00,417 DEBUG [agent.manager.DirectAgentAttache] > >>>>>>>> (DirectAgent-19:null) Seq 1-1910374414: Executing request > >>>>>>>> 2013-03-06 16:22:00,563 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-19:null) SR retrieved for 201 > >>>>>>>> 2013-03-06 16:22:00,576 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-19:null) Checking nfs or SR > >>>>>>>> d558fc56-d287-523e-8a9f-0fe061cb36c5 on > >>>>>>>> XS[7b697b0e-b3fa-6bce-6105-14aa4c399506-172.30.45.13] > >>>>>>>> 2013-03-06 16:22:01,407 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-19:null) Succesfully created VDI for > >>>>>>>> com.cloud.agent.api.storage.CreateCommand. Uuid = > >>>>>>>> 1f842f76-d328-48bc-b638-08be7433e88a > >>>>>>>> 2013-03-06 16:22:01,408 DEBUG [agent.manager.DirectAgentAttache] > >>>>>>>> (DirectAgent-19:null) Seq 1-1910374414: Response Received: > >>>>>>>> 2013-03-06 16:22:01,410 DEBUG [agent.transport.Request] > >>>>>>>> (DirectAgent-19:null) Seq 1-1910374414: Processing: { Ans: , > >> MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 110, > >>>> > >> > [{"storage.CreateAnswer":{"volume":{"id":2,"name":"ROOT-2","mountPoint":"/nfs","path":"1f842f76-d328-48bc-b638-08be7433e88a","size":2147483648,"type":"ROOT","storagePoolType":"PreSetup","storagePoolUuid":"nfs","deviceId":0},"requestTemplateReload":false,"result":true,"wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:01,410 DEBUG [agent.manager.AgentAttache] > >>>>>>>> (DirectAgent-19:null) Seq 1-1910374414: No more commands found > >>>>>>>> 2013-03-06 16:22:01,410 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374414: Received: { Ans: , > MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 110, { CreateAnswer } } > >>>>>>>> 2013-03-06 16:22:01,416 DEBUG [cloud.storage.StorageManagerImpl] > >>>>>>>> (consoleproxy-1:null) Volume Vol[2|vm=2|ROOT] is created on > >>>>>>>> Pool[201|PreSetup] > >>>>>>>> 2013-03-06 16:22:01,435 DEBUG > >>>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] > >>>>>>>> (consoleproxy-1:null) Boot Args for VM[ConsoleProxy|v-2-VM]: > >>>>>>> template=domP > >>>>>>>> type=consoleproxy host=172.30.45.2 port=8250 name=v-2-VM > >> premium=true > >>>>>>>> zone=1 pod=1 guid=Proxy.2 proxy_vm=2 disable_rp_filter=true > >>>>>>>> eth2ip=172.30.45.169 eth2mask=255.255.255.0 gateway=172.30.45.1 > >>>>>>>> eth0ip=169.254.3.56 eth0mask=255.255.0.0 eth1ip=172.30.45.59 > >>>>>>>> eth1mask=255.255.255.0 mgmtcidr=172.30.45.0/24localgw=172.30.45.1 > >>>>>>>> internaldns1=172.30.20.176 internaldns2=172.30.20.177 > >>>>>> dns1=172.30.20.176 > >>>>>>>> dns2=172.30.20.177 > >>>>>>>> 2013-03-06 16:22:01,483 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374415: Sending { Cmd , MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 100111, > >>>> > >> > [{"StartCommand":{"vm":{"id":2,"name":"v-2-VM","bootloader":"PyGrub","type":"ConsoleProxy","cpus":1,"speed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian > >>>>>>>> GNU/Linux 6(64-bit)","bootArgs":" template=domP type=consoleproxy > >>>>>>>> host=172.30.45.2 port=8250 name=v-2-VM premium=true zone=1 pod=1 > >>>>>>>> guid=Proxy.2 proxy_vm=2 disable_rp_filter=true > eth2ip=172.30.45.169 > >>>>>>>> eth2mask=255.255.255.0 gateway=172.30.45.1 eth0ip=169.254.3.56 > >>>>>>>> eth0mask=255.255.0.0 eth1ip=172.30.45.59 eth1mask=255.255.255.0 > >>>>>> mgmtcidr= > >>>>>>>> 172.30.45.0/24 localgw=172.30.45.1 internaldns1=172.30.20.176 > >>>>>>>> internaldns2=172.30.20.177 dns1=172.30.20.176 > >>>> > >> > dns2=172.30.20.177","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"3d2759124df2d5eb","params":{},"uuid":"b6c396f8-25c7-4103-9b86-3944e77686a3","disks":[{"id":2,"name":"ROOT-2","mountPoint":"/nfs","path":"1f842f76-d328-48bc-b638-08be7433e88a","size":2147483648,"type":"ROOT","storagePoolType":"PreSetup","storagePoolUuid":"nfs","deviceId":0}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"0b0846cf-a554-4c46-80d2-ca5275624d1d","ip":"172.30.45.169","netmask":"255.255.255.0","gateway":"172.30.45.1","mac":"06:54:66:00:00:60","dns1":"172.30.20.176","dns2":"172.30.20.177","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":false},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"7348c8a8-9815-4940-99a7-ed9c503d16fc","ip":"169.254.3.56","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:38","broadcastType":"LinkLocal","type":"Control" > , > > "isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"c6222ef4-2750-4d9a-bb2b-f0f0027fb1be","ip":"172.30.45.59","netmask":"255.255.255.0","gateway":"172.30.45.1","mac":"06:12:82:00:00:0a","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false}]},"wait":0}},{"check.CheckSshCommand":{"ip":"169.254.3.56","port":3922,"interval":6,"retries":100,"name":"v-2-VM","wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:01,490 DEBUG [agent.transport.Request] > >>>>>>>> (consoleproxy-1:null) Seq 1-1910374415: Executing: { Cmd , > MgmtId: > >>>>>>>> 233845174730255, via: 1, Ver: v1, Flags: 100111, > >>>> > >> > [{"StartCommand":{"vm":{"id":2,"name":"v-2-VM","bootloader":"PyGrub","type":"ConsoleProxy","cpus":1,"speed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian > >>>>>>>> GNU/Linux 6(64-bit)","bootArgs":" template=domP type=consoleproxy > >>>>>>>> host=172.30.45.2 port=8250 name=v-2-VM premium=true zone=1 pod=1 > >>>>>>>> guid=Proxy.2 proxy_vm=2 disable_rp_filter=true > eth2ip=172.30.45.169 > >>>>>>>> eth2mask=255.255.255.0 gateway=172.30.45.1 eth0ip=169.254.3.56 > >>>>>>>> eth0mask=255.255.0.0 eth1ip=172.30.45.59 eth1mask=255.255.255.0 > >>>>>> mgmtcidr= > >>>>>>>> 172.30.45.0/24 localgw=172.30.45.1 internaldns1=172.30.20.176 > >>>>>>>> internaldns2=172.30.20.177 dns1=172.30.20.176 > >>>> > >> > dns2=172.30.20.177","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"3d2759124df2d5eb","params":{},"uuid":"b6c396f8-25c7-4103-9b86-3944e77686a3","disks":[{"id":2,"name":"ROOT-2","mountPoint":"/nfs","path":"1f842f76-d328-48bc-b638-08be7433e88a","size":2147483648,"type":"ROOT","storagePoolType":"PreSetup","storagePoolUuid":"nfs","deviceId":0}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"0b0846cf-a554-4c46-80d2-ca5275624d1d","ip":"172.30.45.169","netmask":"255.255.255.0","gateway":"172.30.45.1","mac":"06:54:66:00:00:60","dns1":"172.30.20.176","dns2":"172.30.20.177","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":false},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"7348c8a8-9815-4940-99a7-ed9c503d16fc","ip":"169.254.3.56","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:38","broadcastType":"LinkLocal","type":"Control" > , > > "isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"c6222ef4-2750-4d9a-bb2b-f0f0027fb1be","ip":"172.30.45.59","netmask":"255.255.255.0","gateway":"172.30.45.1","mac":"06:12:82:00:00:0a","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false}]},"wait":0}},{"check.CheckSshCommand":{"ip":"169.254.3.56","port":3922,"interval":6,"retries":100,"name":"v-2-VM","wait":0}}] > >>>>>>>> } > >>>>>>>> 2013-03-06 16:22:01,490 DEBUG [agent.manager.DirectAgentAttache] > >>>>>>>> (DirectAgent-25:null) Seq 1-1910374415: Executing request > >>>>>>>> 2013-03-06 16:22:01,636 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) 1. The VM v-2-VM is in Starting state. > >>>>>>>> 2013-03-06 16:22:01,734 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) Created VM > >> d2b354d1-6c06-b6f7-805e-489da21af530 > >>>>>> for > >>>>>>>> v-2-VM > >>>>>>>> 2013-03-06 16:22:01,861 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) PV args are -- quiet > >>>> > >> > console=hvc0%template=domP%type=consoleproxy%host=172.30.45.2%port=8250%name=v-2-VM%premium=true%zone=1%pod=1%guid=Proxy.2%proxy_vm=2%disable_rp_filter=true%eth2ip=172.30.45.169%eth2mask=255.255.255.0%gateway=172.30.45.1%eth0ip=169.254.3.56%eth0mask=255.255.0.0%eth1ip=172.30.45.59%eth1mask=255.255.255.0%mgmtcidr= > >>>> > >> > 172.30.45.0/24%localgw=172.30.45.1%internaldns1=172.30.20.176%internaldns2=172.30.20.177%dns1=172.30.20.176%dns2=172.30.20.177 > >>>>>>>> 2013-03-06 16:22:01,918 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) VBD e879be32-9292-ca96-a273-34dd69112fb1 > >> created > >>>>>>> for > >>>>>>>> Vol[2|ROOT|1f842f76-d328-48bc-b638-08be7433e88a|2147483648] > >>>>>>>> 2013-03-06 16:22:01,919 WARN [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) Catch Exception: class > >>>>>>>> com.cloud.utils.exception.CloudRuntimeException due to > >>>>>>>> com.cloud.utils.exception.CloudRuntimeException: can not find > local > >> sr > >>>>>>>> com.cloud.utils.exception.CloudRuntimeException: can not find > local > >> sr > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.createPatchVbd(XcpOssResource.java:109) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1285) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.execute(XcpOssResource.java:142) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:136) > >>>>>>>> at > >>>> > >> > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191) > >>>>>>>> at > >>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >>>>>>>> at > >>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >>>>>>>> at java.lang.Thread.run(Thread.java:679) > >>>>>>>> 2013-03-06 16:22:01,920 WARN [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) Unable to start v-2-VM due to > >>>>>>>> com.cloud.utils.exception.CloudRuntimeException: can not find > local > >> sr > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.createPatchVbd(XcpOssResource.java:109) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1285) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.execute(XcpOssResource.java:142) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:136) > >>>>>>>> at > >>>> > >> > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191) > >>>>>>>> at > >>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >>>>>>>> at > >>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >>>>>>>> at java.lang.Thread.run(Thread.java:679) > >>>>>>>> 2013-03-06 16:22:01,980 WARN [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) Unable to clean up VBD due to > >>>>>>>> You gave an invalid object reference. The object may have > recently > >>>>>> been > >>>>>>>> deleted. The class parameter gives the type of reference given, > and > >>>>>> the > >>>>>>>> handle parameter echoes the bad value given. > >>>>>>>> at com.xensource.xenapi.Types.checkResponse(Types.java:207) > >>>>>>>> at > >>>>>> com.xensource.xenapi.Connection.dispatch(Connection.java:368) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > >>>>>>>> at com.xensource.xenapi.VBD.unplug(VBD.java:1054) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1092) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1353) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.execute(XcpOssResource.java:142) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497) > >>>>>>>> at > >>>> > >> > com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:136) > >>>>>>>> at > >>>> > >> > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191) > >>>>>>>> at > >>>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >>>>>>>> at > >>>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > >>>>>>>> at > >>>> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >>>>>>>> at java.lang.Thread.run(Thread.java:679) > >>>>>>>> 2013-03-06 16:22:01,982 DEBUG [xen.resource.CitrixResourceBase] > >>>>>>>> (DirectAgent-25:null) The VM is in stopped state, detected problem > >>>>>> during > >>>>>>>> startup : v-2-VM > >>>>>>>> 2013-03-06 16:22:02,344 DEBUG [agent.manager.DirectAgentAttache] > >>>>>>>> (DirectAgent-25:null) Seq 1-1910374415: Cancelling because one of > >> the > >>>>>>>> answers is false and it is stop on error. > >>>> > >>>> > >> > > >