Hi, I have just installed ACS 4.2 on Centos 6.4 using the repo: http://cloudstack.apt-get.eu/rhel/4.2/ for POC purposes (like the DevCloud but on Centos)
On first startup, I am using the Launch wizard to setup a Basic Zone with a KVM host (both KVM and NFS running on management server). The wizard setups everything until it gets to the Primary Storage, where I get following message: "Failed to delete storage pool on host" and the wizard stops there. If I try to add the Primary Storage via "Infrastructure > Primary Storage > Add Primary Storage" I still get the same message. But I can add Secondary Storage without problem. I have double checked the NFS and it mounts ok outside of CloudStack. Note: both Primary and Secondary mount points are not mounted before running the wizard. Can anyone help? Thanks The log extract for the Primary Storage is as follows: 2013-11-11 01:52:38,517 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===END=== 192.168.2.20 -- POST command=addHost&response=json&sessionkey=FOSFGSl%2BMFut7z2dyOA%2BDiqrZ60%3D 2013-11-11 01:52:38,546 DEBUG [cloud.api.ApiServlet] (catalina-exec-17:null) ===START=== 192.168.2.20 -- GET command=createStoragePool&zoneid=303b03d1-507b-457d-b991-ab019745ef01&podId=a45a5bde-aa8b-458c-9d48-8338acddb514&clusterid=52340bd7-2ab0-43c1-ab44-909df68fe788&name=Primary1&scope=cluster&url=nfs%3A%2F%2F192.168.2.40%2Fmount%2Fprimary&response=json&sessionkey=FOSFGSl%2BMFut7z2dyOA%2BDiqrZ60%3D&_=1384048344807 2013-11-11 01:52:38,644 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (catalina-exec-17:null) createPool Params @ scheme - nfs storageHost - 192.168.2.40 hostPath - /mount/primary port - -1 2013-11-11 01:52:38,696 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (catalina-exec-17:null) creating pool null on host 1 2013-11-11 01:52:38,708 DEBUG [agent.transport.Request] (catalina-exec-17:null) Seq 1-249364488: Sending { Cmd , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.CreateStoragePoolCommand":{"add":true,"pool":{"id":1,"uuid":"a7649155-eede-3007-9c9c-f1e054e7256f","host":"192.168.2.40","path":"/mount/primary","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//a7649155-eede-3007-9c9c-f1e054e7256f","wait":0}}] } 2013-11-11 01:52:38,731 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 1-249364488: Processing: { Ans: , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"success","wait":0}}] } 2013-11-11 01:52:38,731 DEBUG [agent.transport.Request] (catalina-exec-17:null) Seq 1-249364488: Received: { Ans: , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 10, { Answer } } 2013-11-11 01:52:38,731 DEBUG [agent.manager.AgentManagerImpl] (catalina-exec-17:null) Details from executing class com.cloud.agent.api.CreateStoragePoolCommand: success 2013-11-11 01:52:38,732 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (catalina-exec-17:null) In createPool Adding the pool to each of the hosts 2013-11-11 01:52:38,735 DEBUG [cloud.storage.StorageManagerImpl] (catalina-exec-17:null) Adding pool null to host 1 2013-11-11 01:52:38,743 DEBUG [agent.transport.Request] (catalina-exec-17:null) Seq 1-249364489: Sending { Cmd , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":1,"uuid":"a7649155-eede-3007-9c9c-f1e054e7256f","host":"192.168.2.40","path":"/mount/primary","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//a7649155-eede-3007-9c9c-f1e054e7256f","wait":0}}] } 2013-11-11 01:52:38,880 DEBUG [agent.transport.Request] (AgentManager-Handler-3:null) Seq 1-249364489: Processing: { Ans: , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: cannot open path '/mnt/a7649155-eede-3007-9c9c-f1e054e7256f': No such file or directory\n\tat com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:526)\n\tat com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:185)\n\tat com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:177)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2656)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1244)\n\tat com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat com.cloud.utils.nio.Task.run(Task.java:83)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:701)\n","wait":0}}] } 2013-11-11 01:52:38,880 DEBUG [agent.transport.Request] (catalina-exec-17:null) Seq 1-249364489: Received: { Ans: , MgmtId: 253719053137455, via: 1, Ver: v1, Flags: 10, { Answer } } 2013-11-11 01:52:38,881 DEBUG [agent.manager.AgentManagerImpl] (catalina-exec-17:null) Details from executing class com.cloud.agent.api.ModifyStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: cannot open path '/mnt/a7649155-eede-3007-9c9c-f1e054e7256f': No such file or directory at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:526) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:185) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:177) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2656) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1244) at com.cloud.agent.Agent.processRequest(Agent.java:525) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2013-11-11 01:52:38,881 WARN [apache.cloudstack.alerts] (catalina-exec-17:null) alertType:: 7 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to attach storage pool1 to the host1 2013-11-11 01:52:38,900 WARN [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (catalina-exec-17:null) Unable to establish a connection between Host[-1-Routing] and org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@7d02890a com.cloud.utils.exception.CloudRuntimeException: Unable establish connection from storage head to storage pool 1 due to com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: cannot open path '/mnt/a7649155-eede-3007-9c9c-f1e054e7256f': No such file or directory at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:526) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:185) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:177) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2656) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1244) at com.cloud.agent.Agent.processRequest(Agent.java:525) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 1 at org.apache.cloudstack.storage.datastore.provider.DefaultHostListener.hostConnect(DefaultHostListener.java:66) at com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:948) at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:393) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:749) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:177) at org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:168) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:514) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305) at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2013-11-11 01:52:38,903 WARN [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (catalina-exec-17:null) No host can access storage pool org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@7d02890a on cluster 1 2013-11-11 01:52:38,909 DEBUG [cloud.storage.StorageManagerImpl] (catalina-exec-17:null) Failed to add data store com.cloud.utils.exception.CloudRuntimeException: Failed to access storage pool at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:404) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:749) at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:177) at org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:168) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:514) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305) at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2013-11-11 01:52:38,914 INFO [cloud.api.ApiServer] (catalina-exec-17:null) Failed to delete storage pool on host 2013-11-11 01:52:38,916 DEBUG [cloud.api.ApiServlet] (catalina-exec-17:null) ===END=== 192.168.2.20 -- GET command=createStoragePool&zoneid=303b03d1-507b-457d-b991-ab019745ef01&podId=a45a5bde-aa8b-458c-9d48-8338acddb514&clusterid=52340bd7-2ab0-43c1-ab44-909df68fe788&name=Primary1&scope=cluster&url=nfs%3A%2F%2F192.168.2.40%2Fmount%2Fprimary&response=json&sessionkey=FOSFGSl%2BMFut7z2dyOA%2BDiqrZ60%3D&_=1384048344807 2013-11-11 01:52:44,053 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Skip capacity scan due to there is no Primary Storage UPintenance mode 2013-11-11 01:52:47,797 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers to update status. 2013-11-11 01:52:47,803 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 networks to update RvR status. 2013-11-11 01:52:47,853 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers to update status. 2013-11-11 01:52:47,857 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 networks to update RvR status. 2013-11-11 01:53:11,316 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-4:null) Ping from 1 2013-11-11 01:53:14,058 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Skip capacity scan due to there is no Primary Storage UPintenance mode 2013-11-11 01:53:17,797 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers to update status. 2013-11-11 01:53:17,801 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 networks to update RvR status. 2013-11-11 01:53:17,855 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers to update status. 2013-11-11 01:53:17,866 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 networks to update RvR status. 2013-11-11 01:53:22,947 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Resetting hosts suitable for reconnect 2013-11-11 01:53:22,951 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Completed resetting hosts suitable for reconnect 2013-11-11 01:53:22,952 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Acquiring hosts for clusters already owned by this management server 2013-11-11 01:53:22,953 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Completed acquiring hosts for clusters already owned by this management server 2013-11-11 01:53:22,953 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Acquiring hosts for clusters not owned by any management server 2013-11-11 01:53:22,955 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:null) Completed acquiring hosts for clusters not owned by any management server 2013-11-11 01:53:33,464 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) StorageCollector is running... 2013-11-11 01:53:33,487 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) VmStatsCollector is running... 2013-11-11 01:53:34,373 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) HostStatsCollector is running...