Primary storage is not in maintenance mode and is available. I have a couple dozen VMs running from this same primary storage and the SSVM and router VM are both running from this same storage. Only the console proxy seems to be having a problem.
$ ./cloudAPI.pl -c listStoragePools -p clusterid=f200c92f-5937-43c7-8bf5-6593f7311400 https://cloud.overstock.com/client/api?apiKey=xxxxxxxxxxxxxxxxx&clusterid=f200c92f-5937-43c7-8bf5-6593f7311400&command=listStoragePools&response=json&signature=xxxxxxxxxxxxx{ "liststoragepoolsresponse" : { "storagepool" : [ { "clusterid" : "f200c92f-5937-43c7-8bf5-6593f7311400", "zonename" : "Offshore", "state" : "Up", "disksizeallocated" : 359196000256, "podid" : "20eff451-3edd-4366-8ab2-60f2390e5134", "podname" : "Offshore", "id" : "OFFSHORE_FC_LUN01", "clustername" : "Offshore", "zoneid" : "8e1c49fc-a5b3-44bb-9d54-932b0047eed0", "name" : "OFFSHORE_FC_LUN01", "disksizeused" : 360047443968, "path" : "/OFFSHORE_FC_LUN01", "disksizetotal" : 1099499044864, "tags" : "", "created" : "2013-06-08T15:45:05-0600", "type" : "PreSetup", "ipaddress" : "localhost" } ], "count" : 1 } } On Jul 20, 2013, at 5:06 PM, David Comerford <davest...@gmail.com> wrote: >> For some reason I was thinking the system VMs lived on the secondary > storage, but I'm probably wrong on that. > The templates for the system VM's live on secondary storage, then they are > deployed to primary storage. > > Allocator seems happy enough to use host id 51 but theres no primary > storage available in cluster id 9. > Is it in maintenance mode or anything? > > 'list storagepools' in cloudmonkey might shed some light. > > Best regards, > David Comerford > ------------------------ > Tel: +353 87 1238295 > Email: davest...@gmail.com > Website: http://dave.ie > GPG key: http://gpg.dave.ie <http://pgp.dave.ie> > > > On 18 July 2013 21:27, Bradley Hieber <mercsni...@gmail.com> wrote: > >> pastebin please >> >> >> On Thu, Jul 18, 2013 at 4:24 PM, Caleb Call <calebc...@me.com> wrote: >> >>> When trying to start the console proxy in one of my zones fails. I'm >>> getting an error that says "no suitable storagePools for in this >> cluster". >>> Which storage pool would this be? For some reason I was thinking the >>> system VMs lived on the secondary storage, but I'm probably wrong on >> that. >>> Either way, I have ~700GB of free space on my primary storage and ~500GB >>> free in my secondary storage, what else is this looking for to be able to >>> start this console proxy? I've included the logs around this event. >>> >>> >>> 2013-07-18 14:02:40,031 DEBUG [cloud.api.ApiServlet] >>> (catalina-exec-13:null) ===START=== 10.4.108.254 -- GET >>> command=startSystemVm&id=d10d7f57 >>> >>> >> -8b1b-45b2-9875-8c18995ac37b&response=json&sessionkey=uMQLwJmB8uyPSQeSiSBXo04naus%3D&_=1374177759969 >>> 2013-07-18 14:02:40,033 DEBUG [ehcache.store.MemoryStore] >>> (catalina-exec-13:null) UserDaoCache: UserDaoMemoryStore hit for 10 >>> 2013-07-18 14:02:40,112 DEBUG [cloud.async.AsyncJobManagerImpl] >>> (catalina-exec-13:null) submit async job-5916, details: AsyncJobVO >>> {id:5916, use >>> rId: 10, accountId: 7, sessionKey: null, instanceType: SystemVm, >>> instanceId: 1024, cmd: com.cloud.api.commands.StartSystemVMCmd, >>> cmdOriginator: >>> null, cmdInfo: >>> >> {"response":"json","id":"d10d7f57-8b1b-45b2-9875-8c18995ac37b","sessionkey":"uMQLwJmB8uyPSQeSiSBXo04naus\u003d","ctxUserId":"10", >>> "_":"1374177759969","ctxAccountId":"7","ctxStartEventId":"18978"}, >>> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, >>> processStat >>> us: 0, resultCode: 0, result: null, initMsid: 145320940120008, >>> completeMsid: null, lastUpdated: null, lastPolled: null, created: null} >>> 2013-07-18 14:02:40,113 DEBUG [cloud.async.AsyncJobManagerImpl] >>> (Job-Executor-43:job-5916) Executing >>> com.cloud.api.commands.StartSystemVMCmd for >>> job-5916 >>> 2013-07-18 14:02:40,114 DEBUG [cloud.api.ApiServlet] >>> (catalina-exec-13:null) ===END=== 10.4.108.254 -- GET >>> command=startSystemVm&id=d10d7f57-8 >>> >>> >> b1b-45b2-9875-8c18995ac37b&response=json&sessionkey=uMQLwJmB8uyPSQeSiSBXo04naus%3D&_=1374177759969 >>> 2013-07-18 14:02:40,125 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916) VM state transitted from :Stopped to Starting >>> with >>> event: StartRequestedvm's original host id: 25 new host id: null host id >>> before state transition: null >>> 2013-07-18 14:02:40,125 DEBUG [cloud.vm.VirtualMachineManagerImpl] >>> (Job-Executor-43:job-5916) Successfully transitioned to start state for >>> VM[Co >>> nsoleProxy|v-1024-VM] reservation id = >> c9a532cb-3976-4284-ab2e-17d7d5dca070 >>> 2013-07-18 14:02:40,138 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) ServiceOfferingDaoCache: >>> ServiceOfferingDaoMemoryStore hit >>> for 8 >>> 2013-07-18 14:02:40,138 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) VMTemplateDaoCache: VMTemplateDaoMemoryStore >> hit >>> for 1 >>> 2013-07-18 14:02:40,138 DEBUG [cloud.vm.VirtualMachineManagerImpl] >>> (Job-Executor-43:job-5916) Trying to deploy VM, vm has dcId: 5 and >> podId: 8 >>> 2013-07-18 14:02:40,138 DEBUG [cloud.vm.VirtualMachineManagerImpl] >>> (Job-Executor-43:job-5916) Deploy avoids pods: null, clusters: null, >> hosts: >>> n >>> ull >>> 2013-07-18 14:02:40,140 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) DataCenterDaoCache: DataCenterDaoMemoryStore >> hit >>> for 5 >>> 2013-07-18 14:02:40,141 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) DeploymentPlanner allocation algorithm: random >>> 2013-07-18 14:02:40,141 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Trying to allocate a host and storage pools >> from >>> dc:5, p >>> od:8,cluster:null, requested cpu: 500, requested ram: 1073741824 >>> 2013-07-18 14:02:40,141 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Is ROOT volume READY (pool already >> allocated)?: >>> No >>> 2013-07-18 14:02:40,141 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) This VM has last host_id specified, trying to >>> choose the >>> same host: 25 >>> 2013-07-18 14:02:40,142 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) The last host of this VM cannot be found >>> 2013-07-18 14:02:40,142 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Cannot choose the last host to deploy this VM >>> 2013-07-18 14:02:40,142 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Searching resources only under specified Pod: >> 8 >>> 2013-07-18 14:02:40,142 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) HostPodDaoCache: HostPodDaoMemoryStore hit >> for 8 >>> 2013-07-18 14:02:40,142 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) DataCenterDaoCache: DataCenterDaoMemoryStore >> hit >>> for 5 >>> 2013-07-18 14:02:40,143 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Listing clusters in order of aggregate >> capacity, >>> that ha >>> ve (atleast one host with) enough CPU and RAM capacity under this Pod: 8 >>> 2013-07-18 14:02:40,143 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) CPUOverprovisioningFactor considered: 5.0 >>> 2013-07-18 14:02:40,150 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Checking resources in Cluster: 9 under Pod: 8 >>> 2013-07-18 14:02:40,150 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Calling HostAllocators to find suitable hosts >>> 2013-07-18 14:02:40,150 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Looking for hosts in >>> dc: 5 >>> pod:8 cluster:9 >>> 2013-07-18 14:02:40,152 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) FirstFitAllocator >> has 1 >>> hos >>> ts to check for allocation: [Host[-51-Routing]] >>> 2013-07-18 14:02:40,154 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Found 1 hosts for >>> allocation after prioritization: [Host[-51-Routing]] >>> 2013-07-18 14:02:40,154 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Looking for >>> speed=500Mhz, Ram=1024 >>> 2013-07-18 14:02:40,155 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Checking if host: 51 >>> has enough capacity for requested CPU: 500 and requested RAM: 1073741824 >> , >>> cpuOverprovisioningFactor: 5.0 >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Hosts's actual total >>> CPU: 70400 and CPU after applying overprovisioning: 352000 >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Free CPU: 303000 , >>> Requested CPU: 500 >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Free RAM: >> 342436691072 >>> , Requested RAM: 1073741824 >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Host has enough CPU >> and >>> RAM available >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) STATS: Can alloc CPU >>> from host: 51, used: 49000, reserved: 0, actual total: 70400, total with >>> overprovisioning: 352000; requested cpu:500,alloc_from_last_host?:false >>> ,considerReservedCapacity?: true >>> 2013-07-18 14:02:40,157 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) STATS: Can alloc MEM >>> from host: 51, used: 62478352384, reserved: 0, total: 404915043456; >>> requested mem: 1073741824,alloc_from_last_host?:false >>> ,considerReservedCapacity?: true >>> 2013-07-18 14:02:40,157 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Found a suitable >> host, >>> adding to list: 51 >>> 2013-07-18 14:02:40,157 DEBUG [allocator.impl.FirstFitAllocator] >>> (Job-Executor-43:job-5916 FirstFitRoutingAllocator) Host Allocator >>> returning 1 suitable hosts >>> 2013-07-18 14:02:40,159 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) No suitable pools found >>> 2013-07-18 14:02:40,159 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) No suitable storagePools found under this >>> Cluster: 9 >>> 2013-07-18 14:02:40,159 DEBUG [cloud.deploy.FirstFitPlanner] >>> (Job-Executor-43:job-5916) Could not find suitable Deployment Destination >>> for this VM under any clusters, returning. >>> 2013-07-18 14:02:40,186 DEBUG [cloud.capacity.CapacityManagerImpl] >>> (Job-Executor-43:job-5916) VM state transitted from :Starting to Stopped >>> with event: OperationFailedvm's original host id: 25 new host id: null >> host >>> id before state transition: null >>> 2013-07-18 14:02:40,186 DEBUG [ehcache.store.MemoryStore] >>> (Job-Executor-43:job-5916) ServiceOfferingDaoCache: >>> ServiceOfferingDaoMemoryStore hit for 8 >>> 2013-07-18 14:02:40,188 WARN >> [cloud.consoleproxy.ConsoleProxyManagerImpl] >>> (Job-Executor-43:job-5916) Exception while trying to start console proxy >>> com.cloud.exception.InsufficientServerCapacityException: Unable to create >>> a deployment for VM[ConsoleProxy|v-1024-VM]Scope=interface >>> com.cloud.dc.DataCenter; id=5 >>> at >>> >> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:734) >>> at >>> >> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:472) >>> at >>> >> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:465) >>> at >>> >> com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:627) >>> at >>> >> com.cloud.server.ManagementServerImpl.startConsoleProxy(ManagementServerImpl.java:1911) >>> at >>> >> com.cloud.server.ManagementServerImpl.startSystemVM(ManagementServerImpl.java:2728) >>> at >>> >> com.cloud.api.commands.StartSystemVMCmd.execute(StartSystemVMCmd.java:107) >>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138) >>> at >>> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >>> at >>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>> at >>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:679) >>> 2013-07-18 14:02:40,189 WARN [cloud.api.ApiDispatcher] >>> (Job-Executor-43:job-5916) class com.cloud.api.ServerApiException : Fail >> to >>> start system vm >>> 2013-07-18 14:02:40,189 DEBUG [cloud.async.AsyncJobManagerImpl] >>> (Job-Executor-43:job-5916) Complete async job-5916, jobStatus: 2, >>> resultCode: 530, result: Error Code: 530 Error text: Fail to start >> system vm >>> >>> >> >> >> -- >> Brad >>
