Ideally we'd need to start the log even earlier, but I can still see: Insufficient space on pool: 6 since its usage percentage: 0.8669012505498059 has crossed the pool.storage.capacity.disablethreshold: 0.85
If you increased pool.storage.capacity.disablethreshold to above 087, then you need to restart your management server(s) for it to take effect. It would be better though, if you add another primary storage pool, or increase the storage in the existing pool. As increasing the threshold isn't a long term solution. Kind regards, Paul Angus paul.an...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -----Original Message----- From: he...@powershield.ru [mailto:he...@powershield.ru] Sent: 06 November 2017 20:53 To: users@cloudstack.apache.org Subject: Re: VR starting problem Hello, Here is some of log. https://pastebin.com/JU0qr69V Interested line: dataCenterId:: 2 // podId:: null // clusterId:: null As you can see, datacenter (zone) showing, but can't see pod and cluster.... Paul Angus писал 2017-11-06 22:44: > Please could you use something like pastebin (https://pastebin.com/) > to paste larger sections of your logs. > The section you have pasted is more likely from XenServer trying to > remove a disk image that was never created in the first place. It may > not be the root cause. > > Kind regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > -----Original Message----- > From: he...@powershield.ru [mailto:he...@powershield.ru] > Sent: 06 November 2017 18:14 > To: users@cloudstack.apache.org > Subject: Re: VR starting problem > > We deleted system VM's and also, can't start. > > Now seeing the error: > > > INFO [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-11:ctx-eaf4ab20 job-97/job-14993 ctx-7391ff7c) > (logid:66199d2c) Unable to contact resource. > com.cloud.exception.StorageUnavailableException: Resource > [StoragePool:6] is unreachable: Unable to create > Vol[2034|vm=2010|ROOT]:com.cloud.utils.exception.CloudRuntimeException: > Catch Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid > for uuid: 5e2f645f-9e5d-45df-a268-dbe9c6c9eef9 failed due to The uuid > you supplied was invalid. > > 2017-11-06 20:16:49,498 DEBUG [c.c.h.x.r.XenServerStorageProcessor] > (DirectAgent-6:ctx-b7ab797e) (logid:d3084120) Catch Exception > com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid: > 5e2f645f-9e5d-45df-a268-dbe9c6c9eef9 failed due to The uuid you > supplied was invalid. > 2017-11-06 20:16:49,501 WARN [c.c.h.x.r.XenServerStorageProcessor] > (DirectAgent-6:ctx-b7ab797e) (logid:d3084120) Unable to create volume; > Pool=volumeTO[uuid=efa30838-f8e4-4186-a299-d2ae716d9081|path=null|datastore=PrimaryDataStoreTO[uuid=SR-UK|name=null|id=6|pooltype=PreSetup]]; > Disk: > com.cloud.utils.exception.CloudRuntimeException: Catch Exception > com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid: > 5e2f645f-9e5d-45df-a268-dbe9c6c9eef9 failed due to The uuid you > supplied was invalid. > > > > > > he...@powershield.ru писал 2017-11-06 20:48: >> Changed few hours ago, and anyway still the same problem. >> >> >> >> Ivan Kudryavtsev писал 2017-11-06 20:40: >>> Hi. Don't mix "used" and "allocated". Just change the specified >>> threshold to 0.98 and go ahead. >>> >>> 7 нояб. 2017 г. 0:22 пользователь <he...@powershield.ru> написал: >>> >>>> No, i re-checked maybe 10 times. I just added some more space, but >>>> the problem is same. All VM's working but if client restart - they >>>> goes down, with the same error with insufficient storage. >>>> >>>> >>>> Ivan Kudryavtsev писал 2017-11-06 20:16: >>>> >>>>> Could it be, that your storage, cluster, pod, zone is in the >>>>> maintanence mode? >>>>> >>>>> 6 нояб. 2017 г. 23:51 пользователь "Ivan Kudryavtsev" < >>>>> kudryavtsev...@bw-sw.com> написал: >>>>> >>>>> Hi. Try to find in early logs what exactly resource is >>>>> insufficient. >>>>> Try >>>>>> to play with threshold of it. >>>>>> >>>>>> 6 нояб. 2017 г. 21:57 пользователь <he...@powershield.ru> написал: >>>>>> >>>>>> Now, i see the problem, then VR restarting. >>>>>>> >>>>>>> 2017-11-06 11:54:15,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-43:ctx-02214ed9 job-14455) (logid:18ad84ad) >>>>>>> Complete async job-14455, jobStatus: FAILED, resultCode: 530, >>>>>>> result: >>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{" >>>>>>> uuidList":[],"errorcode":530,"errortext":"Resource [Host:17] is >>>>>>> unreachable: Host 17: Unable to start instance due to Unable to >>>>>>> start >>>>>>> VM:18985a14-3cb5-46d5-a3f0-3a065cd8b014 due to error in >>>>>>> finalizeStart, not retrying"} >>>>>>> 2017-11-06 15:47:27,814 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-69:ctx-be0b5550 job-14663) (logid:8b609324) >>>>>>> Complete async job-14663, jobStatus: FAILED, resultCode: 530, >>>>>>> result: >>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{" >>>>>>> uuidList":[],"errorcode":530,"errortext":"Unable to start a VM >>>>>>> due to insufficient capacity"} >>>>>>> 2017-11-06 16:54:19,055 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-70:ctx-deaaf8e8 job-14668) (logid:6a7700af) >>>>>>> Complete async job-14668, jobStatus: FAILED, resultCode: 530, >>>>>>> result: >>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{" >>>>>>> uuidList":[],"errorcode":530,"errortext":"Resource [DataCenter:2] >>>>>>> is >>>>>>> unreachable: Unable to reboot domR, it is not in right state >>>>>>> Stopped"} >>>>>>> 2017-11-06 16:58:54,235 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >>>>>>> (API-Job-Executor-71:ctx-b26b3b33 job-14669) (logid:436e6183) >>>>>>> Complete async job-14669, jobStatus: FAILED, resultCode: 530, >>>>>>> result: >>>>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{" >>>>>>> uuidList":[],"errorcode":530,"errortext":"Unable to create a >>>>>>> deployment for VM[DomainRouter|r-1919-VM]"} >>>>>>> >>>>>>> I'm pretty sure, what i have enough resources: >>>>>>> >>>>>>> Primary Storage Used >>>>>>> Allocated: 4.22 TB / 4.88 TB >>>>>>> >>>>>>> Primary Storage Allocated >>>>>>> Allocated: 3.02 TB / 4.88 TB >>>>>>> >>>>>>> Memory Allocated >>>>>>> Allocated: 78.00 GB / 597.93 GB >>>>>>> >>>>>>> CPU Allocated >>>>>>> Allocated: 81.00 GHz / 440.00 GHz >>>>>>> >>>>>>> Shared Network IPs >>>>>>> Allocated: 34 / 253 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Ivan Kudryavtsev писал 2017-11-06 15:28: >>>>>>> >>>>>>> Hi, you should dive into management server logs to find relevant >>>>>>>> messages. >>>>>>>> Otherwise it's impossible to predict what's wrong with the >>>>>>>> system. >>>>>>>> Check >>>>>>>> logs and find exceptions and other errornous stuff. >>>>>>>> >>>>>>>> 6 нояб. 2017 г. 18:14 пользователь <he...@powershield.ru> >>>>>>>> написал: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>>> >>>>>>>>> ACS: 4.8.0 >>>>>>>>> XenServer 6.5SP1 >>>>>>>>> >>>>>>>>> >>>>>>>>> We have got some problem with additional space for our client. >>>>>>>>> >>>>>>>>> We are created new Disk Offering, then we tryed to attach for >>>>>>>>> our client, but got error: Unable to find suitable primary >>>>>>>>> storage when creating volume >>>>>>>>> >>>>>>>>> We checked datadisk and we see, there is not hypervisor (should >>>>>>>>> it be showed? Now saying: None) >>>>>>>>> >>>>>>>>> We tryed to create new instance, but also unlucky. >>>>>>>>> >>>>>>>>> After it, we restarted VR and he can't start. He started and >>>>>>>>> again goes to offline. We tryed to check from XenCenter and got >>>>>>>>> this reason: >>>>>>>>> apache2: >>>>>>>>> Could not reliably determine the server's fully qualified >>>>>>>>> domain name, using 127.0.0.1 for ServerName. >>>>>>>>> >>>>>>>>> We delete VR and now, we can not start it manually and he is >>>>>>>>> not starting automatically about 40min. >>>>>>>>> >>>>>>>>> >>>>>>>>>
