Replying on behalf of Matt. We are able to write data to the Nfs drives. That's not an issue.
Thanks Sriharsha Sent from my iPhone > On Sep 16, 2013, at 19:30, Ahmad Emneina <aemne...@gmail.com> wrote: > > Try to mount your primary storage to a compute host and try to write to it. > Your NFS server might not have come back up properly (settings-wise or all > the relevant services). >> On Sep 16, 2013 6:08 PM, "Matt Foley" <mfo...@hortonworks.com> wrote: >> >> Thank you Chiradeep. Log snippet now available as http://apaste.info/qBIB >> --Matt >> >> On Mon, Sep 16, 2013 at 5:19 PM, Chiradeep Vittal < >> chiradeep.vit...@citrix.com> wrote: >> >>> Attachments are stripped. Can you paste (say at http://apaste.info/) >>> >>> From: Matt Foley <mfo...@hortonworks.com> >>> Date: Monday, September 16, 2013 4:58 PM >>> >>> We had a planned network outage this weekend, which inadvertently >> resulted >>> in making the NFS Shared Primary Storage (used by System VMs) unavailable >>> for a day and a half. (Guest VMs use local storage only, but System VMs >>> use shared storage only.) Cloudstack was not brought down prior to the >>> outage. >>> >>> After network came back, we gracefully brought down all services >> including >>> cloudstack-management, mysql, and NFS, then actually rebooted all servers >>> in the cluster and the NFS server (to make sure no stale file handles), >>> then brought up services in the appropriate order. Also checked mysql >> for >>> table corruption, and found none. Confirmed that the NFS volumes are >>> mountable from all hosts, and in fact Shared Primary Storage is being >>> mounted by cloudstack on hosts as usual, under /mnt/<uuid>. >>> >>> Nevertheless, when try to bring up the cluster, we fail to start the >>> system VMs, with errors "InsufficientServerCapacityException: Unable to >>> create a deployment for VM". The cause is not really insufficient >>> capacity, as actual usage of resources is tiny; these error messages are >>> false explanations of the failure to create primary storage volume for >> the >>> System VMs. >>> >>> Digging into management-server.log, the core issue seems to be the ~160 >>> line snippet from the log attached to this message as >>> cloudstack_debug_2013.09.16.log. The only Shared Primary Storage pool is >>> pool 201, named "cs-primary". It is mounted on all hosts as >>> /mnt/9c6fd9a3-43e5-389a-9594-faecf178b4b9, which is its uuid. The log >>> shows the management server correctly identifying a particular host as >>> being able to access pool 201, then trying to allocate a primary storage >>> volume using the template with uuid f23a16e7-b628-429e-83e1-698935588465. >>> It fails, but I cannot tell why. I suspect its claim that "Template 3 >> has >>> already been downloaded to pool 201" is false, but I don't know how to >>> check this (or fix if wrong). >>> >>> Any guidance for further debugging or fixing this would be GREATLY >>> appreciated. >>> Thanks, >>> --Matt >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >>
