[Openstack] using Glusterfs for instance storage

2013-04-11 Thread John Paul Walters
Hi, We've started implementing a Glusterfs-based solution for instance storage in order to provide live migration. I've run into a strange problem when using a multi-node Gluster setup that I hope someone has a suggestion to resolve. I have a 12 node distributed/replicated Gluster cluster. I

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread Razique Mahroua
Hi JP,my bet is that this is a writing permissions issue. Does nova has the right to write within the mounted directory? Razique Mahroua-Nuage Corazique.mahr...@gmail.comTel: +33 9 72 37 94 15 Le 11 avr. 2013 à 16:36, John Paul Walters jwalt...@isi.edu a écrit :Hi,We've started implementing a

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread John Paul Walters
Hi Razique, Thanks for chiming in. Yes, nova owns the instances directory that it's writing to. In fact, between the multi-node volume and the single node volume, I gave the same permissions: created a directory instances on the gluster volume, and chown nova.nova instances. The individual

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread Sylvain Bauza
Agree. As for other shared FS, this is *highly* important to make sure Nova UID and GID are consistent in between all compute nodes. If this is not the case, then you have to usermod all instances... -Sylvain Le 11/04/2013 16:49, Razique Mahroua a écrit : Hi JP, my bet is that this is a

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread John Paul Walters
Hi Sylvain, I agree, though I've confirmed that the UID and GID are consistent across both the compute nodes and my Glusterfs nodes. JP On Apr 11, 2013, at 11:22 AM, Sylvain Bauza sylvain.ba...@digimind.com wrote: Agree. As for other shared FS, this is *highly* important to make sure Nova

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread Vishvananda Ishaya
You should check your syslog for app armor denied messages. It is possible app armor is getting in the way here. Vish On Apr 11, 2013, at 8:35 AM, John Paul Walters jwalt...@isi.edu wrote: Hi Sylvain, I agree, though I've confirmed that the UID and GID are consistent across both the

Re: [Openstack] using Glusterfs for instance storage

2013-04-11 Thread Razique Mahroua
Also,you can import manually one instance and see if it boots.$ cd/exports/instances/instances/instance-0242$ virsh define libvirt.xml$ virsh start instance-0242and see if it boots, if so, we should start looking somewhere else Razique Mahroua-Nuage Corazique.mahr...@gmail.comTel: +33 9