Re: [Openstack] chardev: opening backend file failed: Permission denied
This is a shot in the dark, but is root squash disabled on the NFS export for all nodes? Sent from my iPhone On Jun 21, 2013, at 11:47 AM, Samuel Winchenbach swinc...@gmail.com wrote: Hi all, I had my cluster working on a single node. glance images and nova instances are stored on an nfs mount. I added two more nodes and enabled live-migration. Now when I try to start to an instance I get: nova-scheduler.log - libvirtError: internal error Process exited while reading console log output: chardev: opening backend file failed: Permission denied Here is the traceback from nova-compute.log: http://pastie.org/pastes/8066860/text I have seen references online to this error, but no solutions. Does anyone know what may be causing and better yet, how to fix it? Thanks, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] chardev: opening backend file failed: Permission denied
no_root_squash is set on that that export. -Sam On Fri, Jun 21, 2013 at 1:12 PM, Aaron Knister aaron.knis...@gmail.comwrote: This is a shot in the dark, but is root squash disabled on the NFS export for all nodes? Sent from my iPhone On Jun 21, 2013, at 11:47 AM, Samuel Winchenbach swinc...@gmail.com wrote: Hi all, I had my cluster working on a single node. glance images and nova instances are stored on an nfs mount. I added two more nodes and enabled live-migration. Now when I try to start to an instance I get: nova-scheduler.log - libvirtError: internal error Process exited while reading console log output: chardev: opening backend file failed: Permission denied Here is the traceback from nova-compute.log: http://pastie.org/pastes/8066860/text I have seen references online to this error, but no solutions. Does anyone know what may be causing and better yet, how to fix it? Thanks, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] chardev: opening backend file failed: Permission denied
Here is some more information. I am using Ubuntu 12.04 LTS, I have turned off apparmor. Here is my mount: dedup:/big_pool/os-grizzly on /os-grizzly type nfs (rw,noatime,nolock,tcp,bg,intr,hard,addr=10.54.90.10) root@test1:/# find /os-grizzly -type d | xargs ls -l -d drwxr-xr-x 4 root root 4 Jun 21 13:39 /os-grizzly drwxr-xr-x 3 glance glance 3 Jun 3 10:25 /os-grizzly/glance drwxr-x--- 2 glance glance 4 Jun 19 15:41 /os-grizzly/glance/images drwxr-xr-x 3 nova nova 3 Jun 21 13:39 /os-grizzly/nova drwxr-xr-x 4 nova nova 5 Jun 21 13:13 /os-grizzly/nova/instances drwxr-xr-x 2 nova nova 4 Jun 19 15:42 /os-grizzly/nova/instances/_base drwxr-xr-x 2 nova nova 6 Jun 19 15:42 /os-grizzly/nova/instances/locks root@test1:/# grep -RE ^[^#] /etc/libvirt/*.conf /etc/libvirt/libvirtd.conf:listen_tls = 0 /etc/libvirt/libvirtd.conf:listen_tcp = 1 /etc/libvirt/libvirtd.conf:unix_sock_group = libvirtd /etc/libvirt/libvirtd.conf:unix_sock_rw_perms = 0770 /etc/libvirt/libvirtd.conf:auth_unix_ro = none /etc/libvirt/libvirtd.conf:auth_unix_rw = none /etc/libvirt/libvirtd.conf:auth_tcp = none /etc/libvirt/qemu.conf:dynamic_ownership = 0 I am stumped. It was working fine before I made the changed (dynamic_ownership, listen_tls, listen_tcp, auth_tcp) and started up the other compute nodes. :/ On Fri, Jun 21, 2013 at 1:27 PM, Samuel Winchenbach swinc...@gmail.comwrote: no_root_squash is set on that that export. -Sam On Fri, Jun 21, 2013 at 1:12 PM, Aaron Knister aaron.knis...@gmail.comwrote: This is a shot in the dark, but is root squash disabled on the NFS export for all nodes? Sent from my iPhone On Jun 21, 2013, at 11:47 AM, Samuel Winchenbach swinc...@gmail.com wrote: Hi all, I had my cluster working on a single node. glance images and nova instances are stored on an nfs mount. I added two more nodes and enabled live-migration. Now when I try to start to an instance I get: nova-scheduler.log - libvirtError: internal error Process exited while reading console log output: chardev: opening backend file failed: Permission denied Here is the traceback from nova-compute.log: http://pastie.org/pastes/8066860/text I have seen references online to this error, but no solutions. Does anyone know what may be causing and better yet, how to fix it? Thanks, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] chardev: opening backend file failed: Permission denied
On Fri, Jun 21, 2013 at 01:57:44PM -0400, Samuel Winchenbach wrote: Here is some more information. I am using Ubuntu 12.04 LTS, I have turned off apparmor. Here is my mount: dedup:/big_pool/os-grizzly on /os-grizzly type nfs (rw,noatime,nolock,tcp,bg,intr,hard,addr=10.54.90.10) root@test1:/# find /os-grizzly -type d | xargs ls -l -d drwxr-xr-x 4 root root 4 Jun 21 13:39 /os-grizzly drwxr-xr-x 3 glance glance 3 Jun 3 10:25 /os-grizzly/glance drwxr-x--- 2 glance glance 4 Jun 19 15:41 /os-grizzly/glance/images drwxr-xr-x 3 nova nova 3 Jun 21 13:39 /os-grizzly/nova drwxr-xr-x 4 nova nova 5 Jun 21 13:13 /os-grizzly/nova/instances drwxr-xr-x 2 nova nova 4 Jun 19 15:42 /os-grizzly/nova/instances/_base drwxr-xr-x 2 nova nova 6 Jun 19 15:42 /os-grizzly/nova/instances/locks root@test1:/# grep -RE ^[^#] /etc/libvirt/*.conf /etc/libvirt/libvirtd.conf:listen_tls = 0 /etc/libvirt/libvirtd.conf:listen_tcp = 1 /etc/libvirt/libvirtd.conf:unix_sock_group = libvirtd /etc/libvirt/libvirtd.conf:unix_sock_rw_perms = 0770 /etc/libvirt/libvirtd.conf:auth_unix_ro = none /etc/libvirt/libvirtd.conf:auth_unix_rw = none /etc/libvirt/libvirtd.conf:auth_tcp = none /etc/libvirt/qemu.conf:dynamic_ownership = 0 I am stumped. It was working fine before I made the changed (dynamic_ownership, listen_tls, listen_tcp, auth_tcp) and started up the other compute nodes. :/ Do not set 'dynamic_ownership' to 0. If you do this, you are required to have a mgmt app which knows how to set ownership on all resources used by QEMU. That option was added as a special hack for oVirt which can do that, but OpenStack does not support this. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] chardev: opening backend file failed: Permission denied
Well... a reboot of the clusters solved that problem. Onto the next one. Thanks for the help. On Fri, Jun 21, 2013 at 2:30 PM, Samuel Winchenbach swinc...@gmail.comwrote: I removed dynamic_ownership=0, restarted libvirtd-bin, qemu-kvm and all the nova-* services. I am still getting the same error: libvirtError: internal error Process exited while reading console log output: chardev: opening backend file failed: Permission denied On Fri, Jun 21, 2013 at 2:19 PM, Samuel Winchenbach swinc...@gmail.comwrote: Also, I am curious Ensure that, NOVA-INST-DIR (set with state_path in nova.conf) is same on all hosts. should I set state_path=/os-grizzly/nova (My NFS mount) or leave it as /var/lib/nova on each node and set instances_path=/os-grizzly/nova/instances? It seems like each node should have it's own,unique state_path but I may be wrong. Sam On Fri, Jun 21, 2013 at 2:15 PM, Samuel Winchenbach swinc...@gmail.comwrote: Ahhh a bug in the documentation? http://docs.openstack.org/trunk/openstack-compute/admin/content/configuring-migrations.html On Fri, Jun 21, 2013 at 2:14 PM, Daniel P. Berrange berra...@redhat.com wrote: On Fri, Jun 21, 2013 at 01:57:44PM -0400, Samuel Winchenbach wrote: Here is some more information. I am using Ubuntu 12.04 LTS, I have turned off apparmor. Here is my mount: dedup:/big_pool/os-grizzly on /os-grizzly type nfs (rw,noatime,nolock,tcp,bg,intr,hard,addr=10.54.90.10) root@test1:/# find /os-grizzly -type d | xargs ls -l -d drwxr-xr-x 4 root root 4 Jun 21 13:39 /os-grizzly drwxr-xr-x 3 glance glance 3 Jun 3 10:25 /os-grizzly/glance drwxr-x--- 2 glance glance 4 Jun 19 15:41 /os-grizzly/glance/images drwxr-xr-x 3 nova nova 3 Jun 21 13:39 /os-grizzly/nova drwxr-xr-x 4 nova nova 5 Jun 21 13:13 /os-grizzly/nova/instances drwxr-xr-x 2 nova nova 4 Jun 19 15:42 /os-grizzly/nova/instances/_base drwxr-xr-x 2 nova nova 6 Jun 19 15:42 /os-grizzly/nova/instances/locks root@test1:/# grep -RE ^[^#] /etc/libvirt/*.conf /etc/libvirt/libvirtd.conf:listen_tls = 0 /etc/libvirt/libvirtd.conf:listen_tcp = 1 /etc/libvirt/libvirtd.conf:unix_sock_group = libvirtd /etc/libvirt/libvirtd.conf:unix_sock_rw_perms = 0770 /etc/libvirt/libvirtd.conf:auth_unix_ro = none /etc/libvirt/libvirtd.conf:auth_unix_rw = none /etc/libvirt/libvirtd.conf:auth_tcp = none /etc/libvirt/qemu.conf:dynamic_ownership = 0 I am stumped. It was working fine before I made the changed (dynamic_ownership, listen_tls, listen_tcp, auth_tcp) and started up the other compute nodes. :/ Do not set 'dynamic_ownership' to 0. If you do this, you are required to have a mgmt app which knows how to set ownership on all resources used by QEMU. That option was added as a special hack for oVirt which can do that, but OpenStack does not support this. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
