Re: [Users] ovirt VM start fails - Perm Denied error
On Wed, Feb 15, 2012 at 02:50:26PM +0530, Deepak C Shetty wrote: On 02/15/2012 01:27 PM, Dan Kenigsberg wrote: On Wed, Feb 15, 2012 at 12:13:39PM +0530, Deepak C Shetty wrote: On 02/14/2012 02:55 PM, Dan Kenigsberg wrote: On Tue, Feb 14, 2012 at 10:36:39AM +0530, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? Hi Dan, Thanks for the hint, after setting virt_use_nfs, it worked for me. Strangely VDSM should have set it, not sure why it didn't. I suppose this is related to the fact that your `semanage` hangs. Please help us understand why. Hello Dan, This is what strace dumped, when i attached to the semanage process. Note that i am only pasting the last few lines.. as the dump was large... Let me know if you need to entire dump... I'd better know if strace seems to be advancing anywhere. Is there a blocking system call? For how long did you wait? (I guess I should have asked for `strace -t`) You are rite, i did not wait enuf for it to finish may be. This time i did.. it took ~ 2.5 mins to finish the semanage cmd. I started with -t this time, so here is the time gap (wait) that i see... 20:12:20 close(5) = 0 20:12:20 open(/etc/selinux/targeted/modules/tmp/netfilter_contexts, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 5 20:12:20 write(5, , 0)= 0 20:12:20 close(5) = 0 20:15:19 open(/etc/selinux/targeted/modules/tmp/commit_num, O_RDONLY) = 5 20:15:19 read(5, 4\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0, 32) = 32 20:15:19 close(5) = 0 I marked the gap with .. let me know if you need the entire dump ? hmm, I wish I'd asked you to use -ff, to see if there's a subthread doing something fishy. Is something interesting written to /var/log/audit /var/log/message at this time? Anyway, I'm not sure that this list has the expertise to properly debug an SELinux problem on your machine. It seems that it merits opening a bug on policycoreutils-python - if you can identify why it takes so long. Dan. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On Tue, Feb 14, 2012 at 10:36:39AM +0530, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? Hi Dan, Thanks for the hint, after setting virt_use_nfs, it worked for me. Strangely VDSM should have set it, not sure why it didn't. I suppose this is related to the fact that your `semanage` hangs. Please help us understand why. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/14/2012 02:55 PM, Dan Kenigsberg wrote: On Tue, Feb 14, 2012 at 10:36:39AM +0530, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? Hi Dan, Thanks for the hint, after setting virt_use_nfs, it worked for me. Strangely VDSM should have set it, not sure why it didn't. I suppose this is related to the fact that your `semanage` hangs. Please help us understand why. Hello Dan, This is what strace dumped, when i attached to the semanage process. Note that i am only pasting the last few lines.. as the dump was large... Let me know if you need to entire dump... write(6, /usr/share/system-config-netboot..., 89) = 89 write(6, /usr/share/system-config-soundca..., 89) = 89 write(6, /usr/share/system-config-kdump/s..., 103) = 103 write(6, /usr/share/system-config-selinux..., 104) = 104 write(6, /usr/share/system-config-rootpas..., 95) = 95 write(6, /usr/share/system-config-securit..., 101) = 101 write(6, /usr/share/system-config-service..., 109) = 109 read(5, , 4096) = 0 close(5)= 0 munmap(0x7f8291b3d000, 4096)= 0 close(6)= 0 close(7)= 0 open(/etc/selinux/targeted/modules/tmp/seusers.final, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 5 write(5, system_u:system_u:s0-s0:c0.c1023..., 106) = 106 close(5)= 0 open(/etc/selinux/targeted/modules/tmp/users_extra, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 5 write(5, user user_u prefix user;\nuser st..., 101) = 101 close(5)= 0 open(/etc/selinux/targeted/modules/tmp/netfilter_contexts, O_WRONLY|O_CREAT|O_TRUNC, 0600) = 5 write(5, , 0) = 0 close(5)= 0 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. Hello Dan, Here is the output you requested... [root@llm56 ~]# ls -lZ /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0 /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso [root@llm56 ~]# getenforce Enforcing [root@llm56 ~]# getsebool virt_use_nfs virt_use_nfs -- off [root@llm56 ~]# groups qemu qemu : qemu kvm Also `su - qemu -s /bin/bash -c 'strings your.iso'` worked for me.. dumped loads of strings on the screen. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/13/2012 08:33 AM, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. Hello Dan, Here is the output you requested... [root@llm56 ~]# ls -lZ /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0 /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso [root@llm56 ~]# getenforce Enforcing [root@llm56 ~]# getsebool virt_use_nfs virt_use_nfs -- off [root@llm56 ~]# groups qemu qemu : qemu kvm Also `su - qemu -s /bin/bash -c 'strings your.iso'` worked for me.. dumped loads of strings on the screen. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users Perhaps I missed that snippet of logs, but is llm56 the hyper-visor? Also, here is another litmus test to help shake out some common NFS issues. Do the following from the hyper-visor: 1. temporarily give user vdsm in /etc/passwd a login shell (e.g. /bin/bash) 2. mkdir /mnt/testmount 3. mount nfs server here:/path/to/iso/storage/domain /mnt/testmount 4. su - vdsm -- Really important. 5. cd 35f880f8-bd0c-4063-b171-2ddaa59e1212/images/---- 6. touch test.txt 7. strings Fedora-16-x86_64-DVD.iso 8. reset vdsm in /etc/passwd to nologin Do 6 and 7 work? Cheers, Keith ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On Mon, Feb 13, 2012 at 08:57:23AM -0500, Keith Robertson wrote: On 02/13/2012 08:33 AM, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. Hello Dan, Here is the output you requested... [root@llm56 ~]# ls -lZ /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0 /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso [root@llm56 ~]# getenforce Enforcing [root@llm56 ~]# getsebool virt_use_nfs virt_use_nfs -- off Vdsm SHOULD configure this to on when it first starts, by running /usr/sbin/semanage boolean -m -S targeted -F /dev/stdin _EOF virt_use_nfs=1 _EOF /usr/sbin/setsebool virt_use_nfs on Please try running this as root to understand why it failed. qemu cannot use NFS when this is off and selinux is enforcing. [root@llm56 ~]# groups qemu qemu : qemu kvm Also `su - qemu -s /bin/bash -c 'strings your.iso'` worked for me.. dumped loads of strings on the screen. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users Perhaps I missed that snippet of logs, but is llm56 the hyper-visor? Also, here is another litmus test to help shake out some common NFS issues. Do the following from the hyper-visor: 1. temporarily give user vdsm in /etc/passwd a login shell (e.g. /bin/bash) 2. mkdir /mnt/testmount 3. mount nfs server here:/path/to/iso/storage/domain /mnt/testmount 4. su - vdsm -- Really important. unrelated comment: if you add '-s /bin/bash' you can avoid steps 1 and 8. 5. cd 35f880f8-bd0c-4063-b171-2ddaa59e1212/images/---- 6. touch test.txt 7. strings Fedora-16-x86_64-DVD.iso 8. reset vdsm in /etc/passwd to nologin Do 6 and 7 work? Cheers, Keith ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/12/2012 10:28 AM, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I hit this same issue last week, on an ovirt-engine that I'd updated from nightly to stable. In order to get around the issue, I completely uninstalled ovirt and removed the /rhev/datacenter directory, which wasn't removed automatically when I uninstalled ovirt. Once I did that, I was able to proceed normally. Jason I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso Here are some of the imp. logs from the engine.log... note that i tried multiple times, so there are multiple entries of ERRORs for each of those tries... 2012-02-13 05:10:04,563 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsDynamicDataVDSCommand] (pool-5-thread-47) FINISH, UpdateVdsDynamicDataVDSCommand, log id: 549dfc2f 2012-02-13 05:10:04,612 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (pool-5-thread-47) Running command: RunVmOnceCommand internal: false. Entities affected : ID: c587bc8e-f800-4ce4-a35e-1dd43480cb58 Type: VM 2012-02-13 05:10:04,617 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (pool-5-thread-47) Cant find VDS to run the VM c587bc8e-f800-4ce4-a35e-1dd43480cb58 on, so this VM will not be run. 2012-02-13 05:10:04,622 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (pool-5-thread-47) Transaction rolled-back for command: org.ovirt.engine.core.bll.RunVmOnceCommand. 2012-02-13 05:11:25,735 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (QuartzScheduler_Worker-10) DbUserCacheManager::refreshAllUserData() - entered 2012-02-13 05:11:52,648 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-1) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:52,791 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-2) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:55,828 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-8) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:56,917 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-2) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:12:11,611 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-10) Running command: UpdateVmCommand internal: false. Entities affected : ID: c587bc8e-f800-4ce4-a35e-1dd43480cb58 Type: VM 2012-02-13 05:12:11,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8080-10) START, IsValidVDSCommand(storagePoolId = 4087fea7-b54a-4318-8d5c-828eff8846f4, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 247cacfc 2012-02-13 05:12:11,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8080-10) FINISH, IsValidVDSCommand, return: true, log id: 247cacfc 2012-02-13 05:12:11,632 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (http--0.0.0.0-8080-10) START, UpdateVMVDSCommand(storagePoolId = 4087fea7-b54a-4318-8d5c-828eff8846f4, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = ----, infoDictionary.size = 1), log id: 1294a3c3 2012-02-13 05:12:11,904 INFO
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/13/2012 08:57 AM, Keith Robertson wrote: On 02/13/2012 08:33 AM, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. Hello Dan, Here is the output you requested... [root@llm56 ~]# ls -lZ /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0 /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso [root@llm56 ~]# getenforce Enforcing [root@llm56 ~]# getsebool virt_use_nfs virt_use_nfs -- off [root@llm56 ~]# groups qemu qemu : qemu kvm Also `su - qemu -s /bin/bash -c 'strings your.iso'` worked for me.. dumped loads of strings on the screen. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users Perhaps I missed that snippet of logs, but is llm56 the hyper-visor? Also, here is another litmus test to help shake out some common NFS issues. Do the following from the hyper-visor: 1. temporarily give user vdsm in /etc/passwd a login shell (e.g. /bin/bash) 2. mkdir /mnt/testmount 3. mount nfs server here:/path/to/iso/storage/domain /mnt/testmount 4. su - vdsm -- Really important. 5. cd 35f880f8-bd0c-4063-b171-2ddaa59e1212/images/---- 6. touch test.txt 7. strings Fedora-16-x86_64-DVD.iso 8. reset vdsm in /etc/passwd to nologin Do 6 and 7 work? Cheers, Keith Here another suggestion, a tool, nfs-check: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob_plain;f=contrib/nfs-check.py;h=1b763c66e053eae2c273048879aa98e634b04d7f;hb=578cfe90239d5c3f6f11f1fc063c148606e991dc -- Cheers Douglas ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] ovirt VM start fails - Perm Denied error
On 02/13/2012 08:10 PM, Dan Kenigsberg wrote: On Mon, Feb 13, 2012 at 08:57:23AM -0500, Keith Robertson wrote: On 02/13/2012 08:33 AM, Deepak C Shetty wrote: On 02/13/2012 03:16 PM, Dan Kenigsberg wrote: On Sun, Feb 12, 2012 at 11:58:05PM +0530, Deepak C Shetty wrote: Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso would you try `ls -lZ` ? Does your /var/log/audit/audit.log shows an selinux problem? What's `getenforce`? And `getsebool virt_use_nfs`? What is `groups qemu`? Does su - qemu -s /bin/bash -c 'strings your.iso' work for you? Regards, Dan. Hello Dan, Here is the output you requested... [root@llm56 ~]# ls -lZ /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. vdsm kvm system_u:object_r:nfs_t:s0 /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso [root@llm56 ~]# getenforce Enforcing [root@llm56 ~]# getsebool virt_use_nfs virt_use_nfs -- off Vdsm SHOULD configure this to on when it first starts, by running /usr/sbin/semanage boolean -m -S targeted -F /dev/stdin _EOF virt_use_nfs=1 _EOF This hangs... tho' i see the below msgs in /var/log/messages... Feb 14 17:39:58 llm56 dbus[1170]: avc: received policyload notice (seqno=5) Feb 14 17:39:58 llm56 setsebool: The virt_use_nfs policy boolean was changed to 1 by root Feb 14 17:39:58 llm56 dbus[1170]: [system] Reloaded configuration Feb 14 17:39:58 llm56 dbus-daemon[1170]: dbus[1170]: avc: received policyload notice (seqno=5) Feb 14 17:39:58 llm56 dbus-daemon[1170]: dbus[1170]: [system] Reloaded configuration I had to kill this from anotehr session, but using setsebool virt_use_nfs 1 worked for me. /usr/sbin/setsebool virt_use_nfs on Please try running this as root to understand why it failed. qemu cannot use NFS when this is off and selinux is enforcing ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] ovirt VM start fails - Perm Denied error
Hi, I have tried this multiple times and i hit the same error. I have 3 storage domains created (iso, data and export) all connected to the DC with DC status as Up and 1 host with status as Up and the same (only) host acting as SPM. I used the engine-iso-uploader utility to upload my .iso to the iso domain. Created a new VM and attached a vdisk of type sparse (thin-prov) and click on Run Once, where i select Attach CD and select my .iso, and change boot order to boot from CD, then disk. But i get this error... VM first-ovirt-vm is down. Exit message internal error process exited while connecting to monitor: qemu-kvm: -drive file=/rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /rhev/data-center/4087fea7-b54a-4318-8d5c-828eff8846f4/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso: Permission denied . I am unable to figure out why.. bcos the user.group perms for the .iso are fine. In fact i logged into the system serving the nfs share and added 0777 perms still i get the same error. Here is the snip of how the perms for .iso look like... ll /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso -rwxr-xr-x. 1 vdsm kvm 3757047808 Feb 13 04:24 /tmp/iso1-domain/35f880f8-bd0c-4063-b171-2ddaa59e1212/images/----/Fedora-16-x86_64-DVD.iso Here are some of the imp. logs from the engine.log... note that i tried multiple times, so there are multiple entries of ERRORs for each of those tries... 2012-02-13 05:10:04,563 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsDynamicDataVDSCommand] (pool-5-thread-47) FINISH, UpdateVdsDynamicDataVDSCommand, log id: 549dfc2f 2012-02-13 05:10:04,612 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (pool-5-thread-47) Running command: RunVmOnceCommand internal: false. Entities affected : ID: c587bc8e-f800-4ce4-a35e-1dd43480cb58 Type: VM 2012-02-13 05:10:04,617 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (pool-5-thread-47) Cant find VDS to run the VM c587bc8e-f800-4ce4-a35e-1dd43480cb58 on, so this VM will not be run. 2012-02-13 05:10:04,622 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (pool-5-thread-47) Transaction rolled-back for command: org.ovirt.engine.core.bll.RunVmOnceCommand. 2012-02-13 05:11:25,735 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (QuartzScheduler_Worker-10) DbUserCacheManager::refreshAllUserData() - entered 2012-02-13 05:11:52,648 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-1) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:52,791 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-2) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:55,828 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-8) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:11:56,917 WARN [org.ovirt.engine.core.bll.GetConfigurationValueQuery] (http--0.0.0.0-8080-2) calling GetConfigurationValueQuery with null version, using default general for version 2012-02-13 05:12:11,611 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-10) Running command: UpdateVmCommand internal: false. Entities affected : ID: c587bc8e-f800-4ce4-a35e-1dd43480cb58 Type: VM 2012-02-13 05:12:11,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8080-10) START, IsValidVDSCommand(storagePoolId = 4087fea7-b54a-4318-8d5c-828eff8846f4, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 247cacfc 2012-02-13 05:12:11,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (http--0.0.0.0-8080-10) FINISH, IsValidVDSCommand, return: true, log id: 247cacfc 2012-02-13 05:12:11,632 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (http--0.0.0.0-8080-10) START, UpdateVMVDSCommand(storagePoolId = 4087fea7-b54a-4318-8d5c-828eff8846f4, ignoreFailoverLimit = false, compatabilityVersion = null, storageDomainId = ----, infoDictionary.size = 1), log id: 1294a3c3 2012-02-13 05:12:11,904 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (http--0.0.0.0-8080-10) FINISH, UpdateVMVDSCommand, log id: 1294a3c3 2012-02-13 05:12:23,502 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IsoPrefixVDSCommand] (http--0.0.0.0-8080-1) START, IsoPrefixVDSCommand(storagePoolId = 4087fea7-b54a-4318-8d5c-828eff8846f4, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 201b9bfd 2012-02-13
