Il 12/02/2015 17:03, Nir Soffer ha scritto: > > > ----- Original Message ----- >> From: "Sandro Bonazzola" <[email protected]> >> To: "Nir Soffer" <[email protected]> >> Cc: "David Teigland" <[email protected]>, "Yeela Kaplan" >> <[email protected]>, "Alon Bar-Lev" <[email protected]>, >> "Allon Mureinik" <[email protected]>, "Yaniv Dary" <[email protected]>, >> "Federico Simoncelli" >> <[email protected]>, "Sahina Bose" <[email protected]>, "Yedidyah Bar >> David" <[email protected]>, "Simone >> Tiraboschi" <[email protected]>, [email protected], "Michal Skrivanek" >> <[email protected]> >> Sent: Thursday, February 12, 2015 5:43:28 PM >> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying hosted-engine >> >> Il 12/02/2015 16:26, Nir Soffer ha scritto: >>> >>> >>> ----- Original Message ----- >>>> From: "Sandro Bonazzola" <[email protected]> >>>> To: "Nir Soffer" <[email protected]>, "David Teigland" >>>> <[email protected]> >>>> Cc: "Yeela Kaplan" <[email protected]>, "Alon Bar-Lev" >>>> <[email protected]>, "Allon Mureinik" <[email protected]>, >>>> "Yaniv Dary" <[email protected]>, "Federico Simoncelli" >>>> <[email protected]>, "Sahina Bose" <[email protected]>, >>>> "Yedidyah Bar David" <[email protected]>, "Simone Tiraboschi" >>>> <[email protected]>, [email protected], "Michal >>>> Skrivanek" <[email protected]> >>>> Sent: Thursday, February 12, 2015 5:22:48 PM >>>> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying hosted-engine >>>> >>>> Il 12/02/2015 16:20, Nir Soffer ha scritto: >>>>> ----- Original Message ----- >>>>>> From: "David Teigland" <[email protected]> >>>>>> To: "Nir Soffer" <[email protected]> >>>>>> Cc: "Sandro Bonazzola" <[email protected]>, "Yeela Kaplan" >>>>>> <[email protected]>, "Alon Bar-Lev" >>>>>> <[email protected]>, "Allon Mureinik" <[email protected]>, "Yaniv >>>>>> Dary" >>>>>> <[email protected]>, "Federico Simoncelli" >>>>>> <[email protected]>, "Sahina Bose" <[email protected]>, "Yedidyah Bar >>>>>> David" <[email protected]>, "Simone >>>>>> Tiraboschi" <[email protected]>, [email protected], "Michal Skrivanek" >>>>>> <[email protected]> >>>>>> Sent: Thursday, February 12, 2015 5:13:49 PM >>>>>> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying >>>>>> hosted-engine >>>>>> >>>>>>>> 2015-02-12 13:08:25+0100 7254 [683]: open error -1 >>>>>>>> /rhev/data-center/mnt/glusterSD/minidell.home:_hosted__engine__glusterfs/a7dba6e3-09ac-430a-8d6e-eea17cf18b8f/images/51156c24-5ea1-42df-bf33-daebf2c4780c/3719b0 >>>>>>>> 06-68a2-4742-964c-413a3f77c0b4.lease >>>>>> >>>>>> open(2) error was EPERM >>>>> >>>>> selinux acting again? >>>>> >>>>> Sandro, can you attach also /var/log/audit/audit.log? >>>> >>>> Sure, but other than what I already reported in >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1191989 >>>> I don't think you'll find much more. >>> >>> Can you reproduce this in permissive mode? >> >> Yes, it's fully reproducible on a host in permissive mode. >> >> Also writing to the storage works: >> >> while true; do echo starting dd `date`; dd if=/dev/zero of=/tmp/test/testfile >> bs=4k count=1k oflag=direct ; echo dd finished `date`; rm -f >> /tmp/test/testfile ; sleep 10; done > > What does "df -T /tmp" show? >
# df -T /tmp /tmp/test/ File system Tipo 1K-blocchi Usati Disponib. Uso% Montato su /dev/sda6 ext4 40185112 5347916 32772812 15% / 192.168.1.107:/hosted_engine_glusterfs fuse.glusterfs 40185088 5347968 32772736 15% /tmp/test > /tmp is typically tempfs, so your dd test never touched any storage. > > Also, copying zeros can be ignored by smart storage, so better copy > "real" file data: > > dd if=/dev/urandom of=testfile bs=1M count=1 > > Now use testfile for writing to storage: > > dd if=testfile of=/path/to/glusterfs/mount/testfile bs=1M count=1 > oflag=direct > > You should do something like: > > 1. mount your gluster volume - same as vdsm does > 2. start the write test, writing to the gluster storage > 3. start your hosted engine flow > 4. when it fails, keep your write test running until for a while If you prefer I can also do a double test, one on /tmp/test and one on /rhev/data-center/mnt/glusterSD/mididell.home\:_hosted__engine__glusterfs/ I'll try to do the test tomorrow. > >> >> >> didn't show any issue while sanlock is failing. >> >> Attached the tar.gz of the ovirt-hosted-engine-setup I'm running and the full >> /var/log directory content. >> >> Note that both glusterd and vdsmd are running on the same host for the >> hyper-converged setup. >> >>> >>>>>> >>>>>>>> 2015-02-12 13:08:25+0100 7254 [683]: r4 release_token open error -2 >>>>>> >>>>>> open(2) error was ENOENT >>>>>> >>>>>>>> 2015-02-12 13:08:31+0100 7260 [16910]: a7dba6e3 aio collect 0 >>>>>>>> 0x7f1d640008c0:0x7f1d640008d0:0x7f1d7ec52000 result -107:0 match res >>>>>> >>>>>> aio result on open file was ENOTCONN >>>>>> >>>>>> Maybe gluster logged some errors about what happened here? >>>>>> >>>>>> >>>> >>>> >>>> -- >>>> Sandro Bonazzola >>>> Better technology. Faster innovation. Powered by community collaboration. >>>> See how it works at redhat.com >>>> >> >> >> -- >> Sandro Bonazzola >> Better technology. Faster innovation. Powered by community collaboration. >> See how it works at redhat.com >> -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
