----- Original Message ----- > From: "Sandro Bonazzola" <[email protected]> > To: "Nir Soffer" <[email protected]> > Cc: "David Teigland" <[email protected]>, "Yeela Kaplan" > <[email protected]>, "Alon Bar-Lev" <[email protected]>, > "Allon Mureinik" <[email protected]>, "Yaniv Dary" <[email protected]>, > "Federico Simoncelli" > <[email protected]>, "Sahina Bose" <[email protected]>, "Yedidyah Bar > David" <[email protected]>, "Simone > Tiraboschi" <[email protected]>, [email protected], "Michal Skrivanek" > <[email protected]> > Sent: Thursday, February 12, 2015 6:11:16 PM > Subject: Re: [ovirt-devel] [HC] Weird issue while deploying hosted-engine > > Il 12/02/2015 17:03, Nir Soffer ha scritto: > > > > > > ----- Original Message ----- > >> From: "Sandro Bonazzola" <[email protected]> > >> To: "Nir Soffer" <[email protected]> > >> Cc: "David Teigland" <[email protected]>, "Yeela Kaplan" > >> <[email protected]>, "Alon Bar-Lev" <[email protected]>, > >> "Allon Mureinik" <[email protected]>, "Yaniv Dary" <[email protected]>, > >> "Federico Simoncelli" > >> <[email protected]>, "Sahina Bose" <[email protected]>, "Yedidyah Bar > >> David" <[email protected]>, "Simone > >> Tiraboschi" <[email protected]>, [email protected], "Michal Skrivanek" > >> <[email protected]> > >> Sent: Thursday, February 12, 2015 5:43:28 PM > >> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying hosted-engine > >> > >> Il 12/02/2015 16:26, Nir Soffer ha scritto: > >>> > >>> > >>> ----- Original Message ----- > >>>> From: "Sandro Bonazzola" <[email protected]> > >>>> To: "Nir Soffer" <[email protected]>, "David Teigland" > >>>> <[email protected]> > >>>> Cc: "Yeela Kaplan" <[email protected]>, "Alon Bar-Lev" > >>>> <[email protected]>, "Allon Mureinik" <[email protected]>, > >>>> "Yaniv Dary" <[email protected]>, "Federico Simoncelli" > >>>> <[email protected]>, "Sahina Bose" <[email protected]>, > >>>> "Yedidyah Bar David" <[email protected]>, "Simone Tiraboschi" > >>>> <[email protected]>, [email protected], "Michal > >>>> Skrivanek" <[email protected]> > >>>> Sent: Thursday, February 12, 2015 5:22:48 PM > >>>> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying > >>>> hosted-engine > >>>> > >>>> Il 12/02/2015 16:20, Nir Soffer ha scritto: > >>>>> ----- Original Message ----- > >>>>>> From: "David Teigland" <[email protected]> > >>>>>> To: "Nir Soffer" <[email protected]> > >>>>>> Cc: "Sandro Bonazzola" <[email protected]>, "Yeela Kaplan" > >>>>>> <[email protected]>, "Alon Bar-Lev" > >>>>>> <[email protected]>, "Allon Mureinik" <[email protected]>, "Yaniv > >>>>>> Dary" > >>>>>> <[email protected]>, "Federico Simoncelli" > >>>>>> <[email protected]>, "Sahina Bose" <[email protected]>, "Yedidyah > >>>>>> Bar > >>>>>> David" <[email protected]>, "Simone > >>>>>> Tiraboschi" <[email protected]>, [email protected], "Michal Skrivanek" > >>>>>> <[email protected]> > >>>>>> Sent: Thursday, February 12, 2015 5:13:49 PM > >>>>>> Subject: Re: [ovirt-devel] [HC] Weird issue while deploying > >>>>>> hosted-engine > >>>>>> > >>>>>>>> 2015-02-12 13:08:25+0100 7254 [683]: open error -1 > >>>>>>>> /rhev/data-center/mnt/glusterSD/minidell.home:_hosted__engine__glusterfs/a7dba6e3-09ac-430a-8d6e-eea17cf18b8f/images/51156c24-5ea1-42df-bf33-daebf2c4780c/3719b0 > >>>>>>>> 06-68a2-4742-964c-413a3f77c0b4.lease > >>>>>> > >>>>>> open(2) error was EPERM > >>>>> > >>>>> selinux acting again? > >>>>> > >>>>> Sandro, can you attach also /var/log/audit/audit.log? > >>>> > >>>> Sure, but other than what I already reported in > >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1191989 > >>>> I don't think you'll find much more. > >>> > >>> Can you reproduce this in permissive mode? > >> > >> Yes, it's fully reproducible on a host in permissive mode. > >> > >> Also writing to the storage works: > >> > >> while true; do echo starting dd `date`; dd if=/dev/zero > >> of=/tmp/test/testfile > >> bs=4k count=1k oflag=direct ; echo dd finished `date`; rm -f > >> /tmp/test/testfile ; sleep 10; done > > > > What does "df -T /tmp" show? > > > > > # df -T /tmp /tmp/test/ > File system Tipo 1K-blocchi Usati > Disponib. Uso% Montato su > /dev/sda6 ext4 40185112 5347916 > 32772812 15% / > 192.168.1.107:/hosted_engine_glusterfs fuse.glusterfs 40185088 5347968 > 32772736 15% /tmp/test > > > > /tmp is typically tempfs, so your dd test never touched any storage. > > > > Also, copying zeros can be ignored by smart storage, so better copy > > "real" file data: > > > > dd if=/dev/urandom of=testfile bs=1M count=1 > > > > Now use testfile for writing to storage: > > > > dd if=testfile of=/path/to/glusterfs/mount/testfile bs=1M count=1 > > oflag=direct > > > > You should do something like: > > > > 1. mount your gluster volume - same as vdsm does > > 2. start the write test, writing to the gluster storage > > 3. start your hosted engine flow > > 4. when it fails, keep your write test running until for a while > > If you prefer I can also do a double test, one on /tmp/test and one on > /rhev/data-center/mnt/glusterSD/mididell.home\:_hosted__engine__glusterfs/ > I'll try to do the test tomorrow.
Writing to /tmp/test seems ok. It will be nice to see a test using random data instead of zeros. I think David should continue to investigate this to understand why sanlock cannot access storage. > >> didn't show any issue while sanlock is failing. > >> > >> Attached the tar.gz of the ovirt-hosted-engine-setup I'm running and the > >> full > >> /var/log directory content. > >> > >> Note that both glusterd and vdsmd are running on the same host for the > >> hyper-converged setup. > >> > >>> > >>>>>> > >>>>>>>> 2015-02-12 13:08:25+0100 7254 [683]: r4 release_token open error -2 > >>>>>> > >>>>>> open(2) error was ENOENT > >>>>>> > >>>>>>>> 2015-02-12 13:08:31+0100 7260 [16910]: a7dba6e3 aio collect 0 > >>>>>>>> 0x7f1d640008c0:0x7f1d640008d0:0x7f1d7ec52000 result -107:0 match res > >>>>>> > >>>>>> aio result on open file was ENOTCONN > >>>>>> > >>>>>> Maybe gluster logged some errors about what happened here? > >>>>>> > >>>>>> > >>>> > >>>> > >>>> -- > >>>> Sandro Bonazzola > >>>> Better technology. Faster innovation. Powered by community > >>>> collaboration. > >>>> See how it works at redhat.com > >>>> > >> > >> > >> -- > >> Sandro Bonazzola > >> Better technology. Faster innovation. Powered by community collaboration. > >> See how it works at redhat.com > >> > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > _______________________________________________ Devel mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/devel
