Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-08 Thread Abi Askushi
Filed the *Bug 1459855* Alex On Thu, Jun 8, 2017 at 1:16 PM, Abi Askushi wrote: > Hi Denis, > > Ok I will file a bug for this. > I am not sure if I will be able to provide troubleshooting info for much > long as I

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-08 Thread Abi Askushi
Hi Denis, Ok I will file a bug for this. I am not sure if I will be able to provide troubleshooting info for much long as I already have put forward the replacement of disks with 512 ones. Alex On Thu, Jun 8, 2017 at 11:48 AM, Denis Chaplygin wrote: > Hello Alex, > > > On

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-08 Thread Denis Chaplygin
Hello Alex, On Wed, Jun 7, 2017 at 11:39 AM, Abi Askushi wrote: > Hi Sahina, > > Did you have the chance to check the logs and have any idea how this may > be addressed? > It seems to be a VDSM issue, as VDSM uses direct IO (and id actualy calls dd) and assumes that

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-07 Thread Abi Askushi
Hi Sahina, Did you have the chance to check the logs and have any idea how this may be addressed? Thanx, Alex On Mon, Jun 5, 2017 at 12:14 PM, Sahina Bose wrote: > Can we have the gluster mount logs and brick logs to check if it's the > same issue? > > On Sun, Jun 4, 2017

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-06 Thread Abi Askushi
Just to note that the mentioned logs below are from the dd with bs=512, which were failing. Attached the full logs from mount and brick. Alex On Tue, Jun 6, 2017 at 3:18 PM, Abi Askushi wrote: > Hi Krutika, > > My comments inline. > > Also attached the strace of: >

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-06 Thread Abi Askushi
Hi Krutika, My comments inline. Also attached the strace of: strace -y -ff -o /root/512-trace-on-root.log dd if=/dev/zero of=/mnt/test2.img oflag=direct bs=512 count=1 and of: strace -y -ff -o /root/4096-trace-on-root.log dd if=/dev/zero of=/mnt/test2.img oflag=direct bs=4096 count=16 I have

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-06 Thread Krutika Dhananjay
I stand corrected. Just realised the strace command I gave was wrong. Here's what you would actually need to execute: strace -y -ff -o -Krutika On Tue, Jun 6, 2017 at 3:20 PM, Krutika Dhananjay wrote: > OK. > > So for the 'Transport endpoint is not connected' issue,

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-06 Thread Krutika Dhananjay
OK. So for the 'Transport endpoint is not connected' issue, could you share the mount and brick logs? Hmmm.. 'Invalid argument' error even on the root partition. What if you change bs to 4096 and run? The logs I showed in my earlier mail shows that gluster is merely returning the error it got

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Abi Askushi
Also when testing with dd i get the following: *Testing on the gluster mount: * dd if=/dev/zero of=/rhev/data-center/mnt/glusterSD/10.100.100.1:_engine/test2.img oflag=direct bs=512 count=1 dd: error writing β/rhev/data-center/mnt/glusterSD/10.100.100.1:_engine/test2.imgβ: *Transport endpoint is

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Abi Askushi
The question that rises is what is needed to make gluster aware of the 4K physical sectors presented to it (the logical sector is also 4K). The offset (127488) at the log does not seem aligned at 4K. Alex On Mon, Jun 5, 2017 at 2:47 PM, Abi Askushi wrote: > Hi Krutika,

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Abi Askushi
Hi Krutika, I am saying that I am facing this issue with 4k drives. I never encountered this issue with 512 drives. Alex On Jun 5, 2017 14:26, "Krutika Dhananjay" wrote: > This seems like a case of O_DIRECT reads and writes gone wrong, judging by > the 'Invalid argument'

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Krutika Dhananjay
This seems like a case of O_DIRECT reads and writes gone wrong, judging by the 'Invalid argument' errors. The two operations that have failed on gluster bricks are: [2017-06-05 09:40:39.428979] E [MSGID: 113072] [posix.c:3453:posix_writev] 0-engine-posix: write failed: offset 0, [Invalid

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Abi Askushi
Hi Sahina, Attached are the logs. Let me know if sth else is needed. I have 5 disks (with 4K physical sector) in RAID5. The RAID has 64K stripe size at the moment. I have prepared the storage as below: pvcreate --dataalignment 256K /dev/sda4 vgcreate --physicalextentsize 256K gluster /dev/sda4

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-05 Thread Sahina Bose
Can we have the gluster mount logs and brick logs to check if it's the same issue? On Sun, Jun 4, 2017 at 11:21 PM, Abi Askushi wrote: > I clean installed everything and ran into the same. > I then ran gdeploy and encountered the same issue when deploying engine. >

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-04 Thread Maor Lipchuk
On Sun, Jun 4, 2017 at 8:51 PM, Abi Askushi wrote: > I clean installed everything and ran into the same. > I then ran gdeploy and encountered the same issue when deploying engine. > Seems that gluster (?) doesn't like 4K sector drives. I am not sure if it > has to do with

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-04 Thread Abi Askushi
I clean installed everything and ran into the same. I then ran gdeploy and encountered the same issue when deploying engine. Seems that gluster (?) doesn't like 4K sector drives. I am not sure if it has to do with alignment. The weird thing is that gluster volumes are all ok, replicating normally

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-04 Thread Abi Askushi
Hi Maor, My disk are of 4K block size and from this bug seems that gluster replica needs 512B block size. Is there a way to make gluster function with 4K drives? Thank you! On Sun, Jun 4, 2017 at 2:34 PM, Maor Lipchuk wrote: > Hi Alex, > > I saw a bug that might be

Re: [ovirt-users] oVirt gluster sanlock issue

2017-06-04 Thread Maor Lipchuk
Hi Alex, I saw a bug that might be related to the issue you encountered at https://bugzilla.redhat.com/show_bug.cgi?id=1386443 Sahina, maybe you have any advise? Do you think that BZ1386443is related? Regards, Maor On Sat, Jun 3, 2017 at 8:45 PM, Abi Askushi wrote: >