On Sat, Jul 16, 2016 at 9:53 PM, Jesper Led Lauridsen TS Infra server < [email protected]> wrote:
> On 07/16/2016 04:10 AM, Pranith Kumar Karampuri wrote: > > > > On Fri, Jul 15, 2016 at 5:20 PM, Jesper Led Lauridsen TS Infra server < > <[email protected]>[email protected]> wrote: > >> Hi, >> >> How do I determine in which log etc. that a healing is in progress or >> startet and how do I if not startet force it. >> >> Additional info, is that I have some problem with a volume if I execute >> 'gluster volume heal <volume> info' the command just hangs but if I >> execute 'gluster volume heal <volume> info split-brain' it return that no >> file are in split-brain. Yet there is and I have successfully recovered >> another one. >> > > If the command hangs there is a chance that operations on the file may > have lead to stale locks. Could you give the output of statedump? > You can follow > https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/ to > generat the files. > > > Thanks for you response. You are right there was a stale lock. But I am > sorry I booted all my cluster nodes, so I guess (without knowing) that > there is no reason to give you the output of a statedump? > > What I can confirm and give of information is: > * All the servers failed to reboot so I had to push the button. They all > failed with the message > "Unmounting pipe file system: Cannot create link /etc/mtab~ > Perhaps there is a stale lock file?" > * After 2 nodes had rebooted the command executed without any problem > and reported a couple off split-brain (Both Directory and Files) > * strace the command showed that it was just looping, so basically the > command didn't hanging. It just couldn't finish. > * I am using "glusterfs-3.6.2-1.el6.x86_64". But hoping to upgrade to > 3.6.9 this weekend. > * The file I refereed to here. Now has the same output on both replicas > when getting getfattr information. The > grusted.afr.glu_rhevtst_dr2_data_01-client-[0,1] and trusted.afr.dirty are > now all zero > If you are anyway looking to upgrade, why not upgrade to 3.7.13, which is the latest stable version? > > > > >> I just have problem with this one. I can determine if there is a healing >> process running or not >> >> I have change 'trusted.afr.glu_rhevtst_dr2_data_01-client-1' to >> 0x000000000000000000000000 on the file located on glustertst03 and executed >> a 'ls -lrt' on the file on the gluster-mount. >> >> [root@glustertst04 ]# getfattr -d -m . -e hex >> /bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> getfattr: Removing leading '/' from absolute path names >> # file: >> bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> >> security.selinux=0x73797374656d5f753a6f626a6563745f723a66696c655f743a733000 >> trusted.afr.dirty=0x000000000000000000000000 >> trusted.afr.glu_rhevtst_dr2_data_01-client-0=0x00004c700000000000000000 >> trusted.afr.glu_rhevtst_dr2_data_01-client-1=0x000000000000000000000000 >> trusted.gfid=0x7575f870875b4c899fd81ef16be3b1a1 >> >> trusted.glusterfs.quota.70145d52-bb80-42ce-b437-64be6ee4a7d4.contri=0x00000001606dc000 >> trusted.pgfid.70145d52-bb80-42ce-b437-64be6ee4a7d4=0x00000001 >> >> [root@glustertst03 ]# getfattr -d -m . -e hex >> /bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> getfattr: Removing leading '/' from absolute path names >> # file: >> bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> >> security.selinux=0x73797374656d5f753a6f626a6563745f723a66696c655f743a733000 >> trusted.afr.dirty=0x000000270000000000000000 >> trusted.afr.glu_rhevtst_dr2_data_01-client-0=0x000000000000000000000000 >> trusted.afr.glu_rhevtst_dr2_data_01-client-1=0x000000000000000000000000 >> trusted.gfid=0x7575f870875b4c899fd81ef16be3b1a1 >> >> trusted.glusterfs.quota.70145d52-bb80-42ce-b437-64be6ee4a7d4.contri=0x0000000160662000 >> trusted.pgfid.70145d52-bb80-42ce-b437-64be6ee4a7d4=0x00000001 >> >> [root@glustertst04 ]# stat >> /var/run/gluster/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> File: >> `/var/run/gluster/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab' >> Size: 21474836480 Blocks: 11548384 IO Block: 131072 regular file >> Device: 31h/49d Inode: 11517990069246079393 Links: 1 >> Access: (0660/-rw-rw----) Uid: ( 36/ vdsm) Gid: ( 36/ kvm) >> Access: 2016-07-15 13:33:47.860224289 +0200 >> Modify: 2016-07-15 13:34:44.396125458 +0200 >> Change: 2016-07-15 13:34:44.397125492 +0200 >> >> [root@glustertst03 ]# stat >> /bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab >> File: >> `/bricks/brick1/glu_rhevtst_dr2_data_01/6bdc67d1-4ae5-47e3-86c3-ef0916996862/images/7669ca25-028e-40a5-9dc8-06c716101709/a1ae3612-bb89-45d8-8041-134c34592eab' >> Size: 21474836480 Blocks: 11547408 IO Block: 4096 regular file >> Device: fd02h/64770d Inode: 159515 Links: 2 >> Access: (0660/-rw-rw----) Uid: ( 36/ vdsm) Gid: ( 36/ kvm) >> Access: 2016-07-13 08:33:00.000561984 +0200 >> Modify: 2016-07-13 08:32:59.969561154 +0200 >> Change: 2016-07-15 12:52:28.414192052 +0200 >> >> Thanks >> Jesper >> _______________________________________________ >> Gluster-users mailing list >> [email protected] >> http://www.gluster.org/mailman/listinfo/gluster-users >> > > > > -- > Pranith > > > > _______________________________________________ > Gluster-users mailing list > [email protected] > http://www.gluster.org/mailman/listinfo/gluster-users > -- Pranith
_______________________________________________ Gluster-users mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-users
