Re: [Gluster-users] Directory in split brain does not heal - Gfs 9.2

2022-08-12 Thread Joe Julian 
Interesting. I've never seen that. It's always been gfid mismatches for 
me since they first started adding gfids to directories.


On 8/12/22 1:27 PM, Strahil Nikolov wrote:

Usually dirs are in split-brain due to content mismatch.
For example, if a file inside a dir can't be healed automaticaly , 
then the dir is "stuck".


Check the files inside the dir for mismatching - fix that first and 
only then try to heal the dir.



Best Regards,
Strahil Nikolov

On Fri, Aug 12, 2022 at 20:26, Péter Károly JUHÁSZ
 wrote:




Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users






Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge:https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Directory in split brain does not heal - Gfs 9.2

2022-08-12 Thread Péter Károly JUHÁSZ 
This always helped for me in this kind of situations:
http://docs.gluster.org/Troubleshooting/resolving-splitbrain/

Joe Julian  于 2022年8月12日周五 18:33写道：

> It could work, but I never imagined, back then, that *directories* could
> get in split-brain.
>
> The most likely reason for that split is that there's a gfid mismatch on
> one of the replicas. I'd go to the brick with the odd gfid, move that
> directory out of the brick path, then do a "find folder" on the client
> mount to rebuild the directory structure. Check the directory to make sure
> all the files are right before deleting the moved odd one.
>
> If you need to fix anything, just copy from the moved directory to a
> client-mount on the same machine.
> On 8/12/22 8:12 AM, Ilias Chasapakis forumZFD wrote:
>
> Dear fellow gluster users,
>
> we are facing a problem with our replica 3 setup. Glusterfs version is 9.2.
>
> We have a problem with a directory that is in split-brain and we cannot
> manage to heal with:
>
> gluster volume heal gfsVol split-brain latest-mtime /folder
>
> The command throws the following error: "failed:Transport endpoint is not
> connected."
>
> So the split brain directory entry remains and and so the whole healing
> process is not completing and other entries get stuck.
>
> I saw there is a python script available
> https://github.com/joejulian/glusterfs-splitbrain Would that be a good
> solution to try? To be honest we are a bit concerned with deleting the gfid
> and the files from the brick manually as it seems it can create
> inconsistencies and break things... I can of course give you more
> information about our setup and situation, but if you already have some
> tip, that would be fantastic.
>
> Best regards
>
> Ilias
>
> --
> forumZFD
> Entschieden für Frieden | Committed to Peace
>
> Ilias Chasapakis
> Referent IT | IT Consultant
>
> Forum Ziviler Friedensdienst e.V. | Forum Civil Peace Service
> Am Kölner Brett 8 | 50825 Köln | Germany
>
> Tel 0221 91273243 | Fax 0221 91273299 | http://www.forumZFD.de
>
> Vorstand nach § 26 BGB, einzelvertretungsberechtigt | Executive Board:
> Oliver Knabe (Vorsitz | Chair), Jens von Bargen, Alexander Mauz
> VR 17651 Amtsgericht Köln
>
> Spenden | Donations: IBAN DE37 3702 0500 0008 2401 01 BIC BFSWDE33XXX
>
>
> 
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://meet.google.com/cpu-eiue-hvk
> Gluster-users mailing 
> listGluster-users@gluster.orghttps://lists.gluster.org/mailman/listinfo/gluster-users
>
> 
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://meet.google.com/cpu-eiue-hvk
> Gluster-users mailing list
> Gluster-users@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
>




Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Directory in split brain does not heal - Gfs 9.2

2022-08-12 Thread Joe Julian 
It could work, but I never imagined, back then, that *directories* could 
get in split-brain.


The most likely reason for that split is that there's a gfid mismatch on 
one of the replicas. I'd go to the brick with the odd gfid, move that 
directory out of the brick path, then do a "find folder" on the client 
mount to rebuild the directory structure. Check the directory to make 
sure all the files are right before deleting the moved odd one.


If you need to fix anything, just copy from the moved directory to a 
client-mount on the same machine.


On 8/12/22 8:12 AM, Ilias Chasapakis forumZFD wrote:


Dear fellow gluster users,

we are facing a problem with our replica 3 setup. Glusterfs version is 
9.2.


We have a problem with a directory that is in split-brain and we 
cannot manage to heal with:



gluster volume heal gfsVol split-brain latest-mtime /folder

The command throws the following error: "failed:Transport endpoint is 
not connected."


So the split brain directory entry remains and and so the whole 
healing process is not completing and other entries get stuck.


I saw there is a python script available 
https://github.com/joejulian/glusterfs-splitbrain 
 Would that be a 
good solution to try? To be honest we are a bit concerned with 
deleting the gfid and the files from the brick manually as it seems it 
can create inconsistencies and break things... I can of course give 
you more information about our setup and situation, but if you already 
have some tip, that would be fantastic.


Best regards

Ilias

--
forumZFD
Entschieden für Frieden | Committed to Peace

Ilias Chasapakis
Referent IT | IT Consultant

Forum Ziviler Friedensdienst e.V. | Forum Civil Peace Service
Am Kölner Brett 8 | 50825 Köln | Germany

Tel 0221 91273243 | Fax 0221 91273299 |http://www.forumZFD.de

Vorstand nach § 26 BGB, einzelvertretungsberechtigt | Executive Board:
Oliver Knabe (Vorsitz | Chair), Jens von Bargen, Alexander Mauz
VR 17651 Amtsgericht Köln

Spenden | Donations: IBAN DE37 3702 0500 0008 2401 01 BIC BFSWDE33XXX





Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge:https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Directory in split brain does not heal - Gfs 9.2

2022-08-12 Thread Ilias Chasapakis forumZFD 

Dear fellow gluster users,

we are facing a problem with our replica 3 setup. Glusterfs version is 9.2.

We have a problem with a directory that is in split-brain and we cannot 
manage to heal with:



gluster volume heal gfsVol split-brain latest-mtime /folder

The command throws the following error: "failed:Transport endpoint is 
not connected."


So the split brain directory entry remains and and so the whole healing 
process is not completing and other entries get stuck.


I saw there is a python script available 
https://github.com/joejulian/glusterfs-splitbrain 
 Would that be a good 
solution to try? To be honest we are a bit concerned with deleting the 
gfid and the files from the brick manually as it seems it can create 
inconsistencies and break things... I can of course give you more 
information about our setup and situation, but if you already have some 
tip, that would be fantastic.


Best regards

Ilias

--
forumZFD
Entschieden für Frieden | Committed to Peace

Ilias Chasapakis
Referent IT | IT Consultant

Forum Ziviler Friedensdienst e.V. | Forum Civil Peace Service
Am Kölner Brett 8 | 50825 Köln | Germany

Tel 0221 91273243 | Fax 0221 91273299 |http://www.forumZFD.de

Vorstand nach § 26 BGB, einzelvertretungsberechtigt | Executive Board:
Oliver Knabe (Vorsitz | Chair), Jens von Bargen, Alexander Mauz
VR 17651 Amtsgericht Köln

Spenden | Donations: IBAN DE37 3702 0500 0008 2401 01 BIC BFSWDE33XXX



OpenPGP_signature
Description: OpenPGP digital signature




Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users