Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Dear Ravi, Thank you for your mail and info. This is great news if these patches can make it into 3.12.12. I will then upgrade asap. Could anyone confirm in case these patches do not make it into 3.12.12? Because I would then rather wait for the next release. I was already told on this list

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hi mabi, there are a couple of AFR patches  from master that I'm currently back porting to the 3.12 branch: afr: heal gfids when file is not present on all bricks afr: don't update readables if inode refresh failed on all children afr: fix bug-1363721.t failure afr: add quorum checks in pre-op

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hello, I just wanted to let you know that last week I have upgraded my two replica nodes from Debian 8 to Debian 9 so now all my 3 nodes (including aribter) are running Debian 9 with a Linux 4 kernel. Unfortunately I still have the exact same issue. Another detail I might have not mentioned

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hi, Now that this issue has happened a few times I noticed a few things which might be helpful for debugging: - This problem happens when files are uploaded via a cloud app called Nextcloud where the files are encrypted by the app itself on the server side (PHP code) but only rarely and

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Thanks Ravi. Let me know when you have time to have a look. It sort of happens around once or twice per week but today it was 24 files in one go which are unsynched and where I need to manually reset the xattrs on the arbiter node. By the way on this volume I use quotas which I set on specifc

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 05/23/2018 12:47 PM, mabi wrote: Hello, I just wanted to ask if you had time to look into this bug I am encountering and if there is anything else I can do? For now in order to get rid of these 3 unsynched files shall I do the same method that was suggested to me in this thread? Sorry

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hello, I just wanted to ask if you had time to look into this bug I am encountering and if there is anything else I can do? For now in order to get rid of these 3 unsynched files shall I do the same method that was suggested to me in this thread? Thanks, Mabi ​ ‐‐‐ Original Message

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hi Ravi, Please fine below the answers to your questions 1) I have never touched the cluster.quorum-type option. Currently it is set as following for this volume: Option Value -- -

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Hi mabi, Some questions: -Did you by any chance change the cluster.quorum-type option from the default values? -Is filename.shareKey supposed to be any empty file? Looks like the file was fallocated with the keep-size option but never written to. (On the 2 data bricks, stat output shows Size

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Thank you Ravi for your fast answer. As requested you will find below the "stat" and "getfattr" of one of the files and its parent directory from all three nodes of my cluster. NODE 1: File:

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 05/15/2018 12:38 PM, mabi wrote: Dear all, I have upgraded my replica 3 GlusterFS cluster (and clients) last Friday from 3.12.7 to 3.12.9 in order to fix this bug but unfortunately I notice that I still have exactly the same problem as initially posted in this thread. It looks like this

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Dear all, I have upgraded my replica 3 GlusterFS cluster (and clients) last Friday from 3.12.7 to 3.12.9 in order to fix this bug but unfortunately I notice that I still have exactly the same problem as initially posted in this thread. It looks like this bug is not resolved as I just got

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Thanks for clarifying that point. Does this mean that the fix for this bug will get backported to the next 3.12 release? ​​ ‐‐‐ Original Message ‐‐‐ On April 9, 2018 2:31 PM, Ravishankar N wrote: > ​​ > > On 04/09/2018 05:54 PM, Dmitry Melekhov wrote: > > >

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 04/09/2018 05:54 PM, Dmitry Melekhov wrote: 09.04.2018 16:18, Ravishankar N пишет: On 04/09/2018 05:40 PM, mabi wrote: Again thanks that worked and I have now no more unsynched files. You mentioned that this bug has been fixed in 3.13, would it be possible to backport it to 3.12? I am

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

09.04.2018 16:18, Ravishankar N пишет: On 04/09/2018 05:40 PM, mabi wrote: Again thanks that worked and I have now no more unsynched files. You mentioned that this bug has been fixed in 3.13, would it be possible to backport it to 3.12? I am asking because 3.13 is not a long-term release

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 04/09/2018 05:40 PM, mabi wrote: Again thanks that worked and I have now no more unsynched files. You mentioned that this bug has been fixed in 3.13, would it be possible to backport it to 3.12? I am asking because 3.13 is not a long-term release and as such I would not like to have to

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Again thanks that worked and I have now no more unsynched files. You mentioned that this bug has been fixed in 3.13, would it be possible to backport it to 3.12? I am asking because 3.13 is not a long-term release and as such I would not like to have to upgrade to 3.13. ‐‐‐ Original

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 04/09/2018 05:09 PM, mabi wrote: Thanks Ravi for your answer. Stupid question but how do I delete the trusted.afr xattrs on this brick? And when you say "this brick", do you mean the brick on the arbitrer node (node 3 in my case)? Sorry I should have been clearer. Yes the brick on the

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Thanks Ravi for your answer. Stupid question but how do I delete the trusted.afr xattrs on this brick? And when you say "this brick", do you mean the brick on the arbitrer node (node 3 in my case)? ​​ ‐‐‐ Original Message ‐‐‐ On April 9, 2018 1:24 PM, Ravishankar N

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

On 04/09/2018 04:36 PM, mabi wrote: As I was suggested in the past by this mailing list a now ran a stat and getfattr on one of the problematic files on all nodes and at the end a stat on the fuse mount directly. The output is below: NODE1: STAT: File:

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

As I was suggested in the past by this mailing list a now ran a stat and getfattr on one of the problematic files on all nodes and at the end a stat on the fuse mount directly. The output is below: NODE1: STAT: File:

Re: [Gluster-users] New 3.12.7 possible split-brain on replica 3

Here would be also the corresponding log entries on a gluster node brick log file: [2018-04-09 06:58:47.363536] W [MSGID: 113093] [posix-gfid-path.c:84:posix_remove_gfid2path_xattr] 0-myvol-private-posix: removing gfid2path xattr failed on