Re: [Gluster-users] Conflict resolution

Hi David, On Wed, Oct 20, 2021 at 6:23 AM David Cunningham wrote: > Hello, > > I've a few questions about conflict resolution in a net-split scenario: > > 1. What are the default values for cluster.server-quorum-type and > cluster.server-quorum-ratio? (at the moment "gluster volume info gvol0"

Re: [Gluster-users] FUSE mount option for local time logging?

s --localtime-logging /home > > And the log in /var/log/glusterfs/home.log is in local time now. Thanks! > That's great Alan, I've sent the fix against issue https://github.com/gluster/glusterfs/issues/2798 -Ravi > Looking forward to your trivial patch getting merged soon... > > On T

Re: [Gluster-users] FUSE mount option for local time logging?

On Tue, Sep 14, 2021 at 6:12 PM Alan Orth wrote: > > I have seen that you can change the backend daemon logging since GlusterFS > v3.12.0¹, and there appears to be an interactive --localtime-logging option > for the glusterfs FUSE mount program, but there is no equivalent mount > option listed

Re: [Gluster-users] Read from fastest node only

o the chosen brick. -Ravi > > On Tue, 10 Aug 2021 at 16:49, Ravishankar N wrote: > >> >> >> On Tue, Aug 10, 2021 at 8:07 AM David Cunningham < >> dcunning...@voisonics.com> wrote: >> >>> Hi Gionatan, >>> >>> Thanks for that

Re: [Gluster-users] Read from fastest node only

On Tue, Aug 10, 2021 at 8:07 AM David Cunningham wrote: > Hi Gionatan, > > Thanks for that reply. Under normal circumstances there would be nothing > that needs to be healed, but how can local-node know this is really the > case without checking the other nodes? > > If using local-node tells

Re: [Gluster-users] Read from fastest node only

On Thu, Jul 29, 2021 at 3:16 PM David Cunningham wrote: > Hello, > > Thanks for all the replies. I'll try to address each point: > > 1. "First readable child... Isn't this the first brick in the subvolume" > Does that mean the first brick in the list returned by "gluster volume > status"? > > 2.

Re: [Gluster-users] Issues with glustershd with release 8.4 and 9.1

cmd = F_SETLK fl_type >>> = F_RDLCK, fl_start = 203, fl_end = 203, user_flock: l_type = F_RDLCK, >>> l_start = 203, l_len = 1 >>> -- = -- >>> connected = 1 >>> total_bytes_read = 6665235356 >>> ping_timeout = 42 >>> total_bytes_writte

Re: [Gluster-users] Issues with glustershd with release 8.4 and 9.1

notice is that normally the bricks that are > not connecting are always in the same (remote) node... i.e. they are both > in node 3 in this case. That seems to be always the case, I have not > encountered a scenario where bricks from different nodes are reporting this > issue (at l

Re: [Gluster-users] Issues with glustershd with release 8.4 and 9.1

On Mon, May 17, 2021 at 4:22 PM Marco Fais wrote: > Hi, > > I am having significant issues with glustershd with releases 8.4 and 9.1. > > My oVirt clusters are using gluster storage backends, and were running > fine with Gluster 7.x (shipped with earlier versions of oVirt Node 4.4.x). > Recently

Re: [Gluster-users] Completely filling up a Disperse volume results in unreadable/unhealable files that must be deleted.

On Wed, May 12, 2021 at 2:14 AM Jeff Byers wrote: > Does anyone have any ideas how to prevent, or perhaps > fix the issue described here: > > Completely filling up a Disperse volume results in > unreadable/unhealable files that must be deleted. >

Re: [Gluster-users] No healing, errno 22

On 16/03/21 11:45 pm, Zenon Panoussis wrote: Yes if the dataset is small, you can try rm -rf of the dir from the mount (assuming no other application is accessing them on the volume) launch heal once so that the heal info becomes zero and then copy it over again . I did approximately so; the

Re: [Gluster-users] No healing, errno 22

On 15/03/21 7:39 pm, Zenon Panoussis wrote: I don't know how to interpret this, but it surely looks as if Maildir/.Sent/cur needs to be healed on all three bricks. That can't be possible, logically it doesn't make sense, because if not even one brick has the data of an object, that object

Re: [Gluster-users] No healing, errno 22

On 15/03/21 5:11 pm, Zenon Panoussis wrote: Indeed, enabling granular was only possible when there were 0 files to heal. Re-disabling it, however, did not impose this limitation. Ah yes, this is expected behavior because even if we disable it, there should be enough information to do the

Re: [Gluster-users] No healing, errno 22

On 15/03/21 3:39 pm, Zenon Panoussis wrote: Does anyone know what healing error 22 "invalid argument" is and how to fix it, or at least how to troubleshoot it? while true; do date; gluster volume heal gv0 statistics heal-count; echo -e "--\n"; sleep 297; done Fri Mar 12 14:58:36

Re: [Gluster-users] Failed to populate loc for thin-arbiter

On 09/03/21 11:43 pm, lejeczek wrote: Hi guys, I have a simple volume but which seems to suffer from some problems. (maybe all volumes in the cluster also) ... [2021-03-09 17:59:08.195634] E [MSGID: 114058] [client-handshake.c:1455:client_query_portmap_cbk] 0-USER-HOME-ta-2: failed to

Re: [Gluster-users] possible memory leak in client/fuse mount

and see if shd and bricks come up on that node. -Ravi Thanks Olaf Op do 26 nov. 2020 om 02:43 schreef Ravishankar N mailto:ravishan...@redhat.com>>: On 25/11/20 7:17 pm, Olaf Buitelaar wrote: Hi Ravi, Thanks for checking. Unfortunately this is our production system,

Re: [Gluster-users] possible memory leak in client/fuse mount

create it? Regards, Ravi Thanks Olaf Op wo 25 nov. 2020 om 14:14 schreef Ravishankar N mailto:ravishan...@redhat.com>>: On 25/11/20 5:50 pm, Olaf Buitelaar wrote: Hi Ashish, Thank you for looking into this. I indeed also suspect it has something todo with the 7.X client, beca

Re: [Gluster-users] possible memory leak in client/fuse mount

On 25/11/20 5:50 pm, Olaf Buitelaar wrote: Hi Ashish, Thank you for looking into this. I indeed also suspect it has something todo with the 7.X client, because on the 6.X clients the issue doesn't really seem to occur. I would love to update everything to 7.X, But since the self-heal

Re: [Gluster-users] Docs on gluster parameters

t; filed that gives a short description. HTH, Ravi Best Regards, Strahil Nikolov В понеделник, 16 ноември 2020 г., 10:36:09 Гринуич+2, Ravishankar N написа: On 14/11/20 3:23 am, Mahdi Adnan wrote: Hi,  Differently, the Gluster docs missing quite a bit regarding th

Re: [Gluster-users] Docs on gluster parameters

On 14/11/20 3:23 am, Mahdi Adnan wrote: Hi,  Differently, the Gluster docs missing quite a bit regarding the available options that can be used in the volumes. Not only that, there are some options that might corrupt data and do not have proper documentation, for example, disable Sharding

Re: [Gluster-users] Self-Heal Daemon not starting after upgrade 6.10 to 7.8

On 02/11/20 8:35 pm, Olaf Buitelaar wrote: Dear Gluster users, I'm trying to upgrade from gluster 6.10 to 7.8, i've currently tried this on 2 hosts, but on both the Self-Heal Daemon refuses to start. It could be because not all not are updated yet, but i'm a bit hesitant to continue, without

Re: [Gluster-users] Gluster linear scale-out performance

On 25/07/20 4:35 am, Artem Russakovskii wrote: Speaking of fio, could the gluster team please help me understand something? We've been having lots of performance issues related to gluster using attached block storage on Linode. At some point, I figured out that Linode has a cap of 500 IOPS

Re: [Gluster-users] Possible SHD stalling

On 08/05/20 7:07 pm, Jaco Kroon wrote: I'm not sure "stuck" is the right word, but looking at the "statistics heal-count" values it goes into a form of "go slow" mode, and ends up adding more entries for heal in some cases at a rate of about 2 every second, sometimes 4 at worst (based on

Re: [Gluster-users] Lightweight read

On 24/04/20 11:42 am, David Cunningham wrote: Hello, My understanding is that GlusterFS checks with all nodes when performing a read. Is it possible to just get the data from the node directly being accessed (in our case using the GlusterFS client), without consulting with the other nodes?

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 17/04/20 10:35 am, Amar Tumballi wrote: This thread has been one of the largest effort to stabilize the systems in recent times. Thanks for patience and number of retries you did, Erik! Thanks indeed! Once https://review.gluster.org/#/c/glusterfs/+/24316/ gets merged on master, I will

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 16/04/20 8:04 pm, Erik Jacobson wrote: Quick update just on how this got set. gluster volume set cm_shared performance.parallel-readdir on Is something we did turn on, thinking it might make our NFS services faster and not knowing about it possibly being negative. Below is a diff of the

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 16/04/20 6:54 pm, Erik Jacobson wrote: The patch by itself is only making changes specific to AFR, so it should not affect other translators. But I wonder how readdir-ahead is enabled in your gnfs stack. All performance xlators are turned off in gnfs except write-behind and AFAIK, there is

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

gluster74, which was your intention, and report back. On Wed, Apr 15, 2020 at 10:33:01AM -0500, Erik Jacobson wrote: Attached the wrong patch by mistake in my previous mail. Sending the correct one now. Early results loook GREAT !! We'll keep beating on it. We applied it to glsuter72 as that is what

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

Attached the wrong patch by mistake in my previous mail. Sending the correct one now. -Ravi On 15/04/20 2:05 pm, Ravishankar N wrote: On 10/04/20 2:06 am, Erik Jacobson wrote: Once again thanks for sticking with us. Here is a reply from Scott Titus. If you have something for us to try

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

logic in AFR. Can you apply the attached patch and see if it fixes the split-brain issue? It should apply cleanly on glusterfs-7.4. Thanks, Ravi >From 4389908252c886c22897d8c52c0ce027a511453f Mon Sep 17 00:00:00 2001 From: Ravishankar N Date: Mon, 24 Dec 2018 13:00:19 +0530 Subject: [PA

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 08/04/20 9:55 pm, Erik Jacobson wrote: 9439138:[2020-04-08 15:48:44.737590] E [afr-common.c:754:afr_inode_event_gen_reset] (-->/usr/lib64/glusterfs/7.2/xlator/cluster/replicate.so(+0x6f735) [0x7fa4fb1cb735] -->/usr/lib64/glusterfs/7.2/xlator/cluster/replicate.so(+0x6f0b9)

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 08/04/20 4:59 am, Erik Jacobson wrote: Apologies for misinterpreting the backtrace. #0  afr_read_txn_refresh_done (frame=0x7ffcf4146478, this=0x7fff64013720, err=5) at afr-read-txn.c:312 #1  0x7fff68938d2b in afr_txn_refresh_done (frame=frame@entry=0x7ffcf4146478,

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

afr_final_errno() is called from many other places other than the inode refresh code path, so the 2 (ENOENT) could be from one of those (mostly afr_lookup_done) but it is puzzling that you are not seeing EIO even once when it is called from afr_inode_refresh_subvol_cbk() code path. Not sure

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 04/04/20 9:12 pm, Erik Jacobson wrote: This leaves us with afr_quorum_errno() returning the error. afr_final_errno() iterates through the 'children', looking for valid errors within the replies for the transaction (refresh transaction?). The function returns the highest valued error, which

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 01/04/20 8:57 am, Erik Jacobson wrote: Here are some back traces. They make my head hurt. Maybe you can suggest something else to try next? In the morning I'll try to unwind this myself too in the source code but I suspect it will be tough for me. (gdb) break

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

From your reply in the other thread, I'm assuming that the file/gfid in question is not in genuine split-brain or needing heal. i.e. for example with that 1 brick down and 2 bricks up test case, if you tried to read the file from say a temporary fuse mount (which is also now connected to only

Re: [Gluster-users] gnfs split brain when 1 server in 3x1 down (high load) - help request

On 29/03/20 9:40 am, Erik Jacobson wrote: Hello all, I am getting split-brain errors in the gnfs nfs.log when 1 gluster server is down in a 3-brick/3-node gluster volume. It only happens under intense load. In the lab, I have a test case that can repeat the problem on a single subvolume

Re: [Gluster-users] interpreting heal info and reported entries

On 30/01/20 9:28 pm, Strahil Nikolov wrote: On January 30, 2020 8:21:18 AM GMT+02:00, Ravishankar N wrote: On 30/01/20 11:41 am, Ravishankar N wrote: I think for some reason setting of AFR xattrs on the parent dir did not happen, which is why the files are stuck in split-brain (instead

Re: [Gluster-users] interpreting heal info and reported entries

On 30/01/20 11:41 am, Ravishankar N wrote: I think for some reason setting of AFR xattrs on the parent dir did not happen, which is why the files are stuck in split-brain (instead of getting recreated on repo2 using the files from repo0 or 1). Can you provide the getfattr output

Re: [Gluster-users] interpreting heal info and reported entries

On 29/01/20 9:56 pm, Cox, Jason wrote: I have glusterfs (v6.6) deployed with 3-way replication used by ovirt (v4.3). I recently updated 1 of the nodes (now at gluster v6.7) and rebooted. When it came back online, glusterfs reported there were entries to be healed under the 2 nodes that

Re: [Gluster-users] Replicated volume load balancing

On 27/01/20 1:14 pm, Stefan wrote: Hi, using GlusterFS 6.7 with a 2+1 replicated volume, how are read requests load balanced, if at all? The reason I ask is that we see consistently higher usage/load on the first brick when compared to the second data brick. Are there any parameters to

Re: [Gluster-users] healing does not heal

On 08/01/20 7:56 pm, lejeczek wrote: On 08/01/2020 11:28, Ravishankar N wrote: On 08/01/20 3:55 pm, lejeczek wrote: On 08/01/2020 02:08, Ravishankar N wrote: On 07/01/20 8:07 pm, lejeczek wrote: Which process should I be gdbing, selfheal's? No the brick process on one of the nodes where

Re: [Gluster-users] healing does not heal

On 08/01/20 3:55 pm, lejeczek wrote: On 08/01/2020 02:08, Ravishankar N wrote: On 07/01/20 8:07 pm, lejeczek wrote: Which process should I be gdbing, selfheal's? No the brick process on one of the nodes where file is missing. okey, would you mind showing exec/cmd for debug? I want

Re: [Gluster-users] healing does not heal

On 07/01/20 8:07 pm, lejeczek wrote: Which process should I be gdbing, selfheal's? No the brick process on one of the nodes where file is missing. Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968

Re: [Gluster-users] healing does not heal

On 07/01/20 4:38 pm, lejeczek wrote: 3. These files which the brick/replica shows appear to exist on only that very brick/replica: Right, so the mknods are failing on the other 2 bricks (as seen from the log) when selfheal is trying to recreate it. 4. I do not see, on the brick/replica in

Re: [Gluster-users] healing does not heal

On 06/01/20 8:12 pm, lejeczek wrote: And when I start this volume, in log on the brick which shows gfids: I assume these messages are from the self-heal daemon's log (glustershd.log). Correct me if I am mistaken. ... [2020-01-06 14:28:24.119506] E [MSGID: 114031]

Re: [Gluster-users] gluster volume heal info takes a long time

On 19/12/19 6:18 pm, Sander Hoentjen wrote: Hi guys, In our setup, `gluster volume heal XXX info` sometimes takes a very long time (6 minutes for example). We noticed this because we run oVirt, and get a message in the Events log "VDSM $hostname command GetGlusterVolumeHealInfoVDS failed:

Re: [Gluster-users] Fwd: VM freeze issue on simple gluster setup.

On 12/12/19 4:01 am, WK wrote: so I can get some sort of resolution on the issue (i.e. is it hardware, Gluster etc) I guess what I really need to know is 1) Node 2 complains that it cant reach node 1 and node 3.  If this was an OS/Hardware networking issue and not internal to Gluster ,

Re: [Gluster-users] Use GlusterFS as storage for images of virtual machines - available issues

On 27/11/19 5:52 pm, Dmitry Melekhov wrote: 27.11.2019 16:16, Ravishankar N пишет: [Re-adding the mailing-list] On 27/11/19 5:35 pm, Gregor Burck wrote: Hi Ravi, https://lists.gluster.org/pipermail/gluster-users/2019-July/036858.html yes, seem so. So I've to took it in the init secure

Re: [Gluster-users] Use GlusterFS as storage for images of virtual machines - available issues

[Re-adding the mailing-list] On 27/11/19 5:35 pm, Gregor Burck wrote: Hi Ravi, https://lists.gluster.org/pipermail/gluster-users/2019-July/036858.html yes, seem so. So I've to took it in the init secure for a normal shotdown in may debian system. I wonder why there is no gracefull funktion

Re: [Gluster-users] Use GlusterFS as storage for images of virtual machines - available issues

On 27/11/19 4:04 pm, Gregor Burck wrote: But when I shutdown or restart one of the nodes, I got hanger. This is, what I didn't understand, I think, the glusterFS is redudant and high available? What I do: 1. all is running 2. I stop one node Please check this thread and see if it helps:

Re: [Gluster-users] [Gluster-Maintainers] [Gluster-devel] Modifying gluster's logging mechanism

On 22/11/19 3:13 pm, Barak Sason Rofman wrote: This is actually one of the main reasons I wanted to bring this up for discussion - will it be fine with the community to run a dedicated tool to reorder the logs offline? I think it is a bad idea to log without ordering and later relying on an

Re: [Gluster-users] Self-heals gone wild

On 08/10/19 11:24 pm, Jamie Lawrence wrote: Hello, I recently stood up a 3x2 (soon to be 3x3) distribute-replicate volume on 5.9, running on Centos 7.7. Volume Name: test_stage1_shared Type: Distributed-Replicate Volume ID: 99674d15-7dce-480e-b642-eaf7da72c1a1 Status: Started Snapshot Count:

Re: [Gluster-users] split-brain errors under heavy load when one brick down

On 16/09/19 7:34 pm, Erik Jacobson wrote: Example errors: ex1 [2019-09-06 18:26:42.665050] E [MSGID: 108008] [afr-read-txn.c:123:afr_read_txn_refresh_done] 0-cm_shared-replicate-1: Failing ACCESS on gfid ee3f5646-9368-4151-92a3-5b8e7db1fbf9: split-brain observed. [Input/output error] Okay

Re: [Gluster-users] replace arbiter

On 14/08/19 7:17 PM, richard lucassen wrote: On Wed, 14 Aug 2019 15:44:14 +0200 richard lucassen wrote: I want to set up a new arbiter server, same version same install. Oops: s/set up a new/replace the/ I want to replace the arbiter server, same version same install. The replace-brick/

Re: [Gluster-users] Continue to work in "degraded mode" (missing brick)

On 07/08/19 9:53 PM, Nux! wrote: Hello, I'm testing a replicated volume with 3 bricks. I've killed a brick, but the volume is still mounted and can see the files from the bricks that are still online and can do operations on them. What I cannot do is create new files in the volume, e.g.:

Re: [Gluster-users] GlusterFS Changing Hash of Large Files?

configured: performance.client-io-threads: off nfs.disable: on transport.address-family: inet auth.allow: 10.5.22.* *From:* Ravishankar N *Sent:* Saturday, July 27, 2019 2:04 AM *To:* Matthew Evans ; glu

Re: [Gluster-users] GlusterFS Changing Hash of Large Files?

On 26/07/19 6:50 PM, Matthew Evans wrote: I've got a new glusterfs 4 node replica cluster running under CentOS 7.  All hosts are backed by SSD drives and are connected to a 1Gbps Ethernet network. 3 nodes are running on CentOS 7 under ESXi on the same physical host, 1 is running on CentOS 7

Re: [Gluster-users] Graceful gluster server retire/poweroff

, Gionatan Danti написа: On 16/07/2019 15:27, Ravishankar N wrote: > Yes, if you simply pkill the gluster brick processes of the node before > switching it off, you won't observe the hang on the clients because they > will receive the disconnect notification immediately. But before th

Re: [Gluster-users] Graceful gluster server retire/poweroff

On 16/07/19 3:16 PM, Gionatan Danti wrote: Hi list, I have a replica 3 test cluster and I have a question about how clients behave to an host shutdown. If I suddenly switch off one of the gluster server, the connected clients see a ~42s stall in I/O: this is expected, as it is the default

Re: [Gluster-users] Pending heal status when deleting files which are marked as to be healed

On 17/06/19 3:45 PM, David Spisla wrote: Hello Gluster Community, my newest observation concerns the self heal daemon: Scenario: 2 Node Gluster v5.5 Cluster with Replica 2 Volume. Just one brick per node. Access via SMB Client from a Win10 machine How to reproduce: I have created a small

Re: [Gluster-users] Proper command for replace-brick on distribute–replicate?

hink docs/Administrator Guide/Managing Volumes.md is the file that needs to be updated. -Ravi Thanks, On Tue, Jun 11, 2019 at 7:32 PM Ravishankar N <mailto:ravishan...@redhat.com>> wrote: On 11/06/19 9:11 PM, Alan Orth wrote: Dear list, In a recent discussion on this

Re: [Gluster-users] Proper command for replace-brick on distribute–replicate?

On 11/06/19 9:11 PM, Alan Orth wrote: Dear list, In a recent discussion on this list Ravi suggested that the documentation for replace-brick¹ was out of date. For a distribute–replicate volume the documentation currently says that we need to kill the old brick's PID, create a temporary

Re: [Gluster-users] No healing on peer disconnect - is it correct?

Adding people how can help you better with the heal part. @Karthik Subrahmanya  @Ravishankar N do take a look and answer this part. Is this behaviour correct? I mean No healing is triggered after peer is reconnected back and VMs. Thanks for explanation. BR! Martin

Re: [Gluster-users] Does replace-brick migrate data?

take quite some time to complete. Regards, Ravi Thanks for your support, ¹ https://joejulian.name/post/dht-misses-are-expensive/ On Fri, May 31, 2019 at 7:57 AM Ravishankar N <mailto:ravishan...@redhat.com>> wrote: On 31/05/19 3:20 AM, Alan Orth wrote: Dear Ravi, I s

Re: [Gluster-users] Does replace-brick migrate data?

ick IDs in the volfiles so they reflect where the data actually is? Or perhaps script something to reset all the xattrs on the files/directories to point to the correct bricks? Thank you for any help or pointers, On Wed, May 29, 2019 at 7:24 AM Ravishankar N mailto:ravishan...@

Re: [Gluster-users] Transport endpoint is not connected

    N/A N/A    Y   7634 Task Status of Volume gvol0 -- There are no active volume tasks On Wed, 29 May 2019 at 16:26, Ravishankar N <mailto:ravishan...@redhat.com>> wrote: On 29/05/19 6:

Re: [Gluster-users] Transport endpoint is not connected

On 29/05/19 6:21 AM, David Cunningham wrote: Hello all, We are seeing a strange issue where a new node gfs3 shows another node gfs2 as not connected on the "gluster volume heal" info: [root@gfs3 bricks]# gluster volume heal gvol0 info Brick gfs1:/nodirectwritedata/gluster/gvol0 Status:

Re: [Gluster-users] Does replace-brick migrate data?

On 29/05/19 9:50 AM, Ravishankar N wrote: On 29/05/19 3:59 AM, Alan Orth wrote: Dear Ravishankar, I'm not sure if Brick4 had pending AFRs because I don't know what that means and it's been a few days so I am not sure I would be able to find that information. When you find some time, have

Re: [Gluster-users] Does replace-brick migrate data?

for files and symlinks for directories are present etc.) . Regards, Ravi Thanks, ¹ https://lists.gluster.org/pipermail/gluster-users/2018-February/033584.html On Fri, May 24, 2019 at 4:59 PM Ravishankar N <mailto:ravishan...@redhat.com>> wrote: On 23/05/19 2:40 AM, Alan O

Re: [Gluster-users] remove-brick failure on distributed with 5.6

Adding a few DHT folks for some possible suggestions. -Ravi On 23/05/19 11:15 PM, bran...@thinkhuge.net wrote: Does anyone know what should be done on a glusterfs v5.6 "gluster volume remove-brick" operation that fails?  I'm trying to remove 1 of 8 distributed smaller nodes for replacement

Re: [Gluster-users] Does replace-brick migrate data?

On 23/05/19 2:40 AM, Alan Orth wrote: Dear list, I seem to have gotten into a tricky situation. Today I brought up a shiny new server with new disk arrays and attempted to replace one brick of a replica 2 distribute/replicate volume on an older server using the `replace-brick` command: #

Re: [Gluster-users] add-brick: failed: Commit failed

gfs1    N/A N/A    Y   28600 Self-heal Daemon on gfs2    N/A N/A    Y   17614 Task Status of Volume gvol0 -- There are no active volume tasks On Wed, 22 May 2019 at 18:06,

Re: [Gluster-users] gluster 5.6: Gfid mismatch detected

heal-count" there are 0 entries left. Files/directories are there. Happened the first time with this setup, but everything ok now. Thx for your fast help :-) Hubert Am Mi., 22. Mai 2019 um 09:32 Uhr schrieb Ravishankar N : On 22/05/19 12:39 PM, Hu Bert wrote: Hi @ll, today i updated and reboo

Re: [Gluster-users] gluster 5.6: Gfid mismatch detected

On 22/05/19 12:39 PM, Hu Bert wrote: Hi @ll, today i updated and rebooted the 3 servers of my replicate 3 setup; after the 3rd one came up again i noticed this error: [2019-05-22 06:41:26.781165] E [MSGID: 108008] [afr-self-heal-common.c:392:afr_gfid_split_brain_source]

Re: [Gluster-users] add-brick: failed: Commit failed

If you are trying this again, please 'gluster volume set $volname client-log-level DEBUG`before attempting the add-brick and attach the gvol0-add-brick-mount.log here. After that, you can change the client-log-level back to INFO. -Ravi On 22/05/19 11:32 AM, Ravishankar N wrote: On 22/05

Re: [Gluster-users] add-brick: failed: Commit failed

to me David. Basically, '/nodirectwritedata/gluster/gvol0' must be empty and must not have any extended attributes set on it. Why fuse_first_lookup() is failing is a bit of a mystery to me at this point. :-( Regards, Ravi Thank you. On Wed, 22 May 2019 at 13:56, Ravishankar N <mailto:ravis

Re: [Gluster-users] add-brick: failed: Commit failed

twritedata/gluster/gvol0 Brick3: gfs3:/nodirectwritedata/gluster/gvol0 (arbiter) Options Reconfigured: performance.client-io-threads: off nfs.disable: on transport.address-family: inet On Wed, 22 May 2019 at 12:43, Ravishankar N <mailto:ravishan...@redhat.com>> wrote: Hi David,

Re: [Gluster-users] add-brick: failed: Commit failed

Hi David, Could you provide the `getfattr -d -m. -e hex /nodirectwritedata/gluster/gvol0` output of all bricks and the output of `gluster volume info`? Thanks, Ravi On 22/05/19 4:57 AM, David Cunningham wrote: Hi Sanju, Here's what glusterd.log says on the new arbiter server when trying to

Re: [Gluster-users] add-brick: failed: Commit failed

On 17/05/19 5:59 AM, David Cunningham wrote: Hello, We're adding an arbiter node to an existing volume and having an issue. Can anyone help? The root cause error appears to be "----0001: failed to resolve (Transport endpoint is not connected)", as below. Was

Re: [Gluster-users] heal: Not able to fetch volfile from glusterd

On 06/05/19 6:43 PM, Łukasz Michalski wrote: Hi, I have problem resolving split-brain in one of my installations. CenOS 7, glusterfs 3.10.12, replica on two nodes: [root@ixmed1 iscsi]# gluster volume status cluster Status of volume: cluster Gluster process TCP

Re: [Gluster-users] Replica 3: Client access via FUSE failed if two bricks are down

On 12/04/19 8:34 PM, Felix Kölzow wrote: Dear Gluster-Community, I created a test-environment to test a gluster volume with replica 3. Afterwards, I am able to manually mount the gluster volume using FUSE. mount command: mount -t glusterfs  -o backup-volfile-servers=gluster01:gluster02

Re: [Gluster-users] Is "replica 4 arbiter 1" allowed to tweak client-quorum?

On 03/04/19 12:18 PM, Ingo Fischer wrote: Hi All, I had a replica 2 cluster to host my VM images from my Proxmox cluster. I got a bit around split brain scenarios by using "nufa" to make sure the files are located on the host where the machine also runs normally. So in fact one replica could

Re: [Gluster-users] Cross-compiling GlusterFS

On 01/04/19 1:20 PM, François Duport wrote: Hi, I try to cross-compile GlusterFS because I don't want my embedded client Todo it and each time I reset my client. So I want the compile application to be in my rom image. That said in appearance I did succeeded in my cross compilation but

Re: [Gluster-users] Docu - how to debug issues

On 20/03/19 10:29 AM, Strahil wrote: Hello Community, Is there a docu page clearing what information is needed to be gathered in advance in order to help the devs resolve issues ? So far I couldn't find one - but I have missed that. volume info, gluster version of the clients/servers and

Re: [Gluster-users] Possible memory leak via wordpress wordfence plugin behavior in 4.1.16

On 09/03/19 7:15 AM, Brian Litzinger wrote: I have 4 machines running glusterfs and wordpress with the wordfence plugin. The wordfence plugin in all 4 instance pounds away writing and re-writing the file: /mnt/glusterfs/www/openvpn.net/wp-content/wflogs/config-synced.php This is leading to

Re: [Gluster-users] Self/Healing process after node maintenance

On 01/22/2019 02:57 PM, Martin Toth wrote: Hi all, I just want to ensure myself how self-healing process exactly works, because I need to turn one of my nodes down for maintenance. I have replica 3 setup. Nothing complicated. 3 nodes, 1 volume, 1 brick per node (ZFS pool). All nodes running

Re: [Gluster-users] Error in Installing Glusterfs-4.1.6 from tar

Wunused-result]                          setreuid (ruid, -1);                          ^   CCLD     fusermount-glusterfs Amudhan On Thu, Dec 27, 2018 at 4:38 PM Ravishankar N mailto:ravishan...@redhat.com>> wrote: On 12/27/2018 04:26 PM, Amudhan P wrote:

Re: [Gluster-users] Replacing arbiter with thin-arbiter

Hello, thin-arbiter is currently a glusterd2 (https://github.com/gluster/glusterd2/) based solution only supporting only creation of new thin-arbiter volumes. If you want to try it out, you would need to do so using the GD2 command line interface. We're still ironing out a few bugs in it

Re: [Gluster-users] WG: Gluster 4.1.6 slow

Hi Michael, On 12/28/2018 12:39 PM, Prof. Dr. Michael Schefczyk wrote: Dear Ravi, Again thank you very much for your response. I did experiment a bit further in my scenario, with the following result: I could leave 4.1.6 for the smaller machines with all SSD. The performance does seem to be

Re: [Gluster-users] Error in Installing Glusterfs-4.1.6 from tar

On 12/27/2018 04:26 PM, Amudhan P wrote: Hi, I am trying to compile & install Glusterfs-4.1.6 using tar file and I am getting this error message when running `make`. ``` CC       afr-self-heal-name.lo CC       afr.lo In file included from afr.c:18:0: afr-common.c: In function

Re: [Gluster-users] glusterd keeps resyncing shards over and over again

On 12/10/2018 05:06 PM, Atin Mukherjee wrote: Even though the subject says the issue is with glusterd, I think the question is more applicable on heal/shards. Added the relevant folks to help out. On Mon, Dec 10, 2018 at 3:43 PM Chris Drescher > wrote: Let

Re: [Gluster-users] Gluster 4.1.6 slow

On 12/10/2018 12:57 PM, Prof. Dr. Michael Schefczyk wrote: Dear All, For years, I am using virtualization at home on two sets of two hosts running current Centos 7, KVM and Gluster. I am used to pull KVM/libvirt/qemu and Gluster from the ovirt repository, as the KVM/libvirt/qemu versions

Re: [Gluster-users] Gluster distributed replicated setup does not serve read from all bricks belonging to the same replica

pending self-heal that would have made hashed mode worse, or is it about as bad as any brick selection policy? Thanks On Thu, Nov 22, 2018 at 7:59 PM Ravishankar N <mailto:ravishan...@redhat.com>> wrote: On 11/22/2018 07:07 PM, Anh Vo wrote: Thanks Ravi, I will try th

Re: [Gluster-users] Gluster distributed replicated setup does not serve read from all bricks belonging to the same replica

'1' not '0'. You can look at afr_read_subvol_select_by_policy() in the source code to understand the preference of selection. Regards, Ravi On Wed, Nov 21, 2018 at 9:57 PM Ravishankar N <mailto:ravishan...@redhat.com>> wrote: Hi, If there are multiple clients , you can chan

Re: [Gluster-users] Gluster distributed replicated setup does not serve read from all bricks belonging to the same replica

Hi, If there are multiple clients , you can change the 'cluster.read-hash-mode' volume option's value to 2. Then different reads should be served from different bricks for different clients. The meaning of various values for 'cluster.read-hash-mode' can be got from `gluster volume set help`.

Re: [Gluster-users] Deleted file sometimes remains in .glusterfs/unlink

On 11/19/2018 08:18 PM, David Spisla wrote: Hello Gluster Community, sometimes it happens that a file accessed via FUSE or SMB will remain in .glusterfs/unlink after delete it. The command 'df -hT' still prints the volume capacity before the files was deleted. Another observation is that

Re: [Gluster-users] Self-healing not healing 27k files on GlusterFS 4.1.5 3 nodes replica

Okay so for all files and dirs, node 2 seems to be the bad copy. Try the following: 1. On both node 1 and node3, set theafr xattr for dir10: setfattr -n trusted.afr.myvol-pro-client-1 -v 0x00010001

Re: [Gluster-users] Self-healing not healing 27k files on GlusterFS 4.1.5 3 nodes replica

On 11/15/2018 09:17 PM, mabi wrote: ‐‐‐ Original Message ‐‐‐ On Thursday, November 15, 2018 1:41 PM, Ravishankar N wrote: Thanks, noted. One more query. Are there files inside each of these directories? Or is it just empty directories? You will find below the content of each

Re: [Gluster-users] Self-healing not healing 27k files on GlusterFS 4.1.5 3 nodes replica

On 11/15/2018 02:11 PM, mabi wrote: Sure, you will find below the getfattr output of all 3 directories from all 3 nodes. Thanks, noted. One more query. Are there files inside each of these directories? Or is it just empty directories? 2. Do you know the file (or directory) names

  1   2   3   4   5   6   >