Re: [Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/us

t;> 0-_data1-client-1: parent translators are ready, attempting connect >> on transport >> [2019-02-08 01:13:35.655197] I [MSGID: 114020] [client.c:2354:notify] >> 0-_data1-client-2: parent translators are ready, attempting connect >> on transport >> [2019-02-08 01:13:

Re: [Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/us

; > > On Sat, Feb 9, 2019 at 12:53 AM Artem Russakovskii > wrote: > >> Hi Nithya, >> >> I can try to disable write-behind as long as it doesn't heavily impact >> performance for us. Which option is it exactly? I don't see it set in my >> list of chan

Re: [Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/us

with >> the crashes that I was having. >> >> *João Baúto* >> --- >> >> *Scientific Computing and Software Platform* >> Champalimaud Research >> Champalimaud Center for the Unknown >> Av. Brasília, Doca de Pedrouços >> 14

Re: [Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/us

Awesome. But is there a release schedule and an ETA for when these will be out in the repos? On Mon, Feb 11, 2019, 9:34 PM Raghavendra Gowdappa wrote: > > > On Tue, Feb 12, 2019 at 10:24 AM Artem Russakovskii > wrote: > >> Great job identifying the issue! >> >>

Re: [Gluster-users] Message repeated over and over after upgrade from 4.1 to 5.3: W [dict.c:761:dict_ref] (-->/usr/lib64/glusterfs/5.3/xlator/performance/quick-read.so(+0x7329) [0x7fd966fcd329] -->/us

before then. We will update on > the list with dates once we decide. > > > On Tue, 12 Feb 2019 at 11:46, Artem Russakovskii > wrote: > >> Awesome. But is there a release schedule and an ETA for when these will >> be out in the repos? >> >> On Mon, Feb 11, 201

Re: [Gluster-users] SLES15 packages for v5.4

I'm seeing it here now: http://download.opensuse.org/repositories/home:/glusterfs:/SLES15-5/SLE_15/x86_64/ . And yay, OpenSUSE 15 is up too http://download.opensuse.org/repositories/home:/glusterfs:/Leap15-5/openSUSE_Leap_15.0/x86_64/ . Finally going to end the log spam and the nasty crashes. Si

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

Noticed the same when upgrading from 5.3 to 5.4, as mentioned. I'm confused though. Is actual replication affected, because the 5.4 server and the 3x 5.3 servers still show heal info as all 4 connected, and the files seem to be replicating correctly as well. So what's actually affected - just the

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

Russakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Tue, Mar 5, 2019 at 10:57 AM Artem Russakovskii wrote: > Noticed the same when upgrading from 5.3 to 5.4, as mentioned. > > I'm confused though. Is actual replication affecte

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

vskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Tue, Mar 5, 2019 at 11:09 AM Artem Russakovskii wrote: > Ended up downgrading to 5.3 just in case. Peer status and volume status > are OK now. > > zypper install --oldpackage glusterfs-

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

and packaging. May be ok to expect it on Monday. > > -Amar > > On Thu, Mar 7, 2019 at 11:54 AM Artem Russakovskii > wrote: > >> Is the next release going to be an imminent hotfix, i.e. something like >> today/tomorrow, or are we talking weeks? >> >> Sincerel

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

.com/ArtemR> On Tue, Mar 12, 2019 at 10:28 AM Artem Russakovskii wrote: > Hi Amar, > > Any updates on this? I'm still not seeing it in OpenSUSE build repos. > Maybe later today? > > Thanks. > > Sincerely, > Artem > > -- > Founder, Android Police <h

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

w: no release notes for 5.4 and 5.5 so far? > >> > https://docs.gluster.org/en/latest/release-notes/ ? > >> > > >> > Am Fr., 15. März 2019 um 14:28 Uhr schrieb Shyam Ranganathan > >> > : > >> > > > >> > > We created a

Re: [Gluster-users] Constant fuse client crashes "fixed" by setting performance.write-behind: off. Any hope for a 4.1.8 release?

I upgraded the node that was crashing to 5.5 yesterday. Today, it got another crash. This is a 1x4 replicate cluster, you can find the config mentioned in my previous reports, and Amar should have it as well. Here's the log: ==> mnt-_data1.log <== The message "I [MSGID: 108031] [afr-common.c:2543:

Re: [Gluster-users] Constant fuse client crashes "fixed" by setting performance.write-behind: off. Any hope for a 4.1.8 release?

Brandon, I've had performance.write-behind: off for weeks ever since it was suggested as a fix, but the crashes kept coming. Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | +ArtemRussakovskii

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

Artem, > > I think you are running into a different crash. The ones reported which > were prevented by turning off write-behind are now fixed. > We will need to look into the one you are seeing to see why it is > happening. > > Regards, > Nithya > > > On Tue, 19 Mar 2019 a

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

ssakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Tue, Mar 19, 2019 at 10:46 PM Amar Tumballi Suryanarayan < atumb...@redhat.com> wrote: > > > On Wed, Mar 20, 2019 at 9:52 AM Artem Russakovskii > wrote: > >> Can I

Re: [Gluster-users] Upgrade 5.3 -> 5.4 on debian: public IP is used instead of LAN IP

<http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Wed, Mar 20, 2019 at 12:57 PM Artem Russakovskii wrote: > Amar, > > I see debuginfo packages now and have

[Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

Hi everyone, I'm preparing the upgrade for our 5.10 gluster cluster running 1x4 replicate volumes to 7.X and decided to upgrade our test cluster first. As soon as I upgraded to 7.0 (and now 7.1), I started seeing the following messages every 10 minutes in the log for one of the volumes: [2019-12

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

ied] > Btw, cluster.op-version is 7, which is the latest. I always rev it up after upgrading the cluster. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

Hi, Any updates here please? Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Mon, Jan 6, 2020 at 4:18 PM Artem Russakovskii wro

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

Still no response? Amar? Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Tue, Jan 14, 2020 at 12:57 PM Artem Russakovskii wrote: >

Re: [Gluster-users] Strange Logs

I've been seeing the same thing happen, and in our case, it's because of running a script that checks gluster from time to time ( https://github.com/jtopjian/scripts/blob/master/gluster/gluster-status.sh in our case). Do you have a job that runs and periodically checks for gluster health? Sincere

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Wed, Feb 12, 2020 at 11:02 PM Amar Tumballi wrote: > > > > On Fri, Feb 7, 2020 at 1:56 PM Artem Russakovskii > wrote: > >> Still no response? Amar? >> >> > Sorry, this got d

[Gluster-users] Feature request: max total size for trashcan

Hi all, I just discovered gluster's trashcan support https://docs.gluster.org/en/latest/Administrator%20Guide/Trash/, but I was surprised to see that we can only limit the max file size and not the size of the trashcan itself (like we can on Windows, for example). This ticket is the feature reque

[Gluster-users] Advice for running out of space on a replicated 4-brick gluster

Hi all, We currently have an 8TB 4-brick replicated volume on our 4 servers, and are at 80% capacity. The max disk size on our host is 10TB. I'm starting to think about what happens closer to 100% and see 2 options. Either we go with another new 4-brick replicated volume and start dealing with sy

Re: [Gluster-users] Feature request: max total size for trashcan

sions on does it actually work at all or not. Good to know it is > helping some usecases. > > -Amar > > On Tue, Feb 18, 2020 at 4:23 AM Artem Russakovskii > wrote: > >> Hi all, >> >> I just discovered gluster's trashcan support >> https://docs.glust

Re: [Gluster-users] Advice for running out of space on a replicated 4-brick gluster

irror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Mon, Feb 17, 2020 at 3:29 PM Strahil Nikolov wrote: > On February 18, 2020 1:16:19 AM GMT+02:00, Artem Russakovskii < > archon...@gmail.com> wrote: > >Hi all, > > > >We c

Re: [Gluster-users] Advice for running out of space on a replicated 4-brick gluster

y, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Tue, Feb 18, 2020 at 11:56 PM Strahil Nikolov wrote: > On February 19, 2020 1:59:12 AM

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.3

Hi, I'm not seeing 7.3 here yet: http://download.opensuse.org/repositories/home:/glusterfs:/Leap15.1-7/openSUSE_Leap_15.1/x86_64/ . Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | @ArtemR

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

logical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Tue, Mar 3, 2020 at 8:25 PM Amar Tumballi wrote: > > > On Sat, Feb 15, 2020 at 7:27 AM Amar Tumballi wrote: > >> >> >> On Sat, Feb 15, 2020 at 3:02 AM Artem Russakovskii >> wrote: >>

Re: [Gluster-users] set larger field width for status command

+1 for this one. Cuts off to the next line for me all the time. Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | @ArtemR On Mon, Mar 2, 2020 at 11:06 AM Brian Andrus

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

e next cycle for 5 and 6 are in April. > > On Sat, Mar 7, 2020 at 1:00 PM Artem Russakovskii > wrote: > >> Great, I'll be glad to test and confirm once the release with this change >> is out. Would you please confirm when it is by replying here? >> >> Tha

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.3

te: > Hi Artem, > > It seems the GlusterFS 7.3 package was created but was not built > successfully due to some error. I have initiated the building process > again. Expect the package to be in the repo soon > > Regards, > Sheetal Pamecha > > > On Mon, Mar 16, 2020 at

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.4

Typo here:"cashes" #1785323: glusterfsd cashes after a few seconds Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | @ArtemR On Thu, Mar 19, 2020 at 11:57 PM Rinku Kot

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

t; >> >> On Mon, Mar 9, 2020 at 1:41 AM Hari Gowtham wrote: >> >>> The release for 7 will happen every month while the release 5 and 6 are >>> done once in every 2 months. >>> We are done for February, so the next cycle for 5 and 6 are in A

[Gluster-users] gluster 5.11 issue: Endless performing entry selfheal / Completed entry selfheal on the same pending heal files after server reboot + very high load

Hi all, I've been observing this problem for a long time now and it's time to finally figure out what's going on. We're running gluster 5.11 and have a 10TB 1 x 4 = 4 replicate volume. I'll include its slightly redacted config below. When I reboot one of the servers and it goes offline for a bit

Re: [Gluster-users] Announcing Gluster release 5.13

Hi, 5.12 and 5.13 are missing from OpenSUSE 15.1: https://download.opensuse.org/repositories/home:/glusterfs:/Leap15.1-5/openSUSE_Leap_15.1/x86_64/ . I see them for OpenSUSE 15.2 though, but we are running 15.1. Please add these builds to 15.1, as they contain important fixes. Thanks. Sincerely

Re: [Gluster-users] Announcing Gluster release 5.13

et | @ArtemR <http://twitter.com/ArtemR> On Sun, Apr 26, 2020 at 3:22 PM Artem Russakovskii wrote: > Hi, > > 5.12 and 5.13 are missing from OpenSUSE 15.1: > https://download.opensuse.org/repositories/home:/glusterfs:/Leap15.1-5/openSUSE_Leap_15.1/x86_64/ > . > > I se

Re: [Gluster-users] remote operation failed [Permission denied] every 10 minutes after upgrading from 5.10 to 7.0

pla.net | @ArtemR <http://twitter.com/ArtemR> On Mon, Mar 23, 2020 at 10:26 PM Hari Gowtham wrote: > The patch is not there in 7.4. It is yet to be backported. It should soon > be backported to the release branches. > > On Tue, Mar 24, 2020 at 3:24 AM Artem Russakovskii > wrote:

Re: [Gluster-users] Announcing Gluster release 5.13

ely, >> Artem >> >> -- >> Founder, Android Police <http://www.androidpolice.com>, APK Mirror >> <http://www.apkmirror.com/>, Illogical Robot LLC >> beerpla.net | @ArtemR <http://twitter.com/ArtemR> >> >> >> On Sun, Apr 26, 2020 at 3:22

Re: [Gluster-users] gluster 5.11 issue: Endless performing entry selfheal / Completed entry selfheal on the same pending heal files after server reboot + very high load

aging heal load spikes. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Sun, Apr 26, 2020 at 3:13 PM Artem Russakovskii wrote: > Hi al

[Gluster-users] Tons of dht: Found anomalies in (null)

Hi, Every time I ls large dirs in our 1x4 replicate gluster volume, I get a ton of these in the logs. If I run the same ls right away again, they won't repeat, but inevitably, in a couple of hours or days, they show up again. What is the meaning of these messages and why am I getting them so fre

[Gluster-users] Extremely slow file listing in folders with many files

Hi all, We have 500GB and 10TB 4x1 replicate xfs-based gluster volumes, and the 10TB one especially is extremely slow to do certain things with (and has been since gluster 3.x when we started). We're currently on 5.13. The number of files isn't even what I'd consider that great - under 100k per d

Re: [Gluster-users] Extremely slow file listing in folders with many files

:24:23 AM GMT+03:00, Artem Russakovskii < > archon...@gmail.com> wrote: > >Hi all, > > > >We have 500GB and 10TB 4x1 replicate xfs-based gluster volumes, and the > >10TB one especially is extremely slow to do certain things with (and > >has > >been since

Re: [Gluster-users] Extremely slow file listing in folders with many files

r xfs filesystem, i.e. > xfs_info of your block device? > > > Regards, > > Felix > On 30/04/2020 17:27, Artem Russakovskii wrote: > > Hi Strahil, in the original email I included both the times for the first > and subsequent reads on the fuse mounted gluster volume as we

Re: [Gluster-users] Extremely slow file listing in folders with many files

rtemR> On Thu, Apr 30, 2020 at 10:54 AM Artem Russakovskii wrote: > getfattr -d -m. -e hex . > # file: . > trusted.afr.SNIP_data1-client-0=0x > trusted.afr.dirty=0x > trusted.gfid=0x44b2db00267a47508b2a8a9

Re: [Gluster-users] Extremely slow file listing in folders with many files

55 AM Strahil Nikolov wrote: > On April 30, 2020 6:27:10 PM GMT+03:00, Artem Russakovskii < > archon...@gmail.com> wrote: > >Hi Strahil, in the original email I included both the times for the > >first > >and subsequent reads on the fuse mounted gluster volume as we

[Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

Hi all, Today, I decided to upgrade one of the four servers (citadel) we have to 7.5 from 5.13. There are 2 volumes, 1x4 replicate, and fuse mounts (I sent the full details earlier in another message). If everything looked OK, I would have proceeded the rolling upgrade for all of them, following t

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

ArtemR <http://twitter.com/ArtemR> On Thu, Apr 30, 2020 at 2:57 PM Artem Russakovskii wrote: > Hi all, > > Today, I decided to upgrade one of the four servers (citadel) we have to > 7.5 from 5.13. There are 2 volumes, 1x4 replicate, and fuse mounts (I sent > the full detail

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

irror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Thu, Apr 30, 2020 at 3:13 PM Artem Russakovskii wrote: > The number of heal pending on citadel, the one that was upgraded to 7.5, > has now gone to 10s of thousands and cont

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

p-version yet. > > > >Thanks. > > > >Sincerely, > >Artem > > > >-- > >Founder, Android Police <http://www.androidpolice.com>, APK Mirror > ><http://www.apkmirror.com/>, Illogical Robot LLC > >beerpla.net | @ArtemR <http://twitter.com/ArtemR&g

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

mirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Fri, May 1, 2020 at 7:25 AM Artem Russakovskii wrote: > I do not have snapshots, no. I have a general file based backup, but also > the other 3 nodes are up. > > OpenSUSE 15.1. > >

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

hout their word first. On Sat, May 2, 2020, 12:47 AM Strahil Nikolov wrote: > On May 1, 2020 8:03:50 PM GMT+03:00, Artem Russakovskii < > archon...@gmail.com> wrote: > >The good news is the downgrade seems to have worked and was painless. > > > >zypper install --oldpack

Re: [Gluster-users] writing to fuse device failed: No such file or directory

I'm on 5.13, and these are the only error messages I'm still seeing (after downgrading from the failed v7 update): [2020-05-04 19:56:29.391121] E [fuse-bridge.c:219:check_and_dump_fuse_W] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x17d)[0x7f0f9a5f324d] (--> /usr/lib64/glusterfs/5.13/xlat

Re: [Gluster-users] writing to fuse device failed: No such file or directory

ported to 5.x, so I am a bit surprised. >> >> Best regards, >> Mabi >> >> >> >> ‐‐‐ Original Message ‐‐‐ >> On Monday, May 4, 2020 9:57 PM, Artem Russakovskii >> wrote: >> >> I'm on 5.13, and these are the only error mess

Re: [Gluster-users] writing to fuse device failed: No such file or directory

se do help us. > > > > > On Wed, May 6, 2020 at 1:16 PM mabi wrote: > >> Hi everyone, >> >> So because upgrading introduces additional problems, does this means I >> should stick with 5.x even if it is EOL? >> >> Or what is a "safe" v

[Gluster-users] Gluster fuse mount freezes entire server until killed and remounted

Hi all, We have been observing a critical issue that started in the last several months and has already randomly affected our servers 3 times. *The symptoms:* - df stops responding and hangs - sometimes apache and nginx stop responding and all requests hang, sometimes they continue wor

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

ttp://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Mon, May 4, 2020 at 10:23 PM Strahil Nikolov wrote: > On May 4, 2020 4:26:32 PM GMT+03:00, Amar Tumballi wrote: > >On Sat, May 2, 2020 at

Re: [Gluster-users] gluster 5.11 issue: Endless performing entry selfheal / Completed entry selfheal on the same pending heal files after server reboot + very high load

the >> self heal actually succeeded without my intervention following a server >> reboot. >> >> The load was still high during this process, but at least the endless >> heal issue is resolved. >> >> I'd still love to hear from the team on managing h

Re: [Gluster-users] Tons of dht: Found anomalies in (null)

<http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Wed, Apr 29, 2020 at 6:00 PM Artem Russakovskii wrote: > Hi, > > Every time I ls large dirs in our 1x4 replicate gluster volume, I get a > ton of these in the logs. >

Re: [Gluster-users] Extremely slow file listing in folders with many files

Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Thu, Apr 30, 2020 at 10:36 PM Strahil Nikolov wrote: > On April 30, 2020 9:05:19 PM GMT+03:00, Artem Russakovskii < > archon...@gmail.com> wrote: > >I did this on the same prod instance just now. > > > &g

Re: [Gluster-users] Extremely slow file listing in folders with many files

beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Mon, May 18, 2020 at 3:10 PM Artem Russakovskii wrote: > Hi, > > Does the gluster team have any feedback about this? Resolving the "Found > anomalies" issues may be key to resolving dir list speed issues. > > S

Re: [Gluster-users] Tons of dht: Found anomalies in (null)

I think there's something else going on here. Our gluster setup is 4x1 replicate, and there should be no such discrepancies and missing unsynced data. In my experience the replication part of gluster works very well and all files replicate correctly to all 4 servers. Yet a single ls could run for

Re: [Gluster-users] Extremely slow file listing in folders with many files

was down when mkdir was going on or > you might have added a new brick recently to the cluster. > > On Tue, May 19, 2020 at 3:56 AM Artem Russakovskii > wrote: > >> I've launched an ls in a folder with ~100k files/dirs 20 minutes ago. >> It's still not done and

Re: [Gluster-users] gluster 5.11 issue: Endless performing entry selfheal / Completed entry selfheal on the same pending heal files after server reboot + very high load

ce.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Fri, May 15, 2020 at 11:20 PM Strahil Nikolov wrote: > On May 16, 2020 1:14:17 AM GMT+03:00, Artem Russakovskii < > archon...@gmail.com> wrote: >

Re: [Gluster-users] Extremely slow file listing in folders with many files

20, 2020 at 10:42 PM Susant Palai wrote: > Artem, could you create a github with all the details. Easier to track > there and collaborate. Please update a snippet of the log file there as > well. > > On Tue, May 19, 2020 at 11:30 PM Artem Russakovskii > wrote: > >>

Re: [Gluster-users] Tons of dht: Found anomalies in (null)

Rolling this into https://github.com/gluster/glusterfs/issues/1256. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Tue, May 19, 2020

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

Fri, May 15, 2020 at 2:51 PM Artem Russakovskii wrote: > Hi, > > I see the team met up recently and one of the discussed items was issues > upgrading to v7. What were the results of this discussion? > > Is the team going to respond to this thread with their thoughts and >

Re: [Gluster-users] State of Gluster project

Has anyone tried to pit Ceph against gluster? I'm curious what the ups and downs are. On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov wrote: > Hey Mahdi, > > For me it looks like Red Hat are focusing more on CEPH than on Gluster. > I hope the project remains active, cause it's very difficult to

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

On Thu, May 21, 2020 at 12:43 PM Artem Russakovskii wrote: > I've also moved this to github: > https://github.com/gluster/glusterfs/issues/1257. > > Sincerely, > Artem > > -- > Founder, Android Police <http://www.androidpolice.com>, APK Mirror > <http://

Re: [Gluster-users] Upgrade from 5.13 to 7.5 full of weird messages

ArtemR <http://twitter.com/ArtemR> On Sun, Jun 21, 2020 at 12:03 PM Mahdi Adnan wrote: > I think if it's reproducible than someone can look into it, can you list > the steps to reproduce it? > > On Sun, Jun 21, 2020 at 9:12 PM Artem Russakovskii > wrote: > >&g

Re: [Gluster-users] "Mismatching layouts" in glusterfs client logs after new brick addition and rebalance

I think it'd be extremely helpful if gluster had a feature to grab all the necessary logs/debug info (maybe a few variations depending on the bug) so that all the user would have to do is enter a simple command and have gluster generate the whole bug report, ready to be sent to to the gluster team.

Re: [Gluster-users] Recovery when 2 of 3 servers goes down!

I had the same requirements (except with 4 servers and no arbiter), and this was the solution: gluster v set VMS cluster.quorum-count 1 gluster v set VMS cluster.quorum-type fixed Sincerely, Artem -- Founder, Android Police , APK Mirror ,

Re: [Gluster-users] Recovery when 2 of 3 servers goes down!

false > gluster vol set VMS cluster.quorum-count 1 > > > Thanks for you replay > > Cheers > > > --- > Gilberto Nunes Ferreira > > > > Em sex., 17 de jul. de 2020 às 16:56, Artem Russakovskii < > archon...@gmail.com> escreveu: > >> I had the same requ

Re: [Gluster-users] Gluster linear scale-out performance

Speaking of fio, could the gluster team please help me understand something? We've been having lots of performance issues related to gluster using attached block storage on Linode. At some point, I figured out that Linode has a cap of 500 IOPS on their block storage

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.7

Hi, https://download.opensuse.org/repositories/home:/glusterfs:/Leap15.1-7/openSUSE_Leap_15.1/x86_64/ is still missing 7.7. Is there an ETA please? Thanks. Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC be

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.7

/master/docs/Install-Guide/Community_Packages.md> > we > are supporting Leap15.2 only. > > Regards, > Shwetha > > On Fri, Jul 31, 2020 at 1:03 AM Artem Russakovskii > wrote: > >> Hi, >> >> >> https://download.opensuse.org/repositories/home:/gl

Re: [Gluster-users] Gluster linear scale-out performance

n other servers, and how that can be optimized/mitigated. Sincerely, Artem -- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | @ArtemR <http://twitter.com/ArtemR> On Thu, Jul 30, 2020 at 8:21 PM

Re: [Gluster-users] performance

I tried putting all web files (specifically WordPress php and static files as well as various cache files) on gluster before, and the results were miserable on a busy site - our usual ~8-10 load quickly turned into 100+ and killed everything. I had to go back to running just the user uploads (whic

Re: [Gluster-users] performance

I'm very curious whether these improvements hold up over the next few days. Please report back. Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | @ArtemR On Wed, Aug 5

Re: [Gluster-users] [Gluster-devel] Announcing Gluster release 7.7

Looks like someone built gluster 7.7 for OpenSUSE 15.1 after all. Yay. Sincerely, Artem -- Founder, Android Police , APK Mirror , Illogical Robot LLC beerpla.net | @ArtemR On Sun, Aug 2, 2020 at 11:46 PM Hu Ber

Re: [Gluster-users] Pending healing...

Remove the "summary" part of the command, which should list the exact file pending heal. Then launch the heal manually. If it still doesn't heal, try running md5sum on the file and see if it heals after that. Sincerely, Artem -- Founder, Android Police , APK Mirror

Re: [Gluster-users] performance

er settings yet to > see if I can improve it further, probably will next week. thinking to > increase the write behind window size further to see what happens, as > well as play with the settings suggested by Strahil. > > On 2020-08-05 5:28 p.m., Artem Russakovski

[Gluster-users] Very slow rsync to gluster volume UNLESS `ls` or `find` scan dir on gluster volume first

Hi, I have been working on setting up a 4 replica gluster with over a million files (~250GB total), and I've seen some really weird stuff happen, even after trying to optimize for small files. I've set up a 4-brick replicate volume (gluster 3.13.2). It took almost 2 days to rsync the data from th

Re: [Gluster-users] Very slow rsync to gluster volume UNLESS `ls` or `find` scan dir on gluster volume first

An update, and a very interesting one! After I started stracing rsync, all I could see was lstat calls, quite slow ones, over and over, which is expected. For example: lstat("uploads/2016/10/nexus2cee_DSC05339_thumb-161x107.jpg", {st_mode=S_IFREG|0664, st_size=4043, ...}) = 0 I googled around an

Re: [Gluster-users] Very slow rsync to gluster volume UNLESS `ls` or `find` scan dir on gluster volume first

>> >> Note that this is just a off-the memory hypothesis, We surely need to >> analyse and debug more thoroughly for a proper explanation. Some one in my >> team would look at it soon. >> >> Regards, >> Amar >> >> On Mon, Feb 5, 2018 at 7:25

[Gluster-users] cluster.readdir-optimize and disappearing files/dirs bug

Hi, As many of you know, gluster suffers from pretty bad performance issues when there are lots of files. One way to at least attempt to improve performance is setting cluster.readdir-optimize to on. However, based on my recent tests (using Gluster 3.13.2), as well as tests of many others (like h

[Gluster-users] [dht-selfheal.c:2328:dht_selfheal_directory] 0-data-dht: Directory selfheal failed: Unable to form layout for directory /

Hi, I noticed when I run gluster volume heal data info, the follow message shows up in the log, along with other stuff: [dht-selfheal.c:2328:dht_selfheal_directory] 0-data-dht: Directory selfheal > failed: Unable to form layout for directory / I'm seeing it on Gluster 4.0.1 and 3.13.2. Here's

Re: [Gluster-users] cluster.readdir-optimize and disappearing files/dirs bug

> not backported to 3.13 branch, but is available on 4.0 through > https://bugzilla.redhat.com/1512437. > > On Tue, Apr 3, 2018 at 11:13 PM, Artem Russakovskii > wrote: > >> Hi, >> >> As many of you know, gluster suffers from pretty bad performance issues >

Re: [Gluster-users] [dht-selfheal.c:2328:dht_selfheal_directory] 0-data-dht: Directory selfheal failed: Unable to form layout for directory /

Thank you for the quick diagnosis. On Wed, Apr 4, 2018, 10:47 PM Raghavendra Gowdappa wrote: > > > On Thu, Apr 5, 2018 at 10:48 AM, Artem Russakovskii > wrote: > >> Hi, >> >> I noticed when I run gluster volume heal data info, the follow message >> show

[Gluster-users] performance.cache-size for high-RAM clients/servers, other tweaks for performance, and improvements to Gluster docs

Hi, I'm trying to squeeze performance out of gluster on 4 80GB RAM 20-CPU machines where Gluster runs on attached block storage (Linode) in (4 replicate bricks), and so far everything I tried results in sub-optimal performance. There are many files - mostly images, several million - and many oper

Re: [Gluster-users] performance.cache-size for high-RAM clients/servers, other tweaks for performance, and improvements to Gluster docs

-- Founder, Android Police <http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Thu, Apr 5, 2018 at 11:13 PM, Artem Ru

Re: [Gluster-users] performance.cache-size for high-RAM clients/servers, other tweaks for performance, and improvements to Gluster docs

= 1 (in [0], left {59, 99}) read(0, "\0\200\0\0\4\200\0\7\220\216Y\343\362\366\231\372?\334N^\303\35\374cC;\vtx\231http://www.androidpolice.com>, APK Mirror <http://www.apkmirror.com/>, Illogical Robot LLC beerpla.net | +ArtemRussakovskii <https://plus.google.com/+ArtemRussakovskii> | @Ar

Re: [Gluster-users] performance.cache-size for high-RAM clients/servers, other tweaks for performance, and improvements to Gluster docs

or 3.10 branches would be preferable for production > > On Fri, Apr 6, 2018 at 4:12 AM, Artem Russakovskii > wrote: > >> Hi again, >> >> I'd like to expand on the performance issues and plead for help. Here's >> one case which shows these odd hic

Re: [Gluster-users] performance.cache-size for high-RAM clients/servers, other tweaks for performance, and improvements to Gluster docs

list with > people sharing their tests. > Not sure I would share your optimism, and rather then going up I > downgraded to 3.12 and have no dir view issue now. Though I had to recreate > the cluster and had to re-add bricks with existing data. > > On Tue, Apr 10, 2018 at 1:47 AM, Ar

[Gluster-users] Getting glusterfs to expand volume size to brick size

Hi, I have a 3-brick replicate volume, but for some reason I can't get it to expand to the size of the bricks. The bricks are 25GB, but even after multiple gluster restarts and remounts, the volume is only about 8GB. I believed I could always extend the bricks (we're using Linode block storage, w

Re: [Gluster-users] Getting glusterfs to expand volume size to brick size

version of Gluster are you running? Were the bricks smaller earlier? > > Regards, > Nithya > > On 15 April 2018 at 00:09, Artem Russakovskii wrote: > >> Hi, >> >> I have a 3-brick replicate volume, but for some reason I can't get it to >> expand to th

Re: [Gluster-users] Getting glusterfs to expand volume size to brick size

ow the values of shared-brick-count in the files in > /var/lib/glusterd/vols/dev_apkmirror_data/ ? > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1541880 > > On 17 April 2018 at 05:17, Artem Russakovskii wrote: > >> Hi Nithya, >> >> I'm on Gluster 4

Re: [Gluster-users] Getting glusterfs to expand volume size to brick size

vskii <https://plus.google.com/+ArtemRussakovskii> | @ArtemR <http://twitter.com/ArtemR> On Mon, Apr 16, 2018 at 9:27 PM, Artem Russakovskii wrote: > pylon:/var/lib/glusterd/vols/dev_apkmirror_data # ack shared-brick-count > dev_apkmirror_data.pylon.mnt-pylon_block3-dev_apkmirror_data.vol >

Re: [Gluster-users] Getting glusterfs to expand volume size to brick size

n wrote: > Ok, it looks like the same problem. > > > @Amar, this fix is supposed to be in 4.0.1. Is it possible to regenerate > the volfiles to fix this? > > Regards, > Nithya > > On 17 April 2018 at 09:57, Artem Russakovskii wrote: > >>

  1   2   >