[Gluster-users] Events API Quotas

[João Baúto]

Hi all,

We rely on quotas to manage the space used by different groups and
currently are setting up a monitoring system that would receive alerts via
webhooks. I have a webhook configured in my distributed cluster and the
events are received successfully.

However, despite having multiple directories exceeding the soft limit, only
one event is received and the quotad.log file has multiple entries
similar to,

[2020-01-20 16:13:34.280507] E
[quotad-aggregator.c:60:quotad_serialize_reply] (-->
/lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f5fa48558ea] (-->
/usr/lib64/glusterfs/7.1/xlator/features/quotad.so(+0x2b70)[0x7f5f957e4b70]
(-->
/usr/lib64/glusterfs/7.1/xlator/features/quotad.so(+0x2cb0)[0x7f5f957e4cb0]
(-->
/usr/lib64/glusterfs/7.1/xlator/features/quotad.so(+0x3165)[0x7f5f957e5165]
(-->
/usr/lib64/glusterfs/7.1/xlator/features/quotad.so(+0x216e)[0x7f5f957e416e]
) 0-: Failed to encode message

I'm running a 4 node distributed cluster with CentOS 7.6 and Gluster 7.1.
The webhook code is the same as the one in
https://docs.gluster.org/en/latest/Administrator%20Guide/Events%20APIs/#webhooks

Any idea on what's happening?

Cheers,
*João Baúto*
---

*Scientific Computing and Software Platform*
Champalimaud Research
Champalimaud Center for the Unknown
Av. Brasília, Doca de Pedrouços
1400-038 Lisbon, Portugal
fchampalimaud.org 


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Understanding gluster performance

[Gionatan Danti]

Hi all,
I would like to better understand gluster performance and how to 
profile/analyze them.


I set up two old test machine each with a quad-core i7 CPU, 8 GB RAM and 
4x 5400 RPM disks in software RAID 10. OS is CentOS 8.1 and I am using 
Gluster 6.7. To avoid being limited by the mechanical HDD, I created a 
small replica 2 volume using /dev/shm/gluster for both bricks.


Testing using fio with 4k random write and fsync=1, I see the following:
- write to /dev/shm: almost 300K IOPs (expected, as I am writing in 
memory);

- write to /mnt/fuse (where the volume is mounted): 250 IOPs;
- as above, but something I get as low as 20-30 IOPs for minutes.

Writing to /mnt/fuse without fsync=1 I get about 5500 IOPs, without the 
strange (and very low) drops. In this case, I seems limited by CPU 
because both glusterd and glusterfsd are near/above 100% core 
utilization.


Further details and volume info can be found below [1]

So, I have some questions:
- why performance are so low with fsync?
- why do I have so low IOPs (20/30) for minutes?
- what is capping the non-fsync test?
- why both glusterd and glusterfd are so CPU intensive? I can understand 
glusterfd itself requiring more CPU, but glusterd should only manage the 
other processes and send the volume information to asking client, right?


Thanks.

[1] [root@blackhole gluster]# gluster volume info tv0
Volume Name: tv0
Type: Replicate
Volume ID: 96534afe-bfde-4d60-a94d-379278ab45c4
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: singularity:/dev/shm/gluster
Brick2: blackhole:/dev/shm/gluster
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

[root@blackhole tmp]# mount | grep glusterfs
localhost:tv0 on /mnt/fuse type fuse.glusterfs 
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)


[root@blackhole glusterfs]# fio --name=test 
--filename=/mnt/fuse/test.img --size=256M --rw=randwrite --fsync=1
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 
4096B-4096B, ioengine=psync, iodepth=1

fio-3.7
Starting 1 process
Jobs: 1 (f=1): [w(1)][11.6%][r=0KiB/s,w=976KiB/s][r=0,w=244 IOPS][eta 
04m:19s]


[root@blackhole tmp]# fio --name=test --filename=/mnt/fuse/test.img 
--size=256M --rw=randwrite --fsync=1
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 
4096B-4096B, ioengine=psync, iodepth=1

fio-3.7
Starting 1 process
Jobs: 1 (f=1): [w(1)][1.8%][r=0KiB/s,w=112KiB/s][r=0,w=28 IOPS][eta 
38m:16s]


[root@blackhole tmp]# fio --name=test --filename=/mnt/fuse/test.img 
--size=256M --rw=randwrite
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 
4096B-4096B, ioengine=psync, iodepth=1

fio-3.7
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=21.3MiB/s][r=0,w=5462 IOPS][eta 
00m:00s]


[root@blackhole gluster]# gluster volume heal tv0 info
Brick singularity:/dev/shm/gluster
Status: Connected
Number of entries: 0
Brick blackhole:/dev/shm/gluster
Status: Connected
Number of entries: 0

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] FUSE: Client receives no error when triggering autocommit with WRITE FOP

[David Spisla]

Dear Gluster Community,

please notice this bug which potentially leads to client-side data loss:
https://bugzilla.redhat.com/show_bug.cgi?id=1793042

Several Trace Logs are provided.

Regards
David Spisla


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Debian Repository changed urls

[Michael Böhm]

Hello,

according to [1] i am using the following url for the apt-repo:

> deb
https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/10/amd64/apt
buster main

I know the 7.2 is not ready there yet, that's not the problem. But since
last Saturday (18.01.20) my servers can't find a Release file there. I've
browsed manually and it seems i have to use:

> deb
https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/buster/amd64/apt
buster main

now - is that right?

Regards

Mika

[1] https://docs.gluster.org/en/latest/Install-Guide/Install/


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] [Gluster-devel] Announcing Gluster release 7.2

[Rinku Kothiya]

Hi,

The Gluster community is pleased to announce the release of Gluster7.2
(packages available at [1]).
Release notes for the release can be found at [2].

Major changes, features and limitations addressed in this release:
None

Apologies for the missing stretch-amd64 packages, we are working on it and
will release it soon.

Thanks,
Gluster community

References:

[1] Packages for 7.1:
https://download.gluster.org/pub/gluster/glusterfs/7/7.2/

[2] Release notes for 7.1:
https://docs.gluster.org/en/latest/release-notes/7.2/


Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users