Hi Jose, There is a known issue with gluster 3.12.x builds (see [1]) so you may be running into this.
The "shared-brick-count" values seem fine on stor1. Please send us "grep -n "share" /var/lib/glusterd/vols/volumedisk1/*" results for the other nodes so we can check if they are the cause. Regards, Nithya [1] https://bugzilla.redhat.com/show_bug.cgi?id=1517260 On 28 February 2018 at 03:03, Jose V. Carrión <[email protected]> wrote: > Hi, > > Some days ago all my glusterfs configuration was working fine. Today I > realized that the total size reported by df command was changed and is > smaller than the aggregated capacity of all the bricks in the volume. > > I checked that all the volumes status are fine, all the glusterd daemons > are running, there is no error in logs, however df shows a bad total size. > > My configuration for one volume: volumedisk1 > [root@stor1 ~]# gluster volume status volumedisk1 detail > > Status of volume: volumedisk1 > ------------------------------------------------------------ > ------------------ > Brick : Brick stor1data:/mnt/glusterfs/vol1/brick1 > TCP Port : 49153 > RDMA Port : 0 > Online : Y > Pid : 13579 > File System : xfs > Device : /dev/sdc1 > Mount Options : rw,noatime > Inode Size : 512 > Disk Space Free : 35.0TB > Total Disk Space : 49.1TB > Inode Count : 5273970048 > Free Inodes : 5273123069 > ------------------------------------------------------------ > ------------------ > Brick : Brick stor2data:/mnt/glusterfs/vol1/brick1 > TCP Port : 49153 > RDMA Port : 0 > Online : Y > Pid : 13344 > File System : xfs > Device : /dev/sdc1 > Mount Options : rw,noatime > Inode Size : 512 > Disk Space Free : 35.0TB > Total Disk Space : 49.1TB > Inode Count : 5273970048 > Free Inodes : 5273124718 > ------------------------------------------------------------ > ------------------ > Brick : Brick stor3data:/mnt/disk_c/glusterfs/vol1/brick1 > TCP Port : 49154 > RDMA Port : 0 > Online : Y > Pid : 17439 > File System : xfs > Device : /dev/sdc1 > Mount Options : rw,noatime > Inode Size : 512 > Disk Space Free : 35.7TB > Total Disk Space : 49.1TB > Inode Count : 5273970048 > Free Inodes : 5273125437 > ------------------------------------------------------------ > ------------------ > Brick : Brick stor3data:/mnt/disk_d/glusterfs/vol1/brick1 > TCP Port : 49155 > RDMA Port : 0 > Online : Y > Pid : 17459 > File System : xfs > Device : /dev/sdd1 > Mount Options : rw,noatime > Inode Size : 512 > Disk Space Free : 35.6TB > Total Disk Space : 49.1TB > Inode Count : 5273970048 > Free Inodes : 5273127036 > > > Then full size for volumedisk1 should be: 49.1TB + 49.1TB + 49.1TB +49.1TB > = *196,4 TB *but df shows: > > [root@stor1 ~]# df -h > Filesystem Size Used Avail Use% Mounted on > /dev/sda2 48G 21G 25G 46% / > tmpfs 32G 80K 32G 1% /dev/shm > /dev/sda1 190M 62M 119M 35% /boot > /dev/sda4 395G 251G 124G 68% /data > /dev/sdb1 26T 601G 25T 3% /mnt/glusterfs/vol0 > /dev/sdc1 50T 15T 36T 29% /mnt/glusterfs/vol1 > stor1data:/volumedisk0 > 76T 1,6T 74T 3% /volumedisk0 > stor1data:/volumedisk1 > *148T* 42T 106T 29% /volumedisk1 > > Exactly 1 brick minus: 196,4 TB - 49,1TB = 148TB > > It's a production system so I hope you can help me. > > Thanks in advance. > > Jose V. > > > Below some other data of my configuration: > > [root@stor1 ~]# gluster volume info > > Volume Name: volumedisk0 > Type: Distribute > Volume ID: 0ee52d94-1131-4061-bcef-bd8cf898da10 > Status: Started > Snapshot Count: 0 > Number of Bricks: 4 > Transport-type: tcp > Bricks: > Brick1: stor1data:/mnt/glusterfs/vol0/brick1 > Brick2: stor2data:/mnt/glusterfs/vol0/brick1 > Brick3: stor3data:/mnt/disk_b1/glusterfs/vol0/brick1 > Brick4: stor3data:/mnt/disk_b2/glusterfs/vol0/brick1 > Options Reconfigured: > performance.cache-size: 4GB > cluster.min-free-disk: 1% > performance.io-thread-count: 16 > performance.readdir-ahead: on > > Volume Name: volumedisk1 > Type: Distribute > Volume ID: 591b7098-800e-4954-82a9-6b6d81c9e0a2 > Status: Started > Snapshot Count: 0 > Number of Bricks: 4 > Transport-type: tcp > Bricks: > Brick1: stor1data:/mnt/glusterfs/vol1/brick1 > Brick2: stor2data:/mnt/glusterfs/vol1/brick1 > Brick3: stor3data:/mnt/disk_c/glusterfs/vol1/brick1 > Brick4: stor3data:/mnt/disk_d/glusterfs/vol1/brick1 > Options Reconfigured: > cluster.min-free-inodes: 6% > performance.cache-size: 4GB > cluster.min-free-disk: 1% > performance.io-thread-count: 16 > performance.readdir-ahead: on > > [root@stor1 ~]# grep -n "share" /var/lib/glusterd/vols/volumedisk1/* > /var/lib/glusterd/vols/volumedisk1/volumedisk1. > stor1data.mnt-glusterfs-vol1-brick1.vol:3: option shared-brick-count 1 > /var/lib/glusterd/vols/volumedisk1/volumedisk1. > stor1data.mnt-glusterfs-vol1-brick1.vol.rpmsave:3: option > shared-brick-count 1 > /var/lib/glusterd/vols/volumedisk1/volumedisk1. > stor2data.mnt-glusterfs-vol1-brick1.vol:3: option shared-brick-count 0 > /var/lib/glusterd/vols/volumedisk1/volumedisk1. > stor2data.mnt-glusterfs-vol1-brick1.vol.rpmsave:3: option > shared-brick-count 0 > /var/lib/glusterd/vols/volumedisk1/volumedisk1.stor3data.mnt-disk_c- > glusterfs-vol1-brick1.vol:3: option shared-brick-count 0 > /var/lib/glusterd/vols/volumedisk1/volumedisk1.stor3data.mnt-disk_c- > glusterfs-vol1-brick1.vol.rpmsave:3: option shared-brick-count 0 > /var/lib/glusterd/vols/volumedisk1/volumedisk1.stor3data.mnt-disk_d- > glusterfs-vol1-brick1.vol:3: option shared-brick-count 0 > /var/lib/glusterd/vols/volumedisk1/volumedisk1.stor3data.mnt-disk_d- > glusterfs-vol1-brick1.vol.rpmsave:3: option shared-brick-count 0 > > > > _______________________________________________ > Gluster-users mailing list > [email protected] > http://lists.gluster.org/mailman/listinfo/gluster-users >
_______________________________________________ Gluster-users mailing list [email protected] http://lists.gluster.org/mailman/listinfo/gluster-users
