Hi Artem, Was the volume size correct before the bricks were expanded?
This sounds like [1] but that should have been fixed in 4.0.0. Can you let us know the values of shared-brick-count in the files in /var/lib/glusterd/vols/dev_apkmirror_data/ ? [1] https://bugzilla.redhat.com/show_bug.cgi?id=1541880 On 17 April 2018 at 05:17, Artem Russakovskii <[email protected]> wrote: > Hi Nithya, > > I'm on Gluster 4.0.1. > > I don't think the bricks were smaller before - if they were, maybe 20GB > because Linode's minimum is 20GB, then I extended them to 25GB, resized > with resize2fs as instructed, and rebooted many times over since. Yet, > gluster refuses to see the full disk size. > > Here's the status detail output: > > gluster volume status dev_apkmirror_data detail > Status of volume: dev_apkmirror_data > ------------------------------------------------------------ > ------------------ > Brick : Brick pylon:/mnt/pylon_block1/dev_apkmirror_data > TCP Port : 49152 > RDMA Port : 0 > Online : Y > Pid : 1263 > File System : ext4 > Device : /dev/sdd > Mount Options : rw,relatime,data=ordered > Inode Size : 256 > Disk Space Free : 23.0GB > Total Disk Space : 24.5GB > Inode Count : 1638400 > Free Inodes : 1625429 > ------------------------------------------------------------ > ------------------ > Brick : Brick pylon:/mnt/pylon_block2/dev_apkmirror_data > TCP Port : 49153 > RDMA Port : 0 > Online : Y > Pid : 1288 > File System : ext4 > Device : /dev/sdc > Mount Options : rw,relatime,data=ordered > Inode Size : 256 > Disk Space Free : 24.0GB > Total Disk Space : 25.5GB > Inode Count : 1703936 > Free Inodes : 1690965 > ------------------------------------------------------------ > ------------------ > Brick : Brick pylon:/mnt/pylon_block3/dev_apkmirror_data > TCP Port : 49154 > RDMA Port : 0 > Online : Y > Pid : 1313 > File System : ext4 > Device : /dev/sde > Mount Options : rw,relatime,data=ordered > Inode Size : 256 > Disk Space Free : 23.0GB > Total Disk Space : 24.5GB > Inode Count : 1638400 > Free Inodes : 1625433 > > > > What's interesting here is that the gluster volume size is exactly 1/3 of > the total (8357M * 3 = 25071M). Yet, each block device is separate, and the > total storage available is 25071M on each brick. > > The fstab is as follows: > /dev/disk/by-id/scsi-0Linode_Volume_pylon_block1 /mnt/pylon_block1 ext4 > defaults 0 2 > /dev/disk/by-id/scsi-0Linode_Volume_pylon_block2 /mnt/pylon_block2 ext4 > defaults 0 2 > /dev/disk/by-id/scsi-0Linode_Volume_pylon_block3 /mnt/pylon_block3 ext4 > defaults 0 2 > > localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data1 glusterfs > defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 > localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data2 glusterfs > defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 > localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data3 glusterfs > defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0 > localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data_ganesha nfs4 > defaults,_netdev,bg,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 0 0 > > The latter entry is for an nfs ganesha test, in case it matters (which, > btw, fails miserably with all kinds of stability issues about broken pipes). > > Note: this is a test server, so all 3 bricks are attached and mounted on > the same server. > > > Sincerely, > Artem > > -- > Founder, Android Police <http://www.androidpolice.com>, APK Mirror > <http://www.apkmirror.com/>, Illogical Robot LLC > beerpla.net | +ArtemRussakovskii > <https://plus.google.com/+ArtemRussakovskii> | @ArtemR > <http://twitter.com/ArtemR> > > On Sun, Apr 15, 2018 at 10:56 PM, Nithya Balachandran <[email protected] > > wrote: > >> What version of Gluster are you running? Were the bricks smaller earlier? >> >> Regards, >> Nithya >> >> On 15 April 2018 at 00:09, Artem Russakovskii <[email protected]> >> wrote: >> >>> Hi, >>> >>> I have a 3-brick replicate volume, but for some reason I can't get it to >>> expand to the size of the bricks. The bricks are 25GB, but even after >>> multiple gluster restarts and remounts, the volume is only about 8GB. >>> >>> I believed I could always extend the bricks (we're using Linode block >>> storage, which allows extending block devices after they're created), and >>> gluster would see the newly available space and extend to use it. >>> >>> Multiple Google searches, and I'm still nowhere. Any ideas? >>> >>> df | ack "block|data" >>> Filesystem 1M-blocks >>> Used Available Use% Mounted on >>> /dev/sdd 25071M >>> 1491M 22284M 7% /mnt/pylon_block1 >>> /dev/sdc 26079M >>> 1491M 23241M 7% /mnt/pylon_block2 >>> /dev/sde 25071M >>> 1491M 22315M 7% /mnt/pylon_block3 >>> localhost:/dev_apkmirror_data 8357M >>> 581M 7428M 8% /mnt/dev_apkmirror_data1 >>> localhost:/dev_apkmirror_data 8357M >>> 581M 7428M 8% /mnt/dev_apkmirror_data2 >>> localhost:/dev_apkmirror_data 8357M >>> 581M 7428M 8% /mnt/dev_apkmirror_data3 >>> >>> >>> >>> gluster volume info >>> >>> Volume Name: dev_apkmirror_data >>> Type: Replicate >>> Volume ID: cd5621ee-7fab-401b-b720-08863717ed56 >>> Status: Started >>> Snapshot Count: 0 >>> Number of Bricks: 1 x 3 = 3 >>> Transport-type: tcp >>> Bricks: >>> Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data >>> Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data >>> Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data >>> Options Reconfigured: >>> disperse.eager-lock: off >>> cluster.lookup-unhashed: auto >>> cluster.read-hash-mode: 0 >>> performance.strict-o-direct: on >>> cluster.shd-max-threads: 12 >>> performance.nl-cache-timeout: 600 >>> performance.nl-cache: on >>> cluster.quorum-count: 1 >>> cluster.quorum-type: fixed >>> network.ping-timeout: 5 >>> network.remote-dio: enable >>> performance.rda-cache-limit: 256MB >>> performance.parallel-readdir: on >>> network.inode-lru-limit: 500000 >>> performance.md-cache-timeout: 600 >>> performance.cache-invalidation: on >>> performance.stat-prefetch: on >>> features.cache-invalidation-timeout: 600 >>> features.cache-invalidation: on >>> performance.io-thread-count: 32 >>> server.event-threads: 4 >>> client.event-threads: 4 >>> performance.read-ahead: off >>> cluster.lookup-optimize: on >>> performance.client-io-threads: on >>> performance.cache-size: 1GB >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> cluster.readdir-optimize: on >>> >>> >>> Thank you. >>> >>> Sincerely, >>> Artem >>> >>> -- >>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror >>> <http://www.apkmirror.com/>, Illogical Robot LLC >>> beerpla.net | +ArtemRussakovskii >>> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR >>> <http://twitter.com/ArtemR> >>> >>> _______________________________________________ >>> Gluster-users mailing list >>> [email protected] >>> http://lists.gluster.org/mailman/listinfo/gluster-users >>> >> >> >
_______________________________________________ Gluster-users mailing list [email protected] http://lists.gluster.org/mailman/listinfo/gluster-users
