Hi Vijay Thanks for your quick response. I am using gluster 3.8.11 on Centos 7 servers glusterfs-3.8.11-1.el7.x86_64
clients are centos 6 but I tested with a centos 7 client as well and results didn't change gluster volume info Volume Name: atlasglust Type: Distribute Volume ID: fbf0ebb8-deab-4388-9d8a-f722618a624b Status: Started Snapshot Count: 0 Number of Bricks: 5 Transport-type: tcp Bricks: Brick1: pplxgluster01.x.y.z:/glusteratlas/brick001/gv0 Brick2: pplxgluster02..x.y.z:/glusteratlas/brick002/gv0 Brick3: pplxgluster03.x.y.z:/glusteratlas/brick003/gv0 Brick4: pplxgluster04.x.y.z:/glusteratlas/brick004/gv0 Brick5: pplxgluster05.x.y.z:/glusteratlas/brick005/gv0 Options Reconfigured: nfs.disable: on performance.readdir-ahead: on transport.address-family: inet auth.allow: x.y.z I am not using directory quota. Please let me know if you require some more info Thanks Kashif On Fri, Jun 9, 2017 at 2:34 PM, Vijay Bellur <[email protected]> wrote: > Can you please provide more details about your volume configuration and > the version of gluster that you are using? > > Regards, > Vijay > > On Fri, Jun 9, 2017 at 5:35 PM, mohammad kashif <[email protected]> > wrote: > >> Hi >> >> I have just moved our 400 TB HPC storage from lustre to gluster. It is >> part of a research institute and users have very small files to big files >> ( few KB to 20GB) . Our setup consists of 5 servers, each with 96TB RAID 6 >> disks. All servers are connected through 10G ethernet but not all clients. >> Gluster volumes are distributed without any replication. There are >> approximately 80 million files in file system. >> I am mounting using glusterfs on clients. >> >> I have copied everything from lustre to gluster but old file system exist >> so I can compare. >> >> The problem, I am facing is extremely slow du on even a small directory. >> Also the time taken is substantially different each time. >> I tried du from same client on a particular directory twice and got >> these results. >> >> time du -sh /data/aa/bb/cc >> 3.7G /data/aa/bb/cc >> real 7m29.243s >> user 0m1.448s >> sys 0m7.067s >> >> time du -sh /data/aa/bb/cc >> 3.7G /data/aa/bb/cc >> real 16m43.735s >> user 0m1.097s >> sys 0m5.802s >> >> 16m and 7m is too long for a 3.7 G directory. I must mention that the >> directory contains huge number of files (208736) >> >> but running du on same directory on old data gives this result >> >> time du -sh /olddata/aa/bb/cc >> 4.0G /olddata/aa/bb/cc >> real 3m1.255s >> user 0m0.755s >> sys 0m38.099s >> >> much better if I run same command again >> >> time du -sh /olddata/aa/bb/cc >> 4.0G /olddata/aa/bb/cc >> real 0m8.309s >> user 0m0.313s >> sys 0m7.755s >> >> Is there anything I can do to improve this performance? I would also like >> hear from some one who is running same kind of setup. >> >> Thanks >> >> Kashif >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> [email protected] >> http://lists.gluster.org/mailman/listinfo/gluster-users >> > >
_______________________________________________ Gluster-users mailing list [email protected] http://lists.gluster.org/mailman/listinfo/gluster-users
