Hi I have just moved our 400 TB HPC storage from lustre to gluster. It is part of a research institute and users have very small files to big files ( few KB to 20GB) . Our setup consists of 5 servers, each with 96TB RAID 6 disks. All servers are connected through 10G ethernet but not all clients. Gluster volumes are distributed without any replication. There are approximately 80 million files in file system. I am mounting using glusterfs on clients.
I have copied everything from lustre to gluster but old file system exist so I can compare. The problem, I am facing is extremely slow du on even a small directory. Also the time taken is substantially different each time. I tried du from same client on a particular directory twice and got these results. time du -sh /data/aa/bb/cc 3.7G /data/aa/bb/cc real 7m29.243s user 0m1.448s sys 0m7.067s time du -sh /data/aa/bb/cc 3.7G /data/aa/bb/cc real 16m43.735s user 0m1.097s sys 0m5.802s 16m and 7m is too long for a 3.7 G directory. I must mention that the directory contains huge number of files (208736) but running du on same directory on old data gives this result time du -sh /olddata/aa/bb/cc 4.0G /olddata/aa/bb/cc real 3m1.255s user 0m0.755s sys 0m38.099s much better if I run same command again time du -sh /olddata/aa/bb/cc 4.0G /olddata/aa/bb/cc real 0m8.309s user 0m0.313s sys 0m7.755s Is there anything I can do to improve this performance? I would also like hear from some one who is running same kind of setup. Thanks Kashif
_______________________________________________ Gluster-users mailing list [email protected] http://lists.gluster.org/mailman/listinfo/gluster-users
