On 07/10/2013 03:18 PM, Joe Julian wrote:
The "small file" complaint is all about latency though. There's very little disk overhead (all inode lookups) to doing a self-heal check. "ls -l" on a 50k file directory and nearly all the delay is from network RTT for self-heal checks (check that with wireshark).
Try it with localhost. Build a small test gluster brick, take networking out of the loop, create 50k files, and launch the self heal. RTT is part of it, but not the majority (last I checked it wasn't a significant fraction relative to other metadata bits).
I did an experiment with 3.3.x a while ago with 2x ramdisks I created a set of files, looped back with losetup, built xfs fs atop them, mirrored them with glusterfs, and then set about to doing metadata/small file heavy workloads. Performance was still abysmal. Pretty sure none of that was RTT. Definitely a stack traversal problem, but I didn't trace it far enough back to be definitively sure where it was.
-- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: [email protected] web : http://scalableinformatics.com http://scalableinformatics.com/siflash phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Gluster-users mailing list [email protected] http://supercolony.gluster.org/mailman/listinfo/gluster-users
