Hi Bart,
Thanks for the patch. For users with that many files in a directory,
using pvfs2-ls is probably a good alternative.
The kernel does readdir requests 32 entries at a time, so increasing
MAX_NUM_DIRENTS won't help for ls. Long listings requires getting the
size of files, which in PVFS is fairly expensive.
Unfortunately, we haven't kept up with the readdirplus implementation,
some bugs have probably crept in since Murali added that tool. If you
were motivated to look at where the servers were crashing, we'd
certainly be interested in helping with the debugging there.
Thanks again,
-sam
On Sep 3, 2008, at 1:46 PM, Bart Taylor wrote:
Hey guys,
Attached is a small patch that increases ls and pvfs2-ls
performance. It increases
some #defines from 32/64 to 512.
I did not see any change for small directories, but listing speeds
were increased
up to 26% for large directories (100,000+ files). Long listing
performance with
ls and pvfs2-ls seem to be generally unchanged. The file system used
is a 5 node
SAN setup with HA running 271.
Also attached are some specifics of the performance we saw.
I did not include an increase for pvfs2-lsplus. Increasing its
MAX_NUM_DIRENTS
value resulted in crashing all of the file system servers a couple
of times.
Bart.
<DIRENT-Results.xls><dirent-
increase.patch>_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
