Thanks Dean! I will take a look at the VFS code paths/performance analysis. Murali
On Wed, 30 Nov 2005, Dean Hildebrand wrote: > I've seen this through VFS. Here are some graphs (ignore the pNFS lines) > with 4MB block size and the standard read/write interfaces. > Overview: > http://www.citi.umich.edu/projects/asci/pnfs/darkstar-exp/ > > Single file: > http://www.citi.umich.edu/projects/asci/pnfs/darkstar-exp/experiments-5_13095_image001.gif > > Separate files: > http://www.citi.umich.edu/projects/asci/pnfs/darkstar-exp/experiments-5_15651_image001.gif > > As the # of data servers increases, aggregate read performance increases > but individual client read performance suffers a little. > > Dean > > On Wed, 30 Nov 2005, Murali Vilayannur wrote: > > > Hi Praveen, > > Sorry for the late response. Unfortunately, we have not seen this kind of > > behavior on many of our setups thru the MPI-I/O/ADIO-PVFS2 interfaces. > > It is conceivable that performance thru the VFS do not scale with > > number of servers. > > > > Could you tell us how the file is being read (i.e using what interfaces, > > what block sizes etc)? > > There still aren't that many tweakable settings just yet, but we are > > working on some I/O performance tuning and we will hopefully have a better > > handle on what settings affect performance. > > Thanks for the reports, > > Murali > > > > On Mon, 21 Nov 2005, Praveen KJ wrote: > > > > > Hi, > > > We have a pvfs2 client reading a 5 GB file sequentially from > > > multiple pvfs2 data servers. > > > There is just one metadata server. All the nodes are connected via > > > GigE network. > > > > > > The problem we are seeing is that the single-client read throughput > > > decreases as the number of servers increase. > > > With 24 servers we see around 5 MB/s read compared to 50 MB/s read > > > with just 2 servers. > > > The setup between each experiment is exactly same, except the > > > number of servers. > > > > > > We have tried a couple of pvfs2 versions including the latest 1.3.1. > > > Is there any setting we have to tweak ? We have taken only pvfs2 > > > defaults till now. > > > Some of the relevant options are pasted below. > > > > > > Thanks, > > > Praveen > > > > > > ------------------------------------------ > > > pvfs2-fs.conf ( except hostnames ) > > > ------------------------------------------ > > > <Defaults> > > > UnexpectedRequests 50 > > > LogFile /tmp/pvfs2-server.log > > > EventLogging none > > > LogStamp usec > > > BMIModules bmi_tcp > > > FlowModules flowproto_multiqueue > > > PerfUpdateInterval 1000 > > > ServerJobBMITimeoutSecs 30 > > > ServerJobFlowTimeoutSecs 30 > > > ClientJobBMITimeoutSecs 300 > > > ClientJobFlowTimeoutSecs 300 > > > ClientRetryLimit 5 > > > ClientRetryDelayMilliSecs 2000 > > > </Defaults> > > > > > > <Filesystem> > > > Name pvfs2-fs > > > ID 1175641402 > > > RootHandle 1048576 > > > <StorageHints> > > > TroveSyncMeta yes > > > TroveSyncData no > > > AttrCacheKeywords datafile_handles,metafile_dist > > > AttrCacheKeywords dir_ent, symlink_target > > > AttrCacheSize 4093 > > > AttrCacheMaxNumElems 32768 > > > </StorageHints> > > > </Filesystem> > > > _______________________________________________ > > > PVFS2-users mailing list > > > [email protected] > > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > > > > _______________________________________________ > > PVFS2-users mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > _______________________________________________ PVFS2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
