Hi Praveen, Sorry for the late response. Unfortunately, we have not seen this kind of behavior on many of our setups thru the MPI-I/O/ADIO-PVFS2 interfaces. It is conceivable that performance thru the VFS do not scale with number of servers.
Could you tell us how the file is being read (i.e using what interfaces, what block sizes etc)? There still aren't that many tweakable settings just yet, but we are working on some I/O performance tuning and we will hopefully have a better handle on what settings affect performance. Thanks for the reports, Murali On Mon, 21 Nov 2005, Praveen KJ wrote: > Hi, > We have a pvfs2 client reading a 5 GB file sequentially from > multiple pvfs2 data servers. > There is just one metadata server. All the nodes are connected via > GigE network. > > The problem we are seeing is that the single-client read throughput > decreases as the number of servers increase. > With 24 servers we see around 5 MB/s read compared to 50 MB/s read > with just 2 servers. > The setup between each experiment is exactly same, except the > number of servers. > > We have tried a couple of pvfs2 versions including the latest 1.3.1. > Is there any setting we have to tweak ? We have taken only pvfs2 > defaults till now. > Some of the relevant options are pasted below. > > Thanks, > Praveen > > ------------------------------------------ > pvfs2-fs.conf ( except hostnames ) > ------------------------------------------ > <Defaults> > UnexpectedRequests 50 > LogFile /tmp/pvfs2-server.log > EventLogging none > LogStamp usec > BMIModules bmi_tcp > FlowModules flowproto_multiqueue > PerfUpdateInterval 1000 > ServerJobBMITimeoutSecs 30 > ServerJobFlowTimeoutSecs 30 > ClientJobBMITimeoutSecs 300 > ClientJobFlowTimeoutSecs 300 > ClientRetryLimit 5 > ClientRetryDelayMilliSecs 2000 > </Defaults> > > <Filesystem> > Name pvfs2-fs > ID 1175641402 > RootHandle 1048576 > <StorageHints> > TroveSyncMeta yes > TroveSyncData no > AttrCacheKeywords datafile_handles,metafile_dist > AttrCacheKeywords dir_ent, symlink_target > AttrCacheSize 4093 > AttrCacheMaxNumElems 32768 > </StorageHints> > </Filesystem> > _______________________________________________ > PVFS2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > _______________________________________________ PVFS2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
