Hi Kshitij - This is the expected behaviour, PVFS2 is not highly optimized for small writes/reads, which is what IOR is typically performing. So you will always see degraded performances here compared to the underlying filesystem's base performance.
There are ways to tune to help optimize for this type of access. If you set your IOR block accesses to something larger such as 64K instead of the default (4K?) I think you would see performances which are closer. This used to be pretty well documented in the FAQ documents for PVFS, i'm not sure where the links are now.. Cheers, Kyle Schochenmaier On Wed, Dec 14, 2011 at 1:09 AM, Kshitij Mehta <[email protected]> wrote: > Well , heres why I wanted to trace in the first place. > > I have a test configuration where we have configured PVFS2 over an SSD > storage. There are two I/O servers that talk to the SSD storage through > Infiniband (There are 2 IB channels going into the SSD, and each storage > server can 'see' one half of the SSD). > > Now I used the IOR benchmark to test the write bandwidth. I first spawn a > process on the I/O server such that it writes data to the underlying ext4 > file system on the SSD instead of PVFS2. I see a bandwidth of ~350 MB/s. > Now I spawn a process on the same I/O server and write data to the PVFS2 > file system configured over the SSD, and I see a write bandwidth of ~180 > MB/s. > > This seems to represent some kind of overhead with PVFS2, but seems too > large. Has anybody else seen similar results? Is the overhead of pvfs2 > documented? > > Do let me know if something is not clear or if you have additional > questions about the above setup. > > Here are some other details: > I/O servers: dual core with 2G main memory each. > PVFS 2.8.2 > > Thanks, > Kshitij > > -----Original Message----- > From: Julian Kunkel [mailto:[email protected]] > Sent: Tuesday, December 13, 2011 3:10 AM > To: Kshitij Mehta > Cc: [email protected] > Subject: Re: [Pvfs2-users] Tracing pvfs2 internals > > Dear Kshitij, > we have a version of OrangeFS which is instrumented with HDTrace, there > you can record detailed information about activity of statemachines and I/O. > For a description see the thesis: > > http://wr.informatik.uni-hamburg.de/_media/research:theses:Tien%20Duc%20Tien_Tracing%20Internal%20Behavior%20in%20PVFS.pdf > > The code is available in our redmine (here is a link to the wiki): > http://redmine.wr.informatik.uni-hamburg.de/projects/piosimhd/wiki > > I consider the tracing implemented in PVFS as rather robust, since it is > our second implementation with PVFS_hints. > However, you might encounter some issues with the build system. > If you want to try it and you need help, just ask. > > Regards, > Julian Kunkel > > > > 2011/12/13 Kshitij Mehta <[email protected]>: > > Hello, > > > > Is there a way I can trace/measure the internal behavior of pvfs2? > > Suppose I have a simple I/O code that writes to pvfs2, I would like to > > find out how much time exactly do various internal operations of Pvfs2 > > take (metadata lookup, creating iovecs, etc.), before data is finally > pushed to disk. > > > > > > > > Is there a configure option (what does `enabletracing` do in the > > config > > file) ? Or is there any other way to determine this ? > > > > > > > > Thanks, > > Kshitij > > > > > > > > > > _______________________________________________ > > Pvfs2-users mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
