Well , heres why I wanted to trace in the first place. I have a test configuration where we have configured PVFS2 over an SSD storage. There are two I/O servers that talk to the SSD storage through Infiniband (There are 2 IB channels going into the SSD, and each storage server can 'see' one half of the SSD).
Now I used the IOR benchmark to test the write bandwidth. I first spawn a process on the I/O server such that it writes data to the underlying ext4 file system on the SSD instead of PVFS2. I see a bandwidth of ~350 MB/s. Now I spawn a process on the same I/O server and write data to the PVFS2 file system configured over the SSD, and I see a write bandwidth of ~180 MB/s. This seems to represent some kind of overhead with PVFS2, but seems too large. Has anybody else seen similar results? Is the overhead of pvfs2 documented? Do let me know if something is not clear or if you have additional questions about the above setup. Here are some other details: I/O servers: dual core with 2G main memory each. PVFS 2.8.2 Thanks, Kshitij -----Original Message----- From: Julian Kunkel [mailto:[email protected]] Sent: Tuesday, December 13, 2011 3:10 AM To: Kshitij Mehta Cc: [email protected] Subject: Re: [Pvfs2-users] Tracing pvfs2 internals Dear Kshitij, we have a version of OrangeFS which is instrumented with HDTrace, there you can record detailed information about activity of statemachines and I/O. For a description see the thesis: http://wr.informatik.uni-hamburg.de/_media/research:theses:Tien%20Duc%20Tien_Tracing%20Internal%20Behavior%20in%20PVFS.pdf The code is available in our redmine (here is a link to the wiki): http://redmine.wr.informatik.uni-hamburg.de/projects/piosimhd/wiki I consider the tracing implemented in PVFS as rather robust, since it is our second implementation with PVFS_hints. However, you might encounter some issues with the build system. If you want to try it and you need help, just ask. Regards, Julian Kunkel 2011/12/13 Kshitij Mehta <[email protected]>: > Hello, > > Is there a way I can trace/measure the internal behavior of pvfs2? > Suppose I have a simple I/O code that writes to pvfs2, I would like to > find out how much time exactly do various internal operations of Pvfs2 > take (metadata lookup, creating iovecs, etc.), before data is finally pushed > to disk. > > > > Is there a configure option (what does `enabletracing` do in the > config > file) ? Or is there any other way to determine this ? > > > > Thanks, > Kshitij > > > > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
