Hi Milo, FUSE will give reasonable performance only if some of kernel's parameters, maximum read-ahead and FUSE's maximum pages per request, is increased (manually). Also, you will have to change the minimum buffer size in FUSE's user-level library (iirc default in 2.7.x is 4K) Without them, your performance will be significantly lower compared to PVFS2's kernel module.
Which version of FUSE are you using? A pre-release version of FUSE (2.8.0-pre3) has big_writes option which increases the buffer size a little bit (128KB) and might give you slightly better performance. You could get even better performance by raising this further and compiling it on your own, but my basic tests showed that it will be at best on-par with PVFS2's kernel module. Sumit. On Fri, Jul 24, 2009 at 11:48 AM, Sam Lang<[email protected]> wrote: > > On Jul 24, 2009, at 10:37 AM, Milo Polte wrote: > >> >> On Jul 24, 2009, at 10:49 AM, Sam Lang wrote: >>>> >>>> dd'ing from /dev/zero to /dev/null on the PVFS2 node: 1.5 GB/s >>>> dd'ing from /dev/zero to raid locally without PVFS2: 262 MB/s >>>> dd'ing from /dev/zero to PVFS2 on the same node or on the same switch >>>> (doesn't seem to matter): 54 MB/s. >>> >>> pvfs2fuse or kmod? >> >> Sorry about not being clear: pvfs2fuse. >> >> I suspect a networking issue but what's odd to me is that read speeds are >> much closer to the local performance than write speeds. Your suggestion of >> reading from a null-aio trove and writing to /dev/null got more than 200 >> MB/s through pvfs2. Both Syncs are set to 'no' in the configuration file, so >> I don't expect a metadata bottleneck. But when running IOZone, rewrite is >> also faster than write. Could there be some bottleneck in the metadata >> updates and growing of files? > > You might be on to something there. In 2.8.1 there's a bug that forces a > sync of metadata (the size of the bstream) even when you don't request it, > and it happens even when you use null-aio. You could try the following > patch to see if that's the actual cause of slowness for you. This is > actually Phil's suggestion, so if it works he deserves the credit there. > > http://www.pvfs.org/fisheye/rdiff/PVFS?csid=MAIN:slang:20090421161045&u&N > > I'm a little skeptical of the fuse interface, just because we haven't done > much testing with it. I would be curious what performance you get with > pvfs2-cp from /dev/zero. Maybe you've already done that and I just missed > it in the email thread. > > -sam > >> >> Thanks. >> >> ~Milo >> > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
