Hi Jason - There are 3 'official' ways to access the filesystem : 1. using pvfs2-cp/ls/etc 2. using the kernel module vfs 3. and using MPI-ROMIO. <am I missing anything?>
In general the kernel interface does tend to lag behind in performance when compared to pvfs2-cp/ls because it uses different buffer sizes. There are a couple things you can do to increase performance of the kernel interface, one which will make a difference is to get the latest version of coreutils (v6+). Before doing the coreutils update you might get a better idea of how this may affect performance via using `dd if=/your/file of=/mnt/pvfs/file bs=64k` The reason I said it may be less than ideal is that PVFS (especially through the kernel interface) does not tend to do well with small IO's, but I can see if your images are large it may still be okay. I'd certainly be interested in your findings. Kyle Schochenmaier On Wed, May 12, 2010 at 8:07 AM, Jason Wood <[email protected]> wrote: > Hi Kyle, > Thanks for your reply. I was looking at the kernel module but it seems to be > considerably slower than using the command-line tools provided plus it > actually makes more sense from a deployment point of view to not use the > kernel module. Is there no way to access data without using the kernel > module? > Why do you think it would not be an ideal setup for a web storage location? > I have looked into a few options and this look promising but if I have > missed something that would make it unsuitable I would have to re-think. > What are the primary usage cases for PVFS2? > Thanks. > On Wed, May 12, 2010 at 1:59 PM, Kyle Schochenmaier <[email protected]> > wrote: >> >> Hi jason - >> >> You'll want to use the pvfs2 kernel module interface (vfs) to acces your >> files, with that you can use normal commands like cp/mv/etc as it provides >> the same interface as other filesystems do at that level. >> I'm not aware of anyone currently or in the past using pvfs2 as a >> webserver storage location, it might not be an ideal setup but that probably >> depends more on your usage, definitely worth a shot though. >> >> Best, >> >> Kyle Schochenmaier >> >> On May 12, 2010 6:55 AM, "Jason Wood" <[email protected]> wrote: >> >> Hi, >> I just found PVFS2 while looking for a solution to storing lots of images >> for a web app and it looks very interesting. I just have a few questions >> which I hope someone can answer. >> I have setup an instance of PVFS2 on CentOS5 using the installation guide >> for a single server instance and it's up and running but I cannot see what >> the correct method for reading a file is. I have tried out pvfs2-cp to copy >> a file in and pvfs2-ls to verify that the file was indeed created, do I have >> to do a pvfs2-cp to copy the file back to the local filesystem to be able to >> access it? >> Are there any major installation of PVFS2 in production at this time? I >> see plenty of support on the community page but I was wondering if there are >> any PVFS2 installations serving any well known sites? >> Thanks, >> Jason >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
