Hi Khalil, Would be good to retain the list email address in the cc so that others can chime in as well.
I have now the system up and running (Y)...
great!
Besides, I would like to know your opinion/suggestions for how to do some performance analysis of PVFS2. Of course, any result will be "|"ed to you once done. My very basic Idea is to write a program to generate huge output, and another one that will be the client to ask pvfs2 the files, probably a multithreaded client running on a multi-processor server (other than the four running pvfs2), then measure with a benchmark tool such as http://sourceforge.net/projects/munin/. or any other, I still haven't started looking this way... I am completely new and ignorant to this field and to pvfs, but I will do my best.
There are tons of benchmark programs each of which measure different things. PVFS is really good at parallel I/O so it would be best if you could run MPI I/O programs (multi-threaded apps running on the same cpu using the kernel/posix interfaces is not a good workload for us because we havent spent time figuring out where performance loss occurs and how to fix them). There is a program called iox.c under pvfs/test/posix/. you can run that MPI program with -m 0 and -s 1 options to get an idea of the aggregate rd/wr bandwidth to a single file. It would be best if you could tell us what your goals are, how your eventual workload looks like etc to save you the trouble of running all these benchmarks before determining if it is appropriate or not for you.
Also, I am wondering if pvfs supports operation such as sort? I see no apparent reference to in install dir no on documentation. In this case, I might try to develop a parallel sort function using pvfs api and openMPI (provided I can understand the system well...)
pvfs is a parallel file system, no more. Any higher-level operation is best done at a level above the FS interface. So if you need some sorting capabilities, you could write one yourself or modify the ones mentioned here to use MPI and run on Linux. http://research.microsoft.com/barc/SortBenchmark/
I would also like to do some comparison with lustre. But since lustre is a kernel patch, I wonder if it can sit with pvfs on the same system. What do you think about it?
it should be possible for lustre and pvfs2 to sit on the same kernel since we take great pains to ensure that pvfs2's kernel module runs on most distro/vanilla kernels. that said, I don't know if that will impact performance for either due to interference/memory pressure, who knows? :)
I also currently installing hadoop (:s) and Ceph....By far pvfs is the easiest to install...
great to hear that! good luck, thanks, Murali
Thanks a lot. Khalil On 12/26/06, Murali Vilayannur <[EMAIL PROTECTED]> wrote: > Hi Khalil, > It looks like the installation etc were successful. In order to > generate the config file run the program pvfs2-genconfig. You can find > this script in /path/to/pvfs2-source/src/apps/admin/pvfs2-genconfig > (or) /path/to/installed/pvfs2/bin/pvfs2-genconfig. > (after replacing it with your installation specific paths). > You can run it by providing the following arguments to it... > > pvfs2-genconfig /etc/pvfs2/fs.conf /etc/pvfs2/server.conf > if /etc/pvfs2 does not exist, you can just create that directory. BTW: > there is no sanctity in choosing /etc/pvfs2 as the standard path for > the config file. it can be anywhere... > hope this helps you get started. > good luck > Murali > > On 12/25/06, Khalil Honsali <[EMAIL PROTECTED] > wrote: > > Greetings all; > > > > I am interested in studying PVFS2 as part of my masters research...My plan > > is to install pvfs2 on a 4x 3Ghz servers... > > > > I have encountered a problem during install process, after checking the > > mailing list archive, I write you to request for help. > > > > I am beginning linux user, so please bare with me. > > > > I have installed pvfs-2.6.1. following the quickstart guide. > > First, I checked & downloaded the required packages. Please see > > PS1---perequisites. > > I have not installed librt, but have libaio installed. > > I have not installed mpich2 but have openmpi installed. > > > > I successfully configure, make and make install pvfs2 from within /usr/src/ > > directory (PS2 - install) > > > > PROBLEM: > > when starting the server configuration to generate the configuration, I > > cannot find the server.conf in /etc/pvfs2/ . (PS3) > > actually the directory does not exist...I wonder If I messed something > > during install....it is 5:40 am, neurone bugs are highly occuring... > > > > Thank you for taking the time to read my post. > > > > Looking forward to hear from you. > > > > > > > > > > > > > > > > PS1---perequisites > > ------------------------------- > > berkeleydb.noarch 1.7.0-1jpp installed > > db4.i386 4.3.29-9.fc6 > > installed > > db4-devel.i386 4.3.29-9.fc6 installed > > glibc.i686 2.5-3 installed > > libaio.i386 0.3.106-3.2 installed > > > > libaio-devel.i386 0.3.106-3.2 installed > > pthreads: /usr/share/man/man7/pthreads.7.gz > > Thread model: posix > > gcc version 4.1.1 20061011 (Red Hat 4.1.1-30) > > openmpi-libs.i386 1.1-7.fc6 installed > > > > PS2 - Install > > ------------------------------------------ > > cp -Rv pvfs-2.6.1/ /usr/src/ > > cd /usr/src/ > > ln -s pvfs2-2.6.1 pvfs2 > > cd pvs2 > > ./configure > > #please see PS2.1 below > > make > > make install > > #please see PS2.2 below > > > > PS3 -- Where is PVFS2?? > > ------------------------------------------------------ > > whereis pvfs2-server > > pvfs2-server: /usr/local/sbin/pvfs2-server > > > > [EMAIL PROTECTED] bin]# ls /usr/local/bin/ > > pvfs2-check-config pvfs2-ls pvfs2-set-debugmask > > pvfs2-chmod pvfs2-lsplus pvfs2-set-eventmask > > pvfs2-chown pvfs2-migrate-collection pvfs2-set-mode > > pvfs2-config pvfs2-mkdir pvfs2-set-sync > > pvfs2-cp pvfs2-mkspace pvfs2-showcoll > > pvfs2-event-mon-example pvfs2-perf-mon-example pvfs2-stat > > pvfs2-fsck pvfs2-perror pvfs2-statfs > > pvfs2-fs-dump pvfs2-ping pvfs2-touch > > pvfs2-genconfig pvfs2-remove-object pvfs2-viewdist > > pvfs2-ln pvfs2-rm pvfs2-xattr > > > > > > PS2.1 - ./Configure LOG > > --------------------------------------- > > > > ***** Displaying PVFS2 Configuration Information ***** > > ------------------------------------------------------ > > PVFS2 configured to build karma gui : no > > PVFS2 configured to use epoll : yes > > PVFS2 configured to perform coverage analysis : no > > PVFS2 configured for aio threaded callbacks : yes > > PVFS2 configured for the 2.6.x kernel module : no > > PVFS2 configured for the 2.4.x kernel module : no > > PVFS2 configured for using the mmap-ra-cache : no > > PVFS2 configured for using trusted connections : no > > PVFS2 configured for a thread-safe client library : yes > > PVFS2 will use workaround for redhat 2.4 kernels : no > > PVFS2 will use workaround for buggy NPTL : no > > PVFS2 server will be built : yes > > > > > > > > PS2.2 - Make Install LOG > > ---------------------------------------- > > [EMAIL PROTECTED] pvfs2]# make install > > install -d /usr/local/share/man/man1 > > install -d /usr/local/share/man/man5 > > install -m 644 ./doc/man/*.1 /usr/local/share/man/man1 > > install -m 644 ./doc/man/*.5 /usr/local/share/man/man5 > > install -d /usr/local/include > > install -m 644 ./include/pvfs2.h /usr/local/include > > install -m 644 ./include/pvfs2-request.h /usr/local/include > > install -m 644 ./include/pvfs2-debug.h /usr/local/include > > install -m 644 ./include/pvfs2-sysint.h /usr/local/include > > install -m 644 ./include/pvfs2- mgmt.h /usr/local/include > > install -m 644 ./include/pvfs2-types.h /usr/local/include > > install -m 644 ./include/pvfs2-util.h /usr/local/include > > install -m 644 ./include/pvfs2-encode-stubs.h /usr/local/include > > install -d /usr/local/lib > > install -m 755 lib/libpvfs2.a /usr/local/lib > > install -d /usr/local/bin > > install -m 755 src/apps/admin/pvfs2-check-config > > src/apps/admin/pvfs2-set-debugmask > > src/apps/admin/pvfs2-set-mode > > src/apps/admin/pvfs2-set-eventmask > > src/apps/admin/pvfs2-set-sync src/apps/admin/pvfs2-ls > > src/apps/admin/pvfs2-lsplus src/apps/admin/pvfs2-ping > > src/apps/admin/pvfs2-rm src/apps/admin/pvfs2-stat > > src/apps/admin/pvfs2-statfs > > src/apps/admin/pvfs2-perf-mon-example > > src/apps/admin/pvfs2-event-mon-example > > src/apps/admin/pvfs2-mkdir src/apps/admin/pvfs2-chmod > > src/apps/admin/pvfs2-chown src/apps/admin/pvfs2-fs-dump > > src/apps/admin/pvfs2-fsck src/apps/admin/pvfs2-cp > > src/apps/admin/pvfs2-viewdist src/apps/admin/pvfs2-xattr > > src/apps/admin/pvfs2-touch > > src/apps/admin/pvfs2-remove-object src/apps/admin/pvfs2-ln > > src/apps/admin/pvfs2-perror /usr/local/bin > > install -m 755 src/apps/admin/pvfs2-config /usr/local/bin > > install -m 755 ./src/apps/admin/pvfs2-genconfig > > /usr/local/bin > > install -d /usr/local/sbin > > install -m 755 src/apps/admin/pvfs2-mkspace > > src/apps/admin/pvfs2-migrate-collection > > src/apps/admin/pvfs2-showcoll /usr/local/bin > > install -m 755 src/server/pvfs2-server /usr/local/sbin > > > > > > > > _______________________________________________ > > Pvfs2-users mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > > > > > >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
