Here are two *very* basic ways that I use to start pvfs2 on my 8 node
fedora cluster. Not fancy, but quick.
MPI
====
mpiexec -n 8 insmod /usr/local/bin/pvfs2-server/pvfs2.ko
mpiexec -n 8 /usr/local/bin/pvfs2-server/sbin/pvfs2-client -p
/usr/local/bin/pvfs2-server/sbin/pvfs2-client-core
mpiexec -n 8 mount -t pvfs2 tcp://iota1:3334/pvfs2-fs /mnt/pvfs2
XARGS
======
cat clients | xargs -i ssh {} 'insmod /usr/local/bin/pvfs2-server/pvfs2.ko'
cat clients | xargs -i ssh {}
'/usr/local/bin/pvfs2-server/sbin/pvfs2-client -p
/usr/local/bin/pvfs2-server/sbin/pvfs2-client-core'
cat clients | xargs -i ssh {} 'mount -t pvfs2 tcp://iota1:3334/pvfs2-fs
/mnt/pvfs2'
To shutdown:
cat clients | xargs -i ssh {} 'umount /mnt/pvfs2'
cat clients | xargs -i ssh {} 'killall pvfs2-client'
sleep 2
cat clients | xargs -i ssh {} 'rmmod pvfs2'
Robert Latham wrote:
On Wed, Nov 15, 2006 at 05:41:45PM +0530, Vikrant Kumar wrote:
1. I understand that a 1.6 release is around the corner. Can anyone
provide a more definite time? Would be really helpful to plan things
on my side.
You probably already saw this, but we released PVFS 2.6.0 today.
2. I have a cluster with 11 nodes connected with Infiniband . Each
node has four Opteron cores. My application uses MPI IO and I would
like to test it up to 32 way. Given the cluster and the application
requirement, what would be the ideal number of servers for this
set-up? I was thinking either four or eight PVFS2 servers, being
shared with the computing-cores on some nodes. I have read the FAQ
entry and I guess this is a very general question. But I just
wanted to get a quick review of the layout from you guys.
8 is a good number for a lot of workloads. You can use fewer
datafiles if you find your workload uses lots of small files, while
still retaining the ablity to have all 8 servers working together for
larger file accesses.
3. Another thing is when a node is rebooted. The provided script
file "pvfs2-server.rc" can take care of the server but what about
client application, the kernel module and the mount point? Is there
a way to automate the startup of all these as well. I think this
would be a critical thing for this bigger installation.
The problem with automating the kernel moudle and mounting process is
that every distribution does it a little differently. This is an area
where we really can't make too many assumptions. The best way to
automate it is to create a script that does the appropriate things
(load kernel module, start up pvfs2-client and pvfs2-client-core,
mount the file system) for your system. Perhaps admins from other
sites might have ideas for you.
Have fun
==rob
--
Dean Hildebrand
Ph.D. Candidate
University of Michigan
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
