Emmanuel Florac <[email protected]> writes:
>> 2. Run a code that uses MPI-IO (or HDF5) to read or write (big) files.
>> If yes, do I have to modify my code and / or re-compile MPI (or
>> HDF5) with specific options to account for PVFS ?
>
> If your PVFS cluster is installed with the MPIO libraries, it should
> work without code modification. You may need some configuration though,
> like running HDF5 with the proper destination storage parameters,
> something like
>
> HDF5_PREFIX=pvfs2:/myorangefs mpiexec /whatever/program
I'm not sure what "MPIO libraries" means, but at least with openmpi and
using ROMIO, you need to configure it to build against pvfs2 with
something like
./configure ... --with-io-romio-flags="--with-file_system=ufs+pvfs2"
(Some openmpi versions need patching, but I think the latest 1.8 is OK.)
I don't know mpich but perhaps Rob Latham will chime in with
authoritative answers.
Then you need to install the parallel version of HDF5 (package
hdf5-{openmpi,mpich} from EPEL for centos) and ensure it's picked up
correctly, of course, and I think had I some trouble at least with
builds against it.
See <http://institutes.lanl.gov/plfs/> for problems and a workaround
concerning hdf-like parallel i/o on gpfs etc., but I don't know how the
current versions compare. (It's a pity there's no direct comparison, at
least between lustre and orangefs, on the same hardware that I know of;
I'd hoped to make one but I didn't get a chance before Lustre was a fait
acomplis here.) Perhaps the new MPI-IO implementation in openmpi 1.8
for pvfs <http://www.open-mpi.org/papers/euro-mpi-2011-ompio/> provides
extra improvement too.
Note that you can test off local disks on your cluster as normal user
jobs -- see the recent answers to me here. If you can install packages
on it, there's orangefs rpm packaging under
<http://copr.fedoraproject.org/coprs/loveshack/orangefs/>. The binaries
are built against openib, but the source has a flag for that.
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
