Weikuan
I previously reported that I was having problems running any MPI jobs between a pair of EM64T machines with RHEL4, Update 3 with the OpenIB modules, (kernel versions 2.6.9-34.ELsmp) and the "mvapich-gen2" code from the OpenIB svn tree. I was having two problems:
- When I tried to run from user
mode, I would get segmentation faults
- When I ran from root, the jobs would fail with the following message: "cpi: pmgr_client_mpd.c:254: mpd_exchange_info: Assertion `len_remote == len_local' failed. ".
The first problem turned out to be a memory problem; I had to increase the size of the max locked-in-memory address space (memlock) in the user limits.
The second problem seemed to be more related to process management than to MPI itself. I remembered that when I modified the "make.mvapich.gen2" build script, there was a parameter for MPD:
# Whether to use an optimized queue pair exchange scheme. This is not
# checked for a setting in in the script. It must be set here explicitly.
# Supported: "-DUSE_MPD_RING", "-DUSE_MPD_BASIC" and "" (to disable)
HAVE_MPD_RING=""
Because I wanted to use MPD to launch jobs, I set HAVE_MPD_RING="-DUSE_MPD_RING" in the build script.
I went back and set the parameter to HAVE_MPD_RING="" to disable it, and rebuilt, which meant that MPD was not installed. Using "mpirun_rsh" I am now able to run the MPI jobs, including "cpi", "mping" and other benchmark tests.
There seems to be a problem with "USE_MPD_RING". Have you seen this before? Should I try with "USE_MPD_BASIC" instead?
-Don Albert-
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
