Hmmm…the OMPI folks point out that directly including mpio.h doesn’t conform to the standard, which is why we don’t separately install it.
> On Sep 11, 2015, at 9:58 AM, Ralph Castain <[email protected]> wrote: > > I’ve asked the OMPI team if we can/should have a separate mpio.h in our > install - will let you know as soon as I hear back. > > Ralph > >> On Sep 11, 2015, at 8:45 AM, Michael Gutteridge >> <[email protected] <mailto:[email protected]>> wrote: >> >> Thanks for all the advice- I followed Jared's suggestion- a basic .c that >> includes the hdf5 headers. That and the HDF examples all compiled and ran >> fine. >> >> Until, that is, I had 'src/common' in the include path. Building my >> examples in the Slurm build tree using an include path with src/common and >> the slurm source top level fails with the same error. >> >> What I see as possibly problematic is 'src/common/mpi.h'. The HDF include >> file ('/usr/include/H5public.h') has a check that handles the MPICH vs. >> OpenMPI differences: >> >> >> 60 #ifdef H5_HAVE_PARALLEL >> 61 # define MPICH_SKIP_MPICXX 1 >> 62 # define OMPI_SKIP_MPICXX 1 >> 63 # include <mpi.h> >> 64 #ifndef MPI_FILE_NULL /*MPIO may be defined in mpi.h already >> */ >> 65 # include <mpio.h> >> 66 #endif >> 67 #endif >> >> >> It appears that OpenMPI has MPIO defined in mpi.h, whereas MPICH defines >> that stuff in mpio.h. As I've got a parallel build, I suspect that the make >> includes src/common/mpi.h, doesn't find "MPI_FILE_NULL" defined and attempts >> to include mpio.h. >> >> I'm not the strongest C user, so I could be wrong on that, though when I >> remove "src/common" from the include path for libsh5util_old it does compile >> successfully. >> >> I think worst case I'll end up with bootstrapping as Paul has indicated >> (thanks for the procedure, BTW). >> >> Thanks for all your time >> >> M >> >> >> >> On Fri, Sep 11, 2015 at 7:30 AM, Van Der Mark, Paul <[email protected] >> <mailto:[email protected]>> wrote: >> I'm glad we were not the only one with that problem. I basically did a >> bootstrap compilation >> 1. compile openmpi without slurm >> 2. compile hdf5 with that openmpi version >> 3. compile slurm with that hpdf5 version >> 4. recompile openmpi with slurm >> 5. for safety recompile hdf5 & slurm >> >> One of the reasons for that slightly confusing setup is because our >> semi-automated building system compiles 3 different versions of openmpi >> and compiles hdf5 for serial and all those openmpi versions at once. >> >> Beet, >> Paul >> >> On Thu, 2015-09-10 at 14:13 -0700, Jared David Baker wrote: >> > Hello Michael, >> > >> > >> > >> > I had this problem the other day when I was building Slurm on Arch. I >> > had hdf-mpi package installed with OpenMPI and there was an >> > inconsistency there as well. Basically the HDF5 implementation was >> > built with mpio in mind, but the compiler invocation does not find an >> > mpio.h as part of the OpenMPI installation. I guess my base point is I >> > have a different HDF5 implementation on our HPC clusters to satisfy >> > the Slurm requirements separate from the MPI enabled versions. Anyway, >> > I’d be suspicious of the OpenMPI/HDF5 packages you’ve installed in >> > Ubuntu and possibly open a bug if the HDF5 uses OpenMPI as the MPI >> > implementation and doesn’t contain the mpio.h file. Can you compile a >> > simple code with your MPI compiler wrapper that includes mpio.h? >> > >> > >> > >> > Best, >> > >> > >> > >> > Jared >> > >> > >> > >> > From: Michael Gutteridge [mailto:[email protected] >> > <mailto:[email protected]>] >> > Sent: Thursday, September 10, 2015 12:41 PM >> > To: slurm-dev >> > Subject: [slurm-dev] Compile of 15.08.0 fails on trusty missing mpio.h >> > >> > >> > >> > >> > Hi >> > >> > >> > >> > >> > >> > This really feels like an obvious one, but I'm having a devil of a >> > time sorting out how to address this. I'm building Slurm 15.08.0 on >> > Ubuntu 14.04 LTS with a minimal set of configure options: >> > >> > >> > >> > >> > >> > --sysconfdir=/etc/slurm-llnl --localstatedir=/var/run/slurm-llnl >> > --with-munge --without-blcr --enable-pam --without-rpath >> > --disable-debug >> > >> > >> > >> > >> > >> > Configure succeeds OK, but make fails a little while in during the >> > build of the HDF5 components: >> > >> > >> > >> > >> > >> > Making all in libsh5util_old >> > >> > >> > make[8]: Entering directory >> > `/home/build/trusty/slurm-llnl/15.08.0/build/slurm-15.08.0/obj-x86_64-linux-gnu/src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old' >> > >> > >> > /bin/bash ../../../../../../libtool --tag=CC --mode=compile gcc >> > -DHAVE_CONFIG_H -I. >> > -I../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old >> > -I../../../../../.. -I../../../../../../slurm -I../../../../../../.. >> > -I../../../../../../../src/common -I. -I/usr/lib/openmpi/include -I >> > /usr/include -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat >> > -Werror=format-security -pthread -fno-gcse -c -o sh5util.lo >> > ../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old/sh5util.c >> > >> > >> > libtool: compile: gcc -DHAVE_CONFIG_H -I. >> > -I../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old >> > -I../../../../../.. -I../../../../../../slurm -I../../../../../../.. >> > -I../../../../../../../src/common -I. -I/usr/lib/openmpi/include -I >> > /usr/include -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat >> > -Werror=format-security -pthread -fno-gcse -c >> > ../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old/sh5util.c >> > -fPIC -DPIC -o .libs/sh5util.o >> > >> > >> > In file included from /usr/include/hdf5.h:24:0, >> > >> > >> > >> > from >> > ../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old/hdf5_api.h:61, >> > >> > >> > >> > from >> > ../../../../../../../src/plugins/acct_gather_profile/hdf5/sh5util/libsh5util_old/sh5util.c:72: >> > >> > >> > /usr/include/H5public.h:65:21: fatal error: mpio.h: No such file or >> > directory >> > >> > >> > # include <mpio.h> >> > >> > >> > ^ >> > >> > >> > compilation terminated. >> > >> > >> > make[8]: *** [sh5util.lo] Error 1 >> > >> > >> > >> > >> > >> > >> > >> > >> > I have both OpenMPI (1.6.5) and MPICH (3.0.4) development packages >> > installed- mpio.h is present in /usr/include/mpich/mpio.h. If I add >> > "CFLAGS=-I/usr/include/mpich" then the build succeeds. That sort of >> > feels kludgy- I'm of the opinion that the configure should have picked >> > that up or that I don't, in fact, have the correct development headers >> > installed. We'd never needed/used MPICH in past... I thought OpenMPI >> > would have supplied mpio.h. >> > >> > >> > >> > >> > >> > Does anyone have any insights into this? Should configure pick it up? >> > Is it a quirk of Ubuntu? >> > >> > >> > >> > >> > >> > Thanks much >> > >> > >> > >> > >> > >> > Michael >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> >> >
