Hi Dave,
Our wrappers are custom written by us for our machines, rather than provided by OpenMPI. I’m not sure how the developers feel, but I’m guessing that’s probably how it should be since it’s so highly dependent on the system environment. Plus, the wrappers are at the ifort / gfortran level, rather than mpifort – this way it also works for all the other libraries that provide compiler-specific bindings (i.e. most packages). Essentially, they check each path in the list of include paths for a GNU or Intel subdirectory and add that as needed – and similarly for the library directories. Of course, this breaks build systems (one relatively common one in particular) that expect the files to be in particular locations in the tree rather than just assuming the compiler knows what it’s doing. We thought about doing “module load openmpi/intel/1.10.2” and so on (and similarly for the other packages – we have about 200-300 in our /apps tree now), but decided against it due to the extra complication for our users – especially since many just do “module load openmpi” and don’t care about the version (yuck). I think there were other reasons as well, but that was before my time. Cheers, Ben From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of Dave Turner Sent: Friday, 4 March 2016 3:28 PM To: Ben Menadue <ben.mena...@nci.org.au> Cc: Open MPI Developers <de...@open-mpi.org> Subject: Re: [OMPI devel] mpif.h on Intel build when run with OMPI_FC=gfortran All, Ben's suggestion seems reasonable to me. How about having the mpifort script choose the correct mpif-sizeof.h header file based on the OMPI_FC compiler given at compile time? Dave On Thu, Mar 3, 2016 at 9:48 PM, Ben Menadue <ben.mena...@nci.org.au <mailto:ben.mena...@nci.org.au> > wrote: Hi Dave, The issue is the way MPI_Sizeof is handled; it's implemented as a series of interfaces that map the MPI_Sizeof call to the right function in the library. I suspect this is needed because that function doesn't take a datatype argument and instead infers this from the argument types - in Fortran, this is only possible if you use an interface to call a different function for each data type. Since "real, dimension(:)" is different from "real, dimension(:, :)" from the perspective of the interface, you need a separate entry for each possible array rank. In Fortran 2008 the maximum rank was increased to 15. This is supported in Intel Fortran and so the mpif-sizeof.h from such a build needs to have interface blocks for up to rank-15 arrays. However, the version of gfortran that you're using doesn’t, and hence it complains when it sees the rank > 7 interfaces in mpif-sizeof.h. To make it compatible between Fortran 2008 compliant and non-compliant compilers, you would need to implement MPI_Sizeof in a totally different fashion - if that’s even possible. FWIW: We maintain two copies of mpif-sizeof.h in separate subdirectories – one for gfortran and one for ifort. Then, there are wrappers around each of the compilers that adds the appropriate subdirectory to the include path. This makes it transparent to our users, and allows us to present a single build tree that works for both compilers. Cheers, Ben From: devel [mailto: <mailto:devel-boun...@open-mpi.org> devel-boun...@open-mpi.org] On Behalf Of Dave Turner Sent: Friday, 4 March 2016 2:23 PM To: Larry Baker < <mailto:ba...@usgs.gov> ba...@usgs.gov> Cc: Open MPI Developers < <mailto:de...@open-mpi.org> de...@open-mpi.org> Subject: Re: [OMPI devel] mpif.h on Intel build when run with OMPI_FC=gfortran All, I think that a GNU build of OpenMPI will allow compiling with both gfortan and ifort, so I think OMPI_FC is useful. I'd like to see it fully supported if possible, so if the higher-dimensions in mpif-sizeof.h are not vital and there is another way of accomplishing the same thing I think it would be useful to address. If not, I would at least like to see some warnings in the documentation of the OMPI_FC section that would list the cases like this where it will fail. Dave On Thu, Mar 3, 2016 at 9:07 PM, Larry Baker <ba...@usgs.gov <mailto:ba...@usgs.gov> > wrote: Dave, Both Gilles and Chris raise important points. You really cannot expect to mix modules from two different Fortran compilers in a single executable. There is no requirement placed on a compiler by the Fortran standard for what object language it should use, how the information in modules is made available across compilation units, or the procedure calling conventions. This makes me wonder, as you do, what the point is of the OMPI_CC and OMPI_FC environment variables? I do think that Intel has tried to make their objects interoperable with GCC objects. That is a link-time issue. You are encountering compile-time issues. Gilles says whatever mpif-sizeof.h was intended to define, it cannot be done in gfortran. Even if mpif-sizeof.h generated for an Intel compiler was standard-conforming (so the errors you encountered are not show stoppers), I'm not sure you would be able to get past the incompatibility between the internal formats used by each compiler to store module definitions and declarations for later USE by another compilation unit. I think your expectations cannot be fulfilled because of the compilers, not because of OpenMPI. Larry Baker US Geological Survey <tel:650-329-5608> 650-329-5608 <mailto:ba...@usgs.gov> ba...@usgs.gov On 3 Mar 2016, at 6:39 PM, Dave Turner wrote: Gilles, I don't see the point of having the OMPI_CC and OMPI_FC environment variables at all if you're saying that we shouldn't expect them to work. I actually do think they work fine if you do a GNU build and use them to specify the Intel compilers. I also think it works fine when you do an Intel build and compile with gcc. So to me it just looks like that one include file is the problem. Dave On Thu, Mar 3, 2016 at 8:02 PM, Gilles Gouaillardet <gil...@rist.or.jp <mailto:gil...@rist.or.jp> > wrote: Dave, you should not expect anything when mixing Fortran compilers (and to be on the safe side, you might not expect much when mixing C/C++ compilers too, for example, if you built ompi with intel and use gcc for your app, gcc might complain about unresolved symbols from the intel runtime) if you compile OpenMPI with gfortran 4.8.5, the automatically generated mpif-sizeof.h contains ! Sad panda. ! ! This compiler does not support the Right Stuff to enable MPI_SIZEOF. ! Specifically: we need support for the INTERFACE keyword, ! ISO_FORTRAN_ENV, and the STORAGE_SIZE() intrinsic on all types. ! Apparently, this compiler does not support both of those things, so ! this file will be (effecitvely) blank (i.e., we didn't bother ! generating the necessary stuff for MPI_SIZEOF because the compiler ! doesn't support ! it). ! ! If you want support for MPI_SIZEOF, please use a different Fortran ! compiler to build Open MPI. intel fortran compilers have the right stuff, so mpif-sizeof.h is usable, and you get something very different. Cheers, Gilles On 3/4/2016 10:17 AM, Dave Turner wrote: My understanding is that OpenMPI built with either Intel or GNU compilers should be able to use the other compilers using the OMPI_CC and OMPI_FC environmental variables. For OpenMPI-1.8.8 built with Intel compilers, if you try to compile any code that includes mpif.h using OMPI_FC=gfortran it fails. The Intel build creates mpi-sizeof.h that dimensions arrays to more than 6 dimensions which gfortran cannot handle. The example below illustrates this. I wasn't able to find any other reports like this on the web, and I don't see any way of specifying a path to an alternate mpif.h include file. This looks to be a bug to me, but please let me know if I missed a config flag somewhere. Dave Turner Selene cat bugtest.F ! Program to illustrate bug when OpenMPI is compiled with Intel ! compilers but run using OMPI_FC=gfortran. PROGRAM BUGTEST INCLUDE "mpif.h" END Selene cat go #!/bin/bash echo "Compile test using default ifort" mpifort --version mpifort bugtest.F -o bugtest_ifort echo "Compile test using gfortan when OpenMPI was compiled with ifort/icc" export OMPI_FC=gfortran mpifort --version mpifort bugtest.F -o bugtest_gfortran Selene ./go Compile test using default ifort ifort (IFORT) 15.0.3 20150407 Copyright (C) 1985-2015 Intel Corporation. All rights reserved. Compile test using gfortan when OpenMPI was compiled with ifort/icc GNU Fortran (Gentoo 4.9.3 p1.4, pie-0.6.4) 4.9.3 Copyright (C) 2015 Free Software Foundation, Inc. GNU Fortran comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Fortran under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING mpif-sizeof.h:75.48: Included at mpif.h:61: Included at bugtest.F:6: COMPLEX(REAL128), DIMENSION(1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions mpif-sizeof.h:82.48: Included at mpif.h:61: Included at bugtest.F:6: COMPLEX(REAL128), DIMENSION(1,1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions mpif-sizeof.h:89.48: Included at mpif.h:61: Included at bugtest.F:6: COMPLEX(REAL128), DIMENSION(1,1,1,1,1,1,1,1,1,*)::x 1 Error: Array specification at (1) has more than 7 dimensions [ More of the same errors have been clipped ] -- Work: davetur...@ksu.edu <mailto:davetur...@ksu.edu> (785) 532-7791 <tel:%28785%29%20532-7791> 2219 Durland, Manhattan KS 66502 Home: drdavetur...@gmail.com <mailto:drdavetur...@gmail.com> cell: (785) 770-5929 <tel:%28785%29%20770-5929> _______________________________________________ devel mailing list de...@open-mpi.org <mailto:de...@open-mpi.org> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel Link to this post: http://www.open-mpi.org/community/lists/devel/2016/03/18671.php -- Work: davetur...@ksu.edu <mailto:davetur...@ksu.edu> (785) 532-7791 <tel:%28785%29%20532-7791> 2219 Durland, Manhattan KS 66502 Home: drdavetur...@gmail.com <mailto:drdavetur...@gmail.com> cell: (785) 770-5929 <tel:%28785%29%20770-5929> _______________________________________________ devel mailing list de...@open-mpi.org <mailto:de...@open-mpi.org> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel Link to this post: http://www.open-mpi.org/community/lists/devel/2016/03/18678.php -- Work: davetur...@ksu.edu <mailto:davetur...@ksu.edu> (785) 532-7791 <tel:%28785%29%20532-7791> 2219 Durland, Manhattan KS 66502 Home: drdavetur...@gmail.com <mailto:drdavetur...@gmail.com> cell: (785) 770-5929 <tel:%28785%29%20770-5929> -- Work: davetur...@ksu.edu <mailto:davetur...@ksu.edu> (785) 532-7791 2219 Durland, Manhattan KS 66502 Home: drdavetur...@gmail.com <mailto:drdavetur...@gmail.com> cell: (785) 770-5929
