Hi Roland, Thanks for your fast response!
I will just update the Einstein Toolkit.. Time flies. I didn't realise my version was now 4 years old! Thanks, Chris On Wed, 30 Sep 2020 at 09:36, Roland Haas <[email protected]> wrote: > Hello Chris, > > my best guess is that the MPI stack on your system got updated. > > MPI_UB has been removed in newer MPI standards and OpenMPI 4 does no > longer offer it by default. > > Carpet should have stopped using it in commit > 5073af2caddf84b64d0aa475968cec4e9845ba73 "CarpetLib: Use > MPI_Type_create_struct instead of MPI_Type_struct" from 2016. > > The best solution would be to update your Einstein Toolkit tree. > > If that is not possible you can try and see if you can downgrade or use > an older MPI stack. > > Finally you can (if this is your workstation or so) ask Cactus to build > the MPI library included in ExternalLibraries/MPI from source (which is > configured to provide the old MPI_UB symbol) by setting: > > MPI_DIR=BUILD > > in your option list and compiling from scratch (make sim-realclean). > > Yours, > Roland > > > Hi everyone, > > > > I've managed to get this compiling and have just completed the example > BBH > > run simulating the system of the first detection, however now that I have > > left it for a few weeks and come back, naturally nothing works. > > > > The error I now have is with MPI_UB and dh.cc when building through > > simfactory. I get the following output: > > > > In file included from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dist.hh:12, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/vect.hh:13, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/bbox.hh:11, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:10: > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc: In > > function ‘ompi_datatype_t* mpi_datatype(const dh::light_dboxes&)’: > > /opt/openmpi4/include/mpi.h:337:63: error: expected ‘}’ before ‘;’ token > > #define THIS_SYMBOL_WAS_REMOVED_IN_MPI30(symbol, new_symbol) 0; > > static_assert(0, #symbol " was removed in MPI-3.0. Use " #new_symbol " > > instead.") > > ^ > > /opt/openmpi4/include/mpi.h:1131:24: note: in expansion of macro > > ‘THIS_SYMBOL_WAS_REMOVED_IN_MPI30’ > > # define MPI_UB THIS_SYMBOL_WAS_REMOVED_IN_MPI30(MPI_UB, > > MPI_Type_create_resized); > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:23: > > note: in expansion of macro ‘MPI_UB’ > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:9: > > note: to match this ‘{’ > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^ > > In file included from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dist.hh:12, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/vect.hh:13, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/bbox.hh:11, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:10: > > /opt/openmpi4/include/mpi.h:337:63: error: expected ‘}’ before ‘;’ token > > #define THIS_SYMBOL_WAS_REMOVED_IN_MPI30(symbol, new_symbol) 0; > > static_assert(0, #symbol " was removed in MPI-3.0. Use " #new_symbol " > > instead.") > > ^ > > /opt/openmpi4/include/mpi.h:1131:24: note: in expansion of macro > > ‘THIS_SYMBOL_WAS_REMOVED_IN_MPI30’ > > # define MPI_UB THIS_SYMBOL_WAS_REMOVED_IN_MPI30(MPI_UB, > > MPI_Type_create_resized); > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:23: > > note: in expansion of macro ‘MPI_UB’ > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2188:46: > > note: to match this ‘{’ > > dist::mpi_struct_descr_t const descr[] = { > > ^ > > In file included from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dist.hh:12, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/vect.hh:13, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/bbox.hh:11, > > from > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:10: > > /opt/openmpi4/include/mpi.h:337:79: error: static assertion failed: > MPI_UB > > was removed in MPI-3.0. Use MPI_Type_create_resized instead. > > #define THIS_SYMBOL_WAS_REMOVED_IN_MPI30(symbol, new_symbol) 0; > > static_assert(0, #symbol " was removed in MPI-3.0. Use " #new_symbol " > > instead.") > > > > ^ > > /opt/openmpi4/include/mpi.h:1131:24: note: in expansion of macro > > ‘THIS_SYMBOL_WAS_REMOVED_IN_MPI30’ > > # define MPI_UB THIS_SYMBOL_WAS_REMOVED_IN_MPI30(MPI_UB, > > MPI_Type_create_resized); > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:23: > > note: in expansion of macro ‘MPI_UB’ > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:29: > > error: expected primary-expression before ‘,’ token > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2188:36: > > warning: unused variable ‘descr’ [-Wunused-variable] > > dist::mpi_struct_descr_t const descr[] = { > > ^~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2177:23: > > warning: unused variable ‘newtype’ [-Wunused-variable] > > static MPI_Datatype newtype; > > ^~~~~~~ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2195:50: > > warning: no return statement in function returning non-void > [-Wreturn-type] > > {1, sizeof s, MPI_UB, "MPI_UB", "MPI_UB"}}; > > ^ > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc: At > > global scope: > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2197:5: > > error: ‘newtype’ does not name a type; did you mean ‘iswctype’? > > newtype = dist::create_mpi_datatype(sizeof descr / sizeof descr[0], > > descr, > > ^~~~~~~ > > iswctype > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2208:5: > > error: ‘initialised’ does not name a type; did you mean > ‘MPI_Initialized’? > > initialised = true; > > ^~~~~~~~~~~ > > MPI_Initialized > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2209:3: > > error: expected declaration before ‘}’ token > > } > > ^ > > > /maybehome/cstevens/Cactus/arrangements/Carpet/CarpetLib/src/dh.cc:2177:23: > > warning: ‘newtype’ defined but not used [-Wunused-variable] > > static MPI_Datatype newtype; > > ^~~~~~~ > > make[3]: *** > > > [/maybehome/cstevens/Cactus/configs/CTGD_Testing/config-data/make.config.rules:281: > > dh.cc.o] Error 1 > > make[2]: *** [/maybehome/cstevens/Cactus/lib/make/make.thornlib:113: > > make.checked] Error 2 > > make[1]: *** [/maybehome/cstevens/Cactus/lib/make/make.configuration:181: > > /maybehome/cstevens/Cactus/configs/CTGD_Testing/lib/libthorn_CarpetLib.a] > > Error 2 > > make: *** [Makefile:260: CTGD_Testing] Error 2 > > > > I did find the following thread: > > > > > http://lists.einsteintoolkit.org/pipermail/users/2018-October/006566.html > > > > and I've done a clean compilation but same deal. I'm using OpenMPI 4.0.5 > > and gcc/g++ etc 8.3.1. I've confirmed which mpicc etc points to the right > > place, and what I think the relevant lines in the option list file are: > > > > CPPFLAGS = -DOMPI_SKIP_MPICXX > > FPPFLAGS = -traditional > > CFLAGS = -g3 -std=gnu11 -Wunused-variable > > CXXFLAGS = -g3 -std=gnu++11 -Wunused-variable > > F77FLAGS = -g3 -fcray-pointer -ffixed-line-length-none > > F90FLAGS = -g3 -fcray-pointer -ffixed-line-length-none# -Wextra > > -Wimplicit-interface#-fallow-argument-mismatch > > > > MPI_DIR = /opt/openmpi4/ > > MPI_LIB_DIRS = /opt/openmpi4/lib > > MPI_INC_DIRS = /opt/openmpi4/include > > MPI_LIBS = mpi open-rte open-pal > > > > I have added CPP = ... CXX = ... etc which point to the appropriate > places. > > > > I also have > > > > cstevens@skynet:~/Cactus$ mpic++ --showme > > g++ -I/opt/openmpi4/include -pthread -Wl,-rpath -Wl,/opt/openmpi4/lib > > -Wl,--enable-new-dtags -L/opt/openmpi4/lib -lmpi > > > > Any help would be greatly appreciated. > > > > Thanks, > > > > Chris > > > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://pgp.mit.edu . > -- Dr Chris Stevens Postdoctoral Fellow Rm 310A Department of Mathematics and Statistics University of Otago Ph: +64 3 479 4567 http://www.chrisdoesmaths.com http://gravity.otago.ac.nz
_______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
