Hi Marco -
I understand that you got the IBV conduit working, but I wanted to leave a
pointer to my earlier experience with Chapel on IBV and MXM conduits:
http://permalink.gmane.org/gmane.comp.lang.chapel.devel/230
I was able to get both IBV and MXM conduits to work with
CHPL_GASNET_SEGMENT=fast ... but had to --disable-pshm when configuring
GASNet - although in a current Chapel, GASNet should build with
--disable-pshm for these conduits by default.
Cheers,
-michael
On Tue, Feb 10, 2015 at 7:19 PM, Marco Postigo <[email protected]> wrote:
> Hello again :)
>
> I've just installed Chapel on a computing cluster of a partner university
> to make some benchmarks of my GLB implementation. Unfortunately I wasn't
> able to use ibv or mxm conduit (only udp works).
>
> When I've tried to simply build Chapel (with make / gmake in Chapels home
> directory) it already failed while trying to building Gasnet because of the
> mxm conduit... so I've checked the Gasnet doc and edited the Makefile
> provided from Chapel in third-party/gasnet to disable mxm support (and
> prevent it from being built). I don't have the error log any more but if
> you're interested I will send it as well.
>
> The main problem is when I'm trying to compile my program having ibv
> activated... the error log will follow at the end of this mail. I have the
> following environment variables set:
>
> export CHPL_HOME=~/opt/chapel
> export CHPL_HOST_PLATFORM=`$CHPL_HOME/util/chplenv/chpl_platform.py`
> export GASNET_ROUTE_OUTPUT=0
> export GASNET_SPAWNFN=S
> export CHPL_COMM=gasnet
> export CHPL_COMM_SUBSTRATE=ibv
> export CHPL_GASNET_SEGMENT=everything
> export MANPATH=$MANPATH:$CHPL_HOME/man
> export PATH=$PATH:$CHPL_HOME/bin/$CHPL_HOST_PLATFORM/
> export SSH_CMD=ssh
> export SSH_OPTIONS=-x
> export CHPL_TASKS=fifo
>
> Operating System: SUSE Linux Enterprise Server 11 (x86_64), VERSION = 11,
> PATCHLEVEL = 3
>
> Other (maybe) useful information:
> gcc (GCC) 4.8.3
> openmpi/gcc/1.8.2
> python/3.4.2
> chapel/1.10.0
>
> Compilation error occuring when trying to compile any program (example is
> here hello2-module.chpl)... I've given the --print-commands parameter as
> well because it may help.
>
> ~/opt/chapel> chpl --print-commands -o hello examples/hello2-module.chpl
> /home/mp27lupo/opt/chapel/util/chplenv/check_huge_pages.py
> /shared/apps/gcc/4.8.3/bin/gcc -DGASNET_PAR -D_REENTRANT -D_GNU_SOURCE
>
> -I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/include
> -I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/include/ibv-conduit
> -I/opt/mellanox/fca/include -std=c99 -Wno-strict-overflow
> -I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include
> -I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none -std=c99
> -DCHPL_TASKS_MODEL_H=\"tasks-fifo.h\"
> -DCHPL_THREADS_MODEL_H=\"threads-pthreads.h\" -DCHPL_WIDE_POINTER_STRUCT
> -DUSE_LOCKS -DONLY_MSPACES -DMSPACES -DCHPL_HAS_GMP -c -o
> /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o -I.
> -I/home/mp27lupo/opt/chapel/runtime//include/localeModels/flat
> -I/home/mp27lupo/opt/chapel/runtime//include/localeModels
> -I/home/mp27lupo/opt/chapel/runtime//include/comm/gasnet
> -I/home/mp27lupo/opt/chapel/runtime//include/comm
> -I/home/mp27lupo/opt/chapel/runtime//include/tasks/fifo
> -I/home/mp27lupo/opt/chapel/runtime//include/threads/pthreads
> -I/home/mp27lupo/opt/chapel/runtime//include/comp-gnu
> -I/home/mp27lupo/opt/chapel/runtime//include/linux64
> -I/home/mp27lupo/opt/chapel/runtime//include
> -I/home/mp27lupo/opt/chapel/runtime//include/qio
> -I/home/mp27lupo/opt/chapel/runtime//include/atomics/intrinsics
> -I/home/mp27lupo/opt/chapel/runtime//include/mem/dlmalloc
> -I/home/mp27lupo/opt/chapel/third-party/utf8-decoder
> -I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include
> -I/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/include
> -I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none
> /tmp/chpl-mp27lupo-27468.deleteme/_main.c
> g++
> -L/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/lib
> -Wl,-rpath
> -Wl,/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/lib
> -L/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/lib
> -Wl,-rpath
> -Wl,/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/lib
> -o /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp
> -L/home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gasnet.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none.gmp.hwloc-none.re2.wide-struct.fs-none
> /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o
> /home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gasnet.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none.gmp.hwloc-none.re2.wide-struct.fs-none/main.o
> -lchpl -lm -lpthread
> -L/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib
> -L/usr/lib64 -L/opt/mellanox/fca/lib -lgasnet-ibv-par -libverbs -lfca
> -lpthread -L/shared/apps/gcc/4.8.3/lib/gcc/x86_64-unknown-linux-gnu/4.8.3
> -lgcc -lm
> -L/home/mp27lupo/opt/chapel/third-party/dlmalloc/install/linux64-gnu-unknown
> -ldlmalloc -lgmp -lre2 -lpthread
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapExchange_mpi':
> gasnet_bootstrap_mpi.c:(.text+0xcd): undefined reference to `ompi_mpi_byte'
> gasnet_bootstrap_mpi.c:(.text+0xf9): undefined reference to `MPI_Allgather'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapBroadcast_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x132): undefined reference to
> `ompi_mpi_byte'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapInit_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x17c): undefined reference to
> `MPI_Initialized'
> gasnet_bootstrap_mpi.c:(.text+0x1a9): undefined reference to `MPI_Init'
> gasnet_bootstrap_mpi.c:(.text+0x1b5): undefined reference to
> `ompi_mpi_comm_world'
> gasnet_bootstrap_mpi.c:(.text+0x1ba): undefined reference to
> `MPI_Comm_group'
> gasnet_bootstrap_mpi.c:(.text+0x1c8): undefined reference to
> `ompi_mpi_comm_world'
> gasnet_bootstrap_mpi.c:(.text+0x1cd): undefined reference to
> `MPI_Comm_create'
> gasnet_bootstrap_mpi.c:(.text+0x1d5): undefined reference to
> `MPI_Group_free'
> gasnet_bootstrap_mpi.c:(.text+0x1e6): undefined reference to
> `MPI_Comm_size'
> gasnet_bootstrap_mpi.c:(.text+0x20a): undefined reference to
> `MPI_Comm_rank'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapFini_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x24a): undefined reference to
> `MPI_Comm_free'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapAbort_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x27e): undefined reference to `MPI_Abort'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapAlltoall_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x2c9): undefined reference to
> `ompi_mpi_byte'
> gasnet_bootstrap_mpi.c:(.text+0x2db): undefined reference to `MPI_Alltoall'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapBroadcast_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x13a): undefined reference to `MPI_Bcast'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapFini_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x265): undefined reference to `MPI_Finalize'
> /home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
> In function `gasneti_bootstrapBarrier_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x2a8): undefined reference to `MPI_Barrier'
> collect2: Fehler: ld gab 1 als Ende-Status zurück
> gmake: *** [/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp] Fehler 1
> error: compiling generated source
>
> Am I doing anything wrong? I've tried to make a hello world mpi program
> which worked fine... I have no clue why it can't find the MPI references in
> gasnet.
>
> Thanks in advance and best regards,
> Marco
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is
> your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Chapel-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/chapel-users
>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
