I haven't worked on ibv enough to have anything much to contribute to that
part of the conversation, but:
Furthermore I would like to suggest to improve the doc of the "chpl -h"
output a bit... I had to do some try and error to get it work because I
was not sure how the --ccflags and --ldflags parameter expects the
flags... tried without leading minus and comma separated etc. Finally
I've found out that I just need to declare it multiple times which was
not what I expected (e.g. knowing from other cmd line tools).
In practice, I tend to use quoted lists of flags:
--ccflags "-fcheck-new -Waddress" --ldflags "--audit --magic"
I hadn't realized that multiple ccflags/ldflags could be used, and agree
that it seems surprising that they would work that way given that most of
our flags override previous settings rather than append to them. I'm not
sure if that's a good or bad thing in this instance.
Also note that 'chpl' accepts -I, -L, -l flags (and .h, .c, .o files) on
the command line directly, so these cases need not be sent through via
--ccflags/-ldflags, if that's more attractive.
Marco, do you have the ability to sign Chapel's contributor agreement(s)?
(ILCA at miniumum; CCLA if you're working on Chapel as part of your job or
a funded situation). Submitting a proposed patch against the --help
output is a trivial way to contribute back to the project :) (though I
think the 'chpl' man page may be the better place to focus given that it's
far less space constrained). Pointers available in paragraph one here:
http://chapel.cray.com/developers.html
-Brad
Thanks again and have a good one!
Best regards,
Marco
Gesendet: Mittwoch, 11. Februar 2015 um 16:30 Uhr
Von: "Greg Titus" <[email protected]>
An: "Marco Postigo" <[email protected]>
Cc: [email protected]
Betreff: Re: Problem with compiling chapel programs if ibv conduit is used
Hello Marco --
Looking at the log, it seems the problem is that no MPI library is
referenced in the link step. When you link your hello world MPI test
program, is there some equivalent to --print-commands you can give to
see what the MPI library is called? For example, if you link your hello
world with gcc, this would be the -v option. There should be a -lmpi
option or something similar. If you give this same option to the Chapel
compiler when compiling your Chapel program it should bring in the
appropriate MPI library.
greg
On 2/10/2015 5:19 PM, Marco Postigo wrote:
> Hello again :)
>
> I've just installed Chapel on a computing cluster of a partner university to
make some benchmarks of my GLB implementation. Unfortunately I wasn't able to use
ibv or mxm conduit (only udp works).
>
> When I've tried to simply build Chapel (with make / gmake in Chapels home
directory) it already failed while trying to building Gasnet because of the mxm
conduit... so I've checked the Gasnet doc and edited the Makefile provided from
Chapel in third-party/gasnet to disable mxm support (and prevent it from being
built). I don't have the error log any more but if you're interested I will send
it as well.
>
> The main problem is when I'm trying to compile my program having ibv
activated... the error log will follow at the end of this mail. I have the
following environment variables set:
>
> export CHPL_HOME=~/opt/chapel
> export CHPL_HOST_PLATFORM=`$CHPL_HOME/util/chplenv/chpl_platform.py`
> export GASNET_ROUTE_OUTPUT=0
> export GASNET_SPAWNFN=S
> export CHPL_COMM=gasnet
> export CHPL_COMM_SUBSTRATE=ibv
> export CHPL_GASNET_SEGMENT=everything
> export MANPATH=$MANPATH:$CHPL_HOME/man
> export PATH=$PATH:$CHPL_HOME/bin/$CHPL_HOST_PLATFORM/
> export SSH_CMD=ssh
> export SSH_OPTIONS=-x
> export CHPL_TASKS=fifo
>
> Operating System: SUSE Linux Enterprise Server 11 (x86_64), VERSION = 11,
PATCHLEVEL = 3
>
> Other (maybe) useful information:
> gcc (GCC) 4.8.3
> openmpi/gcc/1.8.2
> python/3.4.2
> chapel/1.10.0
>
> Compilation error occuring when trying to compile any program (example is
here hello2-module.chpl)... I've given the --print-commands parameter as well
because it may help.
>
> ~/opt/chapel> chpl --print-commands -o hello examples/hello2-module.chpl
> /home/mp27lupo/opt/chapel/util/chplenv/check_huge_pages.py
> /shared/apps/gcc/4.8.3/bin/gcc -DGASNET_PAR -D_REENTRANT -D_GNU_SOURCE
-I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/include
-I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/include/ibv-conduit
-I/opt/mellanox/fca/include -std=c99 -Wno-strict-overflow
-I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include
-I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none -std=c99
-DCHPL_TASKS_MODEL_H=\"tasks-fifo.h\"
-DCHPL_THREADS_MODEL_H=\"threads-pthreads.h\" -DCHPL_WIDE_POINTER_STRUCT -DUSE_LOCKS
-DONLY_MSPACES -DMSPACES -DCHPL_HAS_GMP -c -o /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o -I.
-I/home/mp27lupo/opt/chapel/runtime//include/localeModels/flat
-I/home/mp27lupo/opt/chapel/runtime//include/localeModels
-I/home/mp27lupo/opt/chapel/runtime//include/comm/gasnet
-I/home/mp27lupo/opt/chapel/runtime//include/comm
-I/home/mp27lupo/opt/chapel/runtime//include/tasks/fifo
-I/home/mp27lupo/opt/chapel/runtime//include/threads/pthreads
-I/home/mp27lupo/opt/chapel/runtime//include/comp-gnu
-I/home/mp27lupo/opt/chapel/runtime//include/linux64 -I/home/mp27lupo/opt/chapel/runtime//include
-I/home/mp27lupo/opt/chapel/runtime//include/qio
-I/home/mp27lupo/opt/chapel/runtime//include/atomics/intrinsics
-I/home/mp27lupo/opt/chapel/runtime//include/mem/dlmalloc
-I/home/mp27lupo/opt/chapel/third-party/utf8-decoder
-I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include
-I/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/include
-I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none /tmp/chpl-mp27lupo-27468.deleteme/_main.c
> g++
-L/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/lib
-Wl,-rpath
-Wl,/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/lib
-L/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/lib
-Wl,-rpath
-Wl,/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/lib -o
/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp
-L/home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gasnet.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none.gmp.hwloc-none.re2.wide-struct.fs-none
/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o
/home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gasnet.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none.gmp.hwloc-none.re2.wide-struct.fs-none/main.o
-lchpl -lm -lpthread
-L/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib
-L/usr/lib64 -L/opt/mellanox/fca/lib -lgasnet-ibv-par -libverbs -lfca
-lpthread -L/shared/apps/gcc/4.8.3/lib/gcc/x86_64-unknown-linux-gnu/4.8.3 -lgcc
-lm -L/home/mp27lupo/opt/chapel/third-party/dlmalloc/install/linux64-gnu-unknown
-ldlmalloc -lgmp -lre2 -lpthread
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapExchange_mpi':
> gasnet_bootstrap_mpi.c:(.text+0xcd): undefined reference to `ompi_mpi_byte'
> gasnet_bootstrap_mpi.c:(.text+0xf9): undefined reference to `MPI_Allgather'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapBroadcast_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x132): undefined reference to `ompi_mpi_byte'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapInit_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x17c): undefined reference to `MPI_Initialized'
> gasnet_bootstrap_mpi.c:(.text+0x1a9): undefined reference to `MPI_Init'
> gasnet_bootstrap_mpi.c:(.text+0x1b5): undefined reference to
`ompi_mpi_comm_world'
> gasnet_bootstrap_mpi.c:(.text+0x1ba): undefined reference to `MPI_Comm_group'
> gasnet_bootstrap_mpi.c:(.text+0x1c8): undefined reference to
`ompi_mpi_comm_world'
> gasnet_bootstrap_mpi.c:(.text+0x1cd): undefined reference to `MPI_Comm_create'
> gasnet_bootstrap_mpi.c:(.text+0x1d5): undefined reference to `MPI_Group_free'
> gasnet_bootstrap_mpi.c:(.text+0x1e6): undefined reference to `MPI_Comm_size'
> gasnet_bootstrap_mpi.c:(.text+0x20a): undefined reference to `MPI_Comm_rank'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapFini_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x24a): undefined reference to `MPI_Comm_free'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapAbort_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x27e): undefined reference to `MPI_Abort'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapAlltoall_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x2c9): undefined reference to `ompi_mpi_byte'
> gasnet_bootstrap_mpi.c:(.text+0x2db): undefined reference to `MPI_Alltoall'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapBroadcast_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x13a): undefined reference to `MPI_Bcast'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapFini_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x265): undefined reference to `MPI_Finalize'
>
/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapBarrier_mpi':
> gasnet_bootstrap_mpi.c:(.text+0x2a8): undefined reference to `MPI_Barrier'
> collect2: Fehler: ld gab 1 als Ende-Status zurück
> gmake: *** [/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp] Fehler 1
> error: compiling generated source
>
> Am I doing anything wrong? I've tried to make a hello world mpi program which
worked fine... I have no clue why it can't find the MPI references in gasnet.
>
> Thanks in advance and best regards,
> Marco
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Chapel-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/chapel-users
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
