Hi Marco - Unfortunately most of the Chapel developers are working hard to prepare the next release of Chapel, so we might be slow in getting back to you for now.
I haven't seen before the problem with the sample Cyclic program not working like that. Could you try it with CHPL_TASKS=fifo? Also, I had a little bit of trouble understanding whether or not the Cyclic example works with large or everything? (I understand it is not working intermittently, but which configuration is that? Does it always work with the other configuration?) It might help to make a little table to summarize. I've started with my understanding of your situation, can you complete it? IBV CHPL_GASNET_SEGMENT=? Cyclic test intermittently fails with SEGV IBV CHPL_GASNET_SEGMENT=everything Queue fails with firehose error IBV CHPL_TASKS=fifo CHPL_GASNET_SEGMENT=large Cyclic test ? IBV CHPL_TASKS=fifo CHPL_GASNET_SEGMENT=everything Cyclic test ? Someone will probably need to ask the GASNet team about the firehose error... Regarding the "Conflicting CPU frequencies detected" error, I've seen errors similar to that one when using a cluster with CPU frequency scaling. I ended up configuring SLURM so that it disabled frequency scaling while the job was running. I bet that the MXM conduit has some kind of problem when used on nodes with different CPU frequencies. Thanks, -michael On 3/3/15, 9:47 AM, "Marco Postigo" <[email protected]> wrote: >Hi again, > >it took me some time to get everything working now ... let me explain >what problems I've faced: > >At the beginning everything worked fine although I had to use >CHPL_GASNET_SEGMENT=everything instead of the default "large" with the >IBV conduit. Trying to use the large segmentation algorithm ended up with >segfaults when running with 1 or 2 locales ... > more than 2 locales worked but it had used much more memory than usual. >After that I've debugged my program but couldn't find anything wrong so >I've built Chapel and GASNet with lots of different settings (ibv >spawner, segmentation, tasking layer, lots of > different gasnet options, etc ...) but unfortunately without success. >Then I noticed that the program just runs fine with MPI (and also UDP) >regardless of how many locales are used. >Just to sum up what happened when using CHPL_GASNET_SEGMENT=everything: >The problems started after I've replaced a LinkedList with a Queue that >was based on an array ... elements from this queue have to be send to >other locales at runtime (on every locale exactly one such queue is >instantiated). If there were a lot of data to sent > around between the locales (-> workers stealing/retrieving tasks) then >an error were thrown: >"ibv_reg_mr failed in firehose_move_callback errno=14" followed by a >sigsegv 6 >So I started to try out the large segmentation algorithm again with >different options (as described in your post and some stuff I found on >the GASNet mailing list). But then again, my program didn't worked for nl ><= 2... >I was able to reproduce the error when it crashs with 1 locale with a >slightly modified example from the Chapel spec... the funny thing is that >it won't fail always. Most times I get a SIGSEGV 11 and sometimes there >seems to be a memory leak sometimes causing > that my job is canceled by the batch system after a few runs. > >Sample program: >use CyclicDist; >const Space = {0..numLocales-1}; >const D : domain(1) dmapped Cyclic(startIdx=Space.low) = Space; >var A : [D] int; >forall a in A do > a = a.locale.id; >writeln("A: ", A); > >Output (2 runs - 1 fail, 1 success): >------------------------------------------------- OUTPUT success >------------------------------------------------- >/home/mp27lupo/opt/chapel-1.10.0/third-party/gasnet/install/linux64-gnu-un >known/seg-large/nodbg/bin/gasnetrun_ibv -n 1 -E LESSKEY,MAN >PATH,INFODIR,EGO_CONFDIR,NNTPSERVER,GASNET_ROUTE_OUTPUT,LSB_EXEC_CLUSTER,L >SF_EAUTH_AUX_PASS,HOSTNAME,EGO_TOP,LSB_EFFECTIVE_RSRCREQ,XKE >YSYMDB,LSF_LIM_API_NTRIES,LSF_LOGDIR,CHPL_TASKS,LSB_BATCH_JID,SHELL,HOST,H >ISTSIZE,PROFILEREAD,TMPDIR,SSH_CLIENT,EGO_SERVERDIR,LSB_TRAP >SIGS,CHPL_GASNET_SEGMENT,LS_JOBPID,LSB_ERRORFILE,LSB_JOBRES_CALLBACK,LSB_M >AX_NUM_PROCESSORS,MORE,LSB_JOB_EXECUSER,LSB_JOBID,LSF_SERVER >DIR,LSB_JOBRES_PID,LSB_JOBNAME,SSH_TTY,BSUB_BLOCK_EXEC_HOST,LSF_PJL_TYPE,W >ORK_LOCAL,CHPL_COMM,LSF_LIBDIR,SSH_OPTIONS,USER,LD_LIBRARY_P >ATH,LS_COLORS,SBD_KRB5CCNAME_VAL,LSB_EEXEC_REAL_UID,XNLSPATH,EGO_LIBDIR,EN >V,HOSTTYPE,CPATH,CHPL_MAKE_COMM_SEGMENT,LSF_INVOKE_CMD,CHPL_ >COMM_SUBSTRATE,CHPL_HOST_PLATFORM,LS_EXEC_T,FROM_HEADER,LSF_EAUTH_SERVER,P >AGER,LS_SUBCWD,CSHEDIT,LSF_VERSION,LSB_DJOB_RU_INTERVAL,LSB_ >HOSTS,LSB_UNIXGROUP_INT,XDG_CONFIG_DIRS,LSB_DJOB_HB_INTERVAL,LSB_JOBFILENA >ME,MINICOM,GASNET_BACKTRACE,LSB_JOBINDEX,PATH,MAIL,CPU,SSH_S >ERVERS,CHPL_MAKE_GASNET_SEGMENT,LSB_EXIT_PRE_ABORT,SSH_SENDS_LOCALE,CHPL_C >OMM_DEBUG,LSB_JOBEXIT_STAT,PWD,INPUTRC,JAVA_HOME,_LMFILES_,L >ANG,WORK_SCRATCH,LSB_CHKFILENAME,LSF_EAUTH_CLIENT,MODULEPATH,PYTHONSTARTUP >,LSB_DJOB_HOSTFILE,LOADEDMODULES,SSH_CMD,LSB_JOB_STARTER,X10 >_NTHREADS,LSB_DJOB_NUMPROC,LSB_EXEC_HOSTTYPE,LSF_BINDIR,CHPL_HOME,HOME,SHL >VL,QT_SYSTEM_DIR,OSTYPE,LESS_ADVANCED_PREPROCESSOR,LSB_ACCT_ >FILE,XCURSOR_THEME,BINARY_TYPE_HPC,LS_OPTIONS,LSB_SUB_HOST,EGO_LOCAL_CONFD >IR,__LSF_JOB_TMPDIR__,LSB_SUB_USER,LSFUSER,LSB_QUEUE,LSB_MCP >U_HOSTS,LSB_OUTPUTFILE,LOGNAME,MACHTYPE,LESS,G_FILENAME_ENCODING,TMP,CVS_R >SH,OMPI_MCA_hwloc_base_binding_policy,SSH_CONNECTION,XDG_DAT >A_DIRS,LSB_ECHKPNT_RSH_CMD,MODULESHOME,LSF_CGROUP_TOPDIR_KEY,LESSOPEN,X10_ >HOME,GASNET_IBV_SPAWNER,INFOPATH,LSB_EEXEC_REAL_GID,GASNET_S >PAWNFN,LSF_ENVDIR,LESSCLOSE,LSF_EGO_ENVDIR,EGO_BINDIR,G_BROKEN_FILENAMES,C >OLORTERM,BASH_FUNC_module(),BASH_FUNC_mc(),_ >/home/mp27lupo/prog/bin/example_real -nl 1 -v >QTHREADS: Using 16 Shepherds >QTHREADS: Using 1 Workers per Shepherd >QTHREADS: Guard Pages Enabled >QTHREADS: Using 8388608 byte stack size. >executing on node 0 of 1 node(s): hpa0021 >hello 1 > >------------------------------------------------- OUTPUT fail >------------------------------------------------- >/home/mp27lupo/opt/chapel-1.10.0-debug/third-party/gasnet/install/linux64- >gnu-unknown/seg-large/debug/bin/gasnetrun_ibv -n 1 -E LESSK >EY,MANPATH,INFODIR,EGO_CONFDIR,NNTPSERVER,LSB_EXEC_CLUSTER,LSF_EAUTH_AUX_P >ASS,GASNET_ROUTE_OUTPUT,HOSTNAME,EGO_TOP,LSB_EFFECTIVE_RSRCR >EQ,XKEYSYMDB,LSF_LIM_API_NTRIES,LSF_LOGDIR,LSB_BATCH_JID,SHELL,HOST,CHPL_T >ASKS,HISTSIZE,PROFILEREAD,TMPDIR,SSH_CLIENT,EGO_SERVERDIR,LS >B_TRAPSIGS,LS_JOBPID,LSB_ERRORFILE,CHPL_GASNET_SEGMENT,LSB_JOBRES_CALLBACK >,LSB_MAX_NUM_PROCESSORS,MORE,LSB_JOB_EXECUSER,LSB_JOBID,LSF_ >SERVERDIR,LSB_JOBRES_PID,LSB_JOBNAME,SSH_TTY,BSUB_BLOCK_EXEC_HOST,LSF_PJL_ >TYPE,WORK_LOCAL,LSF_LIBDIR,CHPL_COMM,USER,SSH_OPTIONS,LD_LIB >RARY_PATH,LS_COLORS,SBD_KRB5CCNAME_VAL,LSB_EEXEC_REAL_UID,XNLSPATH,EGO_LIB >DIR,ENV,HOSTTYPE,CPATH,LSF_INVOKE_CMD,CHPL_MAKE_COMM_SEGMENT >,LS_EXEC_T,FROM_HEADER,CHPL_HOST_PLATFORM,CHPL_COMM_SUBSTRATE,LSF_EAUTH_SE >RVER,PAGER,LS_SUBCWD,CSHEDIT,LSF_VERSION,LSB_DJOB_RU_INTERVA >L,LSB_HOSTS,LSB_UNIXGROUP_INT,XDG_CONFIG_DIRS,LSB_DJOB_HB_INTERVAL,LSB_JOB >FILENAME,MINICOM,GASNET_BACKTRACE,LSB_JOBINDEX,PATH,MAIL,CPU >,SSH_SERVERS,LSB_EXIT_PRE_ABORT,SSH_SENDS_LOCALE,CHPL_MAKE_GASNET_SEGMENT, >LSB_JOBEXIT_STAT,CHPL_COMM_DEBUG,PWD,INPUTRC,JAVA_HOME,_LMFI >LES_,LANG,WORK_SCRATCH,LSB_CHKFILENAME,LSF_EAUTH_CLIENT,MODULEPATH,PYTHONS >TARTUP,LSB_DJOB_HOSTFILE,LOADEDMODULES,LSB_JOB_STARTER,X10_N >THREADS,SSH_CMD,LSB_DJOB_NUMPROC,LSB_EXEC_HOSTTYPE,LSF_BINDIR,HOME,SHLVL,Q >T_SYSTEM_DIR,CHPL_HOME,OSTYPE,LESS_ADVANCED_PREPROCESSOR,LSB >_ACCT_FILE,XCURSOR_THEME,BINARY_TYPE_HPC,LS_OPTIONS,LSB_SUB_HOST,EGO_LOCAL >_CONFDIR,__LSF_JOB_TMPDIR__,LSB_SUB_USER,LSFUSER,LSB_QUEUE,L >SB_MCPU_HOSTS,LSB_OUTPUTFILE,LOGNAME,MACHTYPE,LESS,G_FILENAME_ENCODING,TMP >,CVS_RSH,OMPI_MCA_hwloc_base_binding_policy,SSH_CONNECTION,X >DG_DATA_DIRS,LSB_ECHKPNT_RSH_CMD,MODULESHOME,LSF_CGROUP_TOPDIR_KEY,LESSOPE >N,X10_HOME,INFOPATH,GASNET_IBV_SPAWNER,LSB_EEXEC_REAL_GID,LS >F_ENVDIR,GASNET_SPAWNFN,LESSCLOSE,LSF_EGO_ENVDIR,EGO_BINDIR,G_BROKEN_FILEN >AMES,COLORTERM,BASH_FUNC_module(),BASH_FUNC_mc(),_ /home/mp2 >7lupo/prog/bin/example_real -nl 1 -v >QTHREADS: Using 16 Shepherds >QTHREADS: Using 1 Workers per Shepherd >QTHREADS: Guard Pages Enabled >QTHREADS: Using 8388608 byte stack size. >executing on node 0 of 1 node(s): hpa0010 >0: enter barrier for 'barrier before main' >0: enter barrier for 'barrier before broadcasting globals' >0: enter barrier for 'barrier after broadcasting globals' >0: enter barrier for 'pre-user-code hook begin' >0: enter barrier for 'pre-user-code hook end' >*** Caught a fatal signal: SIGSEGV(11) on node 0/1 >*** Caught a fatal signal: SIGSEGV(11) on node 0/1 >Cleaning up orphaned processes... >-------------------------------------------------------------------------- >------------------------------ > >The memory usage is also very interesting: >------------------------------------------------------------ >Exited with exit code 11. >Resource usage summary: >CPU time : 8.12 sec. >Max Memory : 1284 MB >Average Memory : 1284.00 MB >Total Requested Memory : - >Delta Memory : - >(Delta: the difference between total requested memory and actual max >usage.) >Max Swap : 1388 MB >Max Processes : 6 >Max Threads : 7 >The output (if any) follows: >------------------------------------------------------------ > >For comparison running the same program with the same configuration >except that CHPL_COMM_SEGMENT=everything: >------------------------------------------------------------ >Successfully completed. >Resource usage summary: >CPU time : 0.97 sec. >Max Memory : 5 MB >Average Memory : 5.00 MB >Total Requested Memory : - >Delta Memory : - >(Delta: the difference between total requested memory and actual max >usage.) >Max Swap : 166 MB >Max Processes : 6 >Max Threads : 7 >The output (if any) follows: >------------------------------------------------------------ > > >Chapel environment variables that were set: > >export CHPL_HOME=~/opt/chapel >export CHPL_HOST_PLATFORM=`$CHPL_HOME/util/chplenv/chpl_platform.py` >export GASNET_ROUTE_OUTPUT=0 >export GASNET_SPAWNFN=S >export CHPL_COMM=gasnet >export CHPL_COMM_SUBSTRATE=ibv >export CHPL_GASNET_SEGMENT=large >export CHPL_MAKE_GASNET_SEGMENT=large >#export CHPL_MAKE_COMM_SEGMENT=large >export MANPATH=$MANPATH:$CHPL_HOME/man >export PATH=$PATH:$CHPL_HOME/bin/$CHPL_HOST_PLATFORM/ >export SSH_CMD=ssh >export SSH_OPTIONS= >export CHPL_TASKS=qthreads >--------- default variables that were printed by >$CHPL_HOME/util/printchplenv ------ >CHPL_MEM=dlmalloc >CHPL_GMP=gmp >CHPL_HWLOC=hwloc >CHPL_REGEXP=re2 > >Unfortunately I don't have the exact logs any more otherwise I would have >copied the whole output of the printchplenv script... if you need them >just let me know and I will run with IBV again. I had also GASNet tracing >enabled so I could send this logs > as well if you're interested. >Furthermore I've never found out why my program is crashing for 2 locales >... I wasn't able to create a simple program that reproduces this error. > >So finally I've ended up with replacing the GASNet 1.22.0 version with >the newest 1.24.0 release... sadly the ibv error still occured but >luckily now I was able to build the MXM conduit. Now everything seems to >work fine but I'm getting some warnings where > I don't know how to fix them: >[1425347486.475355] sys.c:719 MXM WARN Conflicting CPU >frequencies detected, using: 2601.00 >... >[1425347486.909148] [hpa0273:3056 :0] mxm.c:185 MXM WARN The >'ulimit -s' on the system is set to 'unlimited'. This may have > negative performance implications. Please set the heap size to the >default value (10240) > >Both warnings are printed for each locale that I'm starting. I couldn't >find anything yet how to get rid of these warnings / set at least the >heap size myself so I would appreciate any help anyone could offer :) The >output of printchplenv with mxm conduit: >machine info: Linux hpa0273 3.0.101-0.40-default #1 SMP Thu Sep 18 >13:09:38 UTC 2014 (44b8c95) x86_64 >CHPL_HOME: /home/mp27lupo/opt/chapel >script location: /home/mp27lupo/opt/chapel-1.10.0/util >CHPL_HOST_PLATFORM: linux64 >CHPL_HOST_COMPILER: gnu >CHPL_TARGET_PLATFORM: linux64 >CHPL_TARGET_COMPILER: gnu >CHPL_TARGET_ARCH: unknown >CHPL_LOCALE_MODEL: flat >CHPL_COMM: gasnet > CHPL_COMM_SUBSTRATE: mxm > CHPL_GASNET_SEGMENT: everything >CHPL_TASKS: qthreads >CHPL_LAUNCHER: gasnetrun_ibv >CHPL_TIMERS: generic >CHPL_MEM: cstdlib >CHPL_MAKE: gmake >CHPL_ATOMICS: intrinsics > CHPL_NETWORK_ATOMICS: none >CHPL_GMP: gmp >CHPL_HWLOC: hwloc >CHPL_REGEXP: re2 >CHPL_WIDE_POINTERS: struct >CHPL_LLVM: none >CHPL_AUX_FILESYS: none > >System info: >Operating System: SUSE Linux Enterprise Server 11 (x86_64), VERSION = 11, >PATCHLEVEL = 3 >Other (maybe) useful information: >gcc (GCC) 4.8.3 >openmpi/gcc/1.8.2 >python/3.4.2 >chapel/1.10.0 >Used batch system: LSF > > >Best regards, >Marco > > >Gesendet: Mittwoch, 18. Februar 2015 um 17:57 Uhr >Von: "Michael Ferguson" <[email protected]> >An: "Marco Postigo" <[email protected]> >Cc: "[email protected]" ><[email protected]> >Betreff: Re: Problem with compiling chapel programs if ibv conduit is used >Hi Marco - > >I understand that you got the IBV conduit working, but I wanted to leave >a pointer to my earlier experience with Chapel on IBV and MXM conduits: > >http://permalink.gmane.org/gmane.comp.lang.chapel.devel/230 > >I was able to get both IBV and MXM conduits to work with >CHPL_GASNET_SEGMENT=fast ... but had to --disable-pshm when configuring >GASNet - although in a current Chapel, GASNet should build with >--disable-pshm for these conduits by default. > >Cheers, > >-michael > > >On Tue, Feb 10, 2015 at 7:19 PM, Marco Postigo <[email protected]> >wrote: > >Hello again :) > >I've just installed Chapel on a computing cluster of a partner university >to make some benchmarks of my GLB implementation. Unfortunately I wasn't >able to use ibv or mxm conduit (only udp works). > >When I've tried to simply build Chapel (with make / gmake in Chapels home >directory) it already failed while trying to building Gasnet because of >the mxm conduit... so I've checked the Gasnet doc and edited the Makefile >provided from Chapel in third-party/gasnet > to disable mxm support (and prevent it from being built). I don't have >the error log any more but if you're interested I will send it as well. > >The main problem is when I'm trying to compile my program having ibv >activated... the error log will follow at the end of this mail. I have >the following environment variables set: > >export CHPL_HOME=~/opt/chapel >export CHPL_HOST_PLATFORM=`$CHPL_HOME/util/chplenv/chpl_platform.py` >export GASNET_ROUTE_OUTPUT=0 >export GASNET_SPAWNFN=S >export CHPL_COMM=gasnet >export CHPL_COMM_SUBSTRATE=ibv >export CHPL_GASNET_SEGMENT=everything >export MANPATH=$MANPATH:$CHPL_HOME/man >export PATH=$PATH:$CHPL_HOME/bin/$CHPL_HOST_PLATFORM/ >export SSH_CMD=ssh >export SSH_OPTIONS=-x >export CHPL_TASKS=fifo > >Operating System: SUSE Linux Enterprise Server 11 (x86_64), VERSION = 11, >PATCHLEVEL = 3 > >Other (maybe) useful information: >gcc (GCC) 4.8.3 >openmpi/gcc/1.8.2 >python/3.4.2 >chapel/1.10.0 > >Compilation error occuring when trying to compile any program (example is >here hello2-module.chpl)... I've given the --print-commands parameter as >well because it may help. > >~/opt/chapel> chpl --print-commands -o hello examples/hello2-module.chpl >/home/mp27lupo/opt/chapel/util/chplenv/check_huge_pages.py >/shared/apps/gcc/4.8.3/bin/gcc -DGASNET_PAR -D_REENTRANT -D_GNU_SOURCE > >-I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown >/seg-large/nodbg/include >-I/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown >/seg-large/nodbg/include/ibv-conduit > -I/opt/mellanox/fca/include -std=c99 -Wno-strict-overflow >-I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include >-I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none -std=c99 >-DCHPL_TASKS_MODEL_H=\"tasks-fifo.h\" >-DCHPL_THREADS_MODEL_H=\"threads-pthreads.h\" > -DCHPL_WIDE_POINTER_STRUCT -DUSE_LOCKS -DONLY_MSPACES -DMSPACES >-DCHPL_HAS_GMP -c -o /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o -I. >-I/home/mp27lupo/opt/chapel/runtime//include/localeModels/flat >-I/home/mp27lupo/opt/chapel/runtime//include/localeModels > -I/home/mp27lupo/opt/chapel/runtime//include/comm/gasnet >-I/home/mp27lupo/opt/chapel/runtime//include/comm >-I/home/mp27lupo/opt/chapel/runtime//include/tasks/fifo >-I/home/mp27lupo/opt/chapel/runtime//include/threads/pthreads >-I/home/mp27lupo/opt/chapel/runtime//include/comp-gnu > -I/home/mp27lupo/opt/chapel/runtime//include/linux64 >-I/home/mp27lupo/opt/chapel/runtime//include >-I/home/mp27lupo/opt/chapel/runtime//include/qio >-I/home/mp27lupo/opt/chapel/runtime//include/atomics/intrinsics >-I/home/mp27lupo/opt/chapel/runtime//include/mem/dlmalloc > -I/home/mp27lupo/opt/chapel/third-party/utf8-decoder >-I/home/mp27lupo/opt/chapel/third-party/dlmalloc/include >-I/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/in >clude -I/home/mp27lupo/opt/chapel/runtime//include/hwloc/none >/tmp/chpl-mp27lupo-27468.deleteme/_main.c >g++ >-L/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/li >b -Wl,-rpath >-Wl,/home/mp27lupo/opt/chapel/third-party/gmp/install/linux64-gnu-unknown/ >lib >-L/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/li >b -Wl,-rpath >-Wl,/home/mp27lupo/opt/chapel/third-party/re2/install/linux64-gnu-unknown/ >lib > -o /tmp/chpl-mp27lupo-27468.deleteme/hello.tmp >-L/home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gas >net.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none. >gmp.hwloc-none.re2.wide-struct.fs-none >/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp.o > >/home/mp27lupo/opt/chapel/lib/linux64.gnu.arch-unknown.loc-flat.comm-gasne >t.ibv.large.tasks-fifo.tmr-generic.mem-dlmalloc.atomics-intrinsics.none.gm >p.hwloc-none.re2.wide-struct.fs-none/main.o -lchpl -lm -lpthread >-L/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown >/seg-large/nodbg/lib > -L/usr/lib64 -L/opt/mellanox/fca/lib -lgasnet-ibv-par -libverbs >-lfca -lpthread >-L/shared/apps/gcc/4.8.3/lib/gcc/x86_64-unknown-linux-gnu/4.8.3 -lgcc -lm > >-L/home/mp27lupo/opt/chapel/third-party/dlmalloc/install/linux64-gnu-unkno >wn -ldlmalloc -lgmp -lre2 > -lpthread >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapExchange_mpi': >gasnet_bootstrap_mpi.c:(.text+0xcd): undefined reference to >`ompi_mpi_byte' >gasnet_bootstrap_mpi.c:(.text+0xf9): undefined reference to >`MPI_Allgather' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapBroadcast_mpi': >gasnet_bootstrap_mpi.c:(.text+0x132): undefined reference to >`ompi_mpi_byte' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapInit_mpi': >gasnet_bootstrap_mpi.c:(.text+0x17c): undefined reference to >`MPI_Initialized' >gasnet_bootstrap_mpi.c:(.text+0x1a9): undefined reference to `MPI_Init' >gasnet_bootstrap_mpi.c:(.text+0x1b5): undefined reference to >`ompi_mpi_comm_world' >gasnet_bootstrap_mpi.c:(.text+0x1ba): undefined reference to >`MPI_Comm_group' >gasnet_bootstrap_mpi.c:(.text+0x1c8): undefined reference to >`ompi_mpi_comm_world' >gasnet_bootstrap_mpi.c:(.text+0x1cd): undefined reference to >`MPI_Comm_create' >gasnet_bootstrap_mpi.c:(.text+0x1d5): undefined reference to >`MPI_Group_free' >gasnet_bootstrap_mpi.c:(.text+0x1e6): undefined reference to >`MPI_Comm_size' >gasnet_bootstrap_mpi.c:(.text+0x20a): undefined reference to >`MPI_Comm_rank' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapFini_mpi': >gasnet_bootstrap_mpi.c:(.text+0x24a): undefined reference to >`MPI_Comm_free' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapAbort_mpi': >gasnet_bootstrap_mpi.c:(.text+0x27e): undefined reference to `MPI_Abort' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapAlltoall_mpi': >gasnet_bootstrap_mpi.c:(.text+0x2c9): undefined reference to >`ompi_mpi_byte' >gasnet_bootstrap_mpi.c:(.text+0x2db): undefined reference to >`MPI_Alltoall' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapBroadcast_mpi': >gasnet_bootstrap_mpi.c:(.text+0x13a): undefined reference to `MPI_Bcast' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapFini_mpi': >gasnet_bootstrap_mpi.c:(.text+0x265): undefined reference to >`MPI_Finalize' >/home/mp27lupo/opt/chapel/third-party/gasnet/install/linux64-gnu-unknown/s >eg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): In >function `gasneti_bootstrapBarrier_mpi': >gasnet_bootstrap_mpi.c:(.text+0x2a8): undefined reference to `MPI_Barrier' >collect2: Fehler: ld gab 1 als Ende-Status zurück >gmake: *** [/tmp/chpl-mp27lupo-27468.deleteme/hello.tmp] Fehler 1 >error: compiling generated source > >Am I doing anything wrong? I've tried to make a hello world mpi program >which worked fine... I have no clue why it can't find the MPI references >in gasnet. > >Thanks in advance and best regards, >Marco > >-------------------------------------------------------------------------- >---- >Dive into the World of Parallel Programming. The Go Parallel Website, >sponsored by Intel and developed in partnership with Slashdot Media, is >your >hub for all things parallel software development, from weekly thought >leadership blogs to news, videos, case studies, tutorials and more. Take a >look and join the conversation now. >http://goparallel.sourceforge.net/ <http://goparallel.sourceforge.net/> >_______________________________________________ >Chapel-users mailing list >[email protected] >https://lists.sourceforge.net/lists/listinfo/chapel-users > > > > > > > > ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Chapel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-users
