Hello all,
I'm quite new to Chapel, so I apologize if I'm sending this email to the wrong
mailing lists. I've been having some issues setting up Chapel for multilocale
use on some of our machines here. I've been trying to follow the Multilocale
and Launcher pages in the documentation, however, I still haven't been
successful. I think one of the issues might be that, on some of our machines,
slurm requires an account variable, which may be keeping me from using native
slurm. Is there an environment variable that Chapel can accept that would
specify an account to be used by srun, similar to SLURM_PARTITION that's not
listed in the documentation?
Additionally, when I try to setup chapel to use slurm over ibv, I think I'm
getting linking issues for mpi calls. The setup I used to get these results is
as follows:
% $CHPL_HOME/util/printchplenv
machine info: Linux
CHPL_HOME: /ascldap/users/jcaswel/chapel-1.13.0 *
script location: /home/jcaswel/chapel-1.13.0/util
CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: gnu
CHPL_TARGET_ARCH: unknown
CHPL_LOCALE_MODEL: flat
CHPL_COMM: gasnet *
CHPL_COMM_SUBSTRATE: ibv *
CHPL_GASNET_SEGMENT: large
CHPL_TASKS: qthreads
CHPL_LAUNCHER: slurm-gasnetrun_ibv *
CHPL_TIMERS: generic
CHPL_MEM: jemalloc
CHPL_MAKE: gmake
CHPL_ATOMICS: intrinsics
CHPL_NETWORK_ATOMICS: none
CHPL_GMP: gmp
CHPL_HWLOC: hwloc
CHPL_REGEXP: re2
CHPL_WIDE_POINTERS: struct
CHPL_AUX_FILESYS: none
Additionally, I have set:
CHPL_COMM_SUBSTRATE: ibv
CHPL_LAUNCHER_WALLTIME=00:15:00
As well as the slurm partition. When I do this, I remake the runtime which
completes. However, when I try to compile a program, for instance, the
hello6-taskpar-dist.chpl found in $CHPL_HOME/examples/hello6-taskpar-dist.chpl,
I get the following output:
% chpl -o hello
/ascldap/users/jcaswel/chapel-1.13.0/examples/hello6-taskpar-dist.chpl
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapSNodeBroadcast_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x19b):
undefined reference to `ompi_mpi_byte'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1ba):
undefined reference to `MPI_Isend'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1dd):
undefined reference to `MPI_Waitall'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1fb):
undefined reference to `ompi_mpi_byte'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x212):
undefined reference to `MPI_Recv'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapBroadcast_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x25b):
undefined reference to `ompi_mpi_byte'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x26a):
undefined reference to `MPI_Bcast'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapAlltoall_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x291):
undefined reference to `ompi_mpi_byte'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2ab):
undefined reference to `MPI_Alltoall'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapExchange_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2e0):
undefined reference to `ompi_mpi_byte'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2fa):
undefined reference to `MPI_Allgather'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapBarrier_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x319):
undefined reference to `MPI_Barrier'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapAbort_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x32b):
undefined reference to `MPI_Abort'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapFini_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x357):
undefined reference to `MPI_Comm_free'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x365):
undefined reference to `MPI_Finalize'
$CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o):
In function `gasneti_bootstrapInit_mpi':
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x389):
undefined reference to `MPI_Initialized'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3a3):
undefined reference to `ompi_mpi_comm_world'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3ac):
undefined reference to `MPI_Comm_group'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3b1):
undefined reference to `ompi_mpi_comm_world'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3bf):
undefined reference to `MPI_Comm_create'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3c8):
undefined reference to `MPI_Group_free'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3d9):
undefined reference to `MPI_Comm_size'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x407):
undefined reference to `MPI_Comm_rank'
$CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x43a):
undefined reference to `MPI_Init'
collect2: ld returned 1 exit status
gmake: *** [/tmp/chpl-jcaswel-185203.deleteme/hello.tmp] Error 1
error: compiling generated source
I'm planning on updating to 1.13.1 today as well to see if that resolves the
issue.
Thank you very much in advance,
Jake
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
