For the archives: this thread continued on the chapel-users mailing list. Vassily
On 07/27/16 07:14, Caswell, Jacob wrote: > Hello all, > > I'm quite new to Chapel, so I apologize if I'm sending this email to the > wrong mailing lists. I've been having some issues setting up Chapel for > multilocale use on some of our machines here. I've been trying to follow the > Multilocale and Launcher pages in the documentation, however, I still haven't > been successful. I think one of the issues might be that, on some of our > machines, slurm requires an account variable, which may be keeping me from > using native slurm. Is there an environment variable that Chapel can accept > that would specify an account to be used by srun, similar to SLURM_PARTITION > that's not listed in the documentation? > > Additionally, when I try to setup chapel to use slurm over ibv, I think I'm > getting linking issues for mpi calls. The setup I used to get these results > is as follows: > > % $CHPL_HOME/util/printchplenv > machine info: Linux > CHPL_HOME: /ascldap/users/jcaswel/chapel-1.13.0 * > script location: /home/jcaswel/chapel-1.13.0/util > CHPL_TARGET_PLATFORM: linux64 > CHPL_TARGET_COMPILER: gnu > CHPL_TARGET_ARCH: unknown > CHPL_LOCALE_MODEL: flat > CHPL_COMM: gasnet * > CHPL_COMM_SUBSTRATE: ibv * > CHPL_GASNET_SEGMENT: large > CHPL_TASKS: qthreads > CHPL_LAUNCHER: slurm-gasnetrun_ibv * > CHPL_TIMERS: generic > CHPL_MEM: jemalloc > CHPL_MAKE: gmake > CHPL_ATOMICS: intrinsics > CHPL_NETWORK_ATOMICS: none > CHPL_GMP: gmp > CHPL_HWLOC: hwloc > CHPL_REGEXP: re2 > CHPL_WIDE_POINTERS: struct > CHPL_AUX_FILESYS: none > > Additionally, I have set: > CHPL_COMM_SUBSTRATE: ibv > CHPL_LAUNCHER_WALLTIME=00:15:00 > > As well as the slurm partition. When I do this, I remake the runtime which > completes. However, when I try to compile a program, for instance, the > hello6-taskpar-dist.chpl found in > $CHPL_HOME/examples/hello6-taskpar-dist.chpl, I get the following output: > % chpl -o hello > /ascldap/users/jcaswel/chapel-1.13.0/examples/hello6-taskpar-dist.chpl > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapSNodeBroadcast_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x19b): > undefined reference to `ompi_mpi_byte' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1ba): > undefined reference to `MPI_Isend' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1dd): > undefined reference to `MPI_Waitall' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x1fb): > undefined reference to `ompi_mpi_byte' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x212): > undefined reference to `MPI_Recv' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapBroadcast_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x25b): > undefined reference to `ompi_mpi_byte' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x26a): > undefined reference to `MPI_Bcast' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapAlltoall_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x291): > undefined reference to `ompi_mpi_byte' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2ab): > undefined reference to `MPI_Alltoall' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapExchange_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2e0): > undefined reference to `ompi_mpi_byte' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x2fa): > undefined reference to `MPI_Allgather' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapBarrier_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x319): > undefined reference to `MPI_Barrier' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapAbort_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x32b): > undefined reference to `MPI_Abort' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapFini_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x357): > undefined reference to `MPI_Comm_free' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x365): > undefined reference to `MPI_Finalize' > $CHPL_HOME/third-party/gasnet/install/linux64-gnu-unknown/seg-large/nodbg/lib/libgasnet-ibv-par.a(gasnet_bootstrap_mpi-PAR.o): > In function `gasneti_bootstrapInit_mpi': > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x389): > undefined reference to `MPI_Initialized' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3a3): > undefined reference to `ompi_mpi_comm_world' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3ac): > undefined reference to `MPI_Comm_group' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3b1): > undefined reference to `ompi_mpi_comm_world' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3bf): > undefined reference to `MPI_Comm_create' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3c8): > undefined reference to `MPI_Group_free' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x3d9): > undefined reference to `MPI_Comm_size' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x407): > undefined reference to `MPI_Comm_rank' > $CHPL_HOME/third-party/gasnet/GASNet-1.26.0/other/mpi-spawner/gasnet_bootstrap_mpi.c:(.text+0x43a): > undefined reference to `MPI_Init' > collect2: ld returned 1 exit status > gmake: *** [/tmp/chpl-jcaswel-185203.deleteme/hello.tmp] Error 1 > error: compiling generated source > > I'm planning on updating to 1.13.1 today as well to see if that resolves the > issue. > > Thank you very much in advance, > Jake > > > > ------------------------------------------------------------------------------ > What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic > patterns at an interface-level. Reveals which users, apps, and protocols are > consuming the most bandwidth. Provides multi-vendor support for NetFlow, > J-Flow, sFlow and other flows. Make informed decisions using capacity planning > reports.http://sdm.link/zohodev2dev > > > > _______________________________________________ > Chapel-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/chapel-developers > ------------------------------------------------------------------------------ _______________________________________________ Chapel-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-developers
