Thanks Brad. With the Makefile.mpi-gnu, I'm able to compile the hello6-taskpar-dist.chpl without link errors.
Since I didn't set CHPL_LAUNCHER, I just got one binary. I ran the following command and it looks like it runs well.. right?? [VM-45 ~/chapel_4/chapel-1.13.1]$third-party/gasnet/install/linux64-mpi-gnu-unknown/seg-everything/nodbg/bin/gasnetrun_ofi -np 2 ./hello6-taskpar-dist -nl 2 Hello, world! (from locale 0 of 2 named simpl-l4tm-vm45) Hello, world! (from locale 1 of 2 named simpl-l4tm-vm45) Best, Jintack -----Original Message----- From: Brad Chamberlain [mailto:[email protected]] Sent: Thursday, July 21, 2016 5:07 PM To: Lim, Jintack <[email protected]> Cc: Paul Hargrove <[email protected]>; [email protected]; Michael Ferguson <[email protected]> Subject: RE: [upc-devel] GASNet ofi-conduit in the context of Chapel Hi Jintack -- Michael's suggestion to use 'mpi-gnu' as the target compiler unfortunately only works from the master branch of our GitHub repository, as that option was added after the 1.13 releases. That said, I wouldn't be surprised if you could grab the following one file and drop it in $CHPL_HOME/make/compiler/ to enable it for 1.13: ub.com/chapel-lang/chapel/blob/master/make/compiler/Makefile.mpi-gnu Meanwhile, I'm going to see if I can determine why our Makefiles aren't picking up GASNET_LD as their link step as I expected. -Brad On Thu, 21 Jul 2016, Lim, Jintack wrote: > Hi Brad, Michael and Paul, > > I really appreciate your helpful comments. > > This is the information with make –f <dir>/Makefile print-commands. > CC is /usr/bin/gcc > LD is g++ > GASNET_LD is /usr/bin/mpicc > GASNET_INC_MAKEFILE is > /home/limjint/chapel_4/chapel-1.13.1/third-party/gasnet/install/linux6 > 4-gnu-unknown/seg-everything/nodbg/include/ofi-conduit/ofi-par.mak > > This is the <dir>/Makefile. > http://pastebin.ubuntu.com/20377401/ > I can see that CHPL_TARGET_COMPILER=gnu. > > I also set CHPL_TARGET_COMPILER=mpi-gnu and compiled Chapel again. > I’m not sure MPI is configured to use gcc, but what I got while compiling > Chapel is something like this. > Warning: Unknown compiler: "mpi-gnu" > Warning: No valid option found: arch="native" compiler="mpi-gnu" > version="CompVersion(major=4, minor=9, revision=2, build=0)" > http://pastebin.ubuntu.com/20380446/ > > I also got some trivial compile errors and permission errors before above > error, and just worked around. > http://pastebin.ubuntu.com/20377624/ > http://pastebin.ubuntu.com/20378961/ > > Best, > Jintack > > From: Paul Hargrove [mailto:[email protected]] > Sent: Thursday, July 21, 2016 3:59 PM > To: Michael Ferguson <[email protected]> > Cc: Lim, Jintack <[email protected]>; > [email protected] > Subject: Re: [upc-devel] GASNet ofi-conduit in the context of Chapel > > Brad and Michael, > > It is not clear from your answers if you fully understood Jintack's situation. > However, *I* may be mistaken if the details you skipped over simply aren't > relevant for reasons unknown to me. > Jintack is *not* using mpi-conduit nor is GASNET_CC an MPI compiler. > The conduit in question is ofi-conduit, which is using mpirun as its default > spawner. > > At least ibv, mxm, ofi and psm conduits support launching via ssh or by > mpirun. > By default the configure script enables both of these assuming an mpicc is > found. > If mpi-based spawning is enabled at configure time then *one* .c file is > compiled with mpicc. > All the rest of the GASNet conduit code is compiled with a non-MPI C compiler. > However, due to the one MPI object file the .mak files for those conduits > will set their GASNET_LD=mpicc. > > I am fairly confident that Chapel must already deal with $(GASNET_LD) as > distinct from $(GASNET_CC) since udp-conduit needs a C++ linker. > > I suspect one or more of your suggestions may lead to a work-around > (CHPL_TARGET_COMPILER=mpi-gnu in particular). > However, I am suspicious of the handling of the Makefile fragments. > So, I think Brad's make command with all the print-SOMETHINGs is important to > root out the problem. > > Is it possible that the problem is as simple as Chapel needing "ofi" added to > some list of conduits somewhere? > > -Paul > > > On Thu, Jul 21, 2016 at 3:28 PM, Michael Ferguson > <[email protected]<mailto:[email protected]>> wrote: > Hi - > > You could also try using > > export CHPL_TARGET_COMPILER=mpi-gnu > > Assuming you gave MPI configured to use GCC. If that's the case, if > CHPL_TARGET_COMPILER=mpi-gnu when you build Chapel and use it, you > should see mpicc calls as the linker. > > Brad had lots of other good ideas for how to troubleshoot this problem. > > Cheers, > > -michael > > On 7/20/16, 6:10 PM, "Lim, Jintack" > <[email protected]<mailto:[email protected]>> wrote: > >> Hi GASNet and Chapel community, >> >> I'm trying to run Chapel with GASNet ofi-conduit. >> I got some link errors last time while I was running "make check" >> after compiled Chapel. >> I got feedback from Paul Hargrove and Greg Titus but I still can't >> resolve it. >> http://paste.ubuntu.com/19312464/ >> >> I set CHPL_LAUNCHER=gasnetrun_mpi as Greg suggested. >> (Paul suggested to try to configure using without-mpi-cc GASNet >> option, but it looks like we need mpi based on the chapel lanucher >> setting above?) >> >> Paul said it looks like Chapel is not using mpicc as its linker. >> >> I checked that gasnet_bootstrap_mpi.c was compiled with mpicc while >> building Chapel. >> When building libgasnet-ofi-par.a, gcc is used, but this is the same >> when compiling GASNet only. >> >> So I wonder if I need to configure Chapel with some flags or should I >> put some link option somewhere? >> >> Best, >> Jintack >> >> >> From: Paul Hargrove >> [mailto:[email protected]<mailto:[email protected]>] >> >> Sent: Wednesday, July 13, 2016 6:09 PM >> To: Lim, Jintack <[email protected]<mailto:[email protected]>> >> Cc: [email protected]<mailto:[email protected]>; Jeff Hammond >> <[email protected]<mailto:[email protected]>>; >> [email protected]<mailto:[email protected] >> rge.net> >> Subject: Re: [upc-devel] GASNet ofi-conduit in the context of Chapel >> >> Jintack, >> >> The environment variable GASNET_OFI_SPAWNER is used at >> application-launch to select MPI vs SSH if both were available at >> build time. So, it has no connection to the link failure. Your >> environment variable settings seem to me to be correct. >> >> >> >> The linkage errors seem to indicate that Chapel is not using mpicc as >> its linker (ignoring the $(GASNET_LD) setting). >> >> That is something you will need to take up with the Chapel team if >> you require mpi-based spawning. >> >> The simplest approach to getting past the link problem would be to >> configure using --without-mpi-cc. >> >> Then SSH-based spawning will be your only option and setting of >> GASNET_OFI_SPAWNER will not be needed. >> >> >> >> If you have a PMI-based spawner (including hydra from mpich or srun >> from SLURM), then you may also be able to launch GASNet applications >> that way (not using the gasnetrun_ofi script). You might need some >> additional configure options to help GASNet locate libpmi - let me >> know if you need help pursing that option. >> >> >> >> -Paul >> >> >> >> >> >> On Wed, Jul 13, 2016 at 5:42 PM, Lim, Jintack >> <[email protected]<mailto:[email protected]>> >> wrote: >> >> Hi all, >> >> When I was compiling a Chapel hello world application on top of >> GASNet ofi-conduit, I got some GASNet link errors related to MPI. >> http://paste.ubuntu.com/19312464/ >> I got this either I set GASNET_OFI_SPAWNER to mpi or ssh. >> GASNet version that is included in the Chapel source tree is 1.26.0. >> >> There are instructions to set GASNET environment variables for >> GASNet-udp and ibv conduit in the Chapel documentation >> (http://chapel.cray.com/docs/1.13/usingchapel/multilocale.html), but >> I couldn’t find one for ofi-conduit. >> Could somebody shed some light on how to fix this problem? (such as >> how to set CHPL_LAUNCHER or any other GASNET environment variables to >> fix the link error?) >> >> What I did is >> - CHPL_COMM=gasnet >> - CHPL_COMM_SUBSTRATE=ofi >> - GASNET_OFI_SPAWNER= mpi or ssh >> -GASNET_SSH_SERVERS=”list of my servers” >> >> Please let me know if you need further information. >> >> Best, >> Jintack >> >> >> From: Jeff Hammond >> [mailto:[email protected]<mailto:[email protected]>] >> >> Sent: Wednesday, July 13, 2016 5:19 PM >> To: Lim, Jintack <[email protected]<mailto:[email protected]>> >> Cc: >> [email protected]<mailto:[email protected] >> rge.net> >> Subject: Re: Chapel on GASNet ofi-conduit >> >> I had trouble with Gasnet OFI in the context of UPC. You might want >> post this question to the Berkeley UPC list so the Gasnet team sees >> it. They will know the launcher issues better than the Chapel team >> (no disrespect intended). >> >> >> Jeff >> >> On Wednesday, July 13, 2016, Lim, Jintack >> <[email protected]<mailto:[email protected]>> wrote: >> >> Hi, >> >> I’m trying to install Chapel using GASNet ofi-conduit. I was able to >> compile Chapel itself, however I couldn’t compile >> hello6-taskpar-dist.chpl because of link errors related to MPI. >> Here’s the error message. >> http://paste.ubuntu.com/19312464/ >> >> There are only instructions for GASNet-udp and ibv conduit in the >> documentation >> (http://chapel.cray.com/docs/1.13/usingchapel/multilocale.html), >> so I wonder how to set environment variables (such as launcher) for >> ofi conduit. >> >> Any help would be greatly appreciated. >> >> Best, >> Jintack >> >> >> >> >> >> >> >> >> -- >> Jeff Hammond >> [email protected]<mailto:[email protected]> >> http://jeffhammond.github.io/ >> >> >> >> _______________________________________________ >> upc-devel mailing list >> [email protected]<mailto:[email protected]> >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/upc-devel >> >> >> >> >> >> >> >> >> -- >> Paul H. Hargrove >> [email protected]<mailto:[email protected]> >> >> Computer Languages & Systems Software (CLaSS) Group >> >> Computer Science Department Tel: >> +1-510-495-2352<tel:%2B1-510-495-2352> >> >> Lawrence Berkeley National Laboratory Fax: >> +1-510-486-6900<tel:%2B1-510-486-6900> >> >> >> >> >> >> > > > > -- > Paul H. Hargrove > [email protected]<mailto:[email protected]> > Computer Languages & Systems Software (CLaSS) Group > Computer Science Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > ------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev _______________________________________________ Chapel-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-users
