Hi Jintack --
Michael's suggestion to use 'mpi-gnu' as the target compiler unfortunately
only works from the master branch of our GitHub repository, as that option
was added after the 1.13 releases. That said, I wouldn't be surprised if
you could grab the following one file and drop it in
$CHPL_HOME/make/compiler/ to enable it for 1.13:
ub.com/chapel-lang/chapel/blob/master/make/compiler/Makefile.mpi-gnu
Meanwhile, I'm going to see if I can determine why our Makefiles aren't
picking up GASNET_LD as their link step as I expected.
-Brad
On Thu, 21 Jul 2016, Lim, Jintack wrote:
Hi Brad, Michael and Paul,
I really appreciate your helpful comments.
This is the information with make –f <dir>/Makefile print-commands.
CC is /usr/bin/gcc
LD is g++
GASNET_LD is /usr/bin/mpicc
GASNET_INC_MAKEFILE is
/home/limjint/chapel_4/chapel-1.13.1/third-party/gasnet/install/linux64-gnu-unknown/seg-everything/nodbg/include/ofi-conduit/ofi-par.mak
This is the <dir>/Makefile.
http://pastebin.ubuntu.com/20377401/
I can see that CHPL_TARGET_COMPILER=gnu.
I also set CHPL_TARGET_COMPILER=mpi-gnu and compiled Chapel again.
I’m not sure MPI is configured to use gcc, but what I got while compiling
Chapel is something like this.
Warning: Unknown compiler: "mpi-gnu"
Warning: No valid option found: arch="native" compiler="mpi-gnu"
version="CompVersion(major=4, minor=9, revision=2, build=0)"
http://pastebin.ubuntu.com/20380446/
I also got some trivial compile errors and permission errors before above
error, and just worked around.
http://pastebin.ubuntu.com/20377624/
http://pastebin.ubuntu.com/20378961/
Best,
Jintack
From: Paul Hargrove [mailto:[email protected]]
Sent: Thursday, July 21, 2016 3:59 PM
To: Michael Ferguson <[email protected]>
Cc: Lim, Jintack <[email protected]>; [email protected]
Subject: Re: [upc-devel] GASNet ofi-conduit in the context of Chapel
Brad and Michael,
It is not clear from your answers if you fully understood Jintack's situation.
However, *I* may be mistaken if the details you skipped over simply aren't
relevant for reasons unknown to me.
Jintack is *not* using mpi-conduit nor is GASNET_CC an MPI compiler.
The conduit in question is ofi-conduit, which is using mpirun as its default
spawner.
At least ibv, mxm, ofi and psm conduits support launching via ssh or by mpirun.
By default the configure script enables both of these assuming an mpicc is
found.
If mpi-based spawning is enabled at configure time then *one* .c file is
compiled with mpicc.
All the rest of the GASNet conduit code is compiled with a non-MPI C compiler.
However, due to the one MPI object file the .mak files for those conduits will
set their GASNET_LD=mpicc.
I am fairly confident that Chapel must already deal with $(GASNET_LD) as
distinct from $(GASNET_CC) since udp-conduit needs a C++ linker.
I suspect one or more of your suggestions may lead to a work-around
(CHPL_TARGET_COMPILER=mpi-gnu in particular).
However, I am suspicious of the handling of the Makefile fragments.
So, I think Brad's make command with all the print-SOMETHINGs is important to
root out the problem.
Is it possible that the problem is as simple as Chapel needing "ofi" added to
some list of conduits somewhere?
-Paul
On Thu, Jul 21, 2016 at 3:28 PM, Michael Ferguson
<[email protected]<mailto:[email protected]>> wrote:
Hi -
You could also try using
export CHPL_TARGET_COMPILER=mpi-gnu
Assuming you gave MPI configured to use GCC. If that's the case,
if CHPL_TARGET_COMPILER=mpi-gnu when you build Chapel and use
it, you should see mpicc calls as the linker.
Brad had lots of other good ideas for how to troubleshoot this problem.
Cheers,
-michael
On 7/20/16, 6:10 PM, "Lim, Jintack"
<[email protected]<mailto:[email protected]>> wrote:
Hi GASNet and Chapel community,
I'm trying to run Chapel with GASNet ofi-conduit.
I got some link errors last time while I was running "make check" after
compiled Chapel.
I got feedback from Paul Hargrove and Greg Titus but I still can't
resolve it.
http://paste.ubuntu.com/19312464/
I set CHPL_LAUNCHER=gasnetrun_mpi as Greg suggested.
(Paul suggested to try to configure using without-mpi-cc GASNet option,
but it looks like we need mpi based on the chapel lanucher setting above?)
Paul said it looks like Chapel is not using mpicc as its linker.
I checked that gasnet_bootstrap_mpi.c was compiled with mpicc while
building Chapel.
When building libgasnet-ofi-par.a, gcc is used, but this is the same when
compiling GASNet only.
So I wonder if I need to configure Chapel with some flags or should I put
some link option somewhere?
Best,
Jintack
From: Paul Hargrove [mailto:[email protected]<mailto:[email protected]>]
Sent: Wednesday, July 13, 2016 6:09 PM
To: Lim, Jintack <[email protected]<mailto:[email protected]>>
Cc: [email protected]<mailto:[email protected]>; Jeff Hammond
<[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>
Subject: Re: [upc-devel] GASNet ofi-conduit in the context of Chapel
Jintack,
The environment variable GASNET_OFI_SPAWNER is used at application-launch
to select MPI vs SSH if both were available at build time. So, it has no
connection to the link failure. Your environment variable settings seem
to me to be correct.
The linkage errors seem to indicate that Chapel is not using mpicc as its
linker (ignoring the $(GASNET_LD) setting).
That is something you will need to take up with the Chapel team if you
require mpi-based spawning.
The simplest approach to getting past the link problem would be to
configure using --without-mpi-cc.
Then SSH-based spawning will be your only option and setting of
GASNET_OFI_SPAWNER will not be needed.
If you have a PMI-based spawner (including hydra from mpich or srun from
SLURM), then you may also be able to launch GASNet applications that way
(not using the gasnetrun_ofi script). You might need some additional
configure options to
help GASNet locate libpmi - let me know if you need help pursing that
option.
-Paul
On Wed, Jul 13, 2016 at 5:42 PM, Lim, Jintack
<[email protected]<mailto:[email protected]>>
wrote:
Hi all,
When I was compiling a Chapel hello world application on top of GASNet
ofi-conduit, I got some GASNet
link errors related to MPI.
http://paste.ubuntu.com/19312464/
I got this either I set GASNET_OFI_SPAWNER to mpi or ssh.
GASNet version that is included in the Chapel source tree is 1.26.0.
There are instructions to set GASNET environment variables for GASNet-udp
and ibv conduit in the Chapel
documentation
(http://chapel.cray.com/docs/1.13/usingchapel/multilocale.html), but I
couldn’t find one for ofi-conduit.
Could somebody shed some light on how to fix this problem? (such as how
to set CHPL_LAUNCHER or any
other GASNET environment variables to fix the link error?)
What I did is
- CHPL_COMM=gasnet
- CHPL_COMM_SUBSTRATE=ofi
- GASNET_OFI_SPAWNER= mpi or ssh
-GASNET_SSH_SERVERS=”list of my servers”
Please let me know if you need further information.
Best,
Jintack
From: Jeff Hammond
[mailto:[email protected]<mailto:[email protected]>]
Sent: Wednesday, July 13, 2016 5:19 PM
To: Lim, Jintack <[email protected]<mailto:[email protected]>>
Cc:
[email protected]<mailto:[email protected]>
Subject: Re: Chapel on GASNet ofi-conduit
I had trouble with Gasnet OFI in the context of UPC. You might want post
this question to the Berkeley UPC list so the Gasnet team sees it. They
will know the launcher issues better
than the Chapel team (no disrespect intended).
Jeff
On Wednesday, July 13, 2016, Lim, Jintack
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I’m trying to install Chapel using GASNet ofi-conduit. I was able to
compile Chapel itself, however I couldn’t compile
hello6-taskpar-dist.chpl because of link errors related to
MPI.
Here’s the error message.
http://paste.ubuntu.com/19312464/
There are only instructions for GASNet-udp and ibv conduit in the
documentation
(http://chapel.cray.com/docs/1.13/usingchapel/multilocale.html),
so I wonder how to set environment variables (such as launcher) for ofi
conduit.
Any help would be greatly appreciated.
Best,
Jintack
--
Jeff Hammond
[email protected]<mailto:[email protected]>
http://jeffhammond.github.io/
_______________________________________________
upc-devel mailing list
[email protected]<mailto:[email protected]>
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/upc-devel
--
Paul H. Hargrove
[email protected]<mailto:[email protected]>
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department Tel:
+1-510-495-2352<tel:%2B1-510-495-2352>
Lawrence Berkeley National Laboratory Fax:
+1-510-486-6900<tel:%2B1-510-486-6900>
--
Paul H. Hargrove
[email protected]<mailto:[email protected]>
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
