The odd thing about this environment is that the head node seems to have a slightly different setup than the compute nodes (not sure why exactly, but that's what it is). So hwloc is configured and runs correctly on the head node, but when it is asked to run on the compute nodes it segvs at the call site of the __builtin_ functions.
For Open MPI, hwloc can only be built one way and needs to run on both the compute and head nodes - so configuring hwloc two ways is not an option. Further, to have two different installs I need to be able to build on the compute nodes, which I cannot. I suspect that the ABI compatibility of the libc interface is what is enabling the remainder of the code to work in both environments, and that the __builtin_ functions bypass that ABI to put in system specific code that (for whatever reason) does not match on the compute nodes. I agree it is a somewhat exotic environment, and I am open to other possible solutions. This patch does fix the problem, and so does compiling with a non-gcc compiler - though essentially the same work around. If there was a way to tell gcc directly to not allow __builtin_ then I would do that, but no such option seems to exist. Per Jeff's comment, we could turn it off by default in Open MPI, but it would come at a slight performance penalty as Dave mentioned. Since the number of environments in which this would be a problem are small, I would suggest leaving it enabled in Open MPI and documenting the configure option. For this system we already pass a large number of configure options to get things to work correctly, this would just be one more. -- Josh On Wed, Jun 8, 2011 at 5:58 PM, Samuel Thibault <samuel.thiba...@inria.fr> wrote: > Josh Hursey, le Wed 08 Jun 2011 22:28:53 +0200, a écrit : >> I hit a problem when installing hwloc statically on a machine with a >> slightly different gcc support libraries and OSs on the head/compile >> node versus the compute nodes. The builtin functions would cause hwloc >> to segfault when run on the compute nodes. By disabling the builtin >> operations, and using the more portable techniques seemed to do the >> trick. > > This is very odd. How does it segfault exactly? Unless you are (or > your distribution is) using an -march=something option, gcc is not > supposed to produce code which can not be run on another machine. > > Samuel > _______________________________________________ > hwloc-users mailing list > hwloc-us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users > > -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey
