"Jeff Squyres (jsquyres)" <jsquy...@cisco.com> writes:
> +1
>
> If you're compiling Open MPI from source, you need the -devel package
> so that the libnuma header files are installed (and therefore Open MPI
> [i.e., the hwloc embedded in Open MPI] can include those header files
> and then compile support for libnuma).
>From the log it appears to have built OK against hwloc.
We've been through this before, and I thought the message had been
amended. libnumactl-devel is not an Ubuntu package -- that's
libnuma-dev -- and it's not necessary. This is with the native Ubuntu
openmpi, but I hope libnuma isn't being used by anything other than
libhwloc these days. [I assume the -devel business is from some
pre-hwloc version which dlopen'ed libnuma.so.]
$ mpirun --bind-to-core --report-bindings -np 2 hostname
[albion:01052] MCW rank 0 bound to socket 0[core 0]: [B .]
[albion:01052] MCW rank 1 bound to socket 0[core 1]: [. B]
albion
albion
$ dpkg -l libnuma-dev
dpkg-query: no packages found matching libnuma-dev
I guess the problem is not having hwloc/libnuma1 on the compute host, as
opposed to the build host. Test like this on the host, though the
component name may be different in 1.10:
$ ldd /usr/lib/openmpi/lib/openmpi/mca_paffinity_hwloc.so | egrep 'hwloc|numa'
libhwloc.so.5 => /usr/lib/x86_64-linux-gnu/libhwloc.so.5
(0x00007f690bba8000)
libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1
(0x00007f690b3b9000)
Otherwise I guess there's a useful MCA debugging parameter, but I don't
know what to try in 1.10.
If you want to replace or supplement the system package, I recommend
modifying the source package to use the newer source and build a new
binary package from that; usually the mods are trivial. Then dpkg or
rpm manages the dependencies.
>> On Oct 29, 2015, at 7:01 AM, Ralph Castain <r...@open-mpi.org> wrote:
>>
>> You need to also install libnuma-devel, I’m afraid - just has to do
>> with how they chose to split the library
>>
>>
>>> On Oct 29, 2015, at 2:01 AM, Fabian Wein <fabian.w...@fau.de> wrote:
>>>
>>> Hello,
>>>
>>> I'm new to the list, maybe the issue was somewhere else already
>>> solved. I found a similar question in the mailing list archive but
>>> no solution.
>>>
>>> By comparing to a similar system I found that for my 4 sockets
>>> Opteron system it is necessary to run mpi with -bind-to numa
>>> option.
>>>
>>>
>>> On my ubunut 14.04 system I get
>>>
>>> ----------------------
>>> A request was made to bind a process, but at least one node does NOT
>>> support binding processes to cpus.
>>>
>>> Node: leo
>>> This usually is due to not having libnumactl and libnumactl-devel
>>> installed on the node.
>>> -----------------------
>>>
>>> locate libnuma | grep so
>>>
>>> results in
>>>
>>> /usr/lib/x86_64-linux-gnu/libnuma.so
>>> /usr/lib/x86_64-linux-gnu/libnuma.so.1
>>> /usr/lib64/libnuma.so
>>> /usr/lib64/libnuma.so.1
>>>
>>> When I configure hwloc 1.11.1 it detects numa (it tells me at the
>>> end of configure)
>>> and "grep numa config.status" results in
>>> S["HWLOC_LIBS"]="-lm -lnuma -lxml2 "
>>> S["HWLOC_LINUX_LIBNUMA_LIBS"]="-lnuma"
>>>
>>> When I configure openmpi-1.10.0 it also finds libnuma
>>> grep numa config.status
>>> S["OMPI_WRAPPER_EXTRA_LIBS"]="-lm -lnuma -ldl -lutil "
>>> S["ORTE_WRAPPER_EXTRA_LIBS"]="-lm -lnuma -ldl -lutil "
>>> S["OPAL_WRAPPER_EXTRA_LIBS"]="-lm -lnuma -ldl -lutil "
>>> S["HWLOC_EMBEDDED_LIBS"]="-lm -lnuma"
>>> S["HWLOC_LINUX_LIBNUMA_LIBS"]="-lnuma"
>>> D["WRAPPER_EXTRA_LIBS"]=" \"-lm -lnuma -ldl -lutil \""
>>>
>>> However I have no idea how I could install libnumactl and
>>> libnumactl-devel. I cannot google it.
>>>
>>> I built numactl-2.0.9 manually but
>>>
>>> ~/tmp/numactl-2.0.9$ ll lib*
>>>
>>> only gives libnuma.a and libnuma.so
>>>
>>> Even the source for open-mpi gives no hint
>>> /tmp/openmpi-1.10.0$ grep -r numactl
>>> opal/mca/hwloc/hwloc191/hwloc/README: * libnuma for memory binding
>>> and migration support on Linux (numactl-devel or
>>> orte/mca/rmaps/base/help-orte-rmaps-base.txt:This usually is due to
>>> not having libnumactl and libnumactl-devel
>>> orte/mca/rmaps/base/help-orte-rmaps-base.txt:contained in the
>>> libnumactl and libnumactl-devel packages.
>>> orte/mca/rmaps/base/help-orte-rmaps-base.txt:contained in the
>>> libnumactl and libnumactl-devel packages.
>>>
>>> Please help, I have no idea what to try next. The only options I
>>> currently see are to try with mpich or by intel-mpi.
>>>
>>> Thanks,
>>>
>>> Fabian
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2015/10/27941.php
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/10/27942.php
