Hi Konstantina --
We're a little confused about a few aspects of what you're seeing and
doing, so let me explain some things related to your questions in hopes
that they help. You can follow up with more questions/information if not.
The intention with the current numa locale model is that the top-level
locales (the Locales array, whose size will correspond to your -nl flag)
will correspond to the network nodes; each of these locales will then
allocate a number of sublocales corresponding to the number of NUMA
subdomains present within the nodes themselves (as determined by hwloc).
So this is a 2-level locale model -- the network nodes and the NUMA
subdomains (not cores).
One could also potentially also create a locale model in which individual
cores were exposed as sublocales -- either as children of the
network-level locales, resulting in a 2-level hierarchy with greater
degree than the previous paragraph; or as children of the numa-level
sublocales in the previous paragraph, resulting in a 3-level hierarchy.
But that's not something we're working on now (exposing cores via
sublocales), and doing so would require appropriate coordination and
support from the runtime tasking layer.
As Sung noted, the 1.8 release doesn't support numa and CHPL_COMM=gasnet
(multiple locales) together, so you'll need to be sure to check out the
Chapel trunk in order to run such configurations. The downside of working
from trunk is that it's not guaranteed to be stable, but I've verified
that the head as I write this (r22811) is working fine for the cases I've
tried. Part of our confusion was that in your last mail it sounded as
though you hadn't updated to trunk yet, but were using multiple locales
with some success which doesn't seem possible under 1.8 unless you were
using the flat locale model.
So, in your program, if you want to run on two network nodes with two NUMA
subdomains each, you'd want to use the -nl 2 command line rather than -nl
4. That is, you're enumerating the number of top-level locales, not the
number of leaf locales.
Without knowing what your program is and what error messages you're
seeing, it's hard to advise you how to proceed. You should feel free to
send those as attachments in future messages to help clarify what you're
doing. But in the meantime, let me point you to a sample numa-aware hello
world program in hopes that it helps:
http://svn.code.sf.net/p/chapel/code/trunk/test/localeModels/numa/basics/hello7-taskpar-sublocs.chpl
This code has one parallel loop over the top-level locales and one over
the sublocales, so for example for an -nl 4 run where the hardware had two
numa domains per processor, I'd get 4x2 tasks and messages.
Hope this helps,
-Brad
On Fri, 28 Feb 2014, Konstantina Panagiotopoulou wrote:
> Hello Sung and all,
>
> Thanks for teh quick response. I installed hwloc and now I can compile
> properly.
> I am very interested in the NUMA-type locales. I ll check the trunk for
> more.
>
> By "cluster of many cores" I meant the idea of hierarchical locales
> eg NUMA nodes as locales and their individual cores as sublocales.
> Is there any way to address this type of architecture right now?
>
> For example:
> I have tried ./prog -nl 4 --tasksPerLocale=2 and ./prog -nl 2
> --tasksPerLocale=2 on a configuration with SSH_SERVERS="node1 node2"
> The intention here was to exploit 2 cores/node, but it doesn't seem to work
> that way.
> Actually the first one will give an error and the second one will just use
> two locales(meaning 1 core/node).
>
> Cheers,
> Konstantina
>
------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
