Hi Brice,
>The flag says "when the first touch occurs and the physical memory is
allocated for real, don't allocate on the local node (default), but >rather
allocate where specified by set_membind".
If is it already allocated for real, how set_membind() can allocate on other
node?
So, what's the difference between HWLOC_MEMBIND_BIND and
HWLOC_MEMBIND_FIRSTTOUCH?
>set_membind_nodeset() with BIND and the nodeset containing the nodes where
physical memory should be allocated.
Mm In my tests, I have a process that allocates a memory, thread 0 is bound
on node 0 and thread 1 on node1:
malloc(array)...
omp parallel region
if(tid==0)
set_area_membind_nodeset( first half of array)
if (tid==1)
set_area_membind_nodeset( second half of array)
end parallel region
for(i...)
array(i)
Checking the free memory on two nodes, supposing array is 1 Gb, I see 500 mb
less on first node and 500 mb less on second node and it is ok, because
first touch is done by the process, but on memory already bound on specific
nodes.
Doing the follow test:
omp parallel region
if(tid==0){
malloc(array)...
set_area_membind(HWLOCMEMBIND_BIND, node 0)
}
if (tid==1){
set_area_membind(HWLOCMEMBIND_BIND, node 1)
for(i...)
array(i)
}
end parallel region
Now, checking the free memory on two nodes, I see 1 GB less on second node,
because thread 1 is doing first touch. set_membind() for first thread has no
effect, I suppose, or I dont' understand how it works, because in the second
example only first touch appears to have some effects, indipendently which
hwloc function I'm using.
Sorry, but it is quite difficult to understand .. :(
2011/9/25 Brice Goglin <brice.gog...@inria.fr>
> **
> Le 25/09/2011 11:14, Gabriele Fatigati a écrit :
>
>
> I report my questions in a different way (in the first question i did a
> mistake):
>
>
> 1) I don't understand the means of set_membind() function. Why I should
> to allocate in a node "near" my cpuset and not in my local node (where
> thread or process runs?)
>
>
> It's exactly the same. Your local node is near the cpuset that contains the
> CPUs that are close to this node.
>
>
> 2) Which is the behaviour of HWLOC_MEMBIND_BIND flag?
>
> From the manual:
>
> "Allocate memory on the specified nodes."
>
> It means that I can allocate without binding the memory?
>
>
> It's about physical memory allocation (first touch causing a fault causing
> a page to be allocated), not about virtual memory (malloc).
>
>
> What happens if one thread allocate and another thread in another node
> read/write for the first time this memory? In a little example I see the
> memory is allocated on the second thread, not where first thread does
> malloc(). So, when I have to use HWLOC_MEMBIND_BIND flag? Or it has nothing
> to do with binding?
>
> If the effective allocation is done when first thread touch the memory,
> which is the means of this flag?
>
>
> The flag says "when the first touch occurs and the physical memory is
> allocated for real, don't allocate on the local node (default), but rather
> allocate where specified by set_membind".
>
>
> 2) My goal is to replicate the behaviour of set_area_membind_nodeset() in
> some manner for all futures allocation without call this function each time
> I allocate some memory. Is it possible to do this?
>
>
> set_membind_nodeset() with BIND and the nodeset containing the nodes where
> physical memory should be allocated.
>
> Brice
>
>
> _______________________________________________
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>
--
Ing. Gabriele Fatigati
HPC specialist
SuperComputing Applications and Innovation Department
Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy
www.cineca.it Tel: +39 051 6171722
g.fatigati [AT] cineca.it
