Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
- From: Brice Goglin Sent: 29 January 2019 15:39 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Only the one in brackets is set, others are unset alternatives. If you write "madvise" in that file, it'll become &qu

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
On the 8 numa node machine $cat /sys/kernel/mm/transparent_hugepage/enabled [always] madvise never is set already, so I'm not really sure what should go in there to disable it. JB -Original Message- From: Brice Goglin Sent: 29 January 2019 15:29 To: Biddiscombe, John A. ; Hardware

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
Message- From: Brice Goglin Sent: 29 January 2019 10:35 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Crazy idea: 512 pages could be replaced with a single 2MB huge page. You're not requesting huge pages in your allocation

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
2019 10:35 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Crazy idea: 512 pages could be replaced with a single 2MB huge page. You're not requesting huge pages in your allocation but some systems have transparent huge pages enabled

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 On the 8 numa node machine it sometimes gives the right answer even with 512 pages. Still baffled JB -Original Message- From: hwloc-users On Behalf Of Biddiscombe, John A. Sent: 28 January 2019 16:14 To: Brice

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Biddiscombe, John A.
Brice >Can you print the pattern before and after thread 1 touched its pages, or even >in the middle ? >It looks like somebody is touching too many pages here. Experimenting with different threads touching one or more pages, I get unpredicatable results here on the 8 numa node device, the

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Biddiscombe, John A.
either the syscall is wrong, or the kernel is doing something else. I welcome any suggestions on what might be wrong. Thanks for trying to help. JB -Original Message- From: Brice Goglin Sent: 26 January 2019 10:19 To: Biddiscombe, John A. Cc: Hardware locality user list Subject: Re: [hwloc-u

Re: [hwloc-users] unusual memory binding results

2019-01-25 Thread Biddiscombe, John A.
Dear List/Brice I experimented with disabling the memory touch on threads except for N=1,2,3,4 etc and found a problem in hwloc, which is that the function hwloc_get_area_memlocation was returning '0' when the status of the memory null move operation was -14 (#define EFAULT 14 /* Bad address

Re: [hwloc-users] unusual memory binding results

2019-01-21 Thread Biddiscombe, John A.
Brice Apologies, I didn't explain it very well, I do make sure that if the tile size 256*8 < 4096 (pagesize), then I double the number of tiles per page, I just wanted to keep the explanation simple. here are some code snippets to give you the flavour of it initializing the helper sruct

Re: [hwloc-users] question about hwloc_set_area_membind_nodeset

2017-11-15 Thread Biddiscombe, John A.
ll things magically work if I upgrade to hwloc 2.0 etc etc Thanks JB -Original Message- From: hwloc-users [mailto:hwloc-users-boun...@lists.open-mpi.org] On Behalf Of Biddiscombe, John A. Sent: 13 November 2017 15:37 To: Hardware locality user list <hwloc-users@lists.open-mpi.org

Re: [hwloc-users] question about hwloc_set_area_membind_nodeset

2017-11-13 Thread Biddiscombe, John A.
the original alloc call? thanks JB From: hwloc-users [hwloc-users-boun...@lists.open-mpi.org] on behalf of Biddiscombe, John A. [biddi...@cscs.ch] Sent: 13 November 2017 14:59 To: Hardware locality user list Subject: Re: [hwloc-u

Re: [hwloc-users] question about hwloc_set_area_membind_nodeset

2017-11-13 Thread Biddiscombe, John A.
locality user list Subject: Re: [hwloc-users] question about hwloc_set_area_membind_nodeset Use get_area_memlocation() membind() returns where the pages are *allowed* to go (anywhere) memlocation() returns where the pages are actually allocated. Brice Le 13/11/2017 06:52, Biddiscombe, John

Re: [hwloc-users] question about hwloc_set_area_membind_nodeset

2017-11-13 Thread Biddiscombe, John A.
Thank you to you both. I modified the allocator to allocate one large block using hwloc_alloc and then use one thread per numa domain to touch each page according to the tiling pattern - unfortunately, I hadn't appreciated that now hwloc_get_area_membind_nodeset always returns the full machine

Re: [hwloc-users] HWLOC_VERSION

2017-10-30 Thread Biddiscombe, John A.
, Biddiscombe, John A. a écrit : > Dear list > > According to the release notes Add HWLOC_MEMBIND_BYNODESET flag was > added in 1.11.3 - if I protect some code with > > #if HWLOC_API_VERSION >= 0x00010b00 > then versions 1.11.0, 1.11.1, 1.11.2 still cause build failures. > &

[hwloc-users] HWLOC_VERSION

2017-10-30 Thread Biddiscombe, John A.
Dear list According to the release notes Add HWLOC_MEMBIND_BYNODESET flag was added in 1.11.3 - if I protect some code with #if HWLOC_API_VERSION >= 0x00010b00 then versions 1.11.0, 1.11.1, 1.11.2 still cause build failures. is there some VERSION flag that distinguishes between the patch

Re: [hwloc-users] BGQ question.

2014-03-28 Thread Biddiscombe, John A.
> -Original Message- > From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf > Of Chris Samuel > Sent: 26 March 2014 13:42 > To: Hardware locality user list > Subject: Re: [hwloc-users] BGQ question. > > On Wed, 26 Mar 2014 11:56:08 AM Biddiscombe, John A. wr

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
Chris > Out of interest, why on an I/O node? I'm targeting the BGQ BGAS nodes with flash cards installed. We've done tests with GPFS mounted on the flash and are trying to get comparable results with an in-house driver. JB

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
Goglin [mailto:brice.gog...@inria.fr] Sent: 25 March 2014 09:28 To: Biddiscombe, John A.; Hardware locality user list Subject: Re: [hwloc-users] BGQ question. Can you run hwloc-gather-topology foo and send the resulting foo.tar.bz2 ? If the tarball is too bug, feel free to send it to me in a private

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
for some special projects where we are trying to customise the IO. JB From: Brice Goglin [mailto:brice.gog...@inria.fr] Sent: 25 March 2014 08:43 To: Hardware locality user list; Biddiscombe, John A. Subject: Re: [hwloc-users] BGQ question. Wait, I missed the "io node" part of your first mai

Re: [hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
with something set? Thanks JB From: hwloc-users [mailto:hwloc-users-boun...@open-mpi.org] On Behalf Of Brice Goglin Sent: 25 March 2014 08:04 To: Hardware locality user list Subject: Re: [hwloc-users] BGQ question. Le 25/03/2014 07:51, Biddiscombe, John A. a écrit : I'm compiling hwloc using

[hwloc-users] BGQ question.

2014-03-25 Thread Biddiscombe, John A.
I'm compiling hwloc using clang (bgclang++11 from ANL) to run on IO nodes af a BGQ. It seems to have compiled ok, and when I run lstopo, I get an output like this (below), which looks reasonable, but there are 15 sockets instead of 16. I'm a little worried because the first time I compiled, I