Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
- From: Brice Goglin Sent: 29 January 2019 15:39 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Only the one in brackets is set, others are unset alternatives. If you write "madvise" in that file, it'll become &qu

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Brice Goglin
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 >>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 >>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 >>> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 &

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
locality user list Subject: Re: [hwloc-users] unusual memory binding results Oh, that's very good to know. I guess lots of people using first touch will be affected by this issue. We may want to add a hwloc memory flag doing something similar. Do you have root access to verify that writing

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Brice Goglin
> > Problem seems to be solved for now. Thank you very much for your insights and > suggestions/help. > > JB > > -Original Message- > From: Brice Goglin > Sent: 29 January 2019 10:35 > To: Biddiscombe, John A. ; Hardware locality user list > > Subject:

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
Message- From: Brice Goglin Sent: 29 January 2019 10:35 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Crazy idea: 512 pages could be replaced with a single 2MB huge page. You're not requesting huge pages in your allocation

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
2019 10:35 To: Biddiscombe, John A. ; Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Crazy idea: 512 pages could be replaced with a single 2MB huge page. You're not requesting huge pages in your allocation but some systems have transparent huge pages enabled

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Brice Goglin
On Behalf Of > Biddiscombe, John A. > Sent: 28 January 2019 16:14 > To: Brice Goglin > Cc: Hardware locality user list > Subject: Re: [hwloc-users] unusual memory binding results > > Brice > >> Can you print the pattern before and after thread 1 touched its page

Re: [hwloc-users] unusual memory binding results

2019-01-29 Thread Biddiscombe, John A.
Goglin Cc: Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Brice >Can you print the pattern before and after thread 1 touched its pages, or even >in the middle ? >It looks like somebody is touching too many pages here. Experimenting with differen

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Biddiscombe, John A.
Brice >Can you print the pattern before and after thread 1 touched its pages, or even >in the middle ? >It looks like somebody is touching too many pages here. Experimenting with different threads touching one or more pages, I get unpredicatable results here on the 8 numa node device, the

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Brice Goglin
ent: 28 January 2019 10:56 > To: Biddiscombe, John A. > Cc: Hardware locality user list > Subject: Re: [hwloc-users] unusual memory binding results > > Can you try again disabling the touching in one thread to check whether the > other thread only touched its own pages? (others' st

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Biddiscombe, John A.
To: Biddiscombe, John A. Cc: Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Can you try again disabling the touching in one thread to check whether the other thread only touched its own pages? (others' status should be -2 (ENOENT)) Recent kernels have ways

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Brice Goglin
can > see the memory contents hold the correct CPU ID of the thread that touched > the memory, so either the syscall is wrong, or the kernel is doing something > else. I welcome any suggestions on what might be wrong. > > Thanks for trying to help. > > JB > > -----Original

Re: [hwloc-users] unusual memory binding results

2019-01-28 Thread Biddiscombe, John A.
either the syscall is wrong, or the kernel is doing something else. I welcome any suggestions on what might be wrong. Thanks for trying to help. JB -Original Message- From: Brice Goglin Sent: 26 January 2019 10:19 To: Biddiscombe, John A. Cc: Hardware locality user list Subject: Re: [hwloc-u

Re: [hwloc-users] unusual memory binding results

2019-01-26 Thread Brice Goglin
Le 25/01/2019 à 23:16, Biddiscombe, John A. a écrit : >> move_pages() returning 0 with -14 in the status array? As opposed to >> move_pages() returning -1 with errno set to 14, which would definitely be a >> bug in hwloc. > I think it was move_pages returning zero with -14 in the status array,

Re: [hwloc-users] unusual memory binding results

2019-01-25 Thread Biddiscombe, John A.
call. JB From: Brice Goglin [brice.gog...@inria.fr] Sent: 25 January 2019 21:08 To: Biddiscombe, John A. Cc: Hardware locality user list Subject: Re: [hwloc-users] unusual memory binding results Le 25/01/2019 à 14:17, Biddiscombe, John A. a écrit : > Dea

Re: [hwloc-users] unusual memory binding results

2019-01-25 Thread Brice Goglin
Le 25/01/2019 à 14:17, Biddiscombe, John A. a écrit : > Dear List/Brice > > I experimented with disabling the memory touch on threads except for > N=1,2,3,4 etc and found a problem in hwloc, which is that the function > hwloc_get_area_memlocation was returning '0' when the status of the memory

Re: [hwloc-users] unusual memory binding results

2019-01-25 Thread Biddiscombe, John A.
Dear List/Brice I experimented with disabling the memory touch on threads except for N=1,2,3,4 etc and found a problem in hwloc, which is that the function hwloc_get_area_memlocation was returning '0' when the status of the memory null move operation was -14 (#define EFAULT 14 /* Bad address

Re: [hwloc-users] unusual memory binding results

2019-01-21 Thread Biddiscombe, John A.
>One way to debug would be to disable touching in N-1 thread to check >that everything allocated in on the right node. I shall try that. Thanks ___ hwloc-users mailing list hwloc-users@lists.open-mpi.org

Re: [hwloc-users] unusual memory binding results

2019-01-21 Thread Biddiscombe, John A.
Brice Apologies, I didn't explain it very well, I do make sure that if the tile size 256*8 < 4096 (pagesize), then I double the number of tiles per page, I just wanted to keep the explanation simple. here are some code snippets to give you the flavour of it initializing the helper sruct

Re: [hwloc-users] unusual memory binding results

2019-01-21 Thread Brice Goglin
Le 21/01/2019 à 17:08, Biddiscombe, John A. a écrit : > Dear list, > > I'm allocating a matrix of size (say) 2048*2048 on a node with 2 numa domains > and initializing the matrix by using 2 threads, one pinned on each numa > domain - with the idea that I can create tiles of memory bound to each