Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Bart Van Assche
On Fri, 2018-06-15 at 18:55 +0200, Hannes Reinecke wrote: > On 06/15/2018 04:07 PM, Bart Van Assche wrote: > > On Thu, 2018-06-14 at 15:38 +0200, Hannes Reinecke wrote: > > > For performance reasons we should be able to allocate all memory > > > from a given NUMA node, so this patch adds a new para

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Hannes Reinecke
On 06/15/2018 04:07 PM, Bart Van Assche wrote: On Thu, 2018-06-14 at 15:38 +0200, Hannes Reinecke wrote: For performance reasons we should be able to allocate all memory from a given NUMA node, so this patch adds a new parameter 'rd_numa_node' to allow the user to specify the NUMA node id. When

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Jens Axboe
On 6/15/18 3:23 AM, Mel Gorman wrote: > On Thu, Jun 14, 2018 at 02:47:39PM -0600, Jens Axboe wrote: > Will numactl ... modprobe brd ... solve this problem? It won't, pages are allocated as needed. >>> >>> Then how about a numactl ... dd /dev/ram ... after the modprobe. >> >> Yes

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Jens Axboe
On 6/15/18 1:30 AM, Christoph Hellwig wrote: > On Thu, Jun 14, 2018 at 09:33:35AM -0600, Jens Axboe wrote: >> Next question - what does the memory allocator do if we run out of >> memory on the given node? Should we punt to a different node if that >> happens? Slower, but functional, seems preferab

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Bart Van Assche
On Thu, 2018-06-14 at 15:38 +0200, Hannes Reinecke wrote: > For performance reasons we should be able to allocate all memory > from a given NUMA node, so this patch adds a new parameter > 'rd_numa_node' to allow the user to specify the NUMA node id. > When restricing fio to use the same NUMA node I

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Mel Gorman
On Thu, Jun 14, 2018 at 02:47:39PM -0600, Jens Axboe wrote: > >>> Will numactl ... modprobe brd ... solve this problem? > >> > >> It won't, pages are allocated as needed. > >> > > > > Then how about a numactl ... dd /dev/ram ... after the modprobe. > > Yes of course, or you could do that for ever

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-15 Thread Christoph Hellwig
On Thu, Jun 14, 2018 at 09:33:35AM -0600, Jens Axboe wrote: > Next question - what does the memory allocator do if we run out of > memory on the given node? Should we punt to a different node if that > happens? Slower, but functional, seems preferable to not being able > to get memory. When using

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Hannes Reinecke
On 06/14/2018 10:53 PM, Adam Manzanares wrote: [ .. ] Then how about a numactl ... dd /dev/ram ... after the modprobe. Yes of course, or you could do that for every application that ends up in the path of the doing IO to it. The point of the option is to just make it explicit, and not have to

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Adam Manzanares
On 6/14/18 1:47 PM, Jens Axboe wrote: > On 6/14/18 2:41 PM, Adam Manzanares wrote: >> >> >> On 6/14/18 1:37 PM, Jens Axboe wrote: >>> On 6/14/18 2:32 PM, Adam Manzanares wrote: On 6/14/18 9:09 AM, Hannes Reinecke wrote: > On Thu, 14 Jun 2018 09:33:35 -0600 > Jens Axboe wro

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Jens Axboe
On 6/14/18 2:41 PM, Adam Manzanares wrote: > > > On 6/14/18 1:37 PM, Jens Axboe wrote: >> On 6/14/18 2:32 PM, Adam Manzanares wrote: >>> >>> >>> On 6/14/18 9:09 AM, Hannes Reinecke wrote: On Thu, 14 Jun 2018 09:33:35 -0600 Jens Axboe wrote: > On 6/14/18 9:29 AM, Hannes Reineck

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Adam Manzanares
On 6/14/18 1:37 PM, Jens Axboe wrote: > On 6/14/18 2:32 PM, Adam Manzanares wrote: >> >> >> On 6/14/18 9:09 AM, Hannes Reinecke wrote: >>> On Thu, 14 Jun 2018 09:33:35 -0600 >>> Jens Axboe wrote: >>> On 6/14/18 9:29 AM, Hannes Reinecke wrote: > On Thu, 14 Jun 2018 08:47:33 -0600 > J

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Jens Axboe
On 6/14/18 2:32 PM, Adam Manzanares wrote: > > > On 6/14/18 9:09 AM, Hannes Reinecke wrote: >> On Thu, 14 Jun 2018 09:33:35 -0600 >> Jens Axboe wrote: >> >>> On 6/14/18 9:29 AM, Hannes Reinecke wrote: On Thu, 14 Jun 2018 08:47:33 -0600 Jens Axboe wrote: > On 6/14/18 7:38

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Adam Manzanares
On 6/14/18 9:09 AM, Hannes Reinecke wrote: > On Thu, 14 Jun 2018 09:33:35 -0600 > Jens Axboe wrote: > >> On 6/14/18 9:29 AM, Hannes Reinecke wrote: >>> On Thu, 14 Jun 2018 08:47:33 -0600 >>> Jens Axboe wrote: >>> On 6/14/18 7:38 AM, Hannes Reinecke wrote: > For performance reasons

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Hannes Reinecke
On Thu, 14 Jun 2018 09:33:35 -0600 Jens Axboe wrote: > On 6/14/18 9:29 AM, Hannes Reinecke wrote: > > On Thu, 14 Jun 2018 08:47:33 -0600 > > Jens Axboe wrote: > > > >> On 6/14/18 7:38 AM, Hannes Reinecke wrote: > >>> For performance reasons we should be able to allocate all memory > >>> fro

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Jens Axboe
On 6/14/18 9:29 AM, Hannes Reinecke wrote: > On Thu, 14 Jun 2018 08:47:33 -0600 > Jens Axboe wrote: > >> On 6/14/18 7:38 AM, Hannes Reinecke wrote: >>> For performance reasons we should be able to allocate all memory >>> from a given NUMA node, so this patch adds a new parameter >>> 'rd_numa_node

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Hannes Reinecke
On Thu, 14 Jun 2018 08:47:33 -0600 Jens Axboe wrote: > On 6/14/18 7:38 AM, Hannes Reinecke wrote: > > For performance reasons we should be able to allocate all memory > > from a given NUMA node, so this patch adds a new parameter > > 'rd_numa_node' to allow the user to specify the NUMA node id. >

Re: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Jens Axboe
On 6/14/18 7:38 AM, Hannes Reinecke wrote: > For performance reasons we should be able to allocate all memory > from a given NUMA node, so this patch adds a new parameter > 'rd_numa_node' to allow the user to specify the NUMA node id. > When restricing fio to use the same NUMA node I'm seeing a per

[PATCH] brd: Allow ramdisk to be allocated on selected NUMA node

2018-06-14 Thread Hannes Reinecke
For performance reasons we should be able to allocate all memory from a given NUMA node, so this patch adds a new parameter 'rd_numa_node' to allow the user to specify the NUMA node id. When restricing fio to use the same NUMA node I'm seeing a performance boost of more than 200%. Signed-off-by: H