Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Thursday, June 18, 2015 01:24:01 PM Dan Williams wrote: > Rafael, does patch1 look ok to you? Mostly. acpi_map_pxm_to_online_node() needs a proper kerneldoc comment describing what it does. Thanks, Rafael -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Rafael, does patch1 look ok to you? On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani wrote: > Since NVDIMMs are installed on memory slots, they expose the NUMA > topology of a platform. This patchset adds support of sysfs > 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. > This enables numactl(8) to accept 'block:' and 'file:' paths of > pmem and btt devices as shown in the examples below. > numactl --preferred block:pmem0 --show > numactl --preferred file:/dev/pmem0s --show > > numactl can be used to bind an application to the locality of > a target NVDIMM for better performance. Here is a result of fio > benchmark to ext4/dax on an HP DL380 with 2 sockets for local and > remote settings. > > Local [1] : 4098.3MB/s > Remote [2]: 3718.4MB/s > > [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > > [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio > > > Patch 1/3 applies on top of the acpica branch of the pm tree. > Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of > "libnvdimm: non-volatile memory devices". > > --- > v2: > - Add acpi_map_pxm_to_online_node(), which returns an online node. > - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) > - Check ACPI_NFIT_PROXIMITY_VALID in spa->flags. > > --- > Toshi Kani (3): > 1/3 acpi: Add acpi_map_pxm_to_online_node() > 2/3 libnvdimm: Set numa_node to NVDIMM devices > 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices > > --- > drivers/acpi/nfit.c | 7 +++ > drivers/acpi/numa.c | 40 +--- > drivers/nvdimm/btt.c| 2 ++ > drivers/nvdimm/btt_devs.c | 1 + > drivers/nvdimm/bus.c| 30 ++ > drivers/nvdimm/namespace_devs.c | 1 + > drivers/nvdimm/nd.h | 1 + > drivers/nvdimm/region.c | 1 + > drivers/nvdimm/region_devs.c| 1 + > include/linux/acpi.h| 5 + > include/linux/libnvdimm.h | 2 ++ > 11 files changed, 88 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Thursday, June 18, 2015 01:24:01 PM Dan Williams wrote: Rafael, does patch1 look ok to you? Mostly. acpi_map_pxm_to_online_node() needs a proper kerneldoc comment describing what it does. Thanks, Rafael -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Rafael, does patch1 look ok to you? On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani toshi.k...@hp.com wrote: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Patch 1/3 applies on top of the acpica branch of the pm tree. Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of libnvdimm: non-volatile memory devices. --- v2: - Add acpi_map_pxm_to_online_node(), which returns an online node. - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) - Check ACPI_NFIT_PROXIMITY_VALID in spa-flags. --- Toshi Kani (3): 1/3 acpi: Add acpi_map_pxm_to_online_node() 2/3 libnvdimm: Set numa_node to NVDIMM devices 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices --- drivers/acpi/nfit.c | 7 +++ drivers/acpi/numa.c | 40 +--- drivers/nvdimm/btt.c| 2 ++ drivers/nvdimm/btt_devs.c | 1 + drivers/nvdimm/bus.c| 30 ++ drivers/nvdimm/namespace_devs.c | 1 + drivers/nvdimm/nd.h | 1 + drivers/nvdimm/region.c | 1 + drivers/nvdimm/region_devs.c| 1 + include/linux/acpi.h| 5 + include/linux/libnvdimm.h | 2 ++ 11 files changed, 88 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Thu, 2015-06-11 at 08:38 -0700, Dan Williams wrote: > On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani wrote: > > Since NVDIMMs are installed on memory slots, they expose the NUMA > > topology of a platform. This patchset adds support of sysfs > > 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. > > This enables numactl(8) to accept 'block:' and 'file:' paths of > > pmem and btt devices as shown in the examples below. > > numactl --preferred block:pmem0 --show > > numactl --preferred file:/dev/pmem0s --show > > > > numactl can be used to bind an application to the locality of > > a target NVDIMM for better performance. Here is a result of fio > > benchmark to ext4/dax on an HP DL380 with 2 sockets for local and > > remote settings. > > > > Local [1] : 4098.3MB/s > > Remote [2]: 3718.4MB/s > > > > [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > > > > [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio > > > > > > Patch 1/3 applies on top of the acpica branch of the pm tree. > > Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of > > "libnvdimm: non-volatile memory devices". > > > > --- > > v2: > > - Add acpi_map_pxm_to_online_node(), which returns an online node. > > - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) > > - Check ACPI_NFIT_PROXIMITY_VALID in spa->flags. > > > > --- > > Toshi Kani (3): > > 1/3 acpi: Add acpi_map_pxm_to_online_node() > > 2/3 libnvdimm: Set numa_node to NVDIMM devices > > 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices > > Looks good to me. Once Rafael acks the ACPI core changes I'll pull it > in to libnvdimm-for-next. Great! Thanks Dan, -Toshi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani wrote: > Since NVDIMMs are installed on memory slots, they expose the NUMA > topology of a platform. This patchset adds support of sysfs > 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. > This enables numactl(8) to accept 'block:' and 'file:' paths of > pmem and btt devices as shown in the examples below. > numactl --preferred block:pmem0 --show > numactl --preferred file:/dev/pmem0s --show > > numactl can be used to bind an application to the locality of > a target NVDIMM for better performance. Here is a result of fio > benchmark to ext4/dax on an HP DL380 with 2 sockets for local and > remote settings. > > Local [1] : 4098.3MB/s > Remote [2]: 3718.4MB/s > > [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > > [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio > > > Patch 1/3 applies on top of the acpica branch of the pm tree. > Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of > "libnvdimm: non-volatile memory devices". > > --- > v2: > - Add acpi_map_pxm_to_online_node(), which returns an online node. > - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) > - Check ACPI_NFIT_PROXIMITY_VALID in spa->flags. > > --- > Toshi Kani (3): > 1/3 acpi: Add acpi_map_pxm_to_online_node() > 2/3 libnvdimm: Set numa_node to NVDIMM devices > 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices Looks good to me. Once Rafael acks the ACPI core changes I'll pull it in to libnvdimm-for-next. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani toshi.k...@hp.com wrote: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Patch 1/3 applies on top of the acpica branch of the pm tree. Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of libnvdimm: non-volatile memory devices. --- v2: - Add acpi_map_pxm_to_online_node(), which returns an online node. - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) - Check ACPI_NFIT_PROXIMITY_VALID in spa-flags. --- Toshi Kani (3): 1/3 acpi: Add acpi_map_pxm_to_online_node() 2/3 libnvdimm: Set numa_node to NVDIMM devices 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices Looks good to me. Once Rafael acks the ACPI core changes I'll pull it in to libnvdimm-for-next. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Thu, 2015-06-11 at 08:38 -0700, Dan Williams wrote: On Tue, Jun 9, 2015 at 4:10 PM, Toshi Kani toshi.k...@hp.com wrote: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Patch 1/3 applies on top of the acpica branch of the pm tree. Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of libnvdimm: non-volatile memory devices. --- v2: - Add acpi_map_pxm_to_online_node(), which returns an online node. - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) - Check ACPI_NFIT_PROXIMITY_VALID in spa-flags. --- Toshi Kani (3): 1/3 acpi: Add acpi_map_pxm_to_online_node() 2/3 libnvdimm: Set numa_node to NVDIMM devices 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices Looks good to me. Once Rafael acks the ACPI core changes I'll pull it in to libnvdimm-for-next. Great! Thanks Dan, -Toshi -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Dan Williams writes: > On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer wrote: >> Toshi Kani writes: >> >>> Since NVDIMMs are installed on memory slots, they expose the NUMA >>> topology of a platform. This patchset adds support of sysfs >>> 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. >>> This enables numactl(8) to accept 'block:' and 'file:' paths of >>> pmem and btt devices as shown in the examples below. >>> numactl --preferred block:pmem0 --show >>> numactl --preferred file:/dev/pmem0s --show >>> >>> numactl can be used to bind an application to the locality of >>> a target NVDIMM for better performance. Here is a result of fio >>> benchmark to ext4/dax on an HP DL380 with 2 sockets for local and >>> remote settings. >>> >>> Local [1] : 4098.3MB/s >>> Remote [2]: 3718.4MB/s >>> >>> [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio >>> >>> [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio >>> >> >> Did you post the patches to numactl somewhere? >> > > numactl already supports this today. Ah, I did not know that. I guess I should have RTFM. :) Cheers, Jeff -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 9:20 AM, Elliott, Robert (Server Storage)
wrote:
>> -Original Message-
>> From: Linux-nvdimm [mailto:linux-nvdimm-boun...@lists.01.org] On Behalf Of
>> Dan Williams
>> Sent: Wednesday, June 10, 2015 9:58 AM
>> To: Jeff Moyer
>> Cc: linux-nvdimm; Rafael J. Wysocki; linux-kernel@vger.kernel.org; Linux
>> ACPI
>> Subject: Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
>>
>> On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer wrote:
>> > Toshi Kani writes:
>> >
>> >> Since NVDIMMs are installed on memory slots, they expose the NUMA
>> >> topology of a platform. This patchset adds support of sysfs
>> >> 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
>> >> This enables numactl(8) to accept 'block:' and 'file:' paths of
>> >> pmem and btt devices as shown in the examples below.
>> >> numactl --preferred block:pmem0 --show
>> >> numactl --preferred file:/dev/pmem0s --show
>> >>
>> >> numactl can be used to bind an application to the locality of
>> >> a target NVDIMM for better performance. Here is a result of fio
>> >> benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
>> >> remote settings.
>> >>
>> >> Local [1] : 4098.3MB/s
>> >> Remote [2]: 3718.4MB/s
>> >>
>> >> [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > on-pmem0>
>> >> [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio > on-pmem0>
>> >
>> > Did you post the patches to numactl somewhere?
>> >
>>
>> numactl already supports this today.
>
> numactl does have a bug handling partitions under these devices,
> because it assumes all storage devices have "/devices/pci"
> in their path as it tries to find the parent device for the
> partition. I think we'll propose a numactl patch for that;
> I don't think the drivers can fool it.
>
> Details (from an earlier version of the patch series
> in which btt devices were named /dev/nd1, etc.):
>
> strace shows that numactl is trying to find numa_node in very
> different locations for /dev/nd1p1 vs. /dev/sda1.
>
> strace for /dev/nd1p1
> =
> open("/sys/class/block/nd1p1/dev", O_RDONLY) = 4
> read(4, "259:1\n", 4095)= 6
> close(4)= 0
> close(3)= 0
> readlink("/sys/class/block/nd1p1", "../../devices/LNXSYSTM:00/LNXSYB"...,
> 1024) = 77
> open("/sys/class/block/nd1p1/device/numa_node", O_RDONLY) = -1 ENOENT (No
> such file or directory)
>
> strace for /dev/sda1
>
> open("/sys/class/block/sda1/dev", O_RDONLY) = 4
> read(4, "8:1\n", 4095) = 4
> close(4)= 0
> close(3)= 0
> readlink("/sys/class/block/sda1", "../../devices/pci:00/:00"...,
> 1024) = 91
> open("/sys//devices/pci:00/:00:01.0//numa_node", O_RDONLY) = 3
> read(3, "0\n", 4095)= 2
> close(3)= 0
>
> The "sys/class/block/xxx" paths link to:
> lrwxrwxrwx. 1 root root 0 May 20 20:42 /sys/class/block/nd1p1 ->
> ../../devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0012:00/ndbus0/btt1/block/nd1/nd1p1
> lrwxrwxrwx. 1 root root 0 May 20 20:41 /sys/class/block/sda1 ->
> ../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
>
>
> For /dev/sda1, numactl recognizes "/devices/pci" as
> a special path, and strips off everything after the
> numbers. Faced with:
> ../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
>
> it ends up with this (leaving a sloppy "//" in the path):
> /sys/devices/pci:00/:00:01.0//numa_node
>
> It would also succeed if it ended up with this:
> /sys/devices/pci:00/:00:01.0/:03:00.0/numa_node
>
> For /dev/nd1p1 it does not see that string, so just
> tries to open "/sys/class/block/nd1p1/device/numa_node"
>
> There are no "device/" subdirectories in the tree for
> partition devices (for either sda1 or nd1p1), so this
> fails.
>
>
> From http://oss.sgi.com/projects/libnuma/
> numactl affinity.c:
> /* Somewhat hackish: extract device from symlink path.
>Better would be a direct backlink. This knows slightly too
>much about
RE: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
> -Original Message-
> From: Linux-nvdimm [mailto:linux-nvdimm-boun...@lists.01.org] On Behalf Of
> Dan Williams
> Sent: Wednesday, June 10, 2015 9:58 AM
> To: Jeff Moyer
> Cc: linux-nvdimm; Rafael J. Wysocki; linux-kernel@vger.kernel.org; Linux
> ACPI
> Subject: Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
>
> On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer wrote:
> > Toshi Kani writes:
> >
> >> Since NVDIMMs are installed on memory slots, they expose the NUMA
> >> topology of a platform. This patchset adds support of sysfs
> >> 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
> >> This enables numactl(8) to accept 'block:' and 'file:' paths of
> >> pmem and btt devices as shown in the examples below.
> >> numactl --preferred block:pmem0 --show
> >> numactl --preferred file:/dev/pmem0s --show
> >>
> >> numactl can be used to bind an application to the locality of
> >> a target NVDIMM for better performance. Here is a result of fio
> >> benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
> >> remote settings.
> >>
> >> Local [1] : 4098.3MB/s
> >> Remote [2]: 3718.4MB/s
> >>
> >> [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio on-pmem0>
> >> [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio on-pmem0>
> >
> > Did you post the patches to numactl somewhere?
> >
>
> numactl already supports this today.
numactl does have a bug handling partitions under these devices,
because it assumes all storage devices have "/devices/pci"
in their path as it tries to find the parent device for the
partition. I think we'll propose a numactl patch for that;
I don't think the drivers can fool it.
Details (from an earlier version of the patch series
in which btt devices were named /dev/nd1, etc.):
strace shows that numactl is trying to find numa_node in very
different locations for /dev/nd1p1 vs. /dev/sda1.
strace for /dev/nd1p1
=
open("/sys/class/block/nd1p1/dev", O_RDONLY) = 4
read(4, "259:1\n", 4095)= 6
close(4)= 0
close(3)= 0
readlink("/sys/class/block/nd1p1", "../../devices/LNXSYSTM:00/LNXSYB"..., 1024)
= 77
open("/sys/class/block/nd1p1/device/numa_node", O_RDONLY) = -1 ENOENT (No such
file or directory)
strace for /dev/sda1
open("/sys/class/block/sda1/dev", O_RDONLY) = 4
read(4, "8:1\n", 4095) = 4
close(4)= 0
close(3)= 0
readlink("/sys/class/block/sda1", "../../devices/pci:00/:00"..., 1024)
= 91
open("/sys//devices/pci:00/:00:01.0//numa_node", O_RDONLY) = 3
read(3, "0\n", 4095)= 2
close(3)= 0
The "sys/class/block/xxx" paths link to:
lrwxrwxrwx. 1 root root 0 May 20 20:42 /sys/class/block/nd1p1 ->
../../devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0012:00/ndbus0/btt1/block/nd1/nd1p1
lrwxrwxrwx. 1 root root 0 May 20 20:41 /sys/class/block/sda1 ->
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
For /dev/sda1, numactl recognizes "/devices/pci" as
a special path, and strips off everything after the
numbers. Faced with:
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
it ends up with this (leaving a sloppy "//" in the path):
/sys/devices/pci:00/:00:01.0//numa_node
It would also succeed if it ended up with this:
/sys/devices/pci:00/:00:01.0/:03:00.0/numa_node
For /dev/nd1p1 it does not see that string, so just
tries to open "/sys/class/block/nd1p1/device/numa_node"
There are no "device/" subdirectories in the tree for
partition devices (for either sda1 or nd1p1), so this
fails.
>From http://oss.sgi.com/projects/libnuma/
numactl affinity.c:
/* Somewhat hackish: extract device from symlink path.
Better would be a direct backlink. This knows slightly too
much about the actual sysfs layout. */
char path[1024];
char *fn = NULL;
if (asprintf(, "/sys/class/%s/%s", cls, dev) > 0 &&
readlink(fn, path, sizeof path) > 0) {
regex_t re;
regmatch_t match[2];
char *p;
regcomp(, "(/devices/pci[0-9a-fA-F:/]+\\.[0-9]+)/",
REG_EXTENDED);
ret = regexec(, path, 2, match, 0);
regfree();
if (ret == 0) {
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, 2015-06-10 at 08:57 -0700, Dan Williams wrote:
> On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer wrote:
> > Toshi Kani writes:
> >
> >> Since NVDIMMs are installed on memory slots, they expose the NUMA
> >> topology of a platform. This patchset adds support of sysfs
> >> 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
> >> This enables numactl(8) to accept 'block:' and 'file:' paths of
> >> pmem and btt devices as shown in the examples below.
> >> numactl --preferred block:pmem0 --show
> >> numactl --preferred file:/dev/pmem0s --show
> >>
> >> numactl can be used to bind an application to the locality of
> >> a target NVDIMM for better performance. Here is a result of fio
> >> benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
> >> remote settings.
> >>
> >> Local [1] : 4098.3MB/s
> >> Remote [2]: 3718.4MB/s
> >>
> >> [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio
> >>
> >> [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio
> >>
> >
> > Did you post the patches to numactl somewhere?
> >
>
> numactl already supports this today.
Yes, numactl supports the following sysfs class lookup for numa_node.
This patchset adds numa_node for NVDIMM devices in the same sysfs format
as described in patch 3/3.
/* Generic sysfs class lookup */
static int
affinity_class(struct bitmask *mask, char *cls, const char *dev)
{
:
ret = sysfs_node_read(mask, "/sys/class/%s/%s/device/numa_node",
cls, dev);
Thanks,
-Toshi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer wrote: > Toshi Kani writes: > >> Since NVDIMMs are installed on memory slots, they expose the NUMA >> topology of a platform. This patchset adds support of sysfs >> 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. >> This enables numactl(8) to accept 'block:' and 'file:' paths of >> pmem and btt devices as shown in the examples below. >> numactl --preferred block:pmem0 --show >> numactl --preferred file:/dev/pmem0s --show >> >> numactl can be used to bind an application to the locality of >> a target NVDIMM for better performance. Here is a result of fio >> benchmark to ext4/dax on an HP DL380 with 2 sockets for local and >> remote settings. >> >> Local [1] : 4098.3MB/s >> Remote [2]: 3718.4MB/s >> >> [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio >> >> [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio >> > > Did you post the patches to numactl somewhere? > numactl already supports this today. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Toshi Kani writes: > Since NVDIMMs are installed on memory slots, they expose the NUMA > topology of a platform. This patchset adds support of sysfs > 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. > This enables numactl(8) to accept 'block:' and 'file:' paths of > pmem and btt devices as shown in the examples below. > numactl --preferred block:pmem0 --show > numactl --preferred file:/dev/pmem0s --show > > numactl can be used to bind an application to the locality of > a target NVDIMM for better performance. Here is a result of fio > benchmark to ext4/dax on an HP DL380 with 2 sockets for local and > remote settings. > > Local [1] : 4098.3MB/s > Remote [2]: 3718.4MB/s > > [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio > > [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio > Did you post the patches to numactl somewhere? -Jeff -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, 2015-06-10 at 08:57 -0700, Dan Williams wrote:
On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer jmo...@redhat.com wrote:
Toshi Kani toshi.k...@hp.com writes:
Since NVDIMMs are installed on memory slots, they expose the NUMA
topology of a platform. This patchset adds support of sysfs
'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
This enables numactl(8) to accept 'block:' and 'file:' paths of
pmem and btt devices as shown in the examples below.
numactl --preferred block:pmem0 --show
numactl --preferred file:/dev/pmem0s --show
numactl can be used to bind an application to the locality of
a target NVDIMM for better performance. Here is a result of fio
benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
remote settings.
Local [1] : 4098.3MB/s
Remote [2]: 3718.4MB/s
[1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio
fs-on-pmem0
[2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio
fs-on-pmem0
Did you post the patches to numactl somewhere?
numactl already supports this today.
Yes, numactl supports the following sysfs class lookup for numa_node.
This patchset adds numa_node for NVDIMM devices in the same sysfs format
as described in patch 3/3.
/* Generic sysfs class lookup */
static int
affinity_class(struct bitmask *mask, char *cls, const char *dev)
{
:
ret = sysfs_node_read(mask, /sys/class/%s/%s/device/numa_node,
cls, dev);
Thanks,
-Toshi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Dan Williams dan.j.willi...@intel.com writes: On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer jmo...@redhat.com wrote: Toshi Kani toshi.k...@hp.com writes: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Did you post the patches to numactl somewhere? numactl already supports this today. Ah, I did not know that. I guess I should have RTFM. :) Cheers, Jeff -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer jmo...@redhat.com wrote: Toshi Kani toshi.k...@hp.com writes: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Did you post the patches to numactl somewhere? numactl already supports this today. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
-Original Message-
From: Linux-nvdimm [mailto:linux-nvdimm-boun...@lists.01.org] On Behalf Of
Dan Williams
Sent: Wednesday, June 10, 2015 9:58 AM
To: Jeff Moyer
Cc: linux-nvdimm; Rafael J. Wysocki; linux-kernel@vger.kernel.org; Linux
ACPI
Subject: Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer jmo...@redhat.com wrote:
Toshi Kani toshi.k...@hp.com writes:
Since NVDIMMs are installed on memory slots, they expose the NUMA
topology of a platform. This patchset adds support of sysfs
'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
This enables numactl(8) to accept 'block:' and 'file:' paths of
pmem and btt devices as shown in the examples below.
numactl --preferred block:pmem0 --show
numactl --preferred file:/dev/pmem0s --show
numactl can be used to bind an application to the locality of
a target NVDIMM for better performance. Here is a result of fio
benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
remote settings.
Local [1] : 4098.3MB/s
Remote [2]: 3718.4MB/s
[1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-
on-pmem0
[2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-
on-pmem0
Did you post the patches to numactl somewhere?
numactl already supports this today.
numactl does have a bug handling partitions under these devices,
because it assumes all storage devices have /devices/pci
in their path as it tries to find the parent device for the
partition. I think we'll propose a numactl patch for that;
I don't think the drivers can fool it.
Details (from an earlier version of the patch series
in which btt devices were named /dev/nd1, etc.):
strace shows that numactl is trying to find numa_node in very
different locations for /dev/nd1p1 vs. /dev/sda1.
strace for /dev/nd1p1
=
open(/sys/class/block/nd1p1/dev, O_RDONLY) = 4
read(4, 259:1\n, 4095)= 6
close(4)= 0
close(3)= 0
readlink(/sys/class/block/nd1p1, ../../devices/LNXSYSTM:00/LNXSYB..., 1024)
= 77
open(/sys/class/block/nd1p1/device/numa_node, O_RDONLY) = -1 ENOENT (No such
file or directory)
strace for /dev/sda1
open(/sys/class/block/sda1/dev, O_RDONLY) = 4
read(4, 8:1\n, 4095) = 4
close(4)= 0
close(3)= 0
readlink(/sys/class/block/sda1, ../../devices/pci:00/:00..., 1024)
= 91
open(/sys//devices/pci:00/:00:01.0//numa_node, O_RDONLY) = 3
read(3, 0\n, 4095)= 2
close(3)= 0
The sys/class/block/xxx paths link to:
lrwxrwxrwx. 1 root root 0 May 20 20:42 /sys/class/block/nd1p1 -
../../devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0012:00/ndbus0/btt1/block/nd1/nd1p1
lrwxrwxrwx. 1 root root 0 May 20 20:41 /sys/class/block/sda1 -
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
For /dev/sda1, numactl recognizes /devices/pci as
a special path, and strips off everything after the
numbers. Faced with:
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
it ends up with this (leaving a sloppy // in the path):
/sys/devices/pci:00/:00:01.0//numa_node
It would also succeed if it ended up with this:
/sys/devices/pci:00/:00:01.0/:03:00.0/numa_node
For /dev/nd1p1 it does not see that string, so just
tries to open /sys/class/block/nd1p1/device/numa_node
There are no device/ subdirectories in the tree for
partition devices (for either sda1 or nd1p1), so this
fails.
From http://oss.sgi.com/projects/libnuma/
numactl affinity.c:
/* Somewhat hackish: extract device from symlink path.
Better would be a direct backlink. This knows slightly too
much about the actual sysfs layout. */
char path[1024];
char *fn = NULL;
if (asprintf(fn, /sys/class/%s/%s, cls, dev) 0
readlink(fn, path, sizeof path) 0) {
regex_t re;
regmatch_t match[2];
char *p;
regcomp(re, (/devices/pci[0-9a-fA-F:/]+\\.[0-9]+)/,
REG_EXTENDED);
ret = regexec(re, path, 2, match, 0);
regfree(re);
if (ret == 0) {
free(fn);
assert(match[0].rm_so 0);
assert(match[0].rm_eo 0);
path[match[1].rm_eo + 1] = 0;
p = path + match[0].rm_so;
ret = sysfs_node_read(mask, /sys/%s/numa_node, p);
if (ret 0)
return node_parse_failure(ret, NULL, p);
return ret;
}
}
free(fn);
ret = sysfs_node_read(mask, /sys
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 9:20 AM, Elliott, Robert (Server Storage)
elli...@hp.com wrote:
-Original Message-
From: Linux-nvdimm [mailto:linux-nvdimm-boun...@lists.01.org] On Behalf Of
Dan Williams
Sent: Wednesday, June 10, 2015 9:58 AM
To: Jeff Moyer
Cc: linux-nvdimm; Rafael J. Wysocki; linux-kernel@vger.kernel.org; Linux
ACPI
Subject: Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
On Wed, Jun 10, 2015 at 8:54 AM, Jeff Moyer jmo...@redhat.com wrote:
Toshi Kani toshi.k...@hp.com writes:
Since NVDIMMs are installed on memory slots, they expose the NUMA
topology of a platform. This patchset adds support of sysfs
'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices.
This enables numactl(8) to accept 'block:' and 'file:' paths of
pmem and btt devices as shown in the examples below.
numactl --preferred block:pmem0 --show
numactl --preferred file:/dev/pmem0s --show
numactl can be used to bind an application to the locality of
a target NVDIMM for better performance. Here is a result of fio
benchmark to ext4/dax on an HP DL380 with 2 sockets for local and
remote settings.
Local [1] : 4098.3MB/s
Remote [2]: 3718.4MB/s
[1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-
on-pmem0
[2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-
on-pmem0
Did you post the patches to numactl somewhere?
numactl already supports this today.
numactl does have a bug handling partitions under these devices,
because it assumes all storage devices have /devices/pci
in their path as it tries to find the parent device for the
partition. I think we'll propose a numactl patch for that;
I don't think the drivers can fool it.
Details (from an earlier version of the patch series
in which btt devices were named /dev/nd1, etc.):
strace shows that numactl is trying to find numa_node in very
different locations for /dev/nd1p1 vs. /dev/sda1.
strace for /dev/nd1p1
=
open(/sys/class/block/nd1p1/dev, O_RDONLY) = 4
read(4, 259:1\n, 4095)= 6
close(4)= 0
close(3)= 0
readlink(/sys/class/block/nd1p1, ../../devices/LNXSYSTM:00/LNXSYB...,
1024) = 77
open(/sys/class/block/nd1p1/device/numa_node, O_RDONLY) = -1 ENOENT (No
such file or directory)
strace for /dev/sda1
open(/sys/class/block/sda1/dev, O_RDONLY) = 4
read(4, 8:1\n, 4095) = 4
close(4)= 0
close(3)= 0
readlink(/sys/class/block/sda1, ../../devices/pci:00/:00...,
1024) = 91
open(/sys//devices/pci:00/:00:01.0//numa_node, O_RDONLY) = 3
read(3, 0\n, 4095)= 2
close(3)= 0
The sys/class/block/xxx paths link to:
lrwxrwxrwx. 1 root root 0 May 20 20:42 /sys/class/block/nd1p1 -
../../devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0012:00/ndbus0/btt1/block/nd1/nd1p1
lrwxrwxrwx. 1 root root 0 May 20 20:41 /sys/class/block/sda1 -
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
For /dev/sda1, numactl recognizes /devices/pci as
a special path, and strips off everything after the
numbers. Faced with:
../../devices/pci:00/:00:01.0/:03:00.0/host6/target6:0:0/6:0:0:0/block/sda/sda1
it ends up with this (leaving a sloppy // in the path):
/sys/devices/pci:00/:00:01.0//numa_node
It would also succeed if it ended up with this:
/sys/devices/pci:00/:00:01.0/:03:00.0/numa_node
For /dev/nd1p1 it does not see that string, so just
tries to open /sys/class/block/nd1p1/device/numa_node
There are no device/ subdirectories in the tree for
partition devices (for either sda1 or nd1p1), so this
fails.
From http://oss.sgi.com/projects/libnuma/
numactl affinity.c:
/* Somewhat hackish: extract device from symlink path.
Better would be a direct backlink. This knows slightly too
much about the actual sysfs layout. */
char path[1024];
char *fn = NULL;
if (asprintf(fn, /sys/class/%s/%s, cls, dev) 0
readlink(fn, path, sizeof path) 0) {
regex_t re;
regmatch_t match[2];
char *p;
regcomp(re, (/devices/pci[0-9a-fA-F:/]+\\.[0-9]+)/,
REG_EXTENDED);
ret = regexec(re, path, 2, match, 0);
regfree(re);
if (ret == 0) {
free(fn);
assert(match[0].rm_so 0);
assert(match[0].rm_eo 0);
path[match[1].rm_eo + 1] = 0;
p = path + match[0].rm_so;
ret = sysfs_node_read(mask, /sys/%s/numa_node, p);
if (ret 0)
return
Re: [PATCH v2 0/3] Add NUMA support for NVDIMM devices
Toshi Kani toshi.k...@hp.com writes: Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Did you post the patches to numactl somewhere? -Jeff -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v2 0/3] Add NUMA support for NVDIMM devices
Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio Patch 1/3 applies on top of the acpica branch of the pm tree. Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of "libnvdimm: non-volatile memory devices". --- v2: - Add acpi_map_pxm_to_online_node(), which returns an online node. - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) - Check ACPI_NFIT_PROXIMITY_VALID in spa->flags. --- Toshi Kani (3): 1/3 acpi: Add acpi_map_pxm_to_online_node() 2/3 libnvdimm: Set numa_node to NVDIMM devices 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices --- drivers/acpi/nfit.c | 7 +++ drivers/acpi/numa.c | 40 +--- drivers/nvdimm/btt.c| 2 ++ drivers/nvdimm/btt_devs.c | 1 + drivers/nvdimm/bus.c| 30 ++ drivers/nvdimm/namespace_devs.c | 1 + drivers/nvdimm/nd.h | 1 + drivers/nvdimm/region.c | 1 + drivers/nvdimm/region_devs.c| 1 + include/linux/acpi.h| 5 + include/linux/libnvdimm.h | 2 ++ 11 files changed, 88 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v2 0/3] Add NUMA support for NVDIMM devices
Since NVDIMMs are installed on memory slots, they expose the NUMA topology of a platform. This patchset adds support of sysfs 'numa_node' to I/O-related NVDIMM devices under /sys/bus/nd/devices. This enables numactl(8) to accept 'block:' and 'file:' paths of pmem and btt devices as shown in the examples below. numactl --preferred block:pmem0 --show numactl --preferred file:/dev/pmem0s --show numactl can be used to bind an application to the locality of a target NVDIMM for better performance. Here is a result of fio benchmark to ext4/dax on an HP DL380 with 2 sockets for local and remote settings. Local [1] : 4098.3MB/s Remote [2]: 3718.4MB/s [1] numactl --preferred block:pmem0 --cpunodebind block:pmem0 fio fs-on-pmem0 [2] numactl --preferred block:pmem1 --cpunodebind block:pmem1 fio fs-on-pmem0 Patch 1/3 applies on top of the acpica branch of the pm tree. Patch 2/3-3/3 apply on top of Dan Williams's v5 patch series of libnvdimm: non-volatile memory devices. --- v2: - Add acpi_map_pxm_to_online_node(), which returns an online node. - Manage visibility of sysfs numa_node with is_visible. (Dan Williams) - Check ACPI_NFIT_PROXIMITY_VALID in spa-flags. --- Toshi Kani (3): 1/3 acpi: Add acpi_map_pxm_to_online_node() 2/3 libnvdimm: Set numa_node to NVDIMM devices 3/3 libnvdimm: Add sysfs numa_node to NVDIMM devices --- drivers/acpi/nfit.c | 7 +++ drivers/acpi/numa.c | 40 +--- drivers/nvdimm/btt.c| 2 ++ drivers/nvdimm/btt_devs.c | 1 + drivers/nvdimm/bus.c| 30 ++ drivers/nvdimm/namespace_devs.c | 1 + drivers/nvdimm/nd.h | 1 + drivers/nvdimm/region.c | 1 + drivers/nvdimm/region_devs.c| 1 + include/linux/acpi.h| 5 + include/linux/libnvdimm.h | 2 ++ 11 files changed, 88 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
