Re: [PATCH RFC 01/10] mm: Add pmd support for _PAGE_SPECIAL

2020-02-04 Thread Joao Martins
On 2/3/20 9:34 PM, Matthew Wilcox wrote: > On Fri, Jan 10, 2020 at 07:03:04PM +0000, Joao Martins wrote: >> +++ b/arch/x86/include/asm/pgtable.h >> @@ -293,6 +293,15 @@ static inline int pgd_devmap(pgd_t pgd) >> { >> return 0; >> } >> +#endif >

Re: [PATCH RFC 00/10] device-dax: Support devices without PFN metadata

2020-02-04 Thread Joao Martins
On 2/4/20 1:24 AM, Dan Williams wrote: > On Fri, Jan 10, 2020 at 11:06 AM Joao Martins > wrote: >> >> Hey, > > Hi Joao, > >> >> Presented herewith a small series which allows device-dax to work without >> struct page to be used to back KVM guests m

Re: [PATCH RFC 02/10] mm: Handle pmd entries in follow_pfn()

2020-02-04 Thread Joao Martins
On 2/3/20 9:37 PM, Matthew Wilcox wrote: > On Fri, Jan 10, 2020 at 07:03:05PM +0000, Joao Martins wrote: >> @@ -4366,6 +4366,7 @@ EXPORT_SYMBOL(follow_pte_pmd); >> int follow_pfn(struct vm_area_struct *vma, unsigned long address, >> unsigned long *pfn) >> {

Re: [PATCH RFC 10/10] nvdimm/e820: add multiple namespaces support

2020-02-04 Thread Joao Martins
On 2/4/20 6:20 PM, Barret Rhoden wrote: > Hi - > > On 2/4/20 11:44 AM, Dan Williams wrote: >> On Tue, Feb 4, 2020 at 7:30 AM Barret Rhoden wrote: >>> >>> Hi - >>> >>> On 1/10/20 2:03 PM, Joao Martins wrote: >>>> User can define re

Re: [PATCH RFC 09/10] vfio/type1: Use follow_pfn for VM_FPNMAP VMAs

2020-02-11 Thread Joao Martins
On 2/7/20 9:08 PM, Jason Gunthorpe wrote: > On Fri, Jan 10, 2020 at 07:03:12PM +0000, Joao Martins wrote: >> From: Nikita Leshenko >> >> Unconditionally interpreting vm_pgoff as a PFN is incorrect. >> >> VMAs created by /dev/mem do this, but in general VM_PFNMAP j

[PATCH RFC 10/10] nvdimm/e820: add multiple namespaces support

2020-01-10 Thread Joao Martins
. The config data get/set/size operations are then simply memcpying to this area. Equivalent approach can also be found in the NFIT tests which emulate the same thing. Signed-off-by: Joao Martins --- drivers/nvdimm/e820.c | 212 +- 1 file changed, 191

[PATCH RFC 03/10] mm: Add pud support for _PAGE_SPECIAL

2020-01-10 Thread Joao Martins
on gup_pte_range() and for pmds on gup_huge_pmd(). This allows device-dax to handle 1G hugepages without struct pages. Signed-off-by: Joao Martins --- arch/x86/include/asm/pgtable.h | 18 +- mm/gup.c | 3 +++ mm/huge_memory.c | 8 +--- mm/memory.c

[PATCH RFC 02/10] mm: Handle pmd entries in follow_pfn()

2020-01-10 Thread Joao Martins
pfns on VM_PFNMAP vmas. Signed-off-by: Joao Martins --- mm/memory.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index cfc3668bddeb..db99684d2cb3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4366,6 +4366,7 @@ EXPORT_SYMBOL(follow_pte_pmd

[PATCH RFC 04/10] mm: Handle pud entries in follow_pfn()

2020-01-10 Thread Joao Martins
-developed-by: Nikita Leshenko Signed-off-by: Nikita Leshenko Signed-off-by: Joao Martins --- mm/memory.c | 58 - 1 file changed, 49 insertions(+), 9 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 109643219e1b..f46646630497 100644 --- a/mm

[PATCH RFC 07/10] device-dax: Add support for PFN_SPECIAL flags

2020-01-10 Thread Joao Martins
not assuming callers will pass a dev_pagemap, and avoid returning SIGBUS for the lack of PFN_MAP region pfn flag and finally not setting struct page index/mapping on fault. Signed-off-by: Joao Martins --- drivers/dax/bus.c| 3 ++- drivers/dax/device.c | 40 ++-- 2

[PATCH RFC 00/10] device-dax: Support devices without PFN metadata

2020-01-10 Thread Joao Martins
AX vma. * NUMA: For now excluded setting the target_node; while these two patches are being worked on[1][2]. [1] https://lore.kernel.org/lkml/157401276776.43284.12396353118982684546.st...@dwillia2-desk3.amr.corp.intel.com/ [2] https://lore.kernel.org/lkml/157401277293.43284.3805106435228534675

[PATCH RFC 05/10] device-dax: Do not enforce MADV_DONTFORK on mmap()

2020-01-10 Thread Joao Martins
VM_DONTCOPY prevents a process from mmap-ing the device. Let's not enforce MADV_DONTFORK at mmap(), but rather when it actually gets used (on fault). The assumptions don't change, as it is expected to still retain/madvise MADV_DONTFORK after mmap. Signed-off-by: Joao Martins --- drivers/dax/device.c

[PATCH RFC 09/10] vfio/type1: Use follow_pfn for VM_FPNMAP VMAs

2020-01-10 Thread Joao Martins
From: Nikita Leshenko Unconditionally interpreting vm_pgoff as a PFN is incorrect. VMAs created by /dev/mem do this, but in general VM_PFNMAP just means that the VMA doesn't have an associated struct page and is being managed directly by something other than the core mmu. Use follow_pfn like

[PATCH RFC 01/10] mm: Add pmd support for _PAGE_SPECIAL

2020-01-10 Thread Joao Martins
(). This allows a DAX driver to handle 2M hugepages without struct pages. Signed-off-by: Joao Martins --- arch/x86/include/asm/pgtable.h | 16 +++- mm/gup.c | 3 +++ mm/huge_memory.c | 7 --- mm/memory.c| 3 ++- 4 files

[PATCH RFC 06/10] device-dax: Introduce pfn_flags helper

2020-01-10 Thread Joao Martins
Replace PFN_DEV|PFN_MAP check call sites with two helper functions dax_is_pfn_dev() and dax_is_pfn_map(). Signed-off-by: Joao Martins --- drivers/dax/device.c | 18 ++ 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c

[PATCH RFC 08/10] dax/pmem: Add device-dax support for PFN_MODE_NONE

2020-01-10 Thread Joao Martins
pfn superblock for the pagemap/struct-pages. We only allocate an opaque zeroed object with the chosen align requested, and finally add PFN_SPECIAL to the region pfn_flags. Signed-off-by: Joao Martins --- drivers/dax/pmem/core.c | 36 ++-- 1 file changed, 30 inserti

Re: [PATCH 11/12] device-dax: Add dis-contiguous resource support

2020-04-06 Thread Joao Martins
On 3/23/20 11:55 PM, Dan Williams wrote: [...] > static ssize_t dev_dax_resize(struct dax_region *dax_region, > struct dev_dax *dev_dax, resource_size_t size) > { > resource_size_t avail = dax_region_avail_size(dax_region), to_alloc; > - resource_size_t dev_size =

Re: [PATCH 11/12] device-dax: Add dis-contiguous resource support

2020-03-26 Thread Joao Martins
On 3/25/20 5:48 PM, Dan Williams wrote: > On Wed, Mar 25, 2020 at 3:35 AM Joao Martins > wrote: >> On 3/24/20 4:12 PM, Joao Martins wrote: >>> On 3/23/20 11:55 PM, Dan Williams wrote: >>>> static ssize_t dev_dax_resize(struct dax_region *dax_region, >>

[PATCH ndctl v1 03/10] daxctl: add resize support in reconfigure-device

2020-04-03 Thread Joao Martins
reconfigured 1 device $ daxctl reconfigure-device -s 4G dax0.0 reconfigured 1 device @size (-s) and @mode (-m) are mutually exclusive as the latter relates to assigning memory to System-RAM through kmem as opposed to reconfiguring dynamic dax devices. Signed-off-by: Joao Martins

[PATCH ndctl v1 05/10] daxctl: add command to enable devdax device

2020-04-03 Thread Joao Martins
Add a 'enable-device' command, required prior to reconfiguration of the dax device. Mimics the same functionality as seen in ndctl-enable-namespace. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 40

[PATCH ndctl v1 00/10] daxctl: Support for sub-dividing soft-reserved regions

2020-04-03 Thread Joao Martins
n Williams (1): daxctl: Cleanup whitespace Joao Martins (9): libdaxctl: add daxctl_dev_set_size() daxctl: add resize support in reconfigure-device daxctl: add command to disable devdax device daxctl: add command to enable devdax device libdaxctl: add daxctl_region_create_dev() d

[PATCH ndctl v1 01/10] daxctl: Cleanup whitespace

2020-04-03 Thread Joao Martins
From: Dan Williams Re-indent util_daxctl_region_filter() to match expectations. Signed-off-by: Dan Williams --- util/filter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/filter.c b/util/filter.c index af72793929e2..7d0159f8cef6 100644 --- a/util/filter.c +++

[PATCH ndctl v1 04/10] daxctl: add command to disable devdax device

2020-04-03 Thread Joao Martins
Add a 'disable-device' command, required prior to reconfiguration or destruction of the dax device. Mimmics the same functionality as seen in ndctl-disable-namespace. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c

[PATCH ndctl v1 06/10] libdaxctl: add daxctl_region_create_dev()

2020-04-03 Thread Joao Martins
Add an API to create an empty seed device. This sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 26 ++ daxctl/lib/libdaxctl.sym | 1

[PATCH ndctl v1 02/10] libdaxctl: add daxctl_dev_set_size()

2020-04-03 Thread Joao Martins
Add an API for setting the size of a dax device. This sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 24 daxctl/lib/libdaxctl.sym | 5

[PATCH ndctl v1 10/10] daxctl/test: Add tests for dynamic dax regions

2020-04-03 Thread Joao Martins
beginning and end The tests assume you pass a valid efi_fake_mem parameter marked as EFI_MEMORY_SP e.g. efi_fake_mem=112G@16G:0x4 Naturally it bails out from the test if hmem device driver isn't loaded/builtin or no region is found. Signed-off-by: Joao Martins --- test/Makefile.am

[PATCH ndctl v1 07/10] daxctl: add command to create device

2020-04-03 Thread Joao Martins
r or ndctl-create-namespace: $ daxctl create-device [ { "chardev":"dax0.1", "size":120259084288, "target_node":0, "mode":"devdax" } ] created 1

[PATCH ndctl v1 09/10] daxctl: add command to destroy device

2020-04-03 Thread Joao Martins
-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 66 3 files changed, 68 insertions(+) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 19b33933b91b..29ba63ca17aa 100644

[PATCH ndctl v1 08/10] libdaxctl: add daxctl_region_destroy_dev()

2020-04-03 Thread Joao Martins
Add an API to destroy an empty device. This 'delete' sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 17 + daxctl/lib/libdaxctl.sym | 1

Re: [PATCH 11/12] device-dax: Add dis-contiguous resource support

2020-03-25 Thread Joao Martins
On 3/24/20 4:12 PM, Joao Martins wrote: > On 3/23/20 11:55 PM, Dan Williams wrote: >> static ssize_t dev_dax_resize(struct dax_region *dax_region, >> struct dev_dax *dev_dax, resource_size_t size) >> { >> resource_size_t avail = dax_region_avail

Re: [PATCH 11/12] device-dax: Add dis-contiguous resource support

2020-05-12 Thread Joao Martins
On 3/23/20 11:55 PM, Dan Williams wrote: > @@ -561,13 +580,26 @@ static int __alloc_dev_dax_range(struct dev_dax > *dev_dax, u64 start, > if (start == U64_MAX) > return -EINVAL; > > + ranges = krealloc(dev_dax->ranges, sizeof(*ranges) > + *

Re: [PATCH 11/12] device-dax: Add dis-contiguous resource support

2020-03-24 Thread Joao Martins
On 3/23/20 11:55 PM, Dan Williams wrote: > static ssize_t dev_dax_resize(struct dax_region *dax_region, > struct dev_dax *dev_dax, resource_size_t size) > { > resource_size_t avail = dax_region_avail_size(dax_region), to_alloc; > - resource_size_t dev_size =

Re: [PATCH v2 4/6] ACPI: HMAT: Refactor hmat_register_target_device to hmem_register_device

2020-03-24 Thread Joao Martins
devices(struct > memory_target *target) > if (!IS_ENABLED(CONFIG_DEV_DAX_HMEM)) > return; > > - for (res = target->memregions.child; res; res = res->sibling) > - hmat_register_target_device(target, res); > + for (res = target->memreg

Re: [PATCH v2 6/6] ACPI: HMAT: Attach a device for each soft-reserved range

2020-03-24 Thread Joao Martins
in hmem_register_one(). > +} > + > +static __init int hmem_init(void) > +{ > + walk_iomem_res_desc(IORES_DESC_SOFT_RESERVED, > + IORESOURCE_MEM, 0, -1, NULL, hmem_register_one); > + return 0; > +} > + (...) and then perhaps here returning in the in

Re: [PATCH v2 6/6] ACPI: HMAT: Attach a device for each soft-reserved range

2020-03-24 Thread Joao Martins
On 3/24/20 9:06 PM, Dan Williams wrote: > On Tue, Mar 24, 2020 at 12:41 PM Joao Martins > wrote: >> >> On 3/22/20 4:12 PM, Dan Williams wrote: >>> The hmem enabling in commit 'cf8741ac57ed ("ACPI: NUMA: HMAT: Register >>> "soft reserved" m

Re: [PATCH v4 11/23] device-dax: Kill dax_kmem_res

2020-09-08 Thread Joao Martins
[Sorry for the late response] On 8/21/20 11:06 AM, David Hildenbrand wrote: > On 03.08.20 07:03, Dan Williams wrote: >> @@ -37,109 +45,94 @@ int dev_dax_kmem_probe(struct device *dev) >> * could be mixed in a node with faster memory, causing >> * unavoidable performance issues. >>

Re: [bug report] device-dax: add dis-contiguous resource support

2020-08-31 Thread Joao Martins
On 8/31/20 12:32 PM, Dan Carpenter wrote: > Hello Dan Williams, > > This is a semi-automatic email about new static checker warnings. > > The patch 454c727769f5: "device-dax: add dis-contiguous resource > support" from Aug 26, 2020, leads to the following Smatch complaint: > >

Re: [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges

2020-09-25 Thread Joao Martins
Hey Dan, On 9/25/20 8:11 PM, Dan Williams wrote: > Changes since v4 [1]: > - Rebased on > device-dax-move-instance-creation-parameters-to-struct-dev_dax_data.patch > in -mm [2]. I.e. patches that did not need fixups from v4 are not > included. > > - Folded all fixes > Hmm, perhaps you

Re: [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges

2020-09-25 Thread Joao Martins
On 9/25/20 10:01 PM, Dan Williams wrote: > On Fri, Sep 25, 2020 at 1:52 PM Joao Martins > wrote: >> >> Hey Dan, >> >> On 9/25/20 8:11 PM, Dan Williams wrote: >>> Changes since v4 [1]: >>> - Rebased on >>> device-dax-move-instance-creatio

Re: [PATCH ndctl v2 10/10] daxctl/test: Add tests for dynamic dax regions

2020-07-21 Thread Joao Martins
On 7/13/20 5:08 PM, Joao Martins wrote: > Add a couple tests which exercise the new sysfs based > interface for Soft-Reserved regions (by EFI/HMAT, or > efi_fake_mem). > > The tests exercise the daxctl orchestration surrounding > for creating/disabling/destroying/rec

Re: [PATCH ndctl v1 10/10] daxctl/test: Add tests for dynamic dax regions

2020-07-13 Thread Joao Martins
On 7/12/20 4:04 PM, Dan Williams wrote: > On Fri, Apr 3, 2020 at 1:59 PM Joao Martins wrote: >> >> Add a couple tests which exercise the new sysfs based >> interface for Soft-Reserved regions (by EFI/HMAT, or >> efi_fake_mem). >> >> The tests exerci

[PATCH ndctl v2 02/10] libdaxctl: add daxctl_dev_set_size()

2020-07-13 Thread Joao Martins
Add an API for setting the size of a dax device. This sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 24 daxctl/lib/libdaxctl.sym | 5

[PATCH ndctl v2 08/10] libdaxctl: add daxctl_region_destroy_dev()

2020-07-13 Thread Joao Martins
Add an API to destroy an empty device. This 'delete' sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 17 + daxctl/lib/libdaxctl.sym | 1

[PATCH ndctl v2 06/10] libdaxctl: add daxctl_region_create_dev()

2020-07-13 Thread Joao Martins
Add an API to create an empty seed device. This sysfs attribute is only writeable for a dynamic dax device such as the one exported by hmem. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- daxctl/lib/libdaxctl.c | 26 ++ daxctl/lib/libdaxctl.sym | 1

[PATCH ndctl v2 09/10] daxctl: add command to destroy device

2020-07-13 Thread Joao Martins
-by: Joao Martins Signed-off-by: Dan Williams --- Documentation/daxctl/Makefile.am | 3 +- Documentation/daxctl/daxctl-destroy-device.txt | 63 daxctl/builtin.h | 1 + daxctl/daxctl.c| 1 + daxctl

[PATCH ndctl v2 07/10] daxctl: add command to create device

2020-07-13 Thread Joao Martins
r or ndctl-create-namespace: $ daxctl create-device [ { "chardev":"dax0.1", "size":120259084288, "target_node":0, "mode":"devdax" } ]

[PATCH ndctl v2 00/10] daxctl: Support for sub-dividing soft-reserved regions

2020-07-13 Thread Joao Martins
lia2-desk3.amr.corp.intel.com/ Dan Williams (1): daxctl: Cleanup whitespace Joao Martins (9): libdaxctl: add daxctl_dev_set_size() daxctl: add resize support in reconfigure-device daxctl: add command to disable devdax device daxctl: add command to enable devdax device libdaxctl: add daxctl_region_

[PATCH ndctl v2 04/10] daxctl: add command to disable devdax device

2020-07-13 Thread Joao Martins
Add a 'disable-device' command, required prior to reconfiguration or destruction of the dax device. Mimmics the same functionality as seen in ndctl-disable-namespace. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- Documentation/daxctl/Makefile.am | 3

[PATCH ndctl v2 05/10] daxctl: add command to enable devdax device

2020-07-13 Thread Joao Martins
Add a 'enable-device' command, required prior to reconfiguration of the dax device. Mimics the same functionality as seen in ndctl-enable-namespace. Signed-off-by: Joao Martins Signed-off-by: Dan Williams --- Documentation/daxctl/Makefile.am | 3 +- Documentation/daxctl/daxctl

[PATCH ndctl v2 03/10] daxctl: add resize support in reconfigure-device

2020-07-13 Thread Joao Martins
reconfigured 1 device $ daxctl reconfigure-device -s 4G dax0.0 reconfigured 1 device @size (-s) and @mode (-m) are mutually exclusive as the latter relates to assigning memory to System-RAM through kmem as opposed to reconfiguring dynamic dax devices. Signed-off-by: Joao Martins

[PATCH ndctl v2 01/10] daxctl: Cleanup whitespace

2020-07-13 Thread Joao Martins
From: Dan Williams Re-indent util_daxctl_region_filter() to match expectations. Signed-off-by: Dan Williams --- util/filter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/filter.c b/util/filter.c index af72793929e2..7d0159f8cef6 100644 --- a/util/filter.c +++

[PATCH ndctl v2 10/10] daxctl/test: Add tests for dynamic dax regions

2020-07-13 Thread Joao Martins
untouched. Signed-off-by: Joao Martins --- test/Makefile.am | 1 + test/daxctl-create.sh | 294 ++ 2 files changed, 295 insertions(+) create mode 100755 test/daxctl-create.sh diff --git a/test/Makefile.am b/test/Makefile.am index 1d24a65ded8b

Re: [PATCH v2 22/22] device-dax: Introduce 'mapping' devices

2020-07-16 Thread Joao Martins
On 7/12/20 5:28 PM, Dan Williams wrote: > In support of interrogating the physical address layout of a device with > dis-contiguous ranges, introduce a sysfs directory with 'start', 'end', > and 'page_offset' attributes. The alternative is trying to parse > /proc/iomem, and that file will not

[PATCH v1 1/4] device-dax: Make align a per-device property

2020-07-16 Thread Joao Martins
Introduce @align to struct dev_dax. When creating a new device, we still initialize to the default dax_region @align. Child devices belonging to a region may wish to keep a different alignment property instead of a global region-defined one. Signed-off-by: Joao Martins --- drivers/dax/bus.c

[PATCH ndctl v1 5/8] libdaxctl: add mapping iterator APIs

2020-07-16 Thread Joao Martins
Add the following APIs to be able to iterate over a dynamic device-dax mapping list, as well as fetching each of the mapping attributes. Signed-off-by: Joao Martins --- daxctl/lib/libdaxctl-private.h | 8 daxctl/lib/libdaxctl.c | 91

[PATCH ndctl v1 7/8] libdaxctl: add daxctl_dev_set_mapping()

2020-07-16 Thread Joao Martins
. It also allows an userspace application to pick it's own ranges, should it want to avoid relying on kernel's policy. Signed-off-by: Joao Martins --- daxctl/lib/libdaxctl.c | 27 +++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 2 ++ 3 files changed, 30

[PATCH ndctl v1 6/8] daxctl: include mappings when listing

2020-07-16 Thread Joao Martins
Iterate over the device mappings and print @page_offset, @start, @end and a computed size. Signed-off-by: Joao Martins --- Perhaps hiding it behind a -v|--verbose? --- util/json.c | 54 +- util/json.h | 3 +++ 2 files changed, 56 insertions

[PATCH ndctl v1 8/8] daxctl: Allow restore devices from JSON metadata

2020-07-16 Thread Joao Martins
"start":"0x6", "end":"0x9", "size":"16.00 GiB (17.18 GB)" }, { "page_offset":"0", "start":"0x2", "end":&qu

[PATCH v1 2/4] device-dax: Add an 'align' attribute

2020-07-16 Thread Joao Martins
to 1G. So, we validate all ranges against the new value being attempted, post resizing. Signed-off-by: Joao Martins --- drivers/dax/bus.c | 101 +- 1 file changed, 92 insertions(+), 9 deletions(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c

[PATCH v1 3/4] dax/hmem: Introduce dax_hmem.region_idle parameter

2020-07-16 Thread Joao Martins
Introduce a new module parameter for dax_hmem which initializes all region devices as free, rather than allocating a pagemap for the region by default. All hmem devices created with dax_hmem.region_idle=1 will have full available size for creating dynamic dax devices. Signed-off-by: Joao Martins

[PATCH ndctl v1 3/8] daxctl: add align support in reconfigure-device

2020-07-16 Thread Joao Martins
Add an alignment option to reconfigure-device and use the newly added libdaxctl API to set the alignment of the device prior to setting size. Signed-off-by: Joao Martins --- daxctl/device.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/daxctl/device.c

[PATCH ndctl v1 1/8] daxctl: add daxctl_dev_{get,set}_align()

2020-07-16 Thread Joao Martins
Add support for changing devices alignment. Signed-off-by: Joao Martins --- daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 34 ++ daxctl/lib/libdaxctl.sym | 2 ++ daxctl/libdaxctl.h | 2 ++ 4 files changed, 39

[PATCH ndctl v1 0/8] daxctl: Add device align and range mapping allocation

2020-07-16 Thread Joao Martins
elcome. Joao Joao Martins (8): daxctl: add daxctl_dev_{get,set}_align() util/json: Print device align daxctl: add align support in reconfigure-device daxctl: add align support in create-device libdaxctl: add mapping iterator APIs daxctl: include mappings when listing libdaxct

[PATCH ndctl v1 2/8] util/json: Print device align

2020-07-16 Thread Joao Martins
Fetch device align and include it on listings. Signed-off-by: Joao Martins --- util/json.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/util/json.c b/util/json.c index 21ab25674624..4d9787381d6b 100644 --- a/util/json.c +++ b/util/json.c @@ -278,7 +278,7 @@ struct

[PATCH ndctl v1 4/8] daxctl: add align support in create-device

2020-07-16 Thread Joao Martins
And thus allow changing devices alignment when creating a new child device. Signed-off-by: Joao Martins --- daxctl/device.c | 8 1 file changed, 8 insertions(+) diff --git a/daxctl/device.c b/daxctl/device.c index 9d82ea12aca2..3a844462829b 100644 --- a/daxctl/device.c +++ b/daxctl

Re: [PATCH v2 22/22] device-dax: Introduce 'mapping' devices

2020-07-16 Thread Joao Martins
On 7/16/20 5:00 PM, Dan Williams wrote: > On Thu, Jul 16, 2020 at 6:19 AM Joao Martins > wrote: >> On 7/12/20 5:28 PM, Dan Williams wrote: >>> In support of interrogating the physical address layout of a device with >>> dis-contiguous ranges, introduce a sysf

[PATCH v1 4/4] device-dax: Add a range mapping allocation attribute

2020-07-16 Thread Joao Martins
] where after kexec we want to the same gpa<->phys mappings (as originally created before kexec). [0] https://static.sched.com/hosted_files/kvmforum2019/66/VMM-fast-restart_kvmforum2019.pdf Signed-off-by: Joao Martins --- drivers/dax/bus.c | 64

[PATCH v1 0/4] device-dax: Further improvements to subdivision

2020-07-16 Thread Joao Martins
tps://lore.kernel.org/linux-mm/159457116473.754248.7879464730875147365.st...@dwillia2-desk3.amr.corp.intel.com Joao Joao Martins (4): device-dax: Make align a per-device property device-dax: Add an 'align' attribute dax/hmem: Introduce dax_hmem.region_idle parameter device-dax: Add a range mapp

[PATCH RFC 1/9] memremap: add ZONE_DEVICE support for compound pages

2020-12-08 Thread Joao Martins
-by: Joao Martins --- include/linux/memremap.h | 2 ++ mm/memremap.c| 8 ++-- mm/page_alloc.c | 7 +++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 79c49e7f5c30..f8f26b2cc3da 100644 --- a/include

[PATCH RFC 0/9] mm, sparse-vmemmap: Introduce compound pagemaps

2020-12-08 Thread Joao Martins
1208 (commit a9e26cb5f261). Comments and suggestions very much appreciated! Thanks, Joao Joao Martins (9): memremap: add ZONE_DEVICE support for compound pages sparse-vmemmap: Consolidate arguments in vmemmap section populate sparse-vmemmap: Reuse vmemmap areas for a given page size

[PATCH RFC 3/9] sparse-vmemmap: Reuse vmemmap areas for a given page size

2020-12-08 Thread Joao Martins
. Signed-off-by: Joao Martins --- I wonder, rather than separating vmem_context and mhp_params, that one would just pick the latter. Albeit semantically the ctx aren't necessarily paramters, context passed from multiple sections onlining (i.e. multiple calls to populate_section_memmap). Also provided

[PATCH RFC 9/9] mm: Add follow_devmap_page() for devdax vmas

2020-12-08 Thread Joao Martins
s -> ~19k us Signed-off-by: Joao Martins --- I've special-cased this to device-dax vmas given its similar page size guarantees as hugetlbfs, but I feel this is a bit wrong. I am replicating follow_hugetlb_page() as RFC ought to seek feedback whether this should be generalized if no fundamental

[PATCH RFC 2/9] sparse-vmemmap: Consolidate arguments in vmemmap section populate

2020-12-08 Thread Joao Martins
Replace vmem_altmap with an vmem_context argument. That let us express how the vmemmap is gonna be initialized e.g. passing flags and a page size for reusing pages upon initializing the vmemmap. Signed-off-by: Joao Martins --- include/linux/memory_hotplug.h | 6 +- include/linux/mm.h

[PATCH RFC 4/9] mm/page_alloc: Reuse tail struct pages for compound pagemaps

2020-12-08 Thread Joao Martins
or 262144 for a 1G @align. Signed-off-by: Joao Martins --- mm/memremap.c | 4 +++- mm/page_alloc.c | 23 --- 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/mm/memremap.c b/mm/memremap.c index ecfa74848ac6..3eca07916b9d 100644 --- a/mm/memremap.c +++ b/mm

[PATCH RFC 3/9] sparse-vmemmap: Reuse vmemmap areas for a given mhp_params::align

2020-12-08 Thread Joao Martins
. Signed-off-by: Joao Martins --- I wonder, rather than separating vmem_context and mhp_params, that one would just pick the latter. Albeit semantically the ctx aren't necessarily paramters, context passed from multiple sections onlining (i.e. multiple calls to populate_section_memmap). Also

[PATCH RFC 5/9] device-dax: Compound pagemap support

2020-12-08 Thread Joao Martins
less pages. On emulated NVDIMM guests this can be easily seen, e.g. on a setup with an emulated NVDIMM with 128G in size seeing improvements from ~750ms to ~190ms with 2M pages, and to less than a 1msec with 1G pages. Signed-off-by: Joao Martins --- Probably deserves its own sysfs attribute

[PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages

2020-12-08 Thread Joao Martins
get_user_pages_fast() and pin_user_pages_fast(): $ gup_benchmark -f /dev/dax0.2 -m 16384 -r 10 -S [-u,-a] -n 512 -w (get_user_pages_fast 2M pages) ~75k us -> ~3.6k us (pin_user_pages_fast 2M pages) ~125k us -> ~3.8k us Signed-off-by: Joao Martins --- mm/gup.

[PATCH RFC 7/9] mm/gup: Decrement head page once for group of subpages

2020-12-08 Thread Joao Martins
Rather than decrementing the ref count one by one, we walk the page array and checking which belong to the same compound_head. Later on we decrement the calculated amount of references in a single write to the head page. Signed-off-by: Joao Martins --- mm/gup.c | 41

[PATCH RFC 8/9] RDMA/umem: batch page unpin in __ib_mem_release()

2020-12-08 Thread Joao Martins
sec: 10748.456 usec / round (hugetlbfs) After: 305 rounds in 5.010 sec: 16426.047 usec / round (device-dax) 1073 rounds in 5.004 sec: 4663.622 usec / round (hugetlbfs) We also see similar improvements on a setup with pmem and RDMA hardware. Signed-off-by: Joao Martins --- drivers/infiniband

Re: [PATCH RFC 3/9] sparse-vmemmap: Reuse vmemmap areas for a given mhp_params::align

2020-12-08 Thread Joao Martins
On 12/8/20 5:28 PM, Joao Martins wrote: > Introduce a new flag, MEMHP_REUSE_VMEMMAP, which signals that that > struct pages are onlined with a given alignment, and should reuse the > tail pages vmemmap areas. On that circunstamce we reuse the PFN backing > only the tail pages subsec

Re: [PATCH RFC 2/9] sparse-vmemmap: Consolidate arguments in vmemmap section populate

2020-12-09 Thread Joao Martins
On 12/9/20 6:16 AM, John Hubbard wrote: > On 12/8/20 9:28 AM, Joao Martins wrote: >> Replace vmem_altmap with an vmem_context argument. That let us >> express how the vmemmap is gonna be initialized e.g. passing >> flags and a page size for reusing pages upon initia

Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages

2020-12-09 Thread Joao Martins
On 12/9/20 4:40 AM, John Hubbard wrote: > On 12/8/20 9:28 AM, Joao Martins wrote: >> Much like hugetlbfs or THPs, we treat device pagemaps with >> compound pages like the rest of GUP handling of compound pages. >> >> Rather than incrementing the refcount every 4K,

Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages

2020-12-09 Thread Joao Martins
On 12/9/20 4:24 PM, Jason Gunthorpe wrote: > On Wed, Dec 09, 2020 at 04:02:05PM +0000, Joao Martins wrote: > >> Today (without the series) struct pages are not represented the way they >> are expressed in the page tables, which is what I am hoping to fix in this >>

Re: [PATCH RFC 1/9] memremap: add ZONE_DEVICE support for compound pages

2020-12-09 Thread Joao Martins
On 12/9/20 6:33 AM, Matthew Wilcox wrote: > On Tue, Dec 08, 2020 at 09:59:19PM -0800, John Hubbard wrote: >> On 12/8/20 9:28 AM, Joao Martins wrote: >>> Add a new flag for struct dev_pagemap which designates that a a pagemap >> >> a a >> Ugh. Yeah will fix. &

Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages

2020-12-09 Thread Joao Martins
On 12/9/20 3:15 PM, Jason Gunthorpe wrote: > On Wed, Dec 09, 2020 at 11:05:39AM +0000, Joao Martins wrote: >>> Why is all of this special? Any time we see a PMD/PGD/etc pointing to >>> PFN we can apply this optimization. How come device has its own >>> special pa

Re: [PATCH RFC 7/9] mm/gup: Decrement head page once for group of subpages

2020-12-09 Thread Joao Martins
On 12/8/20 7:34 PM, Jason Gunthorpe wrote: > On Tue, Dec 08, 2020 at 05:28:59PM +0000, Joao Martins wrote: >> Rather than decrementing the ref count one by one, we >> walk the page array and checking which belong to the same >> compound_head. Later on we decrement

Re: [PATCH ndctl v2 10/10] daxctl/test: Add tests for dynamic dax regions

2020-12-10 Thread Joao Martins
On 7/21/20 5:49 PM, Joao Martins wrote: > On 7/13/20 5:08 PM, Joao Martins wrote: >> Add a couple tests which exercise the new sysfs based >> interface for Soft-Reserved regions (by EFI/HMAT, or >> efi_fake_mem). >> >> The tests exercise the daxctl orchestr

Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages

2020-12-09 Thread Joao Martins
On 12/8/20 7:49 PM, Jason Gunthorpe wrote: > On Tue, Dec 08, 2020 at 05:28:58PM +0000, Joao Martins wrote: >> Much like hugetlbfs or THPs, we treat device pagemaps with >> compound pages like the rest of GUP handling of compound pages. >> >> Rather than incrementin

Re: [PATCH RFC 8/9] RDMA/umem: batch page unpin in __ib_mem_release()

2020-12-09 Thread Joao Martins
On 12/8/20 7:29 PM, Jason Gunthorpe wrote: > On Tue, Dec 08, 2020 at 05:29:00PM +0000, Joao Martins wrote: > >> static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, >> int dirty) >> { >> +bool make_dirty = umem->writable &&am

Re: [PATCH RFC 9/9] mm: Add follow_devmap_page() for devdax vmas

2020-12-09 Thread Joao Martins
On 12/8/20 7:57 PM, Jason Gunthorpe wrote: > On Tue, Dec 08, 2020 at 05:29:01PM +0000, Joao Martins wrote: >> Similar to follow_hugetlb_page() add a follow_devmap_page which rather >> than calling follow_page() per 4K page in a PMD/PUD it does so for the >> entire PMD, wher

Re: [PATCH ndctl v1 0/8] daxctl: Add device align and range mapping allocation

2020-12-17 Thread Joao Martins
On 12/16/20 11:42 PM, Dan Williams wrote: > On Wed, Dec 16, 2020 at 2:54 PM Joao Martins > wrote: >> On 12/16/20 10:31 PM, Dan Williams wrote: >>> On Wed, Dec 16, 2020 at 1:51 PM Joao Martins >>> wrote: >>>> On 12/16/20 6:42 PM, Verma, Vishal L wrot

Re: [PATCH RFC 7/9] mm/gup: Decrement head page once for group of subpages

2020-12-17 Thread Joao Martins
On 12/8/20 7:34 PM, Jason Gunthorpe wrote: >> @@ -274,6 +291,7 @@ void unpin_user_pages_dirty_lock(struct page **pages, >> unsigned long npages, >> bool make_dirty) >> { >> unsigned long index; >> +int refs = 1; >> >> /* >> * TODO: this can be

[PATCH daxctl v2 3/5] libdaxctl: add daxctl_dev_set_mapping()

2020-12-17 Thread Joao Martins
. It also allows an userspace application to pick it's own ranges, should it want to avoid relying on kernel's policy. Signed-off-by: Joao Martins --- daxctl/lib/libdaxctl.c | 27 +++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 2 ++ 3 files changed, 30

Re: [PATCH RFC 7/9] mm/gup: Decrement head page once for group of subpages

2020-12-17 Thread Joao Martins
On 12/17/20 8:05 PM, Jason Gunthorpe wrote: > On Thu, Dec 17, 2020 at 07:05:37PM +0000, Joao Martins wrote: >>> No reason not to fix set_page_dirty_lock() too while you are here. >> >> The wack of atomics you mentioned earlier you referred to, I suppose it >> ends bei

[PATCH daxctl v2 4/5] daxctl: allow creating devices from input json

2020-12-17 Thread Joao Martins
;size":34359738368, "target_node":0, "align":1073741824, "mode":"devdax", "mappings":[ { "page_offset":4194304, "start":25769803776, "end":42

[PATCH daxctl v2 5/5] daxctl/test: add a test for daxctl-create with input file

2020-12-17 Thread Joao Martins
The test creates a multi-range device (4 mappings) using the same setup as one of the tests. Afterwards we validate that the size/nr-mappings are the same as the original test. Signed-off-by: Joao Martins --- test/daxctl-create.sh | 31 ++- 1 file changed, 30

[PATCH daxctl v2 0/5] daxctl: range mapping allocation

2020-12-17 Thread Joao Martins
-M and for --input Joao Martins (5): libdaxctl: add mapping iterator APIs daxctl: include mappings when listing libdaxctl: add daxctl_dev_set_mapping() daxctl: allow creating devices from input json daxctl/test: add a test for daxctl-create with input file Documentation/daxctl/daxctl-cre

[PATCH daxctl v2 1/5] libdaxctl: add mapping iterator APIs

2020-12-17 Thread Joao Martins
Add the following APIs to be able to iterate over a dynamic device-dax mapping list, as well as fetching each of the mapping attributes. Signed-off-by: Joao Martins --- daxctl/lib/libdaxctl-private.h | 8 daxctl/lib/libdaxctl.c | 91

[PATCH daxctl v2 2/5] daxctl: include mappings when listing

2020-12-17 Thread Joao Martins
Iterate over the device mappings and print @page_offset, @start, @end and a computed size, but only if user passes -M|--mappings to daxctl list as the output can get verbose with a lot of mapping entries. Signed-off-by: Joao Martins --- Documentation/daxctl/daxctl-list.txt | 4 +++ daxctl

Re: [PATCH RFC 8/9] RDMA/umem: batch page unpin in __ib_mem_release()

2020-12-19 Thread Joao Martins
On 12/9/20 10:59 AM, Joao Martins wrote: > On 12/8/20 7:29 PM, Jason Gunthorpe wrote: >> On Tue, Dec 08, 2020 at 05:29:00PM +0000, Joao Martins wrote: >> >>> static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, >>> int dirty) >>>

  1   2   >