Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-13 Thread Dan Williams
On Thu, Jun 13, 2019 at 11:42 AM Qian Cai wrote: > > On Wed, 2019-06-12 at 12:37 -0700, Dan Williams wrote: > > On Wed, Jun 12, 2019 at 12:16 PM Qian Cai wrote: > > > > > > The linux-next commit "mm/sparsemem: Add helpers track active portions > > > of

Re: [PATCH 08/22] memremap: pass a struct dev_pagemap to ->kill

2019-06-13 Thread Dan Williams
n he introduced the > callback[1]. > > Reviewed-by: Logan Gunthorpe Reviewed-by: Dan Williams Reported-by: Logan Gunthorpe :) > > Logan > > [1] > https://lore.kernel.org/lkml/8f0cae82-130f-8a64-cfbd-fda5fd76b...@deltatee.com/T/#u >

Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 2:47 PM Qian Cai wrote: > > On Wed, 2019-06-12 at 12:38 -0700, Dan Williams wrote: > > On Wed, Jun 12, 2019 at 12:37 PM Dan Williams > > wrote: > > > > > > On Wed, Jun 12, 2019 at 12:16 PM Qian Cai wrote: > > > > > >

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 3:12 PM Ira Weiny wrote: > > On Wed, Jun 12, 2019 at 04:14:21PM -0300, Jason Gunthorpe wrote: > > On Wed, Jun 12, 2019 at 02:09:07PM +0200, Jan Kara wrote: > > > On Wed 12-06-19 08:47:21, Jason Gunthorpe wrote: > > > > On Wed, Jun 12, 2019 at 12:29:17PM +0200, Jan Kara

Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 2:52 PM Dan Williams wrote: > > On Wed, Jun 12, 2019 at 2:47 PM Qian Cai wrote: > > > > On Wed, 2019-06-12 at 12:38 -0700, Dan Williams wrote: > > > On Wed, Jun 12, 2019 at 12:37 PM Dan Williams > > > wrote: > > > > &

Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 2:47 PM Qian Cai wrote: > > On Wed, 2019-06-12 at 12:38 -0700, Dan Williams wrote: > > On Wed, Jun 12, 2019 at 12:37 PM Dan Williams > > wrote: > > > > > > On Wed, Jun 12, 2019 at 12:16 PM Qian Cai wrote: > > > > > >

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 4:32 PM Ira Weiny wrote: > > On Wed, Jun 12, 2019 at 03:54:19PM -0700, Dan Williams wrote: > > On Wed, Jun 12, 2019 at 3:12 PM Ira Weiny wrote: > > > > > > On Wed, Jun 12, 2019 at 04:14:21PM -0300, Jason Gunthorpe wrote: > > > >

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-13 Thread Dan Williams
On Wed, Jun 12, 2019 at 4:32 PM Ira Weiny wrote: > > On Wed, Jun 12, 2019 at 03:54:19PM -0700, Dan Williams wrote: > > On Wed, Jun 12, 2019 at 3:12 PM Ira Weiny wrote: > > > > > > On Wed, Jun 12, 2019 at 04:14:21PM -0300, Jason Gunthorpe wrote: > > > >

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-13 Thread Dan Williams
On Thu, Jun 13, 2019 at 8:14 AM Jason Gunthorpe wrote: > > On Wed, Jun 12, 2019 at 06:14:46PM -0700, Dan Williams wrote: > > > Effectively, we would need a way for an admin to close a specific file > > > descriptor (or set of fds) which point to that file. AFAI

Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-12 Thread Dan Williams
On Wed, Jun 12, 2019 at 12:37 PM Dan Williams wrote: > > On Wed, Jun 12, 2019 at 12:16 PM Qian Cai wrote: > > > > The linux-next commit "mm/sparsemem: Add helpers track active portions > > of a section at boot" [1] causes a crash below when the fir

Re: [PATCH -next] mm/hotplug: skip bad PFNs from pfn_to_online_page()

2019-06-12 Thread Dan Williams
\ > pfn_valid_within(___pfn)) \ > ___page = pfn_to_page(___pfn); \ > ___page; \ Looks ok to me: Acked-by: Dan Williams ...but why is pfn_to_online_page() a multi-line macro instead of a static inline like all the helper routines it invokes?

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-12 Thread Dan Williams
On Wed, Jun 12, 2019 at 3:29 AM Jan Kara wrote: > > On Fri 07-06-19 07:52:13, Ira Weiny wrote: > > On Fri, Jun 07, 2019 at 09:17:29AM -0300, Jason Gunthorpe wrote: > > > On Fri, Jun 07, 2019 at 12:36:36PM +0200, Jan Kara wrote: > > > > > > > Because the pins would be invisible to sysadmin from

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-12 Thread Dan Williams
On Wed, Jun 12, 2019 at 5:09 AM Jan Kara wrote: > > On Wed 12-06-19 08:47:21, Jason Gunthorpe wrote: > > On Wed, Jun 12, 2019 at 12:29:17PM +0200, Jan Kara wrote: > > > > > > > The main objection to the current ODP & DAX solution is that very > > > > > little HW can actually implement it, having

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-12 Thread Dan Williams
On Wed, 2019-06-12 at 10:31 +0200, Arnd Bergmann wrote: > On Tue, Jun 11, 2019 at 7:23 PM Dan Williams wrote: > > On Tue, 2019-06-11 at 10:52 -0600, Subash Abhinov Kasiviswanathan > > wrote: > > > > rmnet should handle muxing the QMAP, QoS, and aggregation and pass &

[PATCH 4/6] libnvdimm/bus: Stop holding nvdimm_bus_list_mutex over __nd_ioctl()

2019-06-11 Thread Dan Williams
-requisite for a follow-on fix. Cc: Fixes: bf9bccc14c05 ("libnvdimm: pmem label sets and namespace instantiation") Cc: Vishal Verma Signed-off-by: Dan Williams --- drivers/nvdimm/bus.c | 96 -- drivers/nvdimm/nd-core.h |3 + 2 fil

[PATCH 5/6] libnvdimm/bus: Fix wait_nvdimm_bus_probe_idle() ABBA deadlock

2019-06-11 Thread Dan Williams
") Cc: Vishal Verma Signed-off-by: Dan Williams --- drivers/nvdimm/bus.c | 17 +++-- drivers/nvdimm/region_devs.c |4 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index dfb93228d6a7..c1d26fca9c4c 100

[PATCH 6/6] driver-core, libnvdimm: Let device subsystems add local lockdep coverage

2019-06-11 Thread Dan Williams
an Signed-off-by: Dan Williams --- drivers/acpi/nfit/core.c| 28 drivers/acpi/nfit/nfit.h| 24 ++ drivers/base/core.c |3 ++ drivers/nvdimm/btt_devs.c | 16 + drivers/nvdimm/bus.c| 28 ++-

[PATCH 3/6] libnvdimm/region: Register badblocks before namespaces

2019-06-11 Thread Dan Williams
") Cc: Cc: Vishal Verma Signed-off-by: Dan Williams --- drivers/nvdimm/region.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c index ef46cc3a71ae..488c47ac4c4a 100644 --- a/drivers/nvdimm/regi

[PATCH 2/6] libnvdimm/bus: Prevent duplicate device_unregister() calls

2019-06-11 Thread Dan Williams
Signed-off-by: Dan Williams --- drivers/nvdimm/bus.c | 25 + 1 file changed, 25 insertions(+) diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index 2dca3034fee0..42713b210f51 100644 --- a/drivers/nvdimm/bus.c +++ b/drivers/nvdimm/bus.c @@ -547,13 +547,38 @@ EXP

[PATCH 1/6] drivers/base: Introduce kill_device()

2019-06-11 Thread Dan Williams
does not change any behavior, but it is a pre-requisite for follow-on fixes and therefore marked for -stable. Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Fixes: 4d88a97aa9e8 ("libnvdimm, nvdimm: dimm driver and base libnvdimm device-driver...") Cc: Signed-off-by: Dan Wi

[PATCH 0/6] libnvdimm: Fix async operations and locking

2019-06-11 Thread Dan Williams
st. Patch4 and patch5 address an ABBA deadlock tripped by the stress test. These patches pass the failing stress test and the existing libnvdimm unit tests with CONFIG_PROVE_LOCKING=y and the new "dev->lockdep_mutex" shadow lock with no lockdep warnings. --- Dan Williams (6)

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-11 Thread Dan Williams
On Tue, 2019-06-11 at 10:52 -0600, Subash Abhinov Kasiviswanathan wrote: > > The general plan (and I believe Daniele Palmas was working on it) > > was > > to eventually make qmi_wwan use rmnet rather than its internal > > sysfs- > > based implementation. qmi_wwan and ipa are at essentially the

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-11 Thread Dan Williams
On Tue, 2019-06-11 at 13:56 +0200, Arnd Bergmann wrote: > On Tue, Jun 11, 2019 at 10:12 AM Johannes Berg > wrote: > > > > As I've made clear before, my work on this has been focused on > > > the IPA transport, > > > and some of this higher-level LTE architecture is new to me. But > > > it > > >

Re: [PATCH v9 11/12] libnvdimm/pfn: Fix fsdax-mode namespace info-block zero-fields

2019-06-07 Thread Dan Williams
On Fri, Jun 7, 2019 at 12:54 PM Andrew Morton wrote: > > On Thu, 6 Jun 2019 15:06:26 -0700 Dan Williams > wrote: > > > On Thu, Jun 6, 2019 at 2:46 PM Andrew Morton > > wrote: > > > > > > On Wed, 05 Jun 2019 14:58:58 -0700 Dan Williams > > &g

[PATCH v3 00/10] EFI Specific Purpose Memory Support

2019-06-07 Thread Dan Williams
/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c221c0b0308f [5]: http://patchwork.ozlabs.org/cover/1096737/ --- Dan Williams (10): acpi/numa: Establish a new drivers/acpi/numa/ directory acpi/numa/hmat: Skip publishing target info for nodes with no online memory efi: Enumerat

Re: [PATCH v9 08/12] mm/sparsemem: Support sub-section hotplug

2019-06-07 Thread Dan Williams
On Fri, Jun 7, 2019 at 1:34 AM Oscar Salvador wrote: > > On Wed, Jun 05, 2019 at 02:58:42PM -0700, Dan Williams wrote: > > The libnvdimm sub-system has suffered a series of hacks and broken > > workarounds for the memory-hotplug implementation's awkward > > section-ali

Re: [Lsf-pc] [LSF/MM TOPIC] The end of the DAX experiment

2019-06-07 Thread Dan Williams
On Wed, Jun 5, 2019 at 9:22 AM Goldwyn Rodrigues wrote: > > Hi Dan/Jerome, > > On 12:20 14/02, Dan Williams wrote: > > On Thu, Feb 14, 2019 at 12:09 PM Matthew Wilcox wrote: > > > > > > On Thu, Feb 14, 2019 at 11:31:24AM -0800, Dan Williams wrote: > > &

Re: [PATCH v9 11/12] libnvdimm/pfn: Fix fsdax-mode namespace info-block zero-fields

2019-06-06 Thread Dan Williams
On Thu, Jun 6, 2019 at 2:46 PM Andrew Morton wrote: > > On Wed, 05 Jun 2019 14:58:58 -0700 Dan Williams > wrote: > > > At namespace creation time there is the potential for the "expected to > > be zero" fields of a 'pfn' info-block to be filled with indete

Re: [PATCH v9 07/12] mm/sparsemem: Prepare for sub-section ranges

2019-06-06 Thread Dan Williams
On Thu, Jun 6, 2019 at 10:21 AM Oscar Salvador wrote: > > On Wed, Jun 05, 2019 at 02:58:37PM -0700, Dan Williams wrote: > > Prepare the memory hot-{add,remove} paths for handling sub-section > > ranges by plumbing the starting page frame and number of pages being > > h

Re: [PATCH RFC 00/10] RDMA/FS DAX truncate proposal

2019-06-06 Thread Dan Williams
On Thu, Jun 6, 2019 at 3:42 AM Jan Kara wrote: > > On Wed 05-06-19 18:45:33, ira.we...@intel.com wrote: > > From: Ira Weiny > > > > ... V1,000,000 ;-) > > > > Pre-requisites: > > John Hubbard's put_user_pages() patch series.[1] > > Jan Kara's ext4_break_layouts() fixes[2] > > > >

[PATCH v9 07/12] mm/sparsemem: Prepare for sub-section ranges

2019-06-05 Thread Dan Williams
. No intended functional changes. Cc: Michal Hocko Cc: Vlastimil Babka Cc: Logan Gunthorpe Cc: Oscar Salvador Reviewed-by: Pavel Tatashin Signed-off-by: Dan Williams --- include/linux/memory_hotplug.h |5 +- mm/memory_hotplug.c| 114 +--- mm/sparse.c

[PATCH v9 12/12] libnvdimm/pfn: Stop padding pmem namespaces to section alignment

2019-06-05 Thread Dan Williams
Now that the mm core supports section-unaligned hotplug of ZONE_DEVICE memory, we no longer need to add padding at pfn/dax device creation time. The kernel will still honor padding established by older kernels. Reported-by: Jeff Moyer Signed-off-by: Dan Williams --- drivers/nvdimm/pfn.h

[PATCH v9 06/12] mm: Kill is_dev_zone() helper

2019-06-05 Thread Dan Williams
Given there are no more usages of is_dev_zone() outside of 'ifdef CONFIG_ZONE_DEVICE' protection, kill off the compilation helper. Cc: Michal Hocko Cc: Logan Gunthorpe Acked-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Pavel Tatashin Signed-off-by: Dan Williams

[PATCH v9 05/12] mm/hotplug: Kill is_dev_zone() usage in __remove_pages()

2019-06-05 Thread Dan Williams
ildenbrand Reviewed-by: Oscar Salvador Signed-off-by: Dan Williams --- mm/memory_hotplug.c |7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 647859a1d119..4b882c57781a 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c

[PATCH v9 09/12] mm: Document ZONE_DEVICE memory-model implications

2019-06-05 Thread Dan Williams
Explain the general mechanisms of 'ZONE_DEVICE' pages and list the users of 'devm_memremap_pages()'. Cc: Jonathan Corbet Reported-by: Mike Rapoport Signed-off-by: Dan Williams --- Documentation/vm/memory-model.rst | 39 + 1 file changed, 39 insertions

[PATCH v9 11/12] libnvdimm/pfn: Fix fsdax-mode namespace info-block zero-fields

2019-06-05 Thread Dan Williams
' for pmem") Cc: Signed-off-by: Dan Williams --- drivers/nvdimm/dax_devs.c |2 +- drivers/nvdimm/pfn.h |1 + drivers/nvdimm/pfn_devs.c | 18 +++--- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/nvdimm/dax_devs.c b/drivers/nvdimm/dax_devs.

[PATCH v9 10/12] mm/devm_memremap_pages: Enable sub-section remap

2019-06-05 Thread Dan Williams
parate remap attempts do not collide within a sub-section (2MB on x86). Cc: Michal Hocko Cc: Toshi Kani Cc: Jérôme Glisse Cc: Logan Gunthorpe Cc: Oscar Salvador Cc: Pavel Tatashin Signed-off-by: Dan Williams --- kernel/memremap.c | 61 +

[PATCH v9 08/12] mm/sparsemem: Support sub-section hotplug

2019-06-05 Thread Dan Williams
Gunthorpe Cc: Oscar Salvador Cc: Pavel Tatashin Signed-off-by: Dan Williams --- include/linux/memory_hotplug.h |2 mm/memory_hotplug.c|7 - mm/page_alloc.c|2 mm/sparse.c| 225 +++- 4 files changed

[PATCH v9 00/12] mm: Sub-section memory hotplug support

2019-06-05 Thread Dan Williams
rnel.org/r/155000671719.348031.2347363160141119237.st...@dwillia2-desk3.amr.corp.intel.com [3]: https://github.com/pmem/ndctl/issues/76 [4]: https://git.kernel.org/pub/scm/linux/kernel/git/djbw/nvdimm.git/log/?h=libnvdimm-pending [5]: https://github.com/pmem/ndctl/commit/7c59b4867e1c --- Dan Williams (12): mm/sparsemem: Introd

[PATCH v9 04/12] mm/sparsemem: Convert kmalloc_section_memmap() to populate_section_memmap()

2019-06-05 Thread Dan Williams
are added to clarify which memmap allocation paths are sub-section capable. Cc: Michal Hocko Cc: David Hildenbrand Cc: Logan Gunthorpe Cc: Oscar Salvador Reviewed-by: Pavel Tatashin Signed-off-by: Dan Williams --- arch/x86/mm/init_64.c |4 ++- include/linux/mm.h|4 ++- mm/sparse

[PATCH v9 01/12] mm/sparsemem: Introduce struct mem_section_usage

2019-06-05 Thread Dan Williams
ckerras Cc: Michael Ellerman Signed-off-by: Dan Williams --- arch/powerpc/include/asm/sparsemem.h |3 + include/linux/mmzone.h | 48 +++- mm/memory_hotplug.c | 18 mm/page_alloc.c |

[PATCH v9 02/12] mm/sparsemem: Add helpers track active portions of a section at boot

2019-06-05 Thread Dan Williams
Gunthorpe Cc: Oscar Salvador Cc: Pavel Tatashin Tested-by: Jane Chu Signed-off-by: Dan Williams --- include/linux/mmzone.h | 29 - mm/page_alloc.c|4 +++- mm/sparse.c| 35 +++ 3 files changed, 66 insertions

[PATCH v9 03/12] mm/hotplug: Prepare shrink_{zone, pgdat}_span for sub-section removal

2019-06-05 Thread Dan Williams
(), can toggle. Cc: Michal Hocko Cc: Vlastimil Babka Cc: Logan Gunthorpe Reviewed-by: Pavel Tatashin Reviewed-by: Oscar Salvador Signed-off-by: Dan Williams --- mm/memory_hotplug.c | 29 - 1 file changed, 8 insertions(+), 21 deletions(-) diff --git a/mm

Re: [PATCH v3] mm/swap: Fix release_pages() when releasing devmap pages

2019-06-04 Thread Dan Williams
On Tue, Jun 4, 2019 at 1:17 PM John Hubbard wrote: > > On 6/4/19 1:11 PM, Dan Williams wrote: > > On Tue, Jun 4, 2019 at 12:48 PM John Hubbard wrote: > >> > >> On 6/4/19 9:48 AM, ira.we...@intel.com wrote: > >>> From: Ira Weiny > >>> >

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-04 Thread Dan Williams
On Tue, 2019-06-04 at 22:04 +0200, Arnd Bergmann wrote: > On Tue, Jun 4, 2019 at 5:18 PM Dan Williams wrote: > > On Tue, 2019-06-04 at 10:13 +0200, Arnd Bergmann wrote: > > > Can you describe what kind of multiplexing is actually going on? > > > I'm still unclear

Re: [PATCH v3] mm/swap: Fix release_pages() when releasing devmap pages

2019-06-04 Thread Dan Williams
t; > processing and should fall to "normal" processing. > > > > This was found via code inspection while determining if release_pages() > > and the new put_user_pages() could be interchangeable.[1] > > > > [1] > > https://lore.kernel.org/lkml/2019052

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-04 Thread Dan Williams
On Mon, 2019-06-03 at 13:04 -0600, Subash Abhinov Kasiviswanathan wrote: > > > I can't (or won't) comment right now on whether IPA needs its own > > > netdev for rmnet to use. The IPA endpoints used for the modem > > > network interfaces are enabled when the netdev is opened and > > > disabled

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-04 Thread Dan Williams
On Tue, 2019-06-04 at 10:13 +0200, Arnd Bergmann wrote: > On Mon, Jun 3, 2019 at 3:32 PM Alex Elder wrote: > > On 6/3/19 5:04 AM, Arnd Bergmann wrote: > > > On Sat, Jun 1, 2019 at 1:59 AM Subash Abhinov Kasiviswanathan > > > > > > - What I'm worried about most here is the flow control handling >

Re: [PATCH v8 09/12] mm/sparsemem: Support sub-section hotplug

2019-06-03 Thread Dan Williams
On Mon, May 13, 2019 at 6:55 AM Oscar Salvador wrote: > > On Mon, May 06, 2019 at 04:40:14PM -0700, Dan Williams wrote: > > > > +void subsection_mask_set(unsigned long *map, unsigned long pfn, > > + unsigned long nr_pages) > > +{ > > +

Re: [PATCH v7 09/12] mm/sparsemem: Support sub-section hotplug

2019-06-03 Thread Dan Williams
On Fri, May 3, 2019 at 5:56 AM Oscar Salvador wrote: > > On Wed, May 01, 2019 at 10:56:10PM -0700, Dan Williams wrote: > > The libnvdimm sub-system has suffered a series of hacks and broken > > workarounds for the memory-hotplug implementation's awkward > > section-ali

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-03 Thread Dan Williams
On Mon, 2019-06-03 at 10:52 -0500, Alex Elder wrote: > On 6/3/19 9:54 AM, Dan Williams wrote: > > > To be perfectly honest, at first I thought having IPA use rmnet > > > was a cargo cult thing like Dan suggested, because I didn't see > > To be clear I only meant cargo-

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-03 Thread Dan Williams
On Fri, 2019-05-31 at 15:47 -0500, Alex Elder wrote: > On 5/31/19 2:19 PM, Arnd Bergmann wrote: > > On Fri, May 31, 2019 at 6:36 PM Alex Elder > > wrote: > > > On 5/31/19 9:58 AM, Dan Williams wrote: > > > > On Thu, 2019-05-30 at 22:53 -0500, Alex Elder wrot

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-06-03 Thread Dan Williams
at 6:36 PM Alex Elder > > > > wrote: > > > > > On 5/31/19 9:58 AM, Dan Williams wrote: > > > > > > On Thu, 2019-05-30 at 22:53 -0500, Alex Elder wrote: > > > > > > > > > > > > My question from the Nov 2018

Re: [PATCH v10 1/3] mm: Shuffle initial free memory to improve memory-side-cache utilization

2019-05-31 Thread Dan Williams
On Fri, May 31, 2019 at 12:33 AM Vlastimil Babka wrote: > > On 2/1/19 6:15 AM, Dan Williams wrote: > > --- a/init/Kconfig > > +++ b/init/Kconfig > > @@ -1714,6 +1714,29 @@ config SLAB_FREELIST_HARDENED > > sacrifies to harden the kernel slab allocator again

Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver

2019-05-31 Thread Dan Williams
On Thu, 2019-05-30 at 22:53 -0500, Alex Elder wrote: > This series presents the driver for the Qualcomm IP Accelerator > (IPA). > > This is version 2 of the series. This version has addressed almost > all of the feedback received in the first version: > >

Re: [PATCH v2 0/6] mm/devm_memremap_pages: Fix page release race

2019-05-30 Thread Dan Williams
On Mon, May 13, 2019 at 12:22 PM Logan Gunthorpe wrote: > > > > On 2019-05-08 11:05 a.m., Logan Gunthorpe wrote: > > > > > > On 2019-05-07 5:55 p.m., Dan Williams wrote: > >> Changes since v1 [1]: > >> - Fix a NULL-pointer deref crash in pci_p2pdma_

[GIT PULL] libnvdimm fixes for v5.2-rc2

2019-05-25 Thread Dan Williams
) libnvdimm fixes v5.2-rc2 - Fix a regression that disabled device-mapper dax support - Remove unnecessary hardened-user-copy overhead (>30%) for dax read(2)/write(2). - Fix some compilation warnings. ---- Dan Williams

Re: [PATCH] mm/swap: Fix release_pages() when releasing devmap pages

2019-05-24 Thread Dan Williams
On Thu, May 23, 2019 at 8:58 PM Dan Williams wrote: > > On Thu, May 23, 2019 at 3:37 PM wrote: > > > > From: Ira Weiny > > > > Device pages can be more than type MEMORY_DEVICE_PUBLIC. > > > > Handle all device pages within release_pages() > &

Re: [PATCH, RFC 2/2] Implement sharing/unsharing of PMDs for FS/DAX

2019-05-24 Thread Dan Williams
On Fri, May 24, 2019 at 9:07 AM Larry Bassel wrote: > On 14 May 19 16:01, Kirill A. Shutemov wrote: > > On Thu, May 09, 2019 at 09:05:33AM -0700, Larry Bassel wrote: [..] > > > diff --git a/mm/memory.c b/mm/memory.c > > > index f7d962d..4c1814c 100644 > > > --- a/mm/memory.c > > > +++

Re: [PATCH] mm/swap: Fix release_pages() when releasing devmap pages

2019-05-24 Thread Dan Williams
On Fri, May 24, 2019 at 8:35 AM Ira Weiny wrote: > > On Thu, May 23, 2019 at 08:58:12PM -0700, Dan Williams wrote: > > On Thu, May 23, 2019 at 3:37 PM wrote: > > > > > > From: Ira Weiny > > > > > > Device pages can be more than type MEMORY_DE

Re: [PATCH] mm/swap: Fix release_pages() when releasing devmap pages

2019-05-23 Thread Dan Williams
new put_user_pages() could be interchangeable. > > Cc: Jérôme Glisse > Cc: Dan Williams > Cc: Michal Hocko > Cc: John Hubbard > Signed-off-by: Ira Weiny > --- > mm/swap.c | 7 +++ > 1 file changed, 3 insertions(+), 4 deletions(-) > > diff --git a/mm/swap.c b/mm/swap.c

Re: [v6 3/3] device-dax: "Hotremove" persistent memory that is used like normal RAM

2019-05-21 Thread Dan Williams
l the next reboot" since there is no guarantee it will work then either and the surefire mechanism to recover the memory from the kmem driver is to not add it in the first place. Perhaps also print out the error code in case it might specify a finer grained reason the memory is pinned. Other than that you can add Reviewed-by: Dan Williams ...as it looks like Andrew will take this through -mm.

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

2019-05-20 Thread Dan Williams
On Mon, May 20, 2019 at 12:27 PM Jerome Glisse wrote: > > On Mon, May 20, 2019 at 11:07:38AM +0530, Anshuman Khandual wrote: > > On 05/18/2019 03:20 AM, Andrew Morton wrote: > > > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual > > > wrote: > > > > > >> The presence of struct page does not

[PATCH v2] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-20 Thread Dan Williams
quot;) Cc: Cc: Jeff Moyer Cc: Ingo Molnar Cc: Christoph Hellwig Cc: Al Viro Cc: Thomas Gleixner Cc: Matthew Wilcox Reported-and-tested-by: Jeff Smits Acked-by: Kees Cook Acked-by: Jan Kara Signed-off-by: Dan Williams --- Changes since v1 [1]: * Update the changelog to clarify which check

Re: [PATCH] mm/dev_pfn: Exclude MEMORY_DEVICE_PRIVATE while computing virtual address

2019-05-20 Thread Dan Williams
On Sun, May 19, 2019 at 10:37 PM Anshuman Khandual wrote: > > On 05/18/2019 03:20 AM, Andrew Morton wrote: > > On Fri, 17 May 2019 16:08:34 +0530 Anshuman Khandual > > wrote: > > > >> The presence of struct page does not guarantee linear mapping for the pfn > >> physical range. Device private

Re: [PATCH] dax: Fix last_page check in __bdev_dax_supported()

2019-05-20 Thread Dan Williams
On Thu, May 16, 2019 at 10:37 PM Vaibhav Jain wrote: > > Dan Williams writes: > > > On Wed, May 15, 2019 at 10:55 PM Vaibhav Jain wrote: > >> > >> Presently __bdev_dax_supported() checks if first sector of last > >> page ( last_page ) on the blo

Re: [PATCH] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-20 Thread Dan Williams
On Mon, May 20, 2019 at 12:52 AM Jan Kara wrote: > > On Sat 18-05-19 21:46:03, Dan Williams wrote: > > On Fri, May 17, 2019 at 12:25 PM Kees Cook wrote: > > > On Fri, May 17, 2019 at 10:28:48AM -0700, Dan Williams wrote: > > > > It seems dax_iomap_actor() is

Re: [PATCH] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-19 Thread Dan Williams
On Fri, May 17, 2019 at 12:25 PM Kees Cook wrote: > On Fri, May 17, 2019 at 10:28:48AM -0700, Dan Williams wrote: > > It seems dax_iomap_actor() is not a path where we'd be worried about > > needing hardened user copy checks. > > I would agree: I think the proposed patch ma

Re: [PATCH] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-17 Thread Dan Williams
On Fri, May 17, 2019 at 8:57 AM Kees Cook wrote: > > On Fri, May 17, 2019 at 08:08:27AM -0700, Dan Williams wrote: > > As far as I can see it's mostly check_heap_object() that is the > > problem, so I'm open to finding a way to just bypass that sub-routine. > > However,

Re: [PATCH] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-17 Thread Dan Williams
On Fri, May 17, 2019 at 1:47 AM Jan Kara wrote: > > Let's add Kees to CC for usercopy expertise... > > On Thu 16-05-19 17:33:38, Dan Williams wrote: > > Jeff discovered that performance improves from ~375K iops to ~519K iops > > on a simple psync-write fio workload

[PATCH] libnvdimm/pmem: Bypass CONFIG_HARDENED_USERCOPY overhead

2019-05-16 Thread Dan Williams
ot;x86, uaccess: introduce copy_from_iter_flushcache...") Cc: Jan Kara Cc: Cc: Jeff Moyer Cc: Ingo Molnar Cc: Christoph Hellwig Cc: Al Viro Cc: Thomas Gleixner Cc: Matthew Wilcox Reported-and-tested-by: Jeff Smits Signed-off-by: Dan Williams --- drivers/nvdimm/pmem.c |9 +++-- 1 file c

Re: [PATCH v2 0/6] mm/devm_memremap_pages: Fix page release race

2019-05-16 Thread Dan Williams
On Thu, May 16, 2019 at 9:45 AM Jane Chu wrote: > > Hi, > > I'm able to reproduce the panic below by running two sets of ndctl > commands that actually serve legitimate purpose in parallel (unlike > the brute force experiment earlier), each set in a indefinite loop. > This time it takes about an

Re: [PATCH] nvdimm: fix compilation warnings with W=1

2019-05-16 Thread Dan Williams
On Thu, May 16, 2019 at 9:05 AM Qian Cai wrote: > > Several places (dimm_devs.c, core.c etc) include label.h but only > label.c uses NSINDEX_SIGNATURE, so move its definition to label.c > instead. > > In file included from drivers/nvdimm/dimm_devs.c:23: > drivers/nvdimm/label.h:41:19: warning:

Re: [PATCH] dax: Fix last_page check in __bdev_dax_supported()

2019-05-16 Thread Dan Williams
On Wed, May 15, 2019 at 10:55 PM Vaibhav Jain wrote: > > Presently __bdev_dax_supported() checks if first sector of last > page ( last_page ) on the block device is aligned to page > boundary. However the code to compute 'last_page' assumes that there > are 8 sectors/page assuming a 4K page-size.

[GIT PULL] libnvdimm fixes for v5.2-rc1

2019-05-15 Thread Dan Williams
: Allow to include DEV_DAX_PMEM as builtin Dan Williams (1): libnvdimm/namespace: Fix label tracking error Vishal Verma (2): dax/pmem: Fix whitespace in dax_pmem tools/testing/nvdimm: add watermarks for dax_pmem* modules drivers/dax/Kconfig | 3

Re: [RESEND PATCH] nvdimm: fix some compilation warnings

2019-05-15 Thread Dan Williams
On Wed, May 15, 2019 at 5:25 PM Verma, Vishal L wrote: > > On Wed, 2019-05-15 at 16:25 -0700, Dan Williams wrote: > > > > > diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c > > > index 4671776f5623..9f02a99cfac0 100644 > > > --- a/drivers/nvdimm

Re: [v5 0/3] "Hotremove" persistent memory

2019-05-15 Thread Dan Williams
On Wed, May 15, 2019 at 11:12 AM Pavel Tatashin wrote: > > > Hi Pavel, > > > > I am working on adding this sort of a workflow into a new daxctl command > > (daxctl-reconfigure-device)- this will allow changing the 'mode' of a > > dax device to kmem, online the resulting memory, and with your

Re: [PATCH v2 12/30] dax: remove block device dependencies

2019-05-15 Thread Dan Williams
On Wed, May 15, 2019 at 12:28 PM Vivek Goyal wrote: > > From: Stefan Hajnoczi > > Although struct dax_device itself is not tied to a block device, some > DAX code assumes there is a block device. Make block devices optional > by allowing bdev to be NULL in commonly used DAX APIs. > > When there

Re: [RESEND PATCH] nvdimm: fix some compilation warnings

2019-05-15 Thread Dan Williams
On Tue, May 14, 2019 at 8:08 AM Qian Cai wrote: > > Several places (dimm_devs.c, core.c etc) include label.h but only > label.c uses NSINDEX_SIGNATURE, so move its definition to label.c > instead. > In file included from drivers/nvdimm/dimm_devs.c:23: > drivers/nvdimm/label.h:41:19: warning:

Re: [PATCH v9 4/7] dm: enable synchronous dax

2019-05-15 Thread Dan Williams
[ add Mike and dm-devel ] Mike, any concerns with the below addition to the device-mapper-dax implementation? On Tue, May 14, 2019 at 7:58 AM Pankaj Gupta wrote: > > This patch sets dax device 'DAXDEV_SYNC' flag if all the target > devices of device mapper support synchrononous DAX. If device

Re: [GIT PULL] device-dax for 5.1: PMEM as RAM

2019-05-15 Thread Dan Williams
On Mon, Mar 11, 2019 at 5:08 PM Linus Torvalds wrote: > > On Mon, Mar 11, 2019 at 8:37 AM Dan Williams wrote: > > > > Another feature the userspace tooling can support for the PMEM as RAM > > case is the ability to complete an Address Range Scrub of the range > >

Re: [PATCH v2 1/6] drivers/base/devres: Introduce devm_release_action()

2019-05-14 Thread Dan Williams
On Tue, May 14, 2019 at 12:12 PM Greg Kroah-Hartman wrote: > > On Tue, May 07, 2019 at 04:56:05PM -0700, Dan Williams wrote: > > The devm_add_action() facility allows a resource allocation routine to > > add custom devm semantics. One such user is devm_memremap_pages(). &g

Re: [PATCH v2 0/6] mm/devm_memremap_pages: Fix page release race

2019-05-14 Thread Dan Williams
On Tue, May 14, 2019 at 11:53 AM Jane Chu wrote: > > On 5/13/2019 12:22 PM, Logan Gunthorpe wrote: > > On 2019-05-08 11:05 a.m., Logan Gunthorpe wrote: > > On 2019-05-07 5:55 p.m., Dan Williams wrote: > > Changes since v1 [1]: > - Fix a NULL-pointer deref crash in

Re: [PATCH v8 00/12] mm: Sub-section memory hotplug support

2019-05-13 Thread Dan Williams
On Mon, May 13, 2019 at 2:02 PM Mike Rapoport wrote: > > Hi Dan, > > On Mon, May 06, 2019 at 04:39:26PM -0700, Dan Williams wrote: > > Changes since v7 [1]: > > Sorry for jumping late No worries, it needs to be rebased on David's "mm/memory_hotplug: Factor out

Re: [PATCH v8 3/6] libnvdimm: add dax_dev sync flag

2019-05-10 Thread Dan Williams
On Fri, May 10, 2019 at 5:45 PM Pankaj Gupta wrote: > > > > > > > > > This patch adds 'DAXDEV_SYNC' flag which is set > > > for nd_region doing synchronous flush. This later > > > is used to disable MAP_SYNC functionality for > > > ext4 & xfs filesystem for devices don't support > > > synchronous

Re: [Qemu-devel] [PATCH v7 2/6] virtio-pmem: Add virtio pmem driver

2019-05-10 Thread Dan Williams
On Wed, May 8, 2019 at 4:19 AM Pankaj Gupta wrote: > > > Hi Dan, > > Thank you for the review. Please see my reply inline. > > > > > Hi Pankaj, > > > > Some minor file placement comments below. > > Sure. > > > > > On Thu, Apr 25, 2019 at 10:02 PM Pankaj Gupta wrote: > > > > > > This patch adds

Re: [PATCH, RFC 1/2] Add config option to enable FS/DAX PMD sharing

2019-05-10 Thread Dan Williams
On Fri, May 10, 2019 at 9:32 AM Elliott, Robert (Servers) wrote: > > > > > -Original Message- > > From: Linux-nvdimm On Behalf Of > > Larry Bassel > > Sent: Thursday, May 09, 2019 11:06 AM > > Subject: [PATCH, RFC 1/2] Add config option to enable FS/DAX PMD > > sharing > > > > If

[PATCH] mm/huge_memory: Fix vmf_insert_pfn_{pmd, pud}() crash, handle unaligned addresses

2019-05-09 Thread Dan Williams
Ma Cc: Aneesh Kumar K.V Cc: Chandan Rajendra Cc: Jan Kara Cc: Andrew Morton Cc: Matthew Wilcox Cc: Souptick Joarder Signed-off-by: Dan Williams --- drivers/dax/device.c|6 ++ fs/dax.c|6 ++ include/linux/huge_mm.h |6 ++ mm/huge_me

[PATCH v2 5/6] PCI/P2PDMA: Track pgmap references per resource, not globally

2019-05-07 Thread Dan Williams
In preparation for fixing a race between devm_memremap_pages_release() and the final put of a page from the device-page-map, allocate a percpu-ref per p2pdma resource mapping. Cc: Logan Gunthorpe Cc: Bjorn Helgaas Cc: Christoph Hellwig Reviewed-by: Ira Weiny Signed-off-by: Dan Williams

[PATCH v2 2/6] mm/devm_memremap_pages: Introduce devm_memunmap_pages

2019-05-07 Thread Dan Williams
Use the new devm_relase_action() facility to allow devm_memremap_pages_release() to be manually triggered. Cc: Logan Gunthorpe Cc: Bjorn Helgaas Cc: Christoph Hellwig Reviewed-by: Ira Weiny Signed-off-by: Dan Williams --- include/linux/memremap.h |6 ++ kernel/memremap.c

[PATCH v2 6/6] mm/devm_memremap_pages: Fix final page put race

2019-05-07 Thread Dan Williams
_pages") Cc: Bjorn Helgaas Cc: "Jérôme Glisse" Cc: Christoph Hellwig Reviewed-by: Ira Weiny Signed-off-by: Dan Williams --- drivers/dax/device.c | 13 +++-- drivers/nvdimm/pmem.c | 17 + drivers/pci/p2pdma.c | 17

Re: [PATCH v2 4/8] mm/memory_hotplug: Create memory block devices after arch_add_memory()

2019-05-07 Thread Dan Williams
On Tue, May 7, 2019 at 11:38 AM David Hildenbrand wrote: > > Only memory to be added to the buddy and to be onlined/offlined by > user space using memory block devices needs (and should have!) memory > block devices. > > Factor out creation of memory block devices Create all devices after >

Re: [PATCH v2 5/8] mm/memory_hotplug: Drop MHP_MEMBLOCK_API

2019-05-07 Thread Dan Williams
On Tue, May 7, 2019 at 11:38 AM David Hildenbrand wrote: > > No longer needed, the callers of arch_add_memory() can handle this > manually. > > Cc: Andrew Morton > Cc: David Hildenbrand > Cc: Michal Hocko > Cc: Oscar Salvador > Cc: Pavel Tatashin > Cc: Wei Yang > Cc: Joonsoo Kim > Cc: Qian

[PATCH v8 05/12] mm/sparsemem: Convert kmalloc_section_memmap() to populate_section_memmap()

2019-05-06 Thread Dan Williams
are added to clarify which memmap allocation paths are sub-section capable. Cc: Michal Hocko Cc: David Hildenbrand Cc: Logan Gunthorpe Cc: Oscar Salvador Reviewed-by: Pavel Tatashin Signed-off-by: Dan Williams --- arch/x86/mm/init_64.c |4 ++- include/linux/mm.h|4 ++- mm/sparse

[PATCH v8 01/12] mm/sparsemem: Introduce struct mem_section_usage

2019-05-06 Thread Dan Williams
ckerras Cc: Michael Ellerman Signed-off-by: Dan Williams --- arch/powerpc/include/asm/sparsemem.h |3 + include/linux/mmzone.h | 48 +++- mm/memory_hotplug.c | 18 mm/page_alloc.c |

Re: [v5 2/3] mm/hotplug: make remove_memory() interface useable

2019-05-06 Thread Dan Williams
On Mon, May 6, 2019 at 10:57 AM Dave Hansen wrote: > > > -static inline void remove_memory(int nid, u64 start, u64 size) {} > > +static inline bool remove_memory(int nid, u64 start, u64 size) > > +{ > > + return -EBUSY; > > +} > > This seems like an appropriate place for a WARN_ONCE(), if

Re: [PATCH v6 00/12] mm: Sub-section memory hotplug support

2019-05-02 Thread Dan Williams
On Thu, May 2, 2019 at 3:46 PM Pavel Tatashin wrote: > > Hi Dan, > > How do you test these patches? Do you have any instructions? Yes, I briefly mentioned this in the cover letter, but here is the test I am using: > > I see for example that check_hotplug_memory_range() still enforces >

Re: [PATCH] kernel: remove the unused device_private_entry_fault export

2019-04-29 Thread Dan Williams
t(vma, addr, page, flags, pmdp); > } > -EXPORT_SYMBOL(device_private_entry_fault); > #endif /* CONFIG_DEVICE_PRIVATE */ Looks good to me. Reviewed-by: Dan Williams

Re: [RFC PATCH 0/5] New fallback workflow for heterogeneous memory system

2019-04-25 Thread Dan Williams
On Thu, Apr 25, 2019 at 1:05 AM Du, Fan wrote: > > > > >-Original Message- > >From: owner-linux...@kvack.org [mailto:owner-linux...@kvack.org] On > >Behalf Of Michal Hocko > >Sent: Thursday, April 25, 2019 3:54 PM > >To: Du, Fan > >Cc: a...@linux-foundation.org; Wu, Fengguang ; >

Re: [PATCH v6 06/12] mm/hotplug: Add mem-hotplug restrictions for remove_memory()

2019-04-24 Thread Dan Williams
On Tue, Apr 23, 2019 at 2:21 PM David Hildenbrand wrote: > > On 17.04.19 20:39, Dan Williams wrote: > > Teach the arch_remove_memory() path to consult the same 'struct > > mhp_restrictions' context as was specified at arch_add_memory() time. > > > > No functional

<    8   9   10   11   12   13   14   15   16   17   >