Re: [RFC] virtio-mem: Add support of memory_hotplug.memmap_on_memory

2021-07-07 Thread David Hildenbrand
On 23.06.21 13:58, Hui Zhu wrote: From: Hui Zhu Sorry for the delay, once I was back from vacation my inbox was overflowing :) We did some virtio-mem resize tests in high memory pressure environment. Memory increases slowly and sometimes fails in these tests. This is a way to reproduce

Re: [RFC] virtio-mem: Add support of memory_hotplug.memmap_on_memory

2021-06-23 Thread David Hildenbrand
On Wednesday, June 23, 2021, Hui Zhu wrote: > From: Hui Zhu > > We did some virtio-mem resize tests in high memory pressure environment. > Memory increases slowly and sometimes fails in these tests. > This is a way to reproduce the issue. > Start a qemu with a small size of memory (132Mb) and

Re: [RFC] virtio-mem: virtio_mem_init: Access bb_size just in BBM mode

2021-06-15 Thread David Hildenbrand
On 15.06.21 08:59, Hui Zhu wrote: From: Hui Zhu /* In BBM, we also want at least two big blocks. */ vm->offline_threshold = max_t(uint64_t, 2 * vm->bbm.bb_size, vm->offline_threshold); This line does not modify vm->offline_threshold depending on the data in

Re: [PATCH v1 05/12] mm/memory_hotplug: remove nid parameter from remove_memory() and friends

2021-06-09 Thread David Hildenbrand
On 08.06.21 13:18, David Hildenbrand wrote: On 08.06.21 13:11, Michael Ellerman wrote: David Hildenbrand writes: There is only a single user remaining. We can simply try to offline all online nodes - which is fast, because we usually span pages and can skip such nodes right away. That makes

Re: [PATCH v1 05/12] mm/memory_hotplug: remove nid parameter from remove_memory() and friends

2021-06-08 Thread David Hildenbrand
On 08.06.21 13:11, Michael Ellerman wrote: David Hildenbrand writes: There is only a single user remaining. We can simply try to offline all online nodes - which is fast, because we usually span pages and can skip such nodes right away. That makes me slightly nervous, because our big powerpc

Re: [PATCH v1 00/12] mm/memory_hotplug: "auto-movable" online policy and memory groups

2021-06-08 Thread David Hildenbrand
On 08.06.21 11:42, Oscar Salvador wrote: On Mon, Jun 07, 2021 at 09:54:18PM +0200, David Hildenbrand wrote: Hi, this series aims at improving in-kernel auto-online support. It tackles the fundamental problems that: Hi David, the idea sounds good to me, and I like that this series takes away

[PATCH v1 12/12] mm/memory_hotplug: improved dynamic memory group aware "auto-movable" online policy

2021-06-07 Thread David Hildenbrand
all number of NUMA nodes. Note: for now, there seems to be no compelling reason to make this behavior configurable. Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 25 ++ include/linux/memory.h | 3 +++ mm/memory_hotplug.c| 60 +++

[PATCH v1 11/12] mm/memory_hotplug: memory group aware "auto-movable" online policy

2021-06-07 Thread David Hildenbrand
ihin a 512 MiB unit will either be MOVABLE or not, not a mixture. We have to pass the memory group to zone_for_pfn_range() to take the memory group into account. Note: for now, there seems to be no compelling reason to make this behavior configurable. Signed-off-by: David Hildenbrand --- dr

[PATCH v1 10/12] virtio-mem: use a single dynamic memory group for a single virtio-mem device

2021-06-07 Thread David Hildenbrand
Let's use a single dynamic memory group. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 22 +++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index e327fb878143..6b9b8b7bf89d 100644

[PATCH v1 09/12] ACPI: memhotplug: use a single static memory group for a single memory device

2021-06-07 Thread David Hildenbrand
fail acpi_memory_enable_device(). Signed-off-by: David Hildenbrand --- drivers/acpi/acpi_memhotplug.c | 35 +- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/drivers/acpi/acpi_memhotplug.c b/drivers/acpi/acpi_memhotplug.c index eb4faf7c5cad

[PATCH v1 08/12] ACPI: memhotplug: memory resources cannot be enabled yet

2021-06-07 Thread David Hildenbrand
We allocate + initialize everything from scratch. In case enabling the device fails, we free all memory resourcs. Signed-off-by: David Hildenbrand --- drivers/acpi/acpi_memhotplug.c | 4 1 file changed, 4 deletions(-) diff --git a/drivers/acpi/acpi_memhotplug.c b/drivers/acpi

[PATCH v1 07/12] mm/memory_hotplug: track present pages in memory groups

2021-06-07 Thread David Hildenbrand
smart auto-online decision for individualmemory blocks within a memory group based on group statistics. Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 10 +- include/linux/memory.h | 4 include/linux/memory_hotplug.h | 13 + mm

[PATCH v1 06/12] drivers/base/memory: "memory groups" to logically group memory blocks

2021-06-07 Thread David Hildenbrand
ant a single unit to be of the same type. For now, memory groups are an internal concept that is not exposed to user space; we might want to change that in the future, though. add_memory() users can specify a mgid instead of a nid when passing the MHP_NID_IS_MGID flag. Signed-off-by: David

[PATCH v1 05/12] mm/memory_hotplug: remove nid parameter from remove_memory() and friends

2021-06-07 Thread David Hildenbrand
...@lists.linux.dev Signed-off-by: David Hildenbrand --- .../platforms/pseries/hotplug-memory.c| 9 - drivers/acpi/acpi_memhotplug.c| 7 +-- drivers/dax/kmem.c| 3 +-- drivers/virtio/virtio_mem.c | 4 ++--

[PATCH v1 04/12] mm/memory_hotplug: remove nid parameter from arch_remove_memory()

2021-06-07 Thread David Hildenbrand
.@vger.kernel.org Cc: linux...@vger.kernel.org Signed-off-by: David Hildenbrand --- arch/arm64/mm/mmu.c| 3 +-- arch/ia64/mm/init.c| 3 +-- arch/powerpc/mm/mem.c | 3 +-- arch/s390/mm/init.c| 3 +-- arch/sh/mm/init.c | 3 +-- arch/x86/mm/in

[PATCH v1 03/12] mm/memory_hotplug: introduce "auto-movable" online policy

2021-06-07 Thread David Hildenbrand
en enabling the new "auto-movable" policy, though. * memory_hotplug.auto_movable_numa_aware considers numa node stats in addition to global stats, and defaults to "true". Note: just like the old policy, the new policy won't take things like unmovable huge pages or memo

[PATCH v1 02/12] mm: track present early pages per zone

2021-06-07 Thread David Hildenbrand
only online/offline complete (individual) memory blocks. Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 14 +++--- include/linux/memory_hotplug.h | 2 +- include/linux/mmzone.h | 7 +++ mm/memory_hotplug.c| 13 ++--- mm/page_alloc.c

[PATCH v1 01/12] mm/memory_hotplug: use "unsigned long" for PFN in zone_for_pfn_range()

2021-06-07 Thread David Hildenbrand
es in the range of multiple TB. Fixes: e5e689302633 ("mm, memory_hotplug: display allowed zones in the preferred ordering") Signed-off-by: David Hildenbrand --- include/linux/memory_hotplug.h | 4 ++-- mm/memory_hotplug.c| 4 ++-- 2 files changed, 4 insertions(+), 4

[PATCH v1 00/12] mm/memory_hotplug: "auto-movable" online policy and memory groups

2021-06-07 Thread David Hildenbrand
: Dan Williams Cc: Anshuman Khandual Cc: Dave Hansen Cc: Vlastimil Babka Cc: Mike Rapoport Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: Pavel Tatashin Cc: virtualization@lists.linux-foundation.org Cc: linux...@kvack.org Cc: linux-a...@vger.kernel.org [1] https://lkml.kernel.org/r/20210

[PATCH v1 7/7] virtio-mem: prioritize unplug from ZONE_MOVABLE in Big Block Mode

2021-06-02 Thread David Hildenbrand
Let's handle unplug in Big Block Mode similar to Sub Block Mode -- prioritize memory blocks onlined to ZONE_MOVABLE. We won't care further about big blocks with mixed zones, as it's rather a corner case that won't matter in practice. Signed-off-by: David Hildenbrand --- drivers/virtio

[PATCH v1 6/7] virtio-mem: simplify high-level unplug handling in Big Block Mode

2021-06-02 Thread David Hildenbrand
with concurrent onlining/offlining either way, so we don;t have to be super correct by failing if an offline big block we'd like to unplug just got (partially) onlined. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 96 ++--- 1 file changed, 24

[PATCH v1 5/7] virtio-mem: prioritize unplug from ZONE_MOVABLE in Sub Block Mode

2021-06-02 Thread David Hildenbrand
NE_MOVABLE at all -- as we didn't allow partially plugged memory blocks in ZONE_MOVABLE before that. That commit already mentioned "In the future, we might want to remember the zone again and use the information when (un)plugging memory." Signed-off-by: David Hildenbrand --- drivers/virt

[PATCH v1 4/7] virtio-mem: simplify high-level unplug handling in Sub Block Mode

2021-06-02 Thread David Hildenbrand
is that we now temporarily unlock the hotplug mutex around cond_resched() when processing offline memory blocks, which doesn't make a real difference as we already have to temporarily unlock in virtio_mem_sbm_unplug_any_sb_offline() when removing a memory block. Signed-off-by: David Hildenbrand

[PATCH v1 3/7] virtio-mem: simplify high-level plug handling in Sub Block Mode

2021-06-02 Thread David Hildenbrand
Let's simplify high-level memory block selection when plugging in Sub Block Mode. No need for two separate loops when selecting memory blocks for plugging memory. Avoid passing the "online" state by simply obtaining the state in virtio_mem_sbm_plug_any_sb(). Signed-off-by: David H

[PATCH v1 1/7] virtio-mem: don't read big block size in Sub Block Mode

2021-06-02 Thread David Hildenbrand
;virtio-mem: Big Block Mode (BBM) memory hotplug") Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 10ec60d81e84..3bf08b5bb359 1006

[PATCH v1 2/7] virtio-mem: use page_zonenum() in virtio_mem_fake_offline()

2021-06-02 Thread David Hildenbrand
Let's use page_zonenum() instead of zone_idx(page_zone()). Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 3bf08b5bb359..1d4b1e25ac8b 100644

[PATCH v1 0/7] virtio-mem: prioritize unplug from ZONE_MOVABLE

2021-06-02 Thread David Hildenbrand
lization@lists.linux-foundation.org Cc: linux...@kvack.org David Hildenbrand (7): virtio-mem: don't read big block size in Sub Block Mode virtio-mem: use page_zonenum() in virtio_mem_fake_offline() virtio-mem: simplify high-level plug handling in Sub Block Mode virtio-mem: simplify high-level unplug h

[PATCH v3 3/6] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-05-26 Thread David Hildenbrand
) and turning memory inaccessible in the hypervisor. We'll handle this in a follow-up patch. Reviewed-by: Mike Rapoport Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand --- fs/proc/kcore.c| 14 +- include/linux/page-flags.h | 12 2 files changed,

[PATCH v3 5/6] virtio-mem: use page_offline_(start|end) when setting PageOffline()

2021-05-26 Thread David Hildenbrand
-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 10ec60d81e84..dc2a2e2b2ff8 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1065,6 +1065,7 @@ static

[PATCH v3 4/6] mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline()

2021-05-26 Thread David Hildenbrand
that care about such races when setting a page PageOffline(). For simplicity, use a rwsem for now; neither drivers nor users are performance sensitive. Acked-by: Michal Hocko Reviewed-by: Mike Rapoport Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 10

[PATCH v3 6/6] fs/proc/kcore: use page_offline_(freeze|thaw)

2021-05-26 Thread David Hildenbrand
Let's properly synchronize with drivers that set PageOffline(). Unfreeze/thaw every now and then, so drivers that want to set PageOffline() can make progress. Acked-by: Mike Rapoport Reviewed-by: Oscar Salvador Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 13 + 1 file

[PATCH v3 2/6] fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM

2021-05-26 Thread David Hildenbrand
Let's resturcture the code, using switch-case, and checking pfn_is_ram() only when we are dealing with KCORE_RAM. Reviewed-by: Mike Rapoport Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 35 +++ 1 file changed, 27 insertions(+), 8 deletions(-) diff

[PATCH v3 0/6] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-05-26 Thread David Hildenbrand
e Kravetz Cc: Aili Yao Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Naoya Horiguchi Cc: linux-hyp...@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: linux-fsde...@vger.kernel.org Cc: linux...@kvack.org David Hildenbra

[PATCH v3 1/6] fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER

2021-05-26 Thread David Hildenbrand
ile at it, also drop vaddr in "struct kcore_list", used by KCORE_REMAP only. Reviewed-by: Mike Rapoport Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 7 ++- include/linux/kcore.h | 3 --- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/fs/proc/k

Re: [PATCH v2 4/6] mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline()

2021-05-17 Thread David Hildenbrand
On 17.05.21 08:43, Mike Rapoport wrote: On Fri, May 14, 2021 at 07:22:45PM +0200, David Hildenbrand wrote: A driver might set a page logically offline -- PageOffline() -- and turn the page inaccessible in the hypervisor; after that, access to page content can be fatal. One example is virtio-mem

Re: [RFC PATCH 13/15] virtio-mem: Handle virtio_device_ready() failure

2021-05-17 Thread David Hildenbrand
turn 0; +out_unreg_device: + unregister_virtio_mem_device(vm); out_unreg_mem: unregister_memory_notifier(>memory_notifier); out_del_resource: I assume this will really be a corner case to hit, right? Failing after essentially being done initializing looks sub-optimal, anyhow: Acked-

[PATCH v2 6/6] fs/proc/kcore: use page_offline_(freeze|thaw)

2021-05-14 Thread David Hildenbrand
Let's properly synchronize with drivers that set PageOffline(). Unfreeze/thaw every now and then, so drivers that want to set PageOffline() can make progress. Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/proc/kcore.c

[PATCH v2 5/6] virtio-mem: use page_offline_(start|end) when setting PageOffline()

2021-05-14 Thread David Hildenbrand
. There will be a virtio spec extension to document this change, including a new feature flag indicating the changed behavior. We really don't want to race against PFN walkers reading random page content. Acked-by: Michael S. Tsirkin Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c

[PATCH v2 4/6] mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline()

2021-05-14 Thread David Hildenbrand
that care about such races when setting a page PageOffline(). For simplicity, use a rwsem for now; neither drivers nor users are performance sensitive. Acked-by: Michal Hocko Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 10 ++ mm/util.c | 40

[PATCH v2 1/6] fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER

2021-05-14 Thread David Hildenbrand
ile at it, also drop vaddr in "struct kcore_list", used by KCORE_REMAP only. Reviewed-by: Mike Rapoport Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 7 ++- include/linux/kcore.h | 3 --- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/fs/proc/k

[PATCH v2 3/6] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-05-14 Thread David Hildenbrand
) and turning memory inaccessible in the hypervisor. We'll handle this in a follow-up patch. Reviewed-by: Mike Rapoport Signed-off-by: David Hildenbrand --- fs/proc/kcore.c| 14 +- include/linux/page-flags.h | 12 2 files changed, 25 insertions(+), 1 deletion

[PATCH v2 2/6] fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM

2021-05-14 Thread David Hildenbrand
Let's resturcture the code, using switch-case, and checking pfn_is_ram() only when we are dealing with KCORE_RAM. Reviewed-by: Mike Rapoport Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 35 +++ 1 file changed, 27 insertions(+), 8 deletions(-) diff

[PATCH v2 0/6] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-05-14 Thread David Hildenbrand
rtualization@lists.linux-foundation.org Cc: linux-fsde...@vger.kernel.org Cc: linux...@kvack.org David Hildenbrand (6): fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM fs/proc/kcore: don't read offline sections, logically offline

Re: [PATCH v1 5/7] mm: introduce page_offline_(begin|end|freeze|unfreeze) to synchronize setting PageOffline()

2021-05-05 Thread David Hildenbrand
On 05.05.21 15:24, Michal Hocko wrote: On Thu 29-04-21 14:25:17, David Hildenbrand wrote: A driver might set a page logically offline -- PageOffline() -- and turn the page inaccessible in the hypervisor; after that, access to page content can be fatal. One example is virtio-mem; while unplugged

Re: [PATCH v1 3/7] mm: rename and move page_is_poisoned()

2021-05-05 Thread David Hildenbrand
Long story short, this should be good enough for the cases we actually can handle? What am I missing? I am not sure I follow. My point is that I fail to see any added value of the check as it doesn't prevent the race (it fundamentally cannot as the page can be poisoned at any time) but the

Re: [PATCH v1 3/7] mm: rename and move page_is_poisoned()

2021-05-05 Thread David Hildenbrand
On 05.05.21 15:13, Michal Hocko wrote: On Thu 29-04-21 14:25:15, David Hildenbrand wrote: Commit d3378e86d182 ("mm/gup: check page posion status for coredump.") introduced page_is_poisoned(), however, v5 [1] of the patch used "page_is_hwpoison()" and something went wro

Re: [PATCH v1 7/7] fs/proc/kcore: use page_offline_(freeze|unfreeze)

2021-05-03 Thread David Hildenbrand
On 03.05.21 13:33, Mike Rapoport wrote: On Mon, May 03, 2021 at 12:13:45PM +0200, David Hildenbrand wrote: On 03.05.21 11:28, Mike Rapoport wrote: On Mon, May 03, 2021 at 10:28:36AM +0200, David Hildenbrand wrote: On 02.05.21 08:34, Mike Rapoport wrote: On Thu, Apr 29, 2021 at 02:25:19PM

Re: [PATCH v1 7/7] fs/proc/kcore: use page_offline_(freeze|unfreeze)

2021-05-03 Thread David Hildenbrand
On 03.05.21 11:28, Mike Rapoport wrote: On Mon, May 03, 2021 at 10:28:36AM +0200, David Hildenbrand wrote: On 02.05.21 08:34, Mike Rapoport wrote: On Thu, Apr 29, 2021 at 02:25:19PM +0200, David Hildenbrand wrote: Let's properly synchronize with drivers that set PageOffline(). Unfreeze every

Re: [PATCH v1 7/7] fs/proc/kcore: use page_offline_(freeze|unfreeze)

2021-05-03 Thread David Hildenbrand
On 02.05.21 08:34, Mike Rapoport wrote: On Thu, Apr 29, 2021 at 02:25:19PM +0200, David Hildenbrand wrote: Let's properly synchronize with drivers that set PageOffline(). Unfreeze every now and then, so drivers that want to set PageOffline() can make progress. Signed-off-by: David Hildenbrand

Re: [PATCH v1 6/7] virtio-mem: use page_offline_(start|end) when setting PageOffline()

2021-05-03 Thread David Hildenbrand
On 02.05.21 08:33, Mike Rapoport wrote: On Thu, Apr 29, 2021 at 02:25:18PM +0200, David Hildenbrand wrote: Let's properly use page_offline_(start|end) to synchronize setting PageOffline(), so we won't have valid page access to unplugged memory regions from /proc/kcore. Signed-off-by: David

Re: [PATCH v1 5/7] mm: introduce page_offline_(begin|end|freeze|unfreeze) to synchronize setting PageOffline()

2021-05-03 Thread David Hildenbrand
On 02.05.21 08:33, Mike Rapoport wrote: On Thu, Apr 29, 2021 at 02:25:17PM +0200, David Hildenbrand wrote: A driver might set a page logically offline -- PageOffline() -- and turn the page inaccessible in the hypervisor; after that, access to page content can be fatal. One example is virtio-mem

[PATCH v1 7/7] fs/proc/kcore: use page_offline_(freeze|unfreeze)

2021-04-29 Thread David Hildenbrand
Let's properly synchronize with drivers that set PageOffline(). Unfreeze every now and then, so drivers that want to set PageOffline() can make progress. Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/fs/proc/kcore.c b

[PATCH v1 5/7] mm: introduce page_offline_(begin|end|freeze|unfreeze) to synchronize setting PageOffline()

2021-04-29 Thread David Hildenbrand
by drivers that care about such races when setting a page PageOffline(). For simplicity, use a rwsem for now; neither drivers nor users are performance sensitive. Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 5 + mm/util.c | 38

[PATCH v1 6/7] virtio-mem: use page_offline_(start|end) when setting PageOffline()

2021-04-29 Thread David Hildenbrand
Let's properly use page_offline_(start|end) to synchronize setting PageOffline(), so we won't have valid page access to unplugged memory regions from /proc/kcore. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 2 ++ mm/util.c | 2 ++ 2 files changed, 4

[PATCH v1 4/7] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-04-29 Thread David Hildenbrand
g unplugged: however, we will properly tear down the identity mapping and handle faults gracefully when accessing this memory from kcore code. Note 2: we can race against drivers setting PageOffline() and turning memory inaccessible in the hypervisor. We'll handle this in a follow-up patch.

[PATCH v1 3/7] mm: rename and move page_is_poisoned()

2021-04-29 Thread David Hildenbrand
ed in other -- kcore -- context. Move the comment to the place where it belongs and simplify. [1] https://lkml.kernel.org/r/20210322193318.377c9ce9@alex-virtual-machine Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 7 +++ mm/gup.c | 6 +- mm/internal

[PATCH v1 2/7] fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM

2021-04-29 Thread David Hildenbrand
Let's resturcture the code, using switch-case, and checking pfn_is_ram() only when we are dealing with KCORE_RAM. Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 35 +++ 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/fs/proc/kcore.c b/fs

[PATCH v1 1/7] fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER

2021-04-29 Thread David Hildenbrand
ile at it, also drop vaddr in "struct kcore_list", used by KCORE_REMAP only. Signed-off-by: David Hildenbrand --- fs/proc/kcore.c | 7 ++- include/linux/kcore.h | 3 --- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 4d2

[PATCH v1 0/7] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages

2021-04-29 Thread David Hildenbrand
ang Cc: Stephen Hemminger Cc: Wei Liu Cc: Naoya Horiguchi Cc: linux-hyp...@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: linux-fsde...@vger.kernel.org Cc: linux...@kvack.org David Hildenbrand (7): fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER fs/proc/kcore: pfn_is_ram c

Re: [PATCH] virtio-balloon: fix a typo in comment of virtballoon_migratepage()

2021-03-29 Thread David Hildenbrand
age that is about to be migrated to newpage. Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH] virtio-balloon: move release_pages_balloon() outside of mutex_unlock(>balloon_lock)

2021-03-26 Thread David Hildenbrand
(vb, vb->deflate_vq); - release_pages_balloon(vb, ); mutex_unlock(>balloon_lock); + release_pages_balloon(vb, ); return num_freed_pages; } I think this should be fine Reviewed-by: David Hildenbrand -- Thanks, David /

Re: [PATCH 6/9] virtio_balloon: remove the balloon-kvm file system

2021-03-09 Thread David Hildenbrand
On 09.03.21 16:53, Christoph Hellwig wrote: Just use the generic anon_inode file system. Signed-off-by: Christoph Hellwig --- drivers/virtio/virtio_balloon.c | 30 +++--- 1 file changed, 3 insertions(+), 27 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c

Re: [PATCH 5/9] vmw_balloon: remove the balloon-vmware file system

2021-03-09 Thread David Hildenbrand
On 09.03.21 16:53, Christoph Hellwig wrote: Just use the generic anon_inode file system. Signed-off-by: Christoph Hellwig --- drivers/misc/vmw_balloon.c | 24 ++-- 1 file changed, 2 insertions(+), 22 deletions(-) diff --git a/drivers/misc/vmw_balloon.c

Re: [PATCH 3/9] powerpc/pseries: remove the ppc-cmm file system

2021-03-09 Thread David Hildenbrand
On 09.03.21 16:53, Christoph Hellwig wrote: Just use the generic anon_inode file system. Signed-off-by: Christoph Hellwig --- arch/powerpc/platforms/pseries/cmm.c | 27 ++- 1 file changed, 2 insertions(+), 25 deletions(-) diff --git

Re: [PATCH 2/9] fs: add an argument-less alloc_anon_inode

2021-03-09 Thread David Hildenbrand
void *priv, int flags, const struct inode *context_inode); +struct inode *alloc_anon_inode(void); #endif /* _LINUX_ANON_INODES_H */ Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtua

Re: [PATCH 1/9] fs: rename alloc_anon_inode to alloc_anon_inode_sb

2021-03-09 Thread David Hildenbrand
_inode(zsmalloc_mnt->mnt_sb); + pool->inode = alloc_anon_inode_sb(zsmalloc_mnt->mnt_sb); if (IS_ERR(pool->inode)) { pool->inode = NULL; return 1; Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH v1 2/2] virtio-mem: support VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE

2021-02-15 Thread David Hildenbrand
Sebastien Boeuf Cc: Pankaj Gupta Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 include/uapi/linux/virtio_mem.h | 10 +++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virti

[PATCH v1 0/2] virito-mem: one fix and VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE

2021-02-15 Thread David Hildenbrand
only support adding/ removing individual Linux memory blocks (e.g., 128MB on x86-64). While we might still be able to allow for reading unplugged memory with file-based memory backends in the future (and I have plans/prototypes for it), at least in the near future we cannot support it. David

[PATCH v1 1/2] virtio-mem: don't read big block size in SBM

2021-02-15 Thread David Hildenbrand
Fixes: 4ba50cd3355d ("virtio-mem: Big Block Mode (BBM) memory hotplug") Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Marek Kedzierski Cc: Hui Zhu Cc: Pankaj Gupta Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 --- 1 file changed, 8

[PATCH v1] mm/memory_hotplug: MEMHP_MERGE_RESOURCE -> MHP_MERGE_RESOURCE

2021-01-26 Thread David Hildenbrand
: Anshuman Khandual Cc: Wei Yang Cc: linux-hyp...@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: xen-de...@lists.xenproject.org Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c| 2 +- drivers/virtio/virtio_mem.c| 2 +- drivers/xen/balloon.c | 2 +- inc

Re: [PATCH] virtio-mem: Assign boolean values to a bool variable

2021-01-20 Thread David Hildenbrand
>>> Want to send your ack on this one? >> >> Sure >> >> Acked-by: David Hildenbrand > > > Added yours and the original Signed-off-by. > > Thanks! Thanks Michael! -- Thanks, David / dhildenb __

Re: [PATCH] virtio-mem: Assign boolean values to a bool variable

2021-01-20 Thread David Hildenbrand
On 20.01.21 12:03, Michael S. Tsirkin wrote: > On Wed, Jan 20, 2021 at 11:04:18AM +0100, David Hildenbrand wrote: >> On 20.01.21 10:57, Michael S. Tsirkin wrote: >>> On Wed, Jan 20, 2021 at 10:40:37AM +0100, David Hildenbrand wrote: >>>> On 20.01.21 08:50,

Re: [PATCH] virtio-mem: Assign boolean values to a bool variable

2021-01-20 Thread David Hildenbrand
On 20.01.21 10:57, Michael S. Tsirkin wrote: > On Wed, Jan 20, 2021 at 10:40:37AM +0100, David Hildenbrand wrote: >> On 20.01.21 08:50, Jiapeng Zhong wrote: >>> Fix the following coccicheck warnings: >>> >>> ./drivers/virtio/virtio_mem.c:2580:2-25: WARNING: As

Re: [PATCH] virtio-mem: Assign boolean values to a bool variable

2021-01-20 Thread David Hildenbrand
On 20.01.21 08:50, Jiapeng Zhong wrote: > Fix the following coccicheck warnings: > > ./drivers/virtio/virtio_mem.c:2580:2-25: WARNING: Assignment > of 0/1 to bool variable. > > Reported-by: Abaci Robot > Signed-off-by: Jiapeng Zhong > --- > drivers/virtio/virtio_mem.c | 2 +- > 1 file

Re: [PATCH 3/6] hugetlb: add free page reporting support

2021-01-07 Thread David Hildenbrand
On 07.01.21 04:38, Liang Li wrote: > On Thu, Jan 7, 2021 at 12:08 AM Michal Hocko wrote: >> >> On Tue 05-01-21 22:49:21, Liang Li wrote: >>> hugetlb manages its page in hstate's free page list, not in buddy >>> system, this patch try to make it works for hugetlbfs. It canbe >>> used for memory

Re: [PATCH 0/6] hugetlbfs: support free page reporting

2021-01-06 Thread David Hildenbrand
On 06.01.21 04:46, Liang Li wrote: > A typical usage of hugetlbfs it's to reserve amount of memory > during the kernel booting stage, and the reserved pages are > unlikely to return to the buddy system. When application need > hugepages, kernel will allocate them from the reserved pool. > when

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-05 Thread David Hildenbrand
On 05.01.21 11:22, Liang Li wrote: That‘s mostly already existing scheduling logic, no? (How many vms can I put onto a specific machine eventually) >>> >>> It depends on how the scheduling component is designed. Yes, you can put >>> 10 VMs with 4C8G(4CPU, 8G RAM) on a host and 20 VMs

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-05 Thread David Hildenbrand
On 05.01.21 03:14, Liang Li wrote: > In our production environment, there are three main applications have such > requirement, one is QEMU [creating a VM with SR-IOV passthrough device], > anther other two are DPDK related applications, DPDK OVS and SPDK vhost, > for best

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

2021-01-05 Thread David Hildenbrand
On 05.01.21 10:20, Michal Hocko wrote: > On Mon 04-01-21 15:00:31, Dave Hansen wrote: >> On 1/4/21 12:11 PM, David Hildenbrand wrote: >>>> Yeah, it certainly can't be the default, but it *is* useful for >>>> thing where we know that there are no cache benefi

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-04 Thread David Hildenbrand
> Am 23.12.2020 um 13:12 schrieb Liang Li : > > On Wed, Dec 23, 2020 at 4:41 PM David Hildenbrand wrote: >> >> [...] >> >>>> I was rather saying that for security it's of little use IMHO. >>>> Application/VM start up time might be

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

2021-01-04 Thread David Hildenbrand
> Am 04.01.2021 um 20:52 schrieb Dave Hansen : > > On 1/4/21 11:27 AM, Matthew Wilcox wrote: >>> On Mon, Jan 04, 2021 at 11:19:13AM -0800, Dave Hansen wrote: >>> On 12/21/20 8:30 AM, Liang Li wrote: --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -137,6

Re: [PATCH] virtio-mem: use false for bool variable

2020-12-29 Thread David Hildenbrand
; > } > > Thanks - the patch subject is a little weird ("false"). I suggest "virtio-mem: use boolean value when setting vm->unplug_all_required" Apart from that Acked-by: David Hildenbrand @Mst, do you want a resend or can you fixup the subject

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-23 Thread David Hildenbrand
[...] >> I was rather saying that for security it's of little use IMHO. >> Application/VM start up time might be improved by using huge pages (and >> pre-zeroing these). Free page reporting might be improved by using >> MADV_FREE instead of MADV_DONTNEED in the hypervisor. >> >>> this feature,

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread David Hildenbrand
> >>> >>> Virtulization >>> = >>> Speed up VM creation and shorten guest boot time, especially for PCI >>> SR-IOV device passthrough scenario. Compared with some of the para >>> vitalization solutions, it is easy to deploy because it’s transparent >>> to guest and can handle DMA

Re: [RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-22 Thread David Hildenbrand
On 22.12.20 09:31, David Hildenbrand wrote: > On 22.12.20 08:49, Liang Li wrote: >> This patch add support of pre zero out free hugepage, we can use >> this feature to speed up page population and page fault handing. >> >> Cc: Alexander Duyck >> Cc: Mel Gorman &g

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread David Hildenbrand
On 21.12.20 17:25, Liang Li wrote: > The first version can be found at: https://lkml.org/lkml/2020/4/12/42 > > Zero out the page content usually happens when allocating pages with > the flag of __GFP_ZERO, this is a time consuming operation, it makes > the population of a large vma area very

Re: [RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-22 Thread David Hildenbrand
On 22.12.20 08:49, Liang Li wrote: > This patch add support of pre zero out free hugepage, we can use > this feature to speed up page population and page fault handing. > > Cc: Alexander Duyck > Cc: Mel Gorman > Cc: Andrea Arcangeli > Cc: Dan Williams > Cc:

Re: [RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-22 Thread David Hildenbrand
On 22.12.20 08:48, Liang Li wrote: > Free page reporting only supports buddy pages, it can't report the > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs The virtio-balloon free page reporting interface accepts a generic sg, so it isn't glue to buddy pages. There is no need

Re: [PATCH v2 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block

2020-11-18 Thread David Hildenbrand
On 18.11.20 05:53, Andrew Morton wrote: On Thu, 12 Nov 2020 14:38:13 +0100 David Hildenbrand wrote: virtio-mem soon wants to use offline_and_remove_memory() memory that exceeds a single Linux memory block (memory_block_size_bytes()). Let's remove that restriction. Let's remember the old

[PATCH v2 29/29] virtio-mem: Big Block Mode (BBM) - safe memory hotunplug

2020-11-12 Thread David Hildenbrand
Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 97 - 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index f1696cdb7b0c..9fc9ec4a25f5 100644

[PATCH v2 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug

2020-11-12 Thread David Hildenbrand
that we might see in the future. So we won't care how many Linux memory blocks a big block actually spans, and how the memory notifier is called. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off

[PATCH v2 28/29] virtio-mem: Big Block Mode (BBM) - basic memory hotunplug

2020-11-12 Thread David Hildenbrand
block sizes. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 156 +++- 1 file changed, 155 insert

[PATCH v2 26/29] virtio-mem: allow to force Big Block Mode (BBM) and set the big block size

2020-11-12 Thread David Hildenbrand
ael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 31 --- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/driv

[PATCH v2 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block

2020-11-12 Thread David Hildenbrand
-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- mm/memory_hotplug.c | 105 +--- 1 file changed, 89 inserti

[PATCH v2 22/29] virito-mem: existing (un)plug functions are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename them accordingly. virtio_mem_plug_request() and virtio_mem_unplug_request() will be handled separately. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 21/29] virtio-mem: memory block ids are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's move first_mb_id/next_mb_id/last_usable_mb_id accordingly. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 46 ++--- 1 file changed, 23 inserti

[PATCH v2 20/29] virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename to "sbs_per_mb" and "sb_size" and move accordingly. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 96 ++--- 1 file

[PATCH v2 19/29] virito-mem: subblock states are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename and move accordingly. While at it, rename sb_bitmap to "sb_states". Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virti

[PATCH v2 24/29] virtio-mem: factor out adding/removing memory from Linux

2020-11-12 Thread David Hildenbrand
that in racy context soon (and we do have proper BUG_ON() statements in the current cases where it must never happen). Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_m

[PATCH v2 23/29] virtio-mem: memory notifier callbacks are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename accordingly. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 29 +++-- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

<    1   2   3   4   5   6   7   8   9   >