Re: [PATCH v2] virtio_balloon: Fix endless deflation and inflation on arm64

2023-10-02 Thread David Hildenbrand
On 25.09.23 01:58, Gavin Shan wrote: Hi David and Michael, On 8/31/23 11:10, Gavin Shan wrote: The deflation request to the target, which isn't unaligned to the guest page size causes endless deflation and inflation actions. For example, we receive the flooding QMP events for the changes on mem

Re: [PATCH v2] virtio_balloon: Fix endless deflation and inflation on arm64

2023-08-31 Thread David Hildenbrand
x27;s size can be stablizied to 0x3ffe soon after the deflation request is sent. { "execute" : "balloon", "arguments": { "value" : 1073672192 } } {"return": {}} {"timestamp": {"seconds": 1693273328, "microseconds": 79307

Re: [PATCH] virtio_balloon: Fix endless deflation and inflation on arm64

2023-08-30 Thread David Hildenbrand
On 29.08.23 03:54, Gavin Shan wrote: The deflation request to the target, which isn't unaligned to the guest page size causes endless deflation and inflation actions. For example, we receive the flooding QMP events for the changes on memory balloon's size after a deflation request to the unaligne

Re: [virtio-comment] virtio queue numbering and optional queues

2023-08-22 Thread David Hildenbrand
On 22.08.23 15:40, Stefan Hajnoczi wrote: On Mon, Aug 21, 2023 at 03:18:50PM -0700, Daniel Verkamp wrote: Hello virtio folks, Hi Daniel, I have CCed those involved in the free page hint and page reporting features. Stefan I noticed a mismatch between the way the specification defines devic

Re: [PATCH] virtio-balloon: correct the comment of virtballoon_migratepage()

2023-08-14 Thread David Hildenbrand
age, struct page *page, enum migrate_mode mode) Reviewed-by: David Hildenbrand -- Cheers, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v1 0/4] virtio-mem: memory unplug/offlining related cleanups

2023-07-14 Thread David Hildenbrand
On 13.07.23 17:03, Michael S. Tsirkin wrote: On Thu, Jul 13, 2023 at 04:55:47PM +0200, David Hildenbrand wrote: Some cleanups+optimizations primarily around offline_and_remove_memory(). Patch #1 drops the "unsafe unplug" feature where we might get stuck in offline_and_remove_memory

[PATCH v1 4/4] virtio-mem: check if the config changed before fake offlining memory

2023-07-13 Thread David Hildenbrand
y and we're not supposed to unplug memory anymore. For example, the hypervisor might detect a low-memory situation while unplugging memory and decide to replug some memory. Continuing trying to unplug memory in that case can be problematic. So let's check on a more regular basis. Signe

[PATCH v1 2/4] virtio-mem: convert most offline_and_remove_memory() errors to -EBUSY

2023-07-13 Thread David Hildenbrand
Just like we do with alloc_contig_range(), let's convert all unknown errors to -EBUSY, but WARN so we can look into the issue. For example, offline_pages() could fail with -EINTR, which would be unexpected in our case. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c

[PATCH v1 3/4] virtio-mem: keep retrying on offline_and_remove_memory() errors in Sub Block Mode (SBM)

2023-07-13 Thread David Hildenbrand
ntig_range() user, if allocating temporary memory fails, or if some memory notifier rejected the offlining request. Let's handle that case better, by simple retrying to offline and remove such memory. Tested using CONFIG_MEMORY_NOTIFIER_ERROR_INJECT. Signed-off-by: David Hildenbrand ---

[PATCH v1 0/4] virtio-mem: memory unplug/offlining related cleanups

2023-07-13 Thread David Hildenbrand
S. Tsirkin" Cc: Jason Wang Cc: Xuan Zhuo David Hildenbrand (4): virtio-mem: remove unsafe unplug in Big Block Mode (BBM) virtio-mem: convert most offline_and_remove_memory() errors to -EBUSY virtio-mem: keep retrying on offline_and_remove_memory() errors in Sub Block Mode (SBM) virt

[PATCH v1 1/4] virtio-mem: remove unsafe unplug in Big Block Mode (BBM)

2023-07-13 Thread David Hildenbrand
ed mechanism; and (b) this mode is not the default and as far as I know, it's unused either way. So let's simply get rid of it. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 51 +++-- 1 file changed, 20 insertions(+), 31 deletions(-

Re: [PATCH v1 3/5] mm/memory_hotplug: make offline_and_remove_memory() timeout instead of failing on fatal signals

2023-06-27 Thread David Hildenbrand
On 27.06.23 16:17, Michal Hocko wrote: On Tue 27-06-23 15:14:11, David Hildenbrand wrote: On 27.06.23 14:40, Michal Hocko wrote: On Tue 27-06-23 13:22:18, David Hildenbrand wrote: John Hubbard writes [1]: Some device drivers add memory to the system via memory hotplug

Re: [PATCH v1 1/5] mm/memory_hotplug: check for fatal signals only in offline_pages()

2023-06-27 Thread David Hildenbrand
On 27.06.23 14:34, Michal Hocko wrote: On Tue 27-06-23 13:22:16, David Hildenbrand wrote: Let's check for fatal signals only. That looks cleaner and still keeps the documented use case for manual user-space triggered memory offlining working. From Documentation/admin-guide/mm/memory-hotplu

Re: [PATCH v1 3/5] mm/memory_hotplug: make offline_and_remove_memory() timeout instead of failing on fatal signals

2023-06-27 Thread David Hildenbrand
On 27.06.23 14:40, Michal Hocko wrote: On Tue 27-06-23 13:22:18, David Hildenbrand wrote: John Hubbard writes [1]: Some device drivers add memory to the system via memory hotplug. When the driver is unloaded, that memory is hot-unplugged. However, memory hot unplug

[PATCH v1 3/5] mm/memory_hotplug: make offline_and_remove_memory() timeout instead of failing on fatal signals

2023-06-27 Thread David Hildenbrand
https://lkml.kernel.org/r/20230620011719.155379-1-jhubb...@nvidia.com Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c| 2 +- include/linux/memory_hotplug.h | 2 +- mm/memory_hotplug.c| 50 -- 3 files changed, 50 insertions(+), 4 d

[PATCH v1 4/5] virtio-mem: set the timeout for offline_and_remove_memory() to 10 seconds

2023-06-27 Thread David Hildenbrand
parameter description and the relationship to unmovable pages. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index f8792223f1db..7468b4a907e3 10

[PATCH v1 5/5] virtio-mem: check if the config changed before (fake) offlining memory

2023-06-27 Thread David Hildenbrand
we're not supposed to unplug memory anymore. Let's optimize for that case, identified while testing the offline_and_remove() memory timeout and simulating it repeatedly running into the timeout. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 8 1 file change

[PATCH v1 2/5] virtio-mem: convert most offline_and_remove_memory() errors to -EBUSY

2023-06-27 Thread David Hildenbrand
Let's prepare for offline_and_remove_memory() to return other error codes that effectively translate to -EBUSY, such as -ETIMEDOUT. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/v

[PATCH v1 0/5] mm/memory_hotplug: make offline_and_remove_memory() timeout instead of failing on fatal signals

2023-06-27 Thread David Hildenbrand
0230620011719.155379-1-jhubb...@nvidia.com David Hildenbrand (5): mm/memory_hotplug: check for fatal signals only in offline_pages() virtio-mem: convert most offline_and_remove_memory() errors to -EBUSY mm/memory_hotplug: make offline_and_remove_memory() timeout instead of failing on fa

[PATCH v1 1/5] mm/memory_hotplug: check for fatal signals only in offline_pages()

2023-06-27 Thread David Hildenbrand
ument there: "the offlining context can be terminated by sending a fatal signal". Signed-off-by: David Hildenbrand --- mm/memory_hotplug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 8e0fa209d533..0d2151df4ee1

Re: [PATCH 08/26] virtio-mem: use array_size

2023-06-26 Thread David Hildenbrand
* C2,...) | alloc( - (E1) * (E2) + array_size(E1, E2) ,...) ) // Signed-off-by: Julia Lawall --- drivers/virtio/virtio_mem.c |6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) Reviewed-by: David Hildenbrand -- Cheers, David / dhildenb

Re: [PATCH 0/7] riscv: Memory Hot(Un)Plug support

2023-05-22 Thread David Hildenbrand
On 21.05.23 11:15, Björn Töpel wrote: Hi David and Anshuman! Björn Töpel writes: David Hildenbrand writes: On 12.05.23 16:57, Björn Töpel wrote: From: Björn Töpel Memory Hot(Un)Plug support for the RISC-V port == [...] Cool stuff! I&#

Re: [PATCH 0/7] riscv: Memory Hot(Un)Plug support

2023-05-17 Thread David Hildenbrand
io-mem-pci"), and can be found here https://github.com/bjoto/qemu/tree/riscv-virtio-mem I will try to upstream that work in parallel with this. Thanks to David Hildenbrand for valuable input for the Qemu side of things. The series is based on the RISC-V fixes tree https://git.kernel.org/

Re: [PATCH v3 6/7] mm/gup: remove vmas parameter from pin_user_pages()

2023-04-17 Thread David Hildenbrand
-by: Lorenzo Stoakes --- Ideally, we'd avoid FOLL_SAME_FILE as well Acked-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/lis

Re: [RFC PATCH 01/19] mm: Introduce vm_account

2023-01-31 Thread David Hildenbrand
On 24.01.23 06:42, Alistair Popple wrote: Kernel drivers that pin pages should account these pages against either user->locked_vm or mm->pinned_vm and fail the pinning if RLIMIT_MEMLOCK is exceeded and CAP_IPC_LOCK isn't held. Currently drivers open-code this accounting and use various methods t

Re: [RFC] memory pressure detection in VMs using PSI mechanism for dynamically inflating/deflating VM memory

2023-01-24 Thread David Hildenbrand
On 24.01.23 00:04, Sudarshan Rajagopalan wrote: [...] One problematic thing is that adding memory to Linux by virtio-mem eventually consumes memory (e.g., the memmap), especially when having to to add a completely new memory block to Linux. Yes we have thought about this issue as well where-in

Re: [RFC] memory pressure detection in VMs using PSI mechanism for dynamically inflating/deflating VM memory

2023-01-23 Thread David Hildenbrand
1. This will be a native userspace daemon that will be running only in the Linux VM which will use virtio-mem driver that uses memory hotplug to add/remove memory. The VM (aka Secondary VM, SVM) will request for memory from the host which is Primary VM, PVM via the backend hypervisor which takes

Re: [RFC] memory pressure detection in VMs using PSI mechanism for dynamically inflating/deflating VM memory

2023-01-17 Thread David Hildenbrand
On 15.01.23 04:57, Sudarshan Rajagopalan wrote: Hello all, Hi, I'll focus on the virtio-mem side of things :) We’re from the Linux memory team here at Qualcomm. We are currently devising a VM memory resizing feature where we dynamically inflate or deflate the Linux VM based on ongoing memor

Re: [PATCH] virtio_balloon: high order allocation

2023-01-02 Thread David Hildenbrand
On 29.12.22 06:31, Soichiro Ueda wrote: Hi David. How does this affect page migration / balloon compaction etc? I guess this patch doesn't affect balloon compaction. When allocating pages using alloc_pages(), it skips compaction by masking out __GFP_RECLAIM if the order is larger than 0. As

Re: [PATCH] virtio_balloon: high order allocation

2022-12-23 Thread David Hildenbrand
On 23.12.22 10:35, Soichiro Ueda wrote: At present, the VirtIO balloon device driver allocates pages one by one using alloc_page(), and frees them using put_page(). This changes it so that the driver allocates high order pages by using alloc_pages(), and frees them using __free_pages() if possib

Re: [PATCH 2/4] virtio-mem: Fix probe failed when modprobe virtio_mem

2022-11-28 Thread David Hildenbrand
/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2793,6 +2793,7 @@ static int virtio_mem_probe(struct virtio_device *vdev) return 0; out_del_vq: + virtio_reset_device(vdev); vdev->config->del_vqs(vdev); out_free_vm: kfree(vm); Apart from that Review

Re: RFC [PATCH v4 2/7] Enable balloon drivers to report inflated memory

2022-10-14 Thread David Hildenbrand
Other problem is that there are drivers that do not use adjust_managed_page_count(). Which ones? Do we care? VMWare and Virtio balloon drivers. I recently proposed to unify them and the objection was that it would break existing users - which is valid so we must care i guess. I'm confused,

Re: RFC [PATCH v4 2/7] Enable balloon drivers to report inflated memory

2022-10-14 Thread David Hildenbrand
On 14.10.22 14:50, Alexander Atanasov wrote: Hello, On 11.10.22 12:23, David Hildenbrand wrote: Sounds to me that all you want is some notifier to be called from adjust_managed_page_count(). What am I missing? Notifier will act as an accumulator to report size of change and it will make

Re: RFC [PATCH v4 2/7] Enable balloon drivers to report inflated memory

2022-10-11 Thread David Hildenbrand
Sounds to me that all you want is some notifier to be called from adjust_managed_page_count(). What am I missing? Notifier will act as an accumulator to report size of change and it will make things easier for the drivers and users wrt locking. Notifier is similar to the memory hotplug notifier

Re: [RFC] how the ballooned memory should be accounted by the drivers inside the guests? (was:[PATCH v6 1/2] Create debugfs file with virtio balloon usage information)

2022-08-09 Thread David Hildenbrand
On 09.08.22 11:36, Alexander Atanasov wrote: > Hello, > > On 2.08.22 16:48, David Hildenbrand wrote: >>>> >>>> In case of Hyper-V I remember a customer BUG report that requested that >>>> exact behavior, however, I'm not a

Re: [RFC] how the ballooned memory should be accounted by the drivers inside the guests? (was:[PATCH v6 1/2] Create debugfs file with virtio balloon usage information)

2022-08-02 Thread David Hildenbrand
>> >> In case of Hyper-V I remember a customer BUG report that requested that >> exact behavior, however, I'm not able to locate the BZ quickly. >> [1] >> https://lists.linuxfoundation.org/pipermail/virtualization/2021-November/057767.html >> (note that I can't easily find the original mail in the

Re: [PATCH v6 1/2] Create debugfs file with virtio balloon usage information

2022-08-01 Thread David Hildenbrand
>>> + >>> + if (!virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) >>> + num_pages = -num_pages; >> With VIRTIO_BALLOON_F_DEFLATE_ON_OOM this will now always report "0". >> >> which would be the same as "num_pages = 0;" and would deserve a comment >> explaining why we don't

Re: [PATCH v6 1/2] Create debugfs file with virtio balloon usage information

2022-08-01 Thread David Hildenbrand
On 26.07.22 16:08, Alexander Atanasov wrote: > Allow the guest to know how much it is ballooned by the host > and how that memory is accounted. > > It is useful when debugging out of memory conditions, > as well for userspace processes that monitor the memory pressure > and for nested virtualizati

Re: [PATCH v6 2/2] Unify how inflated memory is accounted in virtio balloon driver

2022-08-01 Thread David Hildenbrand
On 26.07.22 16:10, Alexander Atanasov wrote: > Always account inflated memory as used for both cases - with and > without deflate on oom. Do not change total ram which can confuse > userspace and users. Sorry, but NAK. This would affect existing users / user space / balloon stats. For example HV

Re: [PATCH v5 1/1] Create debugfs file with virtio balloon usage information

2022-07-25 Thread David Hildenbrand
On 25.07.22 13:27, Alexander Atanasov wrote: > Hi, > > On 18/07/2022 14:35, David Hildenbrand wrote: >> On 14.07.22 15:20, Alexander Atanasov wrote: >>> Allow the guest to know how much it is ballooned by the host. >>> It is useful when debugging out of memory c

Re: [PATCH v5 1/1] Create debugfs file with virtio balloon usage information

2022-07-18 Thread David Hildenbrand
On 14.07.22 15:20, Alexander Atanasov wrote: > Allow the guest to know how much it is ballooned by the host. > It is useful when debugging out of memory conditions. > > When host gets back memory from the guest it is accounted > as used memory in the guest but the guest have no way to know > how m

Re: [PATCH v4 1/1] Create debugfs file with virtio balloon usage information

2022-07-14 Thread David Hildenbrand
On 14.07.22 15:20, Alexander Atanasov wrote: > Hello, > > On 14/07/2022 14:35, David Hildenbrand wrote: >> On 05.07.22 10:36, Alexander Atanasov wrote: >>> Allow the guest to know how much it is ballooned by the host. >>> It is useful when debugging out of mem

Re: [PATCH v4 1/1] Create debugfs file with virtio balloon usage information

2022-07-14 Thread David Hildenbrand
On 05.07.22 10:36, Alexander Atanasov wrote: > Allow the guest to know how much it is ballooned by the host. > It is useful when debugging out of memory conditions. > > When host gets back memory from the guest it is accounted > as used memory in the guest but the guest have no way to know > how m

Re: virtio_balloon regression in 5.19-rc3

2022-06-21 Thread David Hildenbrand
On 20.06.22 20:49, Ben Hutchings wrote: > I've tested a 5.19-rc3 kernel on top of QEMU/KVM with machine type > pc-q35-5.2. It has a virtio balloon device defined in libvirt as: > > >function="0x0"/> > > > but the virtio_balloon driver fails to bind to it: > > virtio_balloo

Re: [PATCH v2 03/19] fs: Add aops->migrate_folio

2022-06-10 Thread David Hildenbrand
On 09.06.22 16:35, Matthew Wilcox wrote: > On Thu, Jun 09, 2022 at 02:50:20PM +0200, David Hildenbrand wrote: >> On 08.06.22 17:02, Matthew Wilcox (Oracle) wrote: >>> diff --git a/Documentation/filesystems/locking.rst >>> b/Documentation/filesystems/locking

[PATCH v1] drivers/virtio: Clarify CONFIG_VIRTIO_MEM for unsupported architectures

2022-06-10 Thread David Hildenbrand
Let's make it clearer that simply unlocking CONFIG_VIRTIO_MEM on an architecture is most probably not sufficient to have it working as expected. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Gavin Shan Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 8 +--- 1

Re: [PATCH v2 03/19] fs: Add aops->migrate_folio

2022-06-09 Thread David Hildenbrand
On 08.06.22 17:02, Matthew Wilcox (Oracle) wrote: > Provide a folio-based replacement for aops->migratepage. Update the > documentation to document migrate_folio instead of migratepage. > > Signed-off-by: Matthew Wilcox (Oracle) > Reviewed-by: Christoph Hellwig > --- > Documentation/filesystem

Re: [PATCH v2 01/19] secretmem: Remove isolate_page

2022-06-09 Thread David Hildenbrand
secretmem_isolate_page, > }; > > static int secretmem_setattr(struct user_namespace *mnt_userns, Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v2 02/19] mm: Convert all PageMovable users to movable_operations

2022-06-09 Thread David Hildenbrand
| 102 ++- > 14 files changed, 94 insertions(+), 365 deletions(-) You probably should have cc'ed the relevant maintainers (including me :P ). For everything except z3fold.c and zsmalloc.c, Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb __

Re: [PATCH 15/20] balloon: Convert to migrate_folio

2022-06-08 Thread David Hildenbrand
On 07.06.22 21:21, Matthew Wilcox wrote: > On Tue, Jun 07, 2022 at 03:24:15PM +0100, Matthew Wilcox wrote: >> On Tue, Jun 07, 2022 at 09:36:21AM +0200, David Hildenbrand wrote: >>> On 06.06.22 22:40, Matthew Wilcox (Oracle) wrote: >>>> const struct address

Re: [PATCH 15/20] balloon: Convert to migrate_folio

2022-06-07 Thread David Hildenbrand
On 06.06.22 22:40, Matthew Wilcox (Oracle) wrote: > This is little more than changing the types over; there's no real work > being done in this function. > > Signed-off-by: Matthew Wilcox (Oracle) > --- > mm/balloon_compaction.c | 15 +++ > 1 file changed, 7 insertions(+), 8 deletion

Re: [PATCH 0/3] recover hardware corrupted page by virtio balloon

2022-06-02 Thread David Hildenbrand
On 02.06.22 11:28, zhenwei pi wrote: > On 6/1/22 15:59, David Hildenbrand wrote: >> On 01.06.22 04:17, zhenwei pi wrote: >>> On 5/31/22 12:08, Jue Wang wrote: >>>> On Mon, May 30, 2022 at 8:49 AM Peter Xu wrote: >>>>> >>>>> On Mon, May

Re: [PATCH 0/3] recover hardware corrupted page by virtio balloon

2022-06-01 Thread David Hildenbrand
On 01.06.22 04:17, zhenwei pi wrote: > On 5/31/22 12:08, Jue Wang wrote: >> On Mon, May 30, 2022 at 8:49 AM Peter Xu wrote: >>> >>> On Mon, May 30, 2022 at 07:33:35PM +0800, zhenwei pi wrote: A VM uses RAM of 2M huge page. Once a MCE(@HVAy in [HVAx,HVAz)) occurs, the 2M([HVAx,HVAz)) of h

Re: [PATCH 3/3] virtio_balloon: Introduce memory recover

2022-05-30 Thread David Hildenbrand
On 25.05.22 01:32, zhenwei pi wrote: > > > On 5/25/22 03:35, Sean Christopherson wrote: >> On Fri, May 20, 2022, zhenwei pi wrote: >>> @@ -59,6 +60,12 @@ enum virtio_balloon_config_read { >>> VIRTIO_BALLOON_CONFIG_READ_CMD_ID = 0, >>> }; >>> >>> +/* the request body to commucate with hos

Re: [PATCH 3/3] virtio_balloon: Introduce memory recover

2022-05-30 Thread David Hildenbrand
> + > struct virtio_balloon { > struct virtio_device *vdev; > struct virtqueue *inflate_vq, *deflate_vq, *stats_vq, *free_page_vq; > @@ -126,6 +133,16 @@ struct virtio_balloon { > /* Free page reporting device */ > struct virtqueue *reporting_vq; > struct page_repor

Re: [PATCH 0/3] recover hardware corrupted page by virtio balloon

2022-05-30 Thread David Hildenbrand
On 27.05.22 08:32, zhenwei pi wrote: > On 5/27/22 02:37, Peter Xu wrote: >> On Wed, May 25, 2022 at 01:16:34PM -0700, Jue Wang wrote: >>> The hypervisor _must_ emulate poisons identified in guest physical >>> address space (could be transported from the source VM), this is to >>> prevent silent dat

Re: [PATCH] virtio_balloon: check virtqueue_add_outbuf() return value

2022-05-30 Thread David Hildenbrand
On 30.05.22 04:16, Bo Liu (刘波)-浪潮信息 wrote: > Adding this patch can avoid unnecessary VM exits and reduce the number of VM > exits > ... in corner cases where virtqueue_add_outbuf() fails? Why do we care about that corner case? Looks like unnecessary code churn to me, unless I am missing somethi

Re: [PATCH 2/3] mm/memory-failure.c: support reset PTE during unpoison

2022-05-29 Thread David Hildenbrand
On 20.05.22 09:06, zhenwei pi wrote: > Origianlly, unpoison_memory() is only used by hwpoison-inject, and > unpoisons a page which is poisoned by hwpoison-inject too. The kernel PTE > entry has no change during software poison/unpoison. > > On a virtualization platform, it's possible to fix hardwa

Re: [PATCH 0/3] recover hardware corrupted page by virtio balloon

2022-05-24 Thread David Hildenbrand
On 20.05.22 09:06, zhenwei pi wrote: > Hi, > > I'm trying to recover hardware corrupted page by virtio balloon, the > workflow of this feature like this: > > Guest 5.MF -> 6.RVQ FE10.Unpoison page > / \/ > ---+

Re: [PATCH 3/4] mm/memofy-failure.c: optimize hwpoison_filter

2022-05-06 Thread David Hildenbrand
On 06.05.22 15:38, zhenwei pi wrote: > > > On 5/6/22 16:59, Naoya Horiguchi wrote: >> On Fri, Apr 29, 2022 at 10:22:05PM +0800, zhenwei pi wrote: >>> In the memory failure procedure, hwpoison_filter has higher priority, >>> if memory_filter() filters the error event, there is no need to do >>> th

Re: [PATCH v10 2/5] mm: page_isolation: check specified range for unmovable pages

2022-04-12 Thread David Hildenbrand
On 12.04.22 17:01, Zi Yan wrote: > On 12 Apr 2022, at 10:49, David Hildenbrand wrote: > >> On 12.04.22 16:07, Zi Yan wrote: >>> On 12 Apr 2022, at 9:10, David Hildenbrand wrote: >>> >>>> On 06.04.22 17:18, Zi Yan wrote: >>>>> From: Zi Y

Re: [PATCH v10 2/5] mm: page_isolation: check specified range for unmovable pages

2022-04-12 Thread David Hildenbrand
On 12.04.22 16:07, Zi Yan wrote: > On 12 Apr 2022, at 9:10, David Hildenbrand wrote: > >> On 06.04.22 17:18, Zi Yan wrote: >>> From: Zi Yan >>> >>> Enable set_migratetype_isolate() to check specified sub-range for >>> unmovable pages

Re: [PATCH v10 2/5] mm: page_isolation: check specified range for unmovable pages

2022-04-12 Thread David Hildenbrand
On 06.04.22 17:18, Zi Yan wrote: > From: Zi Yan > > Enable set_migratetype_isolate() to check specified sub-range for > unmovable pages during isolation. Page isolation is done > at MAX_ORDER_NR_PAEGS granularity, but not all pages within that > granularity are intended to be isolated. For exampl

Re: [PATCH v10 0/5] Use pageblock_order for cma and alloc_contig_range alignment.

2022-04-12 Thread David Hildenbrand
On 06.04.22 17:18, Zi Yan wrote: > From: Zi Yan > > Hi David, Hi! > > This patchset tries to remove the MAX_ORDER-1 alignment requirement for CMA > and alloc_contig_range(). It prepares for my upcoming changes to make > MAX_ORDER adjustable at boot time[1]. It is on top of mmotm-2022-04-05-15-

Re: [PATCH v8 2/5] mm: page_isolation: check specified range for unmovable pages

2022-03-22 Thread David Hildenbrand
On 21.03.22 19:23, Zi Yan wrote: > On 21 Mar 2022, at 13:30, David Hildenbrand wrote: > >> On 17.03.22 16:37, Zi Yan wrote: >>> From: Zi Yan >>> >>> Enable set_migratetype_isolate() to check specified sub-range for >>> unmovable pages dur

Re: [PATCH v8 2/5] mm: page_isolation: check specified range for unmovable pages

2022-03-21 Thread David Hildenbrand
On 17.03.22 16:37, Zi Yan wrote: > From: Zi Yan > > Enable set_migratetype_isolate() to check specified sub-range for > unmovable pages during isolation. Page isolation is done > at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) granularity, but not all > pages within that granularity are intended t

Re: [PATCH v7 2/5] mm: page_isolation: check specified range for unmovable pages

2022-03-14 Thread David Hildenbrand
On 11.03.22 19:36, Zi Yan wrote: > From: Zi Yan > > Enable set_migratetype_isolate() to check specified sub-range for > unmovable pages during isolation. Page isolation is done > at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) granularity, but not all > pages within that granularity are intended t

Re: [PATCH v7 1/5] mm: page_isolation: move has_unmovable_pages() to mm/page_isolation.c

2022-03-14 Thread David Hildenbrand
On 11.03.22 19:36, Zi Yan wrote: > From: Zi Yan > > has_unmovable_pages() is only used in mm/page_isolation.c. Move it from > mm/page_alloc.c and make it static. > > Signed-off-by: Zi Yan > Reviewed-by: Oscar Salvador > Reviewed-by: Mike Rapoport Acked-by: David H

Re: [PATCH] virtio: drop default for virtio-mem

2022-02-25 Thread David Hildenbrand
> tristate "Virtio mem driver" > - default m > depends on X86_64 > depends on VIRTIO > depends on MEMORY_HOTPLUG Yeah, why not Acked-by: David Hildenbrand -- Thanks, David / dhildenb ___

Re: [PATCH] Virtio-balloon: add user space API for sizing

2022-02-15 Thread David Hildenbrand
On 14.02.22 20:59, Kameron Lutes wrote: > This new linux API will allow user space applications to directly > control the size of the virtio-balloon. This is useful in > situations where the guest must quickly respond to drastically > increased memory pressure and cannot wait for the host to adjust

[PATCH v1 0/2] mm: enforce pageblock_order < MAX_ORDER

2022-02-14 Thread David Hildenbrand
yprowski Cc: Robin Murphy Cc: Minchan Kim Cc: Vlastimil Babka Cc: linuxppc-...@lists.ozlabs.org Cc: devicet...@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: io...@lists.linux-foundation.org Cc: linux...@kvack.org David Hildenbrand (2): cma: factor out minimum alignment requirement

[PATCH v1 1/2] cma: factor out minimum alignment requirement

2022-02-14 Thread David Hildenbrand
Let's factor out determining the minimum alignment requirement for CMA and add a helpful comment. No functional change intended. Signed-off-by: David Hildenbrand --- arch/powerpc/include/asm/fadump-internal.h | 5 - arch/powerpc/kernel/fadump.c | 2 +- drive

[PATCH v1 2/2] mm: enforce pageblock_order < MAX_ORDER

2022-02-14 Thread David Hildenbrand
ffected setups should be using CMA or boottime allocations instead. So let's make sure that pageblock_order < MAX_ORDER and simplify. [1] https://lkml.kernel.org/r/87r189a2ks@linux.ibm.com Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 9 +++-- include/li

Re: [PATCH v3] drivers/virtio: Enable virtio mem for ARM64

2022-02-04 Thread David Hildenbrand
On 04.02.22 15:04, Michael S. Tsirkin wrote: > On Fri, Feb 04, 2022 at 02:29:39PM +0100, David Hildenbrand wrote: >> On 04.02.22 14:24, Michael S. Tsirkin wrote: >>> On Wed, Jan 19, 2022 at 09:35:05AM +0100, David Hildenbrand wrote: >>>> On 19.01.22 08:46, Gavi

Re: [PATCH v3] drivers/virtio: Enable virtio mem for ARM64

2022-02-04 Thread David Hildenbrand
On 04.02.22 14:24, Michael S. Tsirkin wrote: > On Wed, Jan 19, 2022 at 09:35:05AM +0100, David Hildenbrand wrote: >> On 19.01.22 08:46, Gavin Shan wrote: >>> Hi Michael, >>> >>> On 1/19/22 3:39 PM, Michael S. Tsirkin wrote: >>>> On Wed, Ja

Re: [PATCH v4 3/7] mm: page_isolation: check specified range for unmovable pages

2022-02-02 Thread David Hildenbrand
On 02.02.22 13:18, Oscar Salvador wrote: > On Wed, Jan 19, 2022 at 02:06:19PM -0500, Zi Yan wrote: >> From: Zi Yan >> >> Enable set_migratetype_isolate() to check specified sub-range for >> unmovable pages during isolation. Page isolation is done >> at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) g

Re: [PATCH] mm/balloon_compaction: make balloon page compaction callbacks static

2022-01-25 Thread David Hildenbrand
relevant code. > > Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v3] drivers/virtio: Enable virtio mem for ARM64

2022-01-19 Thread David Hildenbrand
fig option (CONFIG_VIRTIO_MEM) on the >>> architecture. >>> >>> Signed-off-by: Gavin Shan >>> Acked-by: David Hildenbrand >>> Acked-by: Jonathan Cameron >>> Acked-by: Michael S. Tsirkin >>> --- >>> v3: Pick ack-by tags from

Re: [PATCH] drivers/virtio: Enable virtio mem for ARM64

2022-01-18 Thread David Hildenbrand
On 18.01.22 11:43, Michael S. Tsirkin wrote: > On Tue, Jan 18, 2022 at 09:38:21AM +0100, David Hildenbrand wrote: >> On 18.01.22 02:34, Gavin Shan wrote: >>> This enables virtio-mem device support by allowing to enable the >>> corresponding kernel config option

Re: [PATCH] drivers/virtio: Enable virtio mem for ARM64

2022-01-18 Thread David Hildenbrand
64, but should +theoretically work on all architectures that support memory hotplug and + hotremove. If unsure, say M. Acked-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@l

Re: [PATCH] virtio_mem: break device on remove

2022-01-17 Thread David Hildenbrand
On 17.01.22 09:40, Michael S. Tsirkin wrote: > On Mon, Jan 17, 2022 at 09:31:56AM +0100, David Hildenbrand wrote: >> On 17.01.22 08:55, Michael S. Tsirkin wrote: >>> On Mon, Jan 17, 2022 at 02:40:11PM +0800, Jason Wang wrote: >>>> >>>> 在 2022/1/15 上午5

Re: [PATCH] virtio_mem: break device on remove

2022-01-17 Thread David Hildenbrand
On 17.01.22 08:55, Michael S. Tsirkin wrote: > On Mon, Jan 17, 2022 at 02:40:11PM +0800, Jason Wang wrote: >> >> 在 2022/1/15 上午5:43, Michael S. Tsirkin 写道: >>> A common pattern for device reset is currently: >>> vdev->config->reset(vdev); >>> .. cleanup .. >>> >>> reset prevents new interrupts from

Re: [RFC PATCH v3 7/8] drivers: virtio_mem: use pageblock size as the minimum virtio_mem size.

2022-01-14 Thread David Hildenbrand
On 05.01.22 22:47, Zi Yan wrote: > From: Zi Yan > > alloc_contig_range() now only needs to be aligned to pageblock_order, > drop virtio_mem size requirement that it needs to be the max of > pageblock_order and MAX_ORDER. > > Signed-off-by: Zi Yan > --- > drivers/virtio/virtio_mem.c | 3 +-- >

Re: [RFC PATCH v3 5/8] mm: page_isolation: check specified range for unmovable pages during isolation.

2022-01-14 Thread David Hildenbrand
On 05.01.22 22:47, Zi Yan wrote: > From: Zi Yan > > Enable set_migratetype_isolate() to check specified sub-range for > unmovable pages during isolation. Page isolation is done > at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) granularity, but not all > pages within that granularity are intended t

Re: [RFC PATCH v3 3/8] mm: migrate: allocate the right size of non hugetlb or THP compound pages.

2022-01-13 Thread David Hildenbrand
On 13.01.22 16:46, Zi Yan wrote: > On 12 Jan 2022, at 6:04, David Hildenbrand wrote: > >> On 05.01.22 22:47, Zi Yan wrote: >>> From: Zi Yan >>> >>> alloc_migration_target() is used by alloc_contig_range() and non-LRU >>> movable compound pages can

Re: [RFC PATCH v3 1/8] mm: page_alloc: avoid merging non-fallbackable pageblocks with others.

2022-01-13 Thread David Hildenbrand
On 13.01.22 12:36, Mike Rapoport wrote: > On Wed, Jan 12, 2022 at 11:54:49AM +0100, David Hildenbrand wrote: >> On 05.01.22 22:47, Zi Yan wrote: >>> From: Zi Yan >>> >>> This is done in addition to MIGRATE_ISOLATE pageblock merge avoidance. >>>

Re: [RFC PATCH v3 3/8] mm: migrate: allocate the right size of non hugetlb or THP compound pages.

2022-01-12 Thread David Hildenbrand
On 05.01.22 22:47, Zi Yan wrote: > From: Zi Yan > > alloc_migration_target() is used by alloc_contig_range() and non-LRU > movable compound pages can be migrated. Current code does not allocate the > right page size for such pages. Check THP precisely using > is_transparent_huge() and add allocat

Re: [RFC PATCH v3 2/8] mm: compaction: handle non-lru compound pages properly in isolate_migratepages_block().

2022-01-12 Thread David Hildenbrand
On 05.01.22 22:47, Zi Yan wrote: > From: Zi Yan > > In isolate_migratepages_block(), a !PageLRU tail page can be encountered > when the page is larger than a pageblock. Use compound head page for the > checks inside and skip the entire compound page when isolation succeeds. > This will currentl

Re: [RFC PATCH v3 1/8] mm: page_alloc: avoid merging non-fallbackable pageblocks with others.

2022-01-12 Thread David Hildenbrand
On 05.01.22 22:47, Zi Yan wrote: > From: Zi Yan > > This is done in addition to MIGRATE_ISOLATE pageblock merge avoidance. > It prepares for the upcoming removal of the MAX_ORDER-1 alignment > requirement for CMA and alloc_contig_range(). > > MIGRARTE_HIGHATOMIC should not merge with other migra

Re: [RFC PATCH v2 0/7] Use pageblock_order for cma and alloc_contig_range alignment.

2021-12-10 Thread David Hildenbrand
On 10.12.21 00:04, Zi Yan wrote: > From: Zi Yan > > Hi all, Hi, thanks for working on that! > > This patchset tries to remove the MAX_ORDER - 1 alignment requirement for CMA > and alloc_contig_range(). It prepares for my upcoming changes to make > MAX_ORDER > adjustable at boot time[1]. > >

Re: [PATCH v1 2/2] virtio-mem: prepare fake page onlining code for granularity smaller than MAX_ORDER - 1

2021-12-09 Thread David Hildenbrand
Hi Eric, thanks for the review! >> if (PageDirty(page)) { >> -virtio_mem_clear_fake_offline(pfn + i, max_nr_pages, >> - false); >> -generic_online_page(page, MAX_ORDER - 1); >> +

Re: [PATCH v1 0/2] virtio-mem: prepare for granularity smaller than MAX_ORDER - 1

2021-12-01 Thread David Hildenbrand
On 01.12.21 00:56, Michael S. Tsirkin wrote: > On Fri, Nov 26, 2021 at 02:42:07PM +0100, David Hildenbrand wrote: >> The virtio-mem driver currently supports logical hot(un)plug in >> MAX_ORDER - 1 granularity (4MiB on x86-64) or bigger. We want to support >> pageblock granul

Re: [PATCH v1 0/2] virtio-mem: prepare for granularity smaller than MAX_ORDER - 1

2021-11-30 Thread David Hildenbrand
On 29.11.21 17:47, Zi Yan wrote: > On 26 Nov 2021, at 8:42, David Hildenbrand wrote: > >> The virtio-mem driver currently supports logical hot(un)plug in >> MAX_ORDER - 1 granularity (4MiB on x86-64) or bigger. We want to support >> pageblock granularity (2MiB on x86-64),

[PATCH v1 2/2] virtio-mem: prepare fake page onlining code for granularity smaller than MAX_ORDER - 1

2021-11-26 Thread David Hildenbrand
Let's prepare our fake page onlining code for subblock size smaller than MAX_ORDER - 1: we might get called for ranges not covering properly aligned MAX_ORDER - 1 pages. We have to detect the order to use dynamically. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c

[PATCH v1 1/2] virtio-mem: prepare page onlining code for granularity smaller than MAX_ORDER - 1

2021-11-26 Thread David Hildenbrand
the buddy. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 86 ++--- 1 file changed, 62 insertions(+), 24 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 96e5a8782769..03e1c5743699 100644 --- a/driv

[PATCH v1 0/2] virtio-mem: prepare for granularity smaller than MAX_ORDER - 1

2021-11-26 Thread David Hildenbrand
https://lkml.kernel.org/r/2025193725.737539-1-zi@sent.com Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Zi Yan Cc: Gavin Shan Cc: Hui Zhu Cc: Eric Ren Cc: Sebastien Boeuf Cc: Pankaj Gupta Cc: Wei Yang Cc: virtualization@lists.linux-foundation.org Cc: linux...@kvack.org David

Re: [PATCH AUTOSEL 5.15 7/7] virtio-mem: support VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE

2021-11-26 Thread David Hildenbrand
On 26.11.21 03:30, Sasha Levin wrote: > From: David Hildenbrand > > [ Upstream commit 61082ad6a6e1f999eef7e7e90046486c87933b1e ] > > The initial virtio-mem spec states that while unplugged memory should not > be read, the device still has to allow for reading unplugged m

Re: [PATCH AUTOSEL 5.10 3/4] virtio-mem: support VIRTIO_MEM_F_UNPLUGGED_INACCESSIBLE

2021-11-26 Thread David Hildenbrand
On 26.11.21 03:30, Sasha Levin wrote: > From: David Hildenbrand > > [ Upstream commit 61082ad6a6e1f999eef7e7e90046486c87933b1e ] > > The initial virtio-mem spec states that while unplugged memory should not > be read, the device still has to allow for reading unplugged m

Re: [PATCH v2] virtio_balloon: add param to skip adjusting pages

2021-11-24 Thread David Hildenbrand
I'd appreciate a more generic approach for user space to figure out the "initial memory size" in a virtualized environment than adding some module parameter to virtio-balloon -- if that makes sense. MemTotal as is expresses how much memory the buddy currently manages,

Re: [RFC PATCH 0/3] Use pageblock_order for cma and alloc_contig_range alignment.

2021-11-23 Thread David Hildenbrand
On 17.11.21 04:04, Zi Yan wrote: > On 16 Nov 2021, at 3:58, David Hildenbrand wrote: > >> On 15.11.21 20:37, Zi Yan wrote: >>> From: Zi Yan >>> >>> Hi David, >> >> Hi, >> >> thanks for looking into this. >> Hi, sorry for t

  1   2   3   4   5   6   7   8   9   >