Re: blk-mq crash under KVM in multiqueue block code (with virtio-blk and ext4)

2014-09-17 Thread David Hildenbrand
Does anyone have an idea? The request itself is completely filled with cc That is very weird, the 'rq' is got from hctx-tags, and rq should be valid, and rq-q shouldn't have been changed even though it was double free or double allocation. I am currently asking myself if

Re: blk-mq crash under KVM in multiqueue block code (with virtio-blk and ext4)

2014-09-17 Thread David Hildenbrand
On Wed, 17 Sep 2014 14:00:34 +0200 David Hildenbrand d...@linux.vnet.ibm.com wrote: Does anyone have an idea? The request itself is completely filled with cc That is very weird, the 'rq' is got from hctx-tags, and rq should be valid, and rq-q shouldn't have been changed

Re: blk-mq crash under KVM in multiqueue block code (with virtio-blk and ext4)

2014-09-17 Thread David Hildenbrand
are fully initialized if you mean all requests have been used one time. On Wed, Sep 17, 2014 at 10:11 PM, David Hildenbrand I was playing with a simple patch that just sets cmd_flags and action_flags to What is action_flags? atomic_flags, sorry :) Otherwise e.g. REQ_ATOM_STARTED could

[PATCH] blk-mq: Avoid race condition with uninitialized requests

2014-09-18 Thread David Hildenbrand
of a request. Also move the reset of cmd_flags for the initializing code to the point where a request is freed. So we will never end up with pending flush request indicators that might trigger dereferences of invalid pointers in blk_mq_timeout_check(). Cc: sta...@vger.kernel.org Signed-off-by: David

[PATCH] blk-mq: Avoid race condition with uninitialized requests

2014-09-18 Thread David Hildenbrand
This patch should fix the bug reported in https://lkml.org/lkml/2014/9/11/249. Test is still pending. David Hildenbrand (1): blk-mq: Avoid race condition with uninitialized requests block/blk-mq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 1.8.5.5

Re: [PATCH v4 13/42] virtio_blk: v1.0 support

2014-11-26 Thread David Hildenbrand
for virtio v1.0. Reviewed-by: Thomas Huth th...@linux.vnet.ibm.com Reviewed-by: David Hildenbrand d...@linux.vnet.ibm.com Signed-off-by: Cornelia Huck cornelia.h...@de.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- include/uapi/linux/virtio_blk.h | 15

Re: [PATCH v5 01/45] virtio: use u32, not bitmap for struct virtio_device's features

2014-11-27 Thread David Hildenbrand
From: Rusty Russell ru...@rustcorp.com.au It seemed like a good idea, but it's actually a pain when we get more than 32 feature bits. Just change it to a u32 for now. Cc: Brian Swetland swetl...@google.com Cc: Christian Borntraeger borntrae...@de.ibm.com Signed-off-by: Rusty Russell

Re: [PATCH v5 01/45] virtio: use u32, not bitmap for struct virtio_device's features

2014-11-27 Thread David Hildenbrand
That's the whole reason for the patch. I guess you disagree with it, but it's much easier to deal with simple integers. Well, I can live with it :) clear_bit() and friends are just easier to understand when scanning the code (at least for me).

Re: [PATCH v6 04/46] virtio: add support for 64 bit features.

2014-11-28 Thread David Hildenbrand
-off-by: Cornelia Huck cornelia.h...@de.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com --- Reviewed-by: David Hildenbrand d...@linux.vnet.ibm.com ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https

Re: [PATCH v6 05/46] virtio: assert 32 bit features in transports

2014-11-28 Thread David Hildenbrand
/virtio_mmio.c | 3 +++ drivers/virtio/virtio_pci.c| 3 +++ 7 files changed, 21 insertions(+) Looks sane to me. Reviewed-by: David Hildenbrand d...@linux.vnet.ibm.com ___ Virtualization mailing list Virtualization@lists.linux

Re: [PATCH v7 13/46] virtio: simplify feature bit handling

2014-12-01 Thread David Hildenbrand
Now that we use u64 for bits, we can simply them together. Signed-off-by: Michael S. Tsirkin m...@redhat.com --- drivers/virtio/virtio.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) Reviewed-by: David Hildenbrand d...@linux.vnet.ibm.com

Re: [PATCH v7 16/46] virtio_blk: v1.0 support

2014-12-01 Thread David Hildenbrand
Based on patch by Cornelia Huck. Note: for consistency, and to avoid sparse errors, convert all fields, even those no longer in use for virtio v1.0. Signed-off-by: Cornelia Huck cornelia.h...@de.ibm.com Signed-off-by: Michael S. Tsirkin m...@redhat.com ... -static unsigned

Re: [PATCH v7 16/46] virtio_blk: v1.0 support

2014-12-01 Thread David Hildenbrand
On Mon, Dec 01, 2014 at 09:16:41AM +0100, David Hildenbrand wrote: Based on patch by Cornelia Huck. Note: for consistency, and to avoid sparse errors, convert all fields, even those no longer in use for virtio v1.0. Signed-off-by: Cornelia Huck cornelia.h

Re: [PATCH RFC v2 3/4] virtio: allow finalize_features to fail

2014-12-05 Thread David Hildenbrand
This will make it easy for transports to validate features and return failure. Signed-off-by: Michael S. Tsirkin m...@redhat.com --- include/linux/virtio_config.h | 3 ++- drivers/lguest/lguest_device.c | 4 +++- drivers/misc/mic/card/mic_virtio.c | 4 +++-

Re: [PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2017-01-18 Thread David Hildenbrand
Am 21.12.2016 um 07:52 schrieb Liang Li: This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use {pfn|length} to present the page information instead of the PFNs, to

Re: [PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2017-01-19 Thread David Hildenbrand
> As long as the interface is similar, it seems to make > sense for me - why invent a completely new device that > looks very much like the old one? The only reason would be that this feature could be used independently of virtio-balloon. But this would of course only be the case, if ballooning

Re: [PATCH 1/1] sched: provide common cpu_relax_yield definition

2016-11-16 Thread David Hildenbrand
Signed-off-by: Christian Borntraeger <borntrae...@de.ibm.com> Looks good to me! Reviewed-by: David Hildenbrand <da...@redhat.com> David ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoun

Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-06 Thread David Hildenbrand
Am 30.11.2016 um 09:43 schrieb Liang Li: This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page information to host instead of the PFNs, to

Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread David Hildenbrand
Am 07.12.2016 um 14:35 schrieb Li, Liang Z: Am 30.11.2016 um 09:43 schrieb Liang Li: This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page

Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread David Hildenbrand
I did something similar. Filled the balloon with 15GB for a 16GB idle guest, by using bitmap, the madvise count was reduced to 605. when using the PFNs, the madvise count was 3932160. It means there are quite a lot consecutive bits in the bitmap. I didn't test for a guest with heavy memory

Re: [PATCH] virtio_balloon: prevent uninitialized variable use

2017-03-24 Thread David Hildenbrand
On 23.03.2017 16:17, Arnd Bergmann wrote: > The latest gcc-7.0.1 snapshot reports a new warning: > > virtio/virtio_balloon.c: In function 'update_balloon_stats': > virtio/virtio_balloon.c:258:26: error: 'events[2]' is used uninitialized in > this function [-Werror=uninitialized] >

Re: [virtio-dev] Re: [PATCH v7 kernel 3/5] virtio-balloon: implementation of VIRTIO_BALLOON_F_CHUNK_TRANSFER

2017-03-10 Thread David Hildenbrand
t; >>> TODO: optimize stage 1) by allocating/freeing a chunk of pages >>> instead of a single page each time. >>> >>> Signed-off-by: Liang Li <liang.z...@intel.com> >>> Signed-off-by: Wei Wang <wei.w.w...@intel.com> >>> Suggested-by: Michael S

Re: [PATCH v7 kernel 5/5] This patch contains two parts:

2017-03-06 Thread David Hildenbrand
Am 03.03.2017 um 06:40 schrieb Wei Wang: > From: Liang Li I'd prefer to split this into two parts then and to create proper subjects. If I remember correctly, the general concept was accepted by most reviewers. > > One is to add a new API to mm go get the unused page

Re: [RFC] virtio-mem: paravirtualized memory

2017-07-28 Thread David Hildenbrand
017 16:20, David Hildenbrand wrote: > Hi, > > this is an idea that is based on Andrea Arcangeli's original idea to > host enforce guest access to memory given up using virtio-balloon using > userfaultfd in the hypervisor. While looking into the details, I > realized that host-

Re: [RFC] virtio-mem: paravirtualized memory

2017-07-31 Thread David Hildenbrand
On 31.07.2017 16:12, Michael S. Tsirkin wrote: > On Fri, Jul 28, 2017 at 05:48:07PM +0200, David Hildenbrand wrote: >> In general, a paravirtualized interface (for detection of PMEM regions) >> might have one big advantage: not limited to certain architectures. > > What foll

Re: [RFC] virtio-mem: paravirtualized memory

2017-07-28 Thread David Hildenbrand
On 28.07.2017 17:16, Dan Williams wrote: > On Fri, Jul 28, 2017 at 4:09 AM, David Hildenbrand <da...@redhat.com> wrote: >> Btw, I am thinking about the following addition to the concept: >> >> 1. Add a type to each virtio-mem device. >> >> This describes

Re: [RFC] virtio-mem: paravirtualized memory

2017-07-25 Thread David Hildenbrand
(ping) Hi, this has been on these lists for quite some time now. I want to start preparing a virtio spec for virtio-mem soon. So if you have any more comments/ideas/objections/questions, now is the right time to post them :) Thanks! On 16.06.2017 16:20, David Hildenbrand wrote: >

Re: [PATCH RESEND] mm: don't zero ballooned pages

2017-08-07 Thread David Hildenbrand
> Maybe it is my absolute lack of familiarity with what the host actually > does with balloon pages but I fail to see why the above matters at all. > ksm will not try to merge sub page units (4k for hugetlb or a large base > page). And if you need to hide the guest contents then the host can >

Re: [PATCH RESEND] mm: don't zero ballooned pages

2017-08-07 Thread David Hildenbrand
On 03.08.2017 13:59, Wei Wang wrote: > This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: > enqueue zero page to balloon device")' > > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > shouldn't be given to the host ksmd to scan. Therefore, it is not >

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-21 Thread David Hildenbrand
On 21.06.2017 14:56, Christian Borntraeger wrote: > On 06/20/2017 06:49 PM, David Hildenbrand wrote: >> On 20.06.2017 18:44, Rik van Riel wrote: >>> On Mon, 2017-06-12 at 07:10 -0700, Dave Hansen wrote: >>> >>>> The hypervisor is going to throw away

Re: [RFC] virtio-mem: paravirtualized memory

2017-06-21 Thread David Hildenbrand
On 21.06.2017 13:08, Stefan Hajnoczi wrote: > On Mon, Jun 19, 2017 at 12:26:52PM +0200, David Hildenbrand wrote: >> On 19.06.2017 12:08, Stefan Hajnoczi wrote: >>> On Fri, Jun 16, 2017 at 04:20:02PM +0200, David Hildenbrand wrote: >>>> Important restrictions of this

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread David Hildenbrand
On 20.06.2017 20:17, Michael S. Tsirkin wrote: > On Tue, Jun 20, 2017 at 06:49:33PM +0200, David Hildenbrand wrote: >> On 20.06.2017 18:44, Rik van Riel wrote: >>> On Mon, 2017-06-12 at 07:10 -0700, Dave Hansen wrote: >>> >>>> The hypervisor is going to

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread David Hildenbrand
>> IMHO even simply writing all-zeros to all free pages before starting >> migration (or even when freeing a page) would be a cleaner interface >> than this (because it atomically works with the entity the host cares >> about for migration). But yes, performance is horrible that's why I am >> not

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread David Hildenbrand
On 20.06.2017 18:44, Rik van Riel wrote: > On Mon, 2017-06-12 at 07:10 -0700, Dave Hansen wrote: > >> The hypervisor is going to throw away the contents of these pages, >> right? As soon as the spinlock is released, someone can allocate a >> page, and put good data in it. What keeps the

[RFC] virtio-mem: paravirtualized memory

2017-06-16 Thread David Hildenbrand
Hi, this is an idea that is based on Andrea Arcangeli's original idea to host enforce guest access to memory given up using virtio-balloon using userfaultfd in the hypervisor. While looking into the details, I realized that host-enforcing virtio-balloon would result in way too many problems

Re: [RFC] virtio-mem: paravirtualized memory

2017-06-16 Thread David Hildenbrand
On 16.06.2017 17:04, Michael S. Tsirkin wrote: > On Fri, Jun 16, 2017 at 04:20:02PM +0200, David Hildenbrand wrote: >> Hi, >> >> this is an idea that is based on Andrea Arcangeli's original idea to >> host enforce guest access to memory given up using virtio-b

Re: [RFC] virtio-mem: paravirtualized memory

2017-06-19 Thread David Hildenbrand
On 19.06.2017 12:08, Stefan Hajnoczi wrote: > On Fri, Jun 16, 2017 at 04:20:02PM +0200, David Hildenbrand wrote: >> Important restrictions of this concept: >> - Guests without a virtio-mem guest driver can't see that memory. >> - We will always require some boot memory that

Re: [RFC] virtio-mem: paravirtualized memory

2017-06-18 Thread David Hildenbrand
>> A Linux guest will deflate the balloon (all or some pages) in the >> following scenarios: >> a) page migration > > It inflates it first, doesn't it? Yes, that that is true. I was just listing all scenarios. > >> b) unload virtio-balloon kernel module >> c) hibernate/suspension >> d)

Re: [PATCH] drivers/s390/virtio: Remove the old KVM virtio transport

2017-09-28 Thread David Hildenbrand
3 - > drivers/s390/virtio/kvm_virtio.c | 515 > --- > 3 files changed, 531 deletions(-) > delete mode 100644 drivers/s390/virtio/kvm_virtio.c Yup, looks good to me. Reviewed-by: David Hildenbrand <da...@redhat.com> -- Thanks, David __

Re: [PATCH 1/1] virtio/s390: implement PM operations for virtio_ccw

2017-12-07 Thread David Hildenbrand
virtio_ccw_cio_notify, > .int_class = IRQIO_VIR, > + .freeze = virtio_ccw_freeze, > + .thaw = virtio_ccw_restore, > + .restore = virtio_ccw_restore, > }; > > static int __init pure_hex(char **cp, unsigned in

Re: [PATCH RFCv2 0/4] virtio-mem: paravirtualized memory

2018-05-23 Thread David Hildenbrand
On 23.05.2018 20:24, David Hildenbrand wrote: > This is the Linux driver side of virtio-mem. Compared to the QEMU side, > it is in a pretty complete and clean state. > > virtio-mem is a paravirtualized mechanism of adding/removing memory to/from > a VM. We can do this on a 4MB gr

Re: Design Decision for KVM based anti rootkit

2018-06-18 Thread David Hildenbrand
On 18.06.2018 18:35, Ahmed Soliman wrote: > Shortly after I sent the first email, we found that there is another > way to achieve this kind of communication, via KVM Hypercalls, I think > they are underutilised in kvm, but they exist. > > We also found that they are architecture dependent, but

Re: Design Decision for KVM based anti rootkit

2018-06-18 Thread David Hildenbrand
On 16.06.2018 13:49, Ahmed Soliman wrote: > Following up on these threads: > - https://marc.info/?l=kvm=151929803301378=2 > - http://www.openwall.com/lists/kernel-hardening/2018/02/22/18 > > I lost the original emails so I couldn't reply to them, and also sorry > for being late, it was the end of

Re: [PATCH v34 0/4] Virtio-balloon: support free page reporting

2018-06-27 Thread David Hildenbrand
On 25.06.2018 14:05, Wei Wang wrote: > This patch series is separated from the previous "Virtio-balloon > Enhancement" series. The new feature, VIRTIO_BALLOON_F_FREE_PAGE_HINT, > implemented by this series enables the virtio-balloon driver to report > hints of guest free pages to the host. It

Re: [PATCH RFC V4 3/3] KVM: X86: Adding skeleton for Memory ROE

2018-07-25 Thread David Hildenbrand
> if (kvm_x86_ops->slot_disable_log_dirty) > kvm_x86_ops->slot_disable_log_dirty(kvm, new); > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > index 4ee7bc548a83..82c5780e11d9 100644 > --- a/include/linux/kvm_host.h > +++

Re: [PATCH v34 0/4] Virtio-balloon: support free page reporting

2018-06-29 Thread David Hildenbrand
On 29.06.2018 05:51, Wei Wang wrote: > On 06/27/2018 07:06 PM, David Hildenbrand wrote: >> On 25.06.2018 14:05, Wei Wang wrote: >>> This patch series is separated from the previous "Virtio-balloon >>> Enhancement" series. The new feature, VIRTIO_BALL

Re: [PATCH v34 0/4] Virtio-balloon: support free page reporting

2018-06-29 Thread David Hildenbrand
>> And looking at all the discussions and problems that already happened >> during the development of this series, I think we should rather look >> into how clean free page hinting might solve the same problem. > > I'm not sure I follow the logic. We found that neat tricks > especially re-using

Re: [PATCH v34 0/4] Virtio-balloon: support free page reporting

2018-06-29 Thread David Hildenbrand
>> Why would your suggestion still be applicable? >> >> Your point for now is "I might not want to have page hinting enabled due to >> the overhead, but still a live migration speedup". If that overhead actually >> exists (we'll have to see) or there might be another reason to disable page >>

Re: [PATCH v34 0/4] Virtio-balloon: support free page reporting

2018-06-29 Thread David Hildenbrand
On 29.06.2018 13:31, Wei Wang wrote: > On 06/29/2018 03:46 PM, David Hildenbrand wrote: >>> >>> I'm afraid it can't. For example, when we have a guest booted, without >>> too many memory activities. Assume the guest has 8GB free memory. The >>> arch_free_page

Re: security implications of caching with virtio pmem (was Re: [PATCH v3 0/5] kvm "virtio pmem" device)

2019-02-06 Thread David Hildenbrand
On 04.02.19 23:56, Michael S. Tsirkin wrote: > > On Wed, Jan 09, 2019 at 08:17:31PM +0530, Pankaj Gupta wrote: >> This patch series has implementation for "virtio pmem". >> "virtio pmem" is fake persistent memory(nvdimm) in guest >> which allows to bypass the guest page cache. This also >>

Re: [Qemu-devel] security implications of caching with virtio pmem (was Re: [PATCH v3 0/5] kvm "virtio pmem" device)

2019-02-12 Thread David Hildenbrand
On 11.02.19 23:29, Dave Chinner wrote: > On Mon, Feb 11, 2019 at 02:29:46AM -0500, Pankaj Gupta wrote: >> Hello Dave, >> Are we okay with this? > > Sure. > > I'm not sure I agree with all the analysis presented, but, well, I > haven't looked any deeper because I'm tired of being shouted at and >

Re: [RFC PATCH 00/12] s390: virtio: support protected virtualization

2019-04-12 Thread David Hildenbrand
On 05.04.19 01:16, Halil Pasic wrote: > Enhanced virtualization protection technology may require the use of > bounce buffers for I/O. While support for this was built into the virtio > core, virtio-ccw wasn't changed accordingly. Can you elaborate some more about the general approach (Enhanced

Re: [RFC PATCH 00/12] s390: virtio: support protected virtualization

2019-04-16 Thread David Hildenbrand
On 16.04.19 13:10, Halil Pasic wrote: > On Fri, 12 Apr 2019 15:47:50 +0200 > David Hildenbrand wrote: > >> On 05.04.19 01:16, Halil Pasic wrote: >>> Enhanced virtualization protection technology may require the use of >>> bounce buffers for I/O. While support f

Re: [PATCH v9 2/7] virtio-pmem: Add virtio pmem driver

2019-05-15 Thread David Hildenbrand
> + vpmem->vdev = vdev; > + vdev->priv = vpmem; > + err = init_vq(vpmem); > + if (err) { > + dev_err(>dev, "failed to initialize virtio pmem vq's\n"); > + goto out_err; > + } > + > + virtio_cread(vpmem->vdev, struct virtio_pmem_config, > +

Re: [PATCH v9 2/7] virtio-pmem: Add virtio pmem driver

2019-05-15 Thread David Hildenbrand
On 15.05.19 22:46, David Hildenbrand wrote: >> +vpmem->vdev = vdev; >> +vdev->priv = vpmem; >> +err = init_vq(vpmem); >> +if (err) { >> +dev_err(>dev, "failed to initialize virtio pmem vq's\n"); >> +

Re: [PATCH trivial] mm/balloon_compaction: Grammar s/the its/its/

2019-06-11 Thread David Hildenbrand
dress to allow the driver release the page. > + * its address to allow the driver to release the page. > * @b_dev_info: balloon device decriptor where we will grab a page from. > * > * Driver must call it to properly de-allocate a previous enlisted balloon > page

Re: [PATCH v8 2/6] virtio-pmem: Add virtio pmem driver

2019-05-14 Thread David Hildenbrand
On 10.05.19 17:51, Pankaj Gupta wrote: > This patch adds virtio-pmem driver for KVM guest. > > Guest reads the persistent memory range information from > Qemu over VIRTIO and registers it on nvdimm_bus. It also > creates a nd_region object with the persistent memory > range information so that

Re: [PATCH v8 2/6] virtio-pmem: Add virtio pmem driver

2019-05-14 Thread David Hildenbrand
>> >> I think you should do the same here, vdev->priv is allocated in >> virtio_pmem_probe. >> >> But maybe I am missing something important here :) > > Because virtio_balloon use "kzalloc" for allocation and needs to be freed. > But virtio pmem uses "devm_kzalloc" which takes care of

Re: [PATCH v8 2/6] virtio-pmem: Add virtio pmem driver

2019-05-14 Thread David Hildenbrand
>> >>> + } >>> + >>> + /* When host has read buffer, this completes via host_ack */ >> >> "A host repsonse results in "host_ack" getting called" ... ? >> >>> + wait_event(req->host_acked, req->done); >>> + err = req->ret; >>> +ret: >>> + kfree(req); >>> + return err; >>> +}; >>> + >>>

Re: [PATCH v2] mm/balloon_compaction: Informative allocation warnings

2019-08-21 Thread David Hildenbrand
On 21.08.19 21:10, Nadav Amit wrote: >> On Aug 21, 2019, at 12:06 PM, David Hildenbrand wrote: >> >> On 21.08.19 20:59, Nadav Amit wrote: >>>> On Aug 21, 2019, at 11:57 AM, David Hildenbrand wrote: >>>> >>>> On 21.08.19 11:41, Nadav Amit wro

Re: [PATCH v2] mm/balloon_compaction: Informative allocation warnings

2019-08-21 Thread David Hildenbrand
, it is also > beneficial to suppress these warnings to keep the same behavior that the > balloon had before. > > Since such warnings can still be useful to indicate that the balloon is > over-inflated, print more informative and less frightening warning if > allocation fails inste

Re: [PATCH v2] mm/balloon_compaction: Informative allocation warnings

2019-08-21 Thread David Hildenbrand
On 21.08.19 20:59, Nadav Amit wrote: >> On Aug 21, 2019, at 11:57 AM, David Hildenbrand wrote: >> >> On 21.08.19 11:41, Nadav Amit wrote: >>> There is no reason to print generic warnings when balloon memory >>> allocation fails, as failures are expected and

Re: [PATCH] mm/balloon_compaction: suppress allocation warnings

2019-08-21 Thread David Hildenbrand
On 21.08.19 18:34, Nadav Amit wrote: >> On Aug 21, 2019, at 9:29 AM, David Hildenbrand wrote: >> >> On 21.08.19 18:23, Nadav Amit wrote: >>>> On Aug 21, 2019, at 9:05 AM, David Hildenbrand wrote: >>>> >>>> On 20.08.19 11:16, Nadav Amit

Re: [PATCH] mm/balloon_compaction: suppress allocation warnings

2019-08-21 Thread David Hildenbrand
On 21.08.19 18:23, Nadav Amit wrote: >> On Aug 21, 2019, at 9:05 AM, David Hildenbrand wrote: >> >> On 20.08.19 11:16, Nadav Amit wrote: >>> There is no reason to print warnings when balloon page allocation fails, >>> as they are expected and can be handled g

Re: [PATCH] mm/balloon_compaction: suppress allocation warnings

2019-08-21 Thread David Hildenbrand
On 20.08.19 11:16, Nadav Amit wrote: > There is no reason to print warnings when balloon page allocation fails, > as they are expected and can be handled gracefully. Since VMware > balloon now uses balloon-compaction infrastructure, and suppressed these > warnings before, it is also beneficial to

Re: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-09-16 Thread David Hildenbrand
On 16.09.19 03:41, Wei Wang wrote: > On 09/14/2019 02:36 AM, Tyler Sanderson wrote: >> Hello, I'm curious about the intent of VIRTIO_BALLOON_F_FREE_PAGE_HINT >> (commit >> ).

[PATCH RFC v3 9/9] virtio-mem: Offline and remove completely unplugged memory blocks

2019-09-19 Thread David Hildenbrand
NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- driv

[PATCH RFC v3 7/9] virtio-mem: Allow to offline partially unplugged memory blocks

2019-09-19 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 102 1 file changed, 92 insertions(+), 10 deletion

[PATCH RFC v3 2/9] virtio-mem: Paravirtualized memory hotplug

2019-09-19 Thread David Hildenbrand
Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 17 + drivers/virtio/M

[PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

2019-09-19 Thread David Hildenbrand
Will be needed by virtio-mem to identify the node from a pxm. Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa.c b/drivers/acpi/nu

[PATCH RFC v3 0/9] virtio-mem: paravirtualized memory

2019-09-19 Thread David Hildenbrand
from it. Q: How big should the initial (!virtio-mem) memory of a VM be? A: virtio-mem memory will not go to the DMA zones. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case. [

[PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

2019-09-19 Thread David Hildenbrand
other sign in the struct page that offlining is permitted. Cc: Andrew Morton Cc: Juergen Gross Cc: David Hildenbrand Cc: Pavel Tatashin Cc: Alexander Duyck Cc: Anthony Yznaga Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Oscar Salvador Cc: Michal Hocko Cc: Pingfan Liu Cc: Qian Cai Cc: Dan Wi

[PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory()

2019-09-19 Thread David Hildenbrand
ed by user space again before it gets removed. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Signed-off-by: David Hildenbrand --

[PATCH RFC v3 3/9] virtio-mem: Paravirtualized memory hotunplug part 1

2019-09-19 Thread David Hildenbrand
Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 109 +++- 1 file changed, 107 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 59

[PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

2019-09-19 Thread David Hildenbrand
: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index

[PATCH RFC v3 5/9] virtio-mem: Paravirtualized memory hotunplug part 2

2019-09-19 Thread David Hildenbrand
Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 106 +++- 2 files changed, 1

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

2019-09-23 Thread David Hildenbrand
On 19.09.19 16:22, David Hildenbrand wrote: > Will be needed by virtio-mem to identify the node from a pxm. > > Cc: "Rafael J. Wysocki" > Cc: Len Brown > Cc: linux-a...@vger.kernel.org > Signed-off-by: David Hildenbrand > --- > drivers/acpi/numa.c |

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

2019-09-23 Thread David Hildenbrand
On 23.09.19 12:36, Michal Hocko wrote: > On Mon 23-09-19 12:13:11, David Hildenbrand wrote: >> On 19.09.19 16:22, David Hildenbrand wrote: >>> Will be needed by virtio-mem to identify the node from a pxm. >>> >>> Cc: "Rafael J. Wysocki" >>

Re: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-10-04 Thread David Hildenbrand
On 04.10.19 10:35, Michael S. Tsirkin wrote: > On Fri, Oct 04, 2019 at 10:06:03AM +0200, David Hildenbrand wrote: >> On 04.10.19 01:15, Tyler Sanderson wrote: >>> I was mistaken, the problem with overcommit accounting is not fixed by >>> the change to shrinker interface

[PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-04 Thread David Hildenbrand
ges") Cc: # v3.11+ Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Jiang Liu Cc: Andrew Morton Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/virtio/vir

Re: [PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-10 Thread David Hildenbrand
On 10.12.19 14:31, Michael S. Tsirkin wrote: > On Wed, Dec 04, 2019 at 09:48:07PM +0100, David Hildenbrand wrote: >> In case we have to migrate a ballon page to a newpage of another zone, the >> managed page count of both zones is wrong. Paired with memory offlining >&g

Re: [PATCH v2] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-10 Thread David Hildenbrand
On 05.12.19 10:24, David Hildenbrand wrote: > In case we have to migrate a ballon page to a newpage of another zone, the > managed page count of both zones is wrong. Paired with memory offlining > (which will adjust the managed page count), we can trigger kernel crashes > a

Re: [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

2019-12-13 Thread David Hildenbrand
On 12.12.19 22:43, Rafael J. Wysocki wrote: > On Thursday, December 12, 2019 6:11:25 PM CET David Hildenbrand wrote: >> Will be needed by virtio-mem to identify the node from a pxm. >> >> Cc: "Rafael J. Wysocki" >> Cc: Len Brown >> Cc: linux-a

[PATCH v3] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-11 Thread David Hildenbrand
Cc: Igor Mammedov Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- v2 -> v3: - Refine comment - s/only/online/ in description - Clarify why VIRTIO_BALLOON_F_DEFLATE_ON_OOM has to be checked v1 -> v2: - Adjust count before enquing newpage (and it possibly gets fre

[PATCH RFC v4 02/13] virtio-mem: Paravirtualized memory hotplug

2019-12-12 Thread David Hildenbrand
Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-

[PATCH RFC v4 04/13] mm: Export alloc_contig_range() / free_contig_range()

2019-12-12 Thread David Hildenbrand
: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Acked-by: Michal Hocko # to export contig range allocator API Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file changed, 2

[PATCH RFC v4 03/13] virtio-mem: Paravirtualized memory hotunplug part 1

2019-12-12 Thread David Hildenbrand
Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 109 +++- 1 file changed, 107 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 8f

[PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2019-12-12 Thread David Hildenbrand
es. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case. [1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red

[PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

2019-12-12 Thread David Hildenbrand
Will be needed by virtio-mem to identify the node from a pxm. Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/drivers

[PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2019-12-12 Thread David Hildenbrand
: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 10

[PATCH RFC v4 07/13] virtio-mem: Allow to offline partially unplugged memory blocks

2019-12-12 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 - 1 file changed, 63 insertions(+), 1 deletio

[PATCH RFC v4 09/13] virtio-mem: Offline and remove completely unplugged memory blocks

2019-12-12 Thread David Hildenbrand
NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- driv

[PATCH RFC v4 05/13] virtio-mem: Paravirtualized memory hotunplug part 2

2019-12-12 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 156 2 fi

[PATCH RFC v4 10/13] virtio-mem: Better retry handling

2019-12-12 Thread David Hildenbrand
Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/vi

[PATCH RFC v4 11/13] mm/vmscan: Move count_vm_event(DROP_SLAB) into drop_slab()

2019-12-12 Thread David Hildenbrand
Let's count within the function itself, so every invocation (of future users) will be counted. Cc: Alexander Viro Cc: Andrew Morton Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- fs/drop_caches.c | 4 +--- mm/vmscan.c | 1 + 2 files changed, 2 insertions(+), 3

[PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail

2019-12-12 Thread David Hildenbrand
g Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 30 ++ 1 file changed, 30 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

2019-12-12 Thread David Hildenbrand
off-by: David Hildenbrand --- include/linux/mm.h | 4 ++-- mm/vmscan.c| 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 64799c5cb39f..483300f58be8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2706,8 +2706,8 @@

[PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

2019-12-12 Thread David Hildenbrand
ed by user space again before it gets removed. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Signed-off-by: David Hildenbrand --

Re: [PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-05 Thread David Hildenbrand
On 04.12.19 21:48, David Hildenbrand wrote: > In case we have to migrate a ballon page to a newpage of another zone, the > managed page count of both zones is wrong. Paired with memory offlining > (which will adjust the managed page count), we can trigger kernel crashes > a

[PATCH v2] virtio-balloon: fix managed page counts when migrating pages between zones

2019-12-05 Thread David Hildenbrand
Liu Cc: Andrew Morton Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- v1 -> v2: - Adjust count before enquing newpage (and it possibly gets free form the balloon) - Check if the zone changed --- drivers/virtio/virtio_balloon.c | 7 +++ 1 file chan

  1   2   3   4   5   6   >