[PATCH v16 4/5] mm: support reporting free page blocks

2017-09-29 Thread Wei Wang
ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/mm.h | 6 mm/page_alloc.c|

[PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-09-29 Thread Wei Wang
() in chunks rather than page by page. With this new feature, the above ballooning process takes ~492ms resulting in an improvement of ~88%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang Signed-off-by: Liang Li Suggested

[PATCH v16 1/5] lib/xbitmap: Introduce xbitmap

2017-09-29 Thread Wei Wang
st_bit(). More possible optimizations to add in the future: 1) xb_set_bit_range: set a range of bits 2) when searching a bit, if the bit is not found in the slot, move on to the next slot directly. 3) add Tags to help searching Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew W

[PATCH v16 1/5] lib/xbitmap: Introduce xbitmap

2017-09-29 Thread Wei Wang
optimizations to add in the future: 1) xb_set_bit_range: set a range of bits 2) when searching a bit, if the bit is not found in the slot, move on to the next slot directly. 3) add Tags to help searching Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton Cc: Michal Hocko Cc: Michael S. Tsirkin

[PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

2017-09-29 Thread Wei Wang
to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@kernel.org> --- drivers/virtio/virtio_balloon.c | 249

[PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

2017-09-29 Thread Wei Wang
to the host. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c | 249 +--- include/uapi/linux/virtio_balloon.h | 15 +++ 2 files changed, 244 insertions(+), 20 deletions(-) diff --git

[PATCH v16 0/5] Virtio-balloon Enhancement

2017-09-29 Thread Wei Wang
on patch. 3) commit changes 4) patch re-org Matthew Wilcox (2): lib/xbitmap: Introduce xbitmap radix tree test suite: add tests for xbitmap Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ driv

[PATCH v16 0/5] Virtio-balloon Enhancement

2017-09-29 Thread Wei Wang
on patch. 3) commit changes 4) patch re-org Matthew Wilcox (2): lib/xbitmap: Introduce xbitmap radix tree test suite: add tests for xbitmap Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ driv

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-09-29 Thread Wei Wang
On 09/29/2017 12:01 PM, Michael S. Tsirkin wrote: On Fri, Sep 08, 2017 at 07:09:24PM +0800, Wei Wang wrote: On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-09-29 Thread Wei Wang
On 09/29/2017 12:01 PM, Michael S. Tsirkin wrote: On Fri, Sep 08, 2017 at 07:09:24PM +0800, Wei Wang wrote: On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-26 Thread Wei Wang
On 09/27/2017 12:41 AM, Andi Kleen wrote: 1) vCPU context switching and guest side task switching are not identical. That is, when the vCPU is scheduled out, the guest task on the vCPU may not guest task lifetime has nothing to do with this. It's completely independent of what you do here on

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-26 Thread Wei Wang
On 09/27/2017 12:41 AM, Andi Kleen wrote: 1) vCPU context switching and guest side task switching are not identical. That is, when the vCPU is scheduled out, the guest task on the vCPU may not guest task lifetime has nothing to do with this. It's completely independent of what you do here on

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-26 Thread Wei Wang
On 09/25/2017 10:57 PM, Andi Kleen wrote: +static void auto_switch_lbr_msrs(struct vcpu_vmx *vmx) +{ + int i; + struct perf_lbr_stack lbr_stack; + + perf_get_lbr_stack(_stack); + + add_atomic_switch_msr(vmx, MSR_LBR_SELECT, 0, 0); + add_atomic_switch_msr(vmx,

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-26 Thread Wei Wang
On 09/25/2017 10:57 PM, Andi Kleen wrote: +static void auto_switch_lbr_msrs(struct vcpu_vmx *vmx) +{ + int i; + struct perf_lbr_stack lbr_stack; + + perf_get_lbr_stack(_stack); + + add_atomic_switch_msr(vmx, MSR_LBR_SELECT, 0, 0); + add_atomic_switch_msr(vmx,

Re: [PATCH v1 0/4] Enable LBR for the guest

2017-09-26 Thread Wei Wang
On 09/25/2017 10:59 PM, Andi Kleen wrote: On Mon, Sep 25, 2017 at 12:44:52PM +0800, Wei Wang wrote: This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e

Re: [PATCH v1 0/4] Enable LBR for the guest

2017-09-26 Thread Wei Wang
On 09/25/2017 10:59 PM, Andi Kleen wrote: On Mon, Sep 25, 2017 at 12:44:52PM +0800, Wei Wang wrote: This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e

Re: [PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

2017-09-25 Thread Wei Wang
On 09/25/2017 07:54 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: +static void update_msr_autoload_count_max(void) +{ + u64 vmx_msr; + int n; + + /* +* According to the Intel SDM, if Bits 27:25 of MSR_IA32_VMX_MISC is +* n, then (n + 1) * 512

Re: [PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

2017-09-25 Thread Wei Wang
On 09/25/2017 07:54 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: +static void update_msr_autoload_count_max(void) +{ + u64 vmx_msr; + int n; + + /* +* According to the Intel SDM, if Bits 27:25 of MSR_IA32_VMX_MISC is +* n, then (n + 1) * 512

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-25 Thread Wei Wang
On 09/25/2017 05:16 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang <wei.w.w...@intel.com> This has to be enabled separately for each guest, because

Re: [PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-25 Thread Wei Wang
On 09/25/2017 05:16 PM, Paolo Bonzini wrote: On 25/09/2017 06:44, Wei Wang wrote: Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang This has to be enabled separately for each guest, because it may prevent live migration

[PATCH v1 2/4] KVM/vmx: auto switch MSR_IA32_DEBUGCTLMSR

2017-09-24 Thread Wei Wang
Passthrough the MSR_IA32_DEBUGCTLMSR to the guest, and take advantage of the hardware VT-x feature to auto switch the msr upon VMExit and VMEntry. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/kvm/vmx.c | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff

[PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

2017-09-24 Thread Wei Wang
. This can be calculated by (MSR_IA32_VMX_MISC[27:25] + 1) * 512. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/kvm/vmx.c | 72 +++--- 1 file changed, 63 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm

[PATCH v1 2/4] KVM/vmx: auto switch MSR_IA32_DEBUGCTLMSR

2017-09-24 Thread Wei Wang
Passthrough the MSR_IA32_DEBUGCTLMSR to the guest, and take advantage of the hardware VT-x feature to auto switch the msr upon VMExit and VMEntry. Signed-off-by: Wei Wang --- arch/x86/kvm/vmx.c | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx.c

[PATCH v1 1/4] KVM/vmx: re-write the msr auto switch feature

2017-09-24 Thread Wei Wang
. This can be calculated by (MSR_IA32_VMX_MISC[27:25] + 1) * 512. Signed-off-by: Wei Wang --- arch/x86/kvm/vmx.c | 72 +++--- 1 file changed, 63 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 0726ca7..8434fc8

[PATCH v1 0/4] Enable LBR for the guest

2017-09-24 Thread Wei Wang
This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e. load and saved) on VMExit and VMEntry. Test: Try "perf record -b ./test_program" on guest. W

[PATCH v1 3/4] perf/x86: add a function to get the lbr stack

2017-09-24 Thread Wei Wang
for the guest. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/events/intel/lbr.c | 23 +++ arch/x86/include/asm/perf_event.h | 14 ++ 2 files changed, 37 insertions(+) diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c index 8

[PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-24 Thread Wei Wang
Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang <wei.w.w...@intel.com> --- arch/x86/kvm/vmx.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/arch/x86/kvm/vmx.c

[PATCH v1 3/4] perf/x86: add a function to get the lbr stack

2017-09-24 Thread Wei Wang
for the guest. Signed-off-by: Wei Wang --- arch/x86/events/intel/lbr.c | 23 +++ arch/x86/include/asm/perf_event.h | 14 ++ 2 files changed, 37 insertions(+) diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c index 8a6bbac..ea547ec 100644 --- a/arch

[PATCH v1 0/4] Enable LBR for the guest

2017-09-24 Thread Wei Wang
This patch series enables the Last Branch Recording feature for the guest. Instead of trapping each LBR stack MSR access, the MSRs are passthroughed to the guest. Those MSRs are switched (i.e. load and saved) on VMExit and VMEntry. Test: Try "perf record -b ./test_program" on guest. W

[PATCH v1 4/4] KVM/vmx: enable lbr for the guest

2017-09-24 Thread Wei Wang
Passthrough the LBR stack to the guest, and auto switch the stack MSRs upon VMEntry and VMExit. Signed-off-by: Wei Wang --- arch/x86/kvm/vmx.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index

Re: [PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

2017-09-12 Thread Wei Wang
On 09/11/2017 08:54 PM, Matthew Wilcox wrote: On Mon, Aug 28, 2017 at 06:08:29PM +0800, Wei Wang wrote: From: Matthew Wilcox <mawil...@microsoft.com> The eXtensible Bitmap is a sparse bitmap representation which is efficient for set bits which tend to cluster. It supports up to 'unsigne

Re: [PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

2017-09-12 Thread Wei Wang
On 09/11/2017 08:54 PM, Matthew Wilcox wrote: On Mon, Aug 28, 2017 at 06:08:29PM +0800, Wei Wang wrote: From: Matthew Wilcox The eXtensible Bitmap is a sparse bitmap representation which is efficient for set bits which tend to cluster. It supports up to 'unsigned long' worth of bits

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-09-08 Thread Wei Wang
On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon

Re: [virtio-dev] Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-09-08 Thread Wei Wang
On 09/08/2017 11:36 AM, Michael S. Tsirkin wrote: On Tue, Aug 29, 2017 at 11:09:18AM +0800, Wei Wang wrote: On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon

Re: [PATCH v15 4/5] mm: support reporting free page blocks

2017-08-28 Thread Wei Wang
On 08/28/2017 09:33 PM, Michal Hocko wrote: On Mon 28-08-17 18:08:32, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [PATCH v15 4/5] mm: support reporting free page blocks

2017-08-28 Thread Wei Wang
On 08/28/2017 09:33 PM, Michal Hocko wrote: On Mon 28-08-17 18:08:32, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-28 Thread Wei Wang
On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon (i.e. inflated/deflated) pages using scatter-gather lists to the host. The implementation of the previous

Re: [PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-28 Thread Wei Wang
On 08/29/2017 02:03 AM, Michael S. Tsirkin wrote: On Mon, Aug 28, 2017 at 06:08:31PM +0800, Wei Wang wrote: Add a new feature, VIRTIO_BALLOON_F_SG, which enables the transfer of balloon (i.e. inflated/deflated) pages using scatter-gather lists to the host. The implementation of the previous

[PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

2017-08-28 Thread Wei Wang
st_bit(). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> --- include/linux/radix-tree.h | 3

[PATCH v15 1/5] lib/xbitmap: Introduce xbitmap

2017-08-28 Thread Wei Wang
Wilcox Signed-off-by: Wei Wang Cc: Andrew Morton Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/radix-tree.h | 3 + include/linux/xbitmap.h| 61 lib/Makefile | 2 +- lib/radix-tree.c | 22 +- lib/xbitmap.c | 176

[PATCH v15 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

2017-08-28 Thread Wei Wang
to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> --- drivers/virtio/virtio_balloon.c | 247 +--- include/uapi/linux/virtio_balloon.h | 15 +++ 2 files changed, 242 insertions(+), 20 deleti

[PATCH v15 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

2017-08-28 Thread Wei Wang
to the host. Signed-off-by: Wei Wang Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 247 +--- include/uapi/linux/virtio_balloon.h | 15 +++ 2 files changed, 242 insertions(+), 20 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b

[PATCH v15 4/5] mm: support reporting free page blocks

2017-08-28 Thread Wei Wang
ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-28 Thread Wei Wang
() in chunks rather than page by page. With this new feature, the above ballooning process takes ~597ms resulting in an improvement of ~86%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

[PATCH v15 4/5] mm: support reporting free page blocks

2017-08-28 Thread Wei Wang
ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/mm.h | 5 + mm/page_alloc.c|

[PATCH v15 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-28 Thread Wei Wang
() in chunks rather than page by page. With this new feature, the above ballooning process takes ~597ms resulting in an improvement of ~86%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang Signed-off-by: Liang Li Suggested

[PATCH v15 2/5] lib/xbitmap: add xb_find_next_bit() and xb_zero()

2017-08-28 Thread Wei Wang
xb_find_next_bit() is used to find the next "1" or "0" bit in the given range. xb_zero() is used to zero the given range of bits. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Matthew Wilcox <mawil...@m

[PATCH v15 2/5] lib/xbitmap: add xb_find_next_bit() and xb_zero()

2017-08-28 Thread Wei Wang
xb_find_next_bit() is used to find the next "1" or "0" bit in the given range. xb_zero() is used to zero the given range of bits. Signed-off-by: Wei Wang Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/xbitmap.h |

[PATCH v15 0/5] Virtio-balloon Enhancement

2017-08-28 Thread Wei Wang
tead of two. 2) re-write the virtio-balloon implementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1): lib/xbitmap: Introduce xbitmap Wei Wang (4): lib/xbitmap: add xb_find_next_bit() and xb_zero() virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page

[PATCH v15 0/5] Virtio-balloon Enhancement

2017-08-28 Thread Wei Wang
tead of two. 2) re-write the virtio-balloon implementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1): lib/xbitmap: Introduce xbitmap Wei Wang (4): lib/xbitmap: add xb_find_next_bit() and xb_zero() virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page

Re: [PATCH v14 4/5] mm: support reporting free page blocks

2017-08-21 Thread Wei Wang
On 08/18/2017 09:46 PM, Michal Hocko wrote: On Thu 17-08-17 11:26:55, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [PATCH v14 4/5] mm: support reporting free page blocks

2017-08-21 Thread Wei Wang
On 08/18/2017 09:46 PM, Michal Hocko wrote: On Thu 17-08-17 11:26:55, Wei Wang wrote: This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the calle

Re: [virtio-dev] Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-20 Thread Wei Wang
On 08/19/2017 02:10 AM, Michael S. Tsirkin wrote: On Fri, Aug 18, 2017 at 04:36:06PM +0800, Wei Wang wrote: On 08/18/2017 10:28 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Signed-off

Re: [virtio-dev] Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-20 Thread Wei Wang
On 08/19/2017 02:10 AM, Michael S. Tsirkin wrote: On Fri, Aug 18, 2017 at 04:36:06PM +0800, Wei Wang wrote: On 08/18/2017 10:28 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Signed-off

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-20 Thread Wei Wang
On 08/19/2017 02:26 AM, Michael S. Tsirkin wrote: On Fri, Aug 18, 2017 at 04:41:41PM +0800, Wei Wang wrote: On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-20 Thread Wei Wang
On 08/19/2017 02:26 AM, Michael S. Tsirkin wrote: On Fri, Aug 18, 2017 at 04:41:41PM +0800, Wei Wang wrote: On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-18 Thread Wei Wang
On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add some text here explaining the report_free_page_signal thing. I also really think we need some kind of ID

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-18 Thread Wei Wang
On 08/18/2017 10:13 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Please add some text here explaining the report_free_page_signal thing. I also really think we need some kind of ID

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-18 Thread Wei Wang
On 08/18/2017 10:28 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> --- dr

Re: [PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-18 Thread Wei Wang
On 08/18/2017 10:28 AM, Michael S. Tsirkin wrote: On Thu, Aug 17, 2017 at 11:26:56AM +0800, Wei Wang wrote: Add a new vq to report hints of guest free pages to the host. Signed-off-by: Wei Wang Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 167

Re: [PATCH v14 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-18 Thread Wei Wang
On 08/18/2017 10:22 AM, Michael S. Tsirkin wrote: +static void send_balloon_page_sg(struct virtio_balloon *vb, +struct virtqueue *vq, +void *addr, +uint32_t size) +{ + unsigned int len; +

Re: [PATCH v14 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-18 Thread Wei Wang
On 08/18/2017 10:22 AM, Michael S. Tsirkin wrote: +static void send_balloon_page_sg(struct virtio_balloon *vb, +struct virtqueue *vq, +void *addr, +uint32_t size) +{ + unsigned int len; +

[PATCH v14 4/5] mm: support reporting free page blocks

2017-08-16 Thread Wei Wang
This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the caller's responsibility to either detect or prevent the use of such pages. Signed-off-by:

[PATCH v14 4/5] mm: support reporting free page blocks

2017-08-16 Thread Wei Wang
This patch adds support to walk through the free page blocks in the system and report them via a callback function. Some page blocks may leave the free list after zone->lock is released, so it is the caller's responsibility to either detect or prevent the use of such pages. Signed-off-by:

[PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-16 Thread Wei Wang
Add a new vq to report hints of guest free pages to the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> --- drivers/virtio/virtio_balloon.c | 167 +++- include/uapi/linux/virtio_balloon.h |

[PATCH v14 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-16 Thread Wei Wang
() in chunks rather than page by page. With this new feature, the above ballooning process takes ~541ms resulting in an improvement of ~87%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

[PATCH v14 5/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

2017-08-16 Thread Wei Wang
Add a new vq to report hints of guest free pages to the host. Signed-off-by: Wei Wang Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 167 +++- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 147 insertions(+), 21 deletions

[PATCH v14 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-08-16 Thread Wei Wang
() in chunks rather than page by page. With this new feature, the above ballooning process takes ~541ms resulting in an improvement of ~87%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang Signed-off-by: Liang Li Suggested

[PATCH v14 2/5] lib/xbitmap: add xb_find_next_bit() and xb_zero()

2017-08-16 Thread Wei Wang
xb_find_next_bit() is used to find the next "1" or "0" bit in the given range. xb_zero() is used to zero the given range of bits. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Matthew Wilcox <mawil...@m

[PATCH v14 1/5] lib/xbitmap: Introduce xbitmap

2017-08-16 Thread Wei Wang
st_bit(). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> --- include/linux/radix-tree.h | 3

[PATCH v14 0/5] Virtio-balloon Enhancement

2017-08-16 Thread Wei Wang
plementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1): lib/xbitmap: Introduce xbitmap Wei Wang (4): lib/xbitmap: add xb_find_next_bit() and xb_zero() virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_FREE_PAG

[PATCH v14 0/5] Virtio-balloon Enhancement

2017-08-16 Thread Wei Wang
plementation patch. 3) commit changes 4) patch re-org Matthew Wilcox (1): lib/xbitmap: Introduce xbitmap Wei Wang (4): lib/xbitmap: add xb_find_next_bit() and xb_zero() virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_FREE_PAG

[PATCH v14 2/5] lib/xbitmap: add xb_find_next_bit() and xb_zero()

2017-08-16 Thread Wei Wang
xb_find_next_bit() is used to find the next "1" or "0" bit in the given range. xb_zero() is used to zero the given range of bits. Signed-off-by: Wei Wang Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/xbitmap.h |

[PATCH v14 1/5] lib/xbitmap: Introduce xbitmap

2017-08-16 Thread Wei Wang
Wilcox Signed-off-by: Wei Wang Cc: Andrew Morton Cc: Michal Hocko Cc: Michael S. Tsirkin --- include/linux/radix-tree.h | 3 + include/linux/xbitmap.h| 61 lib/Makefile | 2 +- lib/radix-tree.c | 22 +- lib/xbitmap.c | 176

Re: [virtio-dev] [PATCH v13 0/5] Virtio-balloon Enhancement

2017-08-16 Thread Wei Wang
On 08/16/2017 01:57 PM, Adam Tao wrote: On Thu, Aug 03, 2017 at 02:38:14PM +0800, Wei Wang wrote: This patch series enhances the existing virtio-balloon with the following new features: 1) fast ballooning: transfer ballooned pages between the guest and host in chunks using sgs, instead of one

Re: [virtio-dev] [PATCH v13 0/5] Virtio-balloon Enhancement

2017-08-16 Thread Wei Wang
On 08/16/2017 01:57 PM, Adam Tao wrote: On Thu, Aug 03, 2017 at 02:38:14PM +0800, Wei Wang wrote: This patch series enhances the existing virtio-balloon with the following new features: 1) fast ballooning: transfer ballooned pages between the guest and host in chunks using sgs, instead of one

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-13 Thread Wei Wang
t merged before submitting my patch? Thanks. Wei On Sun, Aug 13, 2017 at 9:24 AM, David Ahern <dsah...@gmail.com> wrote: > On 8/12/17 1:42 PM, Wei Wang wrote: >> Hi Ido, >> >>>> - if ((rt->dst.dev == dev || !dev) && >>>> + if ((rt->dst

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-13 Thread Wei Wang
rged before submitting my patch? Thanks. Wei On Sun, Aug 13, 2017 at 9:24 AM, David Ahern wrote: > On 8/12/17 1:42 PM, Wei Wang wrote: >> Hi Ido, >> >>>> - if ((rt->dst.dev == dev || !dev) && >>>> + if ((rt->dst.dev == dev || !dev

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
ei On Sat, Aug 12, 2017 at 11:01 AM, Ido Schimmel <ido...@idosch.org> wrote: > Hi Wei, > > On Fri, Aug 11, 2017 at 05:10:02PM -0700, Wei Wang wrote: >> I think we have a potential fix for this issue. >> Martin and I found that when addrconf_dst_alloc() creates a rt6, it

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
Wei On Sat, Aug 12, 2017 at 11:01 AM, Ido Schimmel wrote: > Hi Wei, > > On Fri, Aug 11, 2017 at 05:10:02PM -0700, Wei Wang wrote: >> I think we have a potential fix for this issue. >> Martin and I found that when addrconf_dst_alloc() creates a rt6, it is >> possible t

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
On Fri, Aug 11, 2017 at 8:37 PM, David Ahern <dsah...@gmail.com> wrote: > On 8/11/17 6:25 PM, Wei Wang wrote: >> By "a patch to fix that" do you mean after your patch, for every rt6, >> rt6->rt6i_idev will be the same as rt6->dst.dev? > > FIB entries sho

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
On Fri, Aug 11, 2017 at 8:37 PM, David Ahern wrote: > On 8/11/17 6:25 PM, Wei Wang wrote: >> By "a patch to fix that" do you mean after your patch, for every rt6, >> rt6->rt6i_idev will be the same as rt6->dst.dev? > > FIB entries should have them the same

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
M, John Stultz <john.stu...@linaro.org> wrote: > On Fri, Aug 11, 2017 at 5:31 PM, John Stultz <john.stu...@linaro.org> wrote: >> On Fri, Aug 11, 2017 at 5:10 PM, Wei Wang <wei...@google.com> wrote: >>>> If after Cong's fix, the issue still happens, could you hel

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
Aug 11, 2017 at 5:31 PM, John Stultz wrote: >> On Fri, Aug 11, 2017 at 5:10 PM, Wei Wang wrote: >>>> If after Cong's fix, the issue still happens, could you help try the >>>> patch attached and collect all logs when you try the reproduce the >>>&

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
On Fri, Aug 11, 2017 at 8:07 PM, John Stultz <john.stu...@linaro.org> wrote: > On Fri, Aug 11, 2017 at 5:31 PM, John Stultz <john.stu...@linaro.org> wrote: >> On Fri, Aug 11, 2017 at 5:10 PM, Wei Wang <wei...@google.com> wrote: >>>> If after Cong's fix, the

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-12 Thread Wei Wang
On Fri, Aug 11, 2017 at 8:07 PM, John Stultz wrote: > On Fri, Aug 11, 2017 at 5:31 PM, John Stultz wrote: >> On Fri, Aug 11, 2017 at 5:10 PM, Wei Wang wrote: >>>> If after Cong's fix, the issue still happens, could you help try the >>>> patch attached

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
good so far. > Great. Thanks. > Do you still want feedback on the previous changes? If this patch is good, then I don't really need the previous feedback. Thanks. Wei On Fri, Aug 11, 2017 at 5:31 PM, John Stultz <john.stu...@linaro.org> wrote: > On Fri, Aug 11, 2017 at 5:10 PM

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
good so far. > Great. Thanks. > Do you still want feedback on the previous changes? If this patch is good, then I don't really need the previous feedback. Thanks. Wei On Fri, Aug 11, 2017 at 5:31 PM, John Stultz wrote: > On Fri, Aug 11, 2017 at 5:10 PM, Wei Wang wrote: >>> If af

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
On Fri, Aug 11, 2017 at 5:19 PM, David Ahern <dsah...@gmail.com> wrote: > On 8/11/17 6:10 PM, Wei Wang wrote: >> I think we have a potential fix for this issue. >> Martin and I found that when addrconf_dst_alloc() creates a rt6, it is >> possible that rt6->dst.dev p

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
On Fri, Aug 11, 2017 at 5:19 PM, David Ahern wrote: > On 8/11/17 6:10 PM, Wei Wang wrote: >> I think we have a potential fix for this issue. >> Martin and I found that when addrconf_dst_alloc() creates a rt6, it is >> possible that rt6->dst.dev points to loopback device

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
if it fixes the issue on your side before I submit an official patch. Thanks very much for the help from everyone. Wei On Fri, Aug 11, 2017 at 10:25 AM, Wei Wang <wei...@google.com> wrote: > On Fri, Aug 11, 2017 at 9:48 AM, Cong Wang <xiyou.wangc...@gmail.com> wrote: >> Hi

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
if it fixes the issue on your side before I submit an official patch. Thanks very much for the help from everyone. Wei On Fri, Aug 11, 2017 at 10:25 AM, Wei Wang wrote: > On Fri, Aug 11, 2017 at 9:48 AM, Cong Wang wrote: >> Hi, >> >> On Thu, Aug 10, 2017 at 11:12 AM, Jo

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
On Fri, Aug 11, 2017 at 9:48 AM, Cong Wang <xiyou.wangc...@gmail.com> wrote: > Hi, > > On Thu, Aug 10, 2017 at 11:12 AM, John Stultz <john.stu...@linaro.org> wrote: >> On Wed, Aug 9, 2017 at 10:41 PM, Wei Wang <wei...@google.com> wrote: >>> Hi John, >&g

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-11 Thread Wei Wang
On Fri, Aug 11, 2017 at 9:48 AM, Cong Wang wrote: > Hi, > > On Thu, Aug 10, 2017 at 11:12 AM, John Stultz wrote: >> On Wed, Aug 9, 2017 at 10:41 PM, Wei Wang wrote: >>> Hi John, >>> >>> Is it possible to try the attached patch? >> >> Thanks

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-10 Thread Wei Wang
On Thu, Aug 10, 2017 at 11:12 AM, John Stultz <john.stu...@linaro.org> wrote: > On Wed, Aug 9, 2017 at 10:41 PM, Wei Wang <wei...@google.com> wrote: >> Hi John, >> >> Is it possible to try the attached patch? > > Thanks so much for the quick turn aroun

Re: unregister_netdevice: waiting for eth0 to become free. Usage count = 1

2017-08-10 Thread Wei Wang
On Thu, Aug 10, 2017 at 11:12 AM, John Stultz wrote: > On Wed, Aug 9, 2017 at 10:41 PM, Wei Wang wrote: >> Hi John, >> >> Is it possible to try the attached patch? > > Thanks so much for the quick turn around! > > So I dropped all the reverts you suggested, an

Re: [virtio-dev] Re: [PATCH v13 4/5] mm: support reporting free page blocks

2017-08-10 Thread Wei Wang
On 08/10/2017 03:05 PM, Michal Hocko wrote: On Tue 08-08-17 14:34:25, Wei Wang wrote: On 08/08/2017 02:12 PM, Wei Wang wrote: On 08/03/2017 05:11 PM, Michal Hocko wrote: On Thu 03-08-17 14:38:18, Wei Wang wrote: This is just too ugly and wrong actually. Never provide struct page pointers

Re: [virtio-dev] Re: [PATCH v13 4/5] mm: support reporting free page blocks

2017-08-10 Thread Wei Wang
On 08/10/2017 03:05 PM, Michal Hocko wrote: On Tue 08-08-17 14:34:25, Wei Wang wrote: On 08/08/2017 02:12 PM, Wei Wang wrote: On 08/03/2017 05:11 PM, Michal Hocko wrote: On Thu 03-08-17 14:38:18, Wei Wang wrote: This is just too ugly and wrong actually. Never provide struct page pointers

Re: [PATCH v13 1/5] Introduce xbitmap

2017-08-10 Thread Wei Wang
On 08/10/2017 05:36 AM, Andrew Morton wrote: On Thu, 3 Aug 2017 14:38:15 +0800 Wei Wang <wei.w.w...@intel.com> wrote: From: Matthew Wilcox <mawil...@microsoft.com> The eXtensible Bitmap is a sparse bitmap representation which is efficient for set bits which tend to cluster. It

Re: [PATCH v13 1/5] Introduce xbitmap

2017-08-10 Thread Wei Wang
On 08/10/2017 05:36 AM, Andrew Morton wrote: On Thu, 3 Aug 2017 14:38:15 +0800 Wei Wang wrote: From: Matthew Wilcox The eXtensible Bitmap is a sparse bitmap representation which is efficient for set bits which tend to cluster. It supports up to 'unsigned long' worth of bits

<    3   4   5   6   7   8   9   10   11   >