Re: [PATCH 2/2] virtio_balloon: free some memory from baloon on OOM

2014-10-15 Thread Denis V. Lunev
On 14/10/14 13:10, Michael S. Tsirkin wrote: On Tue, Oct 14, 2014 at 10:14:05AM +1030, Rusty Russell wrote: Michael S. Tsirkin m...@redhat.com writes: On Mon, Oct 13, 2014 at 04:02:52PM +1030, Rusty Russell wrote: Denis V. Lunev d...@parallels.com writes: From: Raushaniya Maksudova

Re: [PATCH net-next RFC 1/3] virtio: support for urgent descriptors

2014-10-15 Thread Rusty Russell
Jason Wang jasow...@redhat.com writes: Below should be useful for some experiments Jason is doing. I thought I'd send it out for early review/feedback. event idx feature allows us to defer interrupts until a specific # of descriptors were used. Sometimes it might be useful to get an

Re: [PATCH 2/2] virtio_balloon: free some memory from baloon on OOM

2014-10-15 Thread Rusty Russell
Michael S. Tsirkin m...@redhat.com writes: On Tue, Oct 14, 2014 at 10:14:05AM +1030, Rusty Russell wrote: Michael S. Tsirkin m...@redhat.com writes: On Mon, Oct 13, 2014 at 04:02:52PM +1030, Rusty Russell wrote: Denis V. Lunev d...@parallels.com writes: From: Raushaniya Maksudova

Re: [PATCH] virtio_balloon: Convert vballon kthread into a workqueue

2014-10-15 Thread Rusty Russell
Petr Mladek pmla...@suse.cz writes: Workqueues have clean and rich API for all basic operations. The code is usually easier and better readable. It can be easily tuned for the given purpose. OK, sure. -static void fill_balloon(struct virtio_balloon *vb, size_t num) +static void

Re: [PATCH v4 03/25] virtio-pci: move freeze/restore to virtio core

2014-10-15 Thread Paul Bolle
...@de.ibm.com This patch landed in today's linux-next (next-20141015). include/linux/virtio.h | 6 + drivers/virtio/virtio.c | 54 + drivers/virtio/virtio_pci.c | 54 ++--- 3 files changed, 62

Re: [PATCH net-next RFC 0/3] virtio-net: Conditionally enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 07:06 AM, Michael S. Tsirkin wrote: On Tue, Oct 14, 2014 at 02:53:27PM -0400, David Miller wrote: From: Jason Wang jasow...@redhat.com Date: Sat, 11 Oct 2014 15:16:43 +0800 We free old transmitted packets in ndo_start_xmit() currently, so any packet must be orphaned

[RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Jason Wang
Accumulate the sent packets and sent bytes in local variables and perform a single u64_stats_update_begin/end() after. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Signed-off-by: Jason Wang jasow...@redhat.com --- drivers/net/virtio_net.c | 12 1

[RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
This patch introduces virtio_enable_cb_avail() to publish avail idx and used event. This could be used by batched buffer submitting to reduce the number of tx interrupts. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Signed-off-by: Jason Wang jasow...@redhat.com

[RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Jason Wang
According to David, proper accounting and queueing (at all levels, not just TCP sockets) is more important than trying to skim a bunch of cycles by avoiding TX interrupts. Having an event to free the SKB is absolutely essential for the stack to operate correctly. This series tries to enable tx

[RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Jason Wang
With the help of xmit_more and virtqueue_enable_cb_avail(), this patch enable tx interrupt only for the final skb in the chain if needed. This will help to mitigate tx interrupts. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Signed-off-by: Jason Wang

[RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
Orphan skb in ndo_start_xmit() breaks socket accounting and packet queuing. This in fact breaks lots of things such as pktgen and several TCP optimizations. And also make BQL can't be implemented for virtio-net. This patch tries to solve this issue by enabling tx interrupt. To avoid introducing

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This patch introduces virtio_enable_cb_avail() to publish avail idx and used event. This could be used by batched buffer submitting to reduce the number of tx interrupts. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S.

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This patch checks the new event idx to make sure used event idx never goes back. This is used to synchronize the calls between virtqueue_enable_cb_delayed() and virtqueue_enable_cb(). Cc: Rusty Russell ru...@rustcorp.com.au Cc:

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: Accumulate the sent packets and sent bytes in local variables and perform a single u64_stats_update_begin/end() after. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Signed-off-by: Jason Wang

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Eric Dumazet
On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: Accumulate the sent packets and sent bytes in local variables and perform a single u64_stats_update_begin/end() after. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Signed-off-by: Jason Wang

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Eric Dumazet
On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: ... +static int free_old_xmit_skbs(struct send_queue *sq, int budget) +{ + struct sk_buff *skb; + unsigned int len; + struct virtnet_info *vi = sq-vq-vdev-priv; + struct virtnet_stats *stats = this_cpu_ptr(vi-stats); +

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Jason Wang
On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This patch checks the new event idx to make sure used event idx never goes back. This is used to synchronize the calls between virtqueue_enable_cb_delayed() and virtqueue_enable_cb().

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: Orphan skb in ndo_start_xmit() breaks socket accounting and packet queuing. This in fact breaks lots of things such as pktgen and several TCP optimizations. And also make BQL can't be implemented for virtio-net. This patch tries to

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This patch introduces virtio_enable_cb_avail() to publish avail idx and used event. This could be used by batched buffer submitting to reduce the number of tx interrupts. Cc: Rusty

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: With the help of xmit_more and virtqueue_enable_cb_avail(), this patch enable tx interrupt only for the final skb in the chain if needed. This will help to mitigate tx interrupts. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 05:37 PM, Eric Dumazet wrote: On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: ... +static int free_old_xmit_skbs(struct send_queue *sq, int budget) +{ +struct sk_buff *skb; +unsigned int len; +struct virtnet_info *vi = sq-vq-vdev-priv; +struct

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 06:18 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: Orphan skb in ndo_start_xmit() breaks socket accounting and packet queuing. This in fact breaks lots of things such as pktgen and several TCP optimizations. And also make BQL can't

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This patch checks the new event idx to make sure used event idx never goes back. This is used to synchronize the calls

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Jason Wang
On 10/15/2014 06:22 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: With the help of xmit_more and virtqueue_enable_cb_avail(), this patch enable tx interrupt only for the final skb in the chain if needed. This will help to mitigate tx interrupts.

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This patch introduces virtio_enable_cb_avail() to publish avail idx and used event. This could be used by batched buffer

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:31:19PM +0800, Jason Wang wrote: On 10/15/2014 06:22 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: With the help of xmit_more and virtqueue_enable_cb_avail(), this patch enable tx interrupt only for the final skb in the

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Jason Wang
On 10/15/2014 06:32 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This patch checks the new event idx to make sure used event idx never

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: From: Of Michael S. Tsirkin On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: Accumulate the sent packets and sent bytes in local variables and perform a single u64_stats_update_begin/end() after. Cc: Rusty

RE: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread David Laight
From: Michael S. Tsirkin On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: From: Of Michael S. Tsirkin On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: Accumulate the sent packets and sent bytes in local variables and perform a single

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
On 10/15/2014 06:41 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This patch introduces virtio_enable_cb_avail() to publish avail idx and

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 06:43 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 06:25:25PM +0800, Jason Wang wrote: On 10/15/2014 06:18 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: Orphan skb in ndo_start_xmit() breaks socket accounting and packet

Re: [PATCH RFC] virtio_net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 05:53 AM, Michael S. Tsirkin wrote: On newer hosts that support delayed tx interrupts, we probably don't have much to gain from orphaning packets early. Based on patch by Jason Wang. Note: this will likely degrade performance for hosts without event idx support. Various

Re: [RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Jason Wang
On 10/15/2014 06:25 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:24PM +0800, Jason Wang wrote: According to David, proper accounting and queueing (at all levels, not just TCP sockets) is more important than trying to skim a bunch of cycles by avoiding TX interrupts. He also

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:44:41PM +0800, Jason Wang wrote: On 10/15/2014 06:32 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:58:15PM +0800, Jason Wang wrote: On 10/15/2014 06:41 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This

Re: [PATCH RFC] virtio_net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 07:04:20PM +0800, Jason Wang wrote: On 10/15/2014 05:53 AM, Michael S. Tsirkin wrote: On newer hosts that support delayed tx interrupts, we probably don't have much to gain from orphaning packets early. Based on patch by Jason Wang. Note: this will likely

Re: [RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 07:14:14PM +0800, Jason Wang wrote: On 10/15/2014 06:25 PM, Michael S. Tsirkin wrote: On Wed, Oct 15, 2014 at 03:25:24PM +0800, Jason Wang wrote: According to David, proper accounting and queueing (at all levels, not just TCP sockets) is more important than trying to

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 10:51:32AM +, David Laight wrote: From: Michael S. Tsirkin On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: From: Of Michael S. Tsirkin On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: Accumulate the sent packets and sent bytes in

[PATCH] virtio_net: fix use after free

2014-10-15 Thread Michael S. Tsirkin
commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 net: Remove ndo_xmit_flush netdev operation, use signalling instead. added code that looks at skb-xmit_more after the skb has been put in TX VQ. Since some paths process the ring and free the skb immediately, this can cause use after free. Fix

RE: [PATCH] virtio_net: fix use after free

2014-10-15 Thread David Laight
From: Michael S. Tsirkin commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 net: Remove ndo_xmit_flush netdev operation, use signalling instead. added code that looks at skb-xmit_more after the skb has been put in TX VQ. Since some paths process the ring and free the skb immediately, this

Re: [PATCH] virtio_net: fix use after free

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 01:24:57PM +, David Laight wrote: From: Michael S. Tsirkin commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 net: Remove ndo_xmit_flush netdev operation, use signalling instead. added code that looks at skb-xmit_more after the skb has been put in TX VQ.

[PATCH RFC v2 1/3] virtio_net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On newer hosts that support delayed tx interrupts, we probably don't have much to gain from orphaning packets early. Based on patch by Jason Wang. Note: this might degrade performance for hosts without event idx support. Should be addressed by the next patch. Signed-off-by: Michael S. Tsirkin

[PATCH RFC v2 3/3] virtio-net: optimize free_old_xmit_skbs stats

2014-10-15 Thread Michael S. Tsirkin
From: Jason Wang jasow...@redhat.com We already have counters for sent packets and sent bytes. Use them to reduce the number of u64_stats_update_begin/end(). Take care not to bother with stats update when called speculatively. Based on a patch by Jason Wang. Cc: Rusty Russell

[PATCH RFC v2 2/3] virtio_net: bql

2014-10-15 Thread Michael S. Tsirkin
Improve tx batching using byte queue limits. Should be especially effective for MQ. Signed-off-by: Michael S. Tsirkin m...@redhat.com --- drivers/net/virtio_net.c | 20 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/drivers/net/virtio_net.c

[PATCH 1/2] virtio_balloon: return the amount of freed memory from leak_balloon()

2014-10-15 Thread Denis V. Lunev
From: Raushaniya Maksudova rmaksud...@parallels.com This value would be useful in the next patch to provide the amount of the freed memory for OOM killer. Signed-off-by: Raushaniya Maksudova rmaksud...@parallels.com Signed-off-by: Denis V. Lunev d...@openvz.org CC: Rusty Russell

[PATCH v3 0/2] shrink virtio baloon on OOM in guest

2014-10-15 Thread Denis V. Lunev
Excessive virtio_balloon inflation can cause invocation of OOM-killer, when Linux is under severe memory pressure. Various mechanisms are responsible for correct virtio_balloon memory management. Nevertheless it is often the case that these control tools does not have enough time to react on fast

[PATCH 2/2] virtio_balloon: free some memory from balloon on OOM

2014-10-15 Thread Denis V. Lunev
From: Raushaniya Maksudova rmaksud...@parallels.com Excessive virtio_balloon inflation can cause invocation of OOM-killer, when Linux is under severe memory pressure. Various mechanisms are responsible for correct virtio_balloon memory management. Nevertheless it is often the case that these

Re: [PATCH] virtio_net: fix use after free

2014-10-15 Thread David Miller
From: Michael S. Tsirkin m...@redhat.com Date: Wed, 15 Oct 2014 16:23:28 +0300 You used __netif_subqueue_stopped but that seems to use a slightly more expensive test_bit internally. More expensive in what sense? It should be roughly the same as x y sans the volatile. Anyways I'm ambivalent