Re: [PATCH v4 03/25] virtio-pci: move freeze/restore to virtio core

2014-10-15 Thread Paul Bolle
is patch landed in today's linux-next (next-20141015). > include/linux/virtio.h | 6 + > drivers/virtio/virtio.c | 54 > + > drivers/virtio/virtio_pci.c | 54 > ++--- > 3 file

Re: [PATCH net-next RFC 0/3] virtio-net: Conditionally enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 07:06 AM, Michael S. Tsirkin wrote: > On Tue, Oct 14, 2014 at 02:53:27PM -0400, David Miller wrote: >> > From: Jason Wang >> > Date: Sat, 11 Oct 2014 15:16:43 +0800 >> > >>> > > We free old transmitted packets in ndo_start_xmit() currently, so any >>> > > packet must be orphaned als

[RFC PATCH net-next 4/6] virtio-net: return the number of packets sent in free_old_xmit_skbs()

2014-10-15 Thread Jason Wang
This patch returns the number of packets sent in free_old_xmit_skbs(), this is necessary for calling this function in napi. Cc: Rusty Russell Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- drivers/net/virtio_net.c |4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git

[RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Jason Wang
Accumulate the sent packets and sent bytes in local variables and perform a single u64_stats_update_begin/end() after. Cc: Rusty Russell Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- drivers/net/virtio_net.c | 12 1 files changed, 8 insertions(+), 4 deletions(-) diff --g

[RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
This patch introduces virtio_enable_cb_avail() to publish avail idx and used event. This could be used by batched buffer submitting to reduce the number of tx interrupts. Cc: Rusty Russell Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- drivers/virtio/virtio_ring.c | 22

[RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Jason Wang
According to David, proper accounting and queueing (at all levels, not just TCP sockets) is more important than trying to skim a bunch of cycles by avoiding TX interrupts. Having an event to free the SKB is absolutely essential for the stack to operate correctly. This series tries to enable tx int

[RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Jason Wang
With the help of xmit_more and virtqueue_enable_cb_avail(), this patch enable tx interrupt only for the final skb in the chain if needed. This will help to mitigate tx interrupts. Cc: Rusty Russell Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- drivers/net/virtio_net.c | 10 +++---

[RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
Orphan skb in ndo_start_xmit() breaks socket accounting and packet queuing. This in fact breaks lots of things such as pktgen and several TCP optimizations. And also make BQL can't be implemented for virtio-net. This patch tries to solve this issue by enabling tx interrupt. To avoid introducing ex

[RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Jason Wang
This patch checks the new event idx to make sure used event idx never goes back. This is used to synchronize the calls between virtqueue_enable_cb_delayed() and virtqueue_enable_cb(). Cc: Rusty Russell Cc: Michael S. Tsirkin Signed-off-by: Jason Wang --- drivers/virtio/virtio_ring.c |7 +++

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: > This patch introduces virtio_enable_cb_avail() to publish avail idx > and used event. This could be used by batched buffer submitting to > reduce the number of tx interrupts. > > Cc: Rusty Russell > Cc: Michael S. Tsirkin > Signed-off

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: > This patch checks the new event idx to make sure used event idx never > goes back. This is used to synchronize the calls between > virtqueue_enable_cb_delayed() and virtqueue_enable_cb(). > > Cc: Rusty Russell > Cc: Michael S. Tsirkin

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: > Accumulate the sent packets and sent bytes in local variables and perform a > single u64_stats_update_begin/end() after. > > Cc: Rusty Russell > Cc: Michael S. Tsirkin > Signed-off-by: Jason Wang Not sure how much it's worth but sin

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Eric Dumazet
On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: > Accumulate the sent packets and sent bytes in local variables and perform a > single u64_stats_update_begin/end() after. > > Cc: Rusty Russell > Cc: Michael S. Tsirkin > Signed-off-by: Jason Wang > --- > drivers/net/virtio_net.c | 12 +++

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Eric Dumazet
On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: ... > +static int free_old_xmit_skbs(struct send_queue *sq, int budget) > +{ > + struct sk_buff *skb; > + unsigned int len; > + struct virtnet_info *vi = sq->vq->vdev->priv; > + struct virtnet_stats *stats = this_cpu_ptr(vi->sta

RE: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread David Laight
From: Of Michael S. Tsirkin > On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: > > Accumulate the sent packets and sent bytes in local variables and perform a > > single u64_stats_update_begin/end() after. > > > > Cc: Rusty Russell > > Cc: Michael S. Tsirkin > > Signed-off-by: Jason Wa

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Jason Wang
On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: >> This patch checks the new event idx to make sure used event idx never >> goes back. This is used to synchronize the calls between >> virtqueue_enable_cb_delayed() and virtqueue_enable_

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: > Orphan skb in ndo_start_xmit() breaks socket accounting and packet > queuing. This in fact breaks lots of things such as pktgen and several > TCP optimizations. And also make BQL can't be implemented for > virtio-net. > > This patch tri

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: >> This patch introduces virtio_enable_cb_avail() to publish avail idx >> and used event. This could be used by batched buffer submitting to >> reduce the number of tx interrupts. >> >> C

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: > With the help of xmit_more and virtqueue_enable_cb_avail(), this patch > enable tx interrupt only for the final skb in the chain if > needed. This will help to mitigate tx interrupts. > > Cc: Rusty Russell > Cc: Michael S. Tsirkin > S

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 05:37 PM, Eric Dumazet wrote: > On Wed, 2014-10-15 at 15:25 +0800, Jason Wang wrote: > > ... > >> +static int free_old_xmit_skbs(struct send_queue *sq, int budget) >> +{ >> +struct sk_buff *skb; >> +unsigned int len; >> +struct virtnet_info *vi = sq->vq->vdev->priv; >> +

Re: [RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 03:25:24PM +0800, Jason Wang wrote: > According to David, proper accounting and queueing (at all levels, not > just TCP sockets) is more important than trying to skim a bunch of > cycles by avoiding TX interrupts. He also mentioned we should find other ways to batch > Havi

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 06:18 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: >> > Orphan skb in ndo_start_xmit() breaks socket accounting and packet >> > queuing. This in fact breaks lots of things such as pktgen and several >> > TCP optimizations. And also make B

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: > On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: > >> This patch checks the new event idx to make sure used event idx never > >> goes back. This is used to synchronize the

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Jason Wang
On 10/15/2014 06:22 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: >> With the help of xmit_more and virtqueue_enable_cb_avail(), this patch >> enable tx interrupt only for the final skb in the chain if >> needed. This will help to mitigate tx interrupts

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: > On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: > >> This patch introduces virtio_enable_cb_avail() to publish avail idx > >> and used event. This could be used by batched

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:25:25PM +0800, Jason Wang wrote: > On 10/15/2014 06:18 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: > >> > Orphan skb in ndo_start_xmit() breaks socket accounting and packet > >> > queuing. This in fact breaks lots of thing

Re: [RFC PATCH net-next 6/6] virtio-net: enable tx interrupt only for the final skb in the chain

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:31:19PM +0800, Jason Wang wrote: > On 10/15/2014 06:22 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 03:25:30PM +0800, Jason Wang wrote: > >> With the help of xmit_more and virtqueue_enable_cb_avail(), this patch > >> enable tx interrupt only for the final skb

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Jason Wang
On 10/15/2014 06:32 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: >> On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: >>> On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: This patch checks the new event idx to make sure used event idx n

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: > From: Of Michael S. Tsirkin > > On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: > > > Accumulate the sent packets and sent bytes in local variables and perform > > > a > > > single u64_stats_update_begin/end() after. > > >

RE: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread David Laight
From: Michael S. Tsirkin > On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: > > From: Of Michael S. Tsirkin > > > On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: > > > > Accumulate the sent packets and sent bytes in local variables and > > > > perform a > > > > single u64_

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Jason Wang
On 10/15/2014 06:41 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: >> On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: >>> On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: This patch introduces virtio_enable_cb_avail() to publish avail i

Re: [RFC PATCH net-next 5/6] virtio-net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 06:43 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 06:25:25PM +0800, Jason Wang wrote: >> > On 10/15/2014 06:18 PM, Michael S. Tsirkin wrote: >>> > > On Wed, Oct 15, 2014 at 03:25:29PM +0800, Jason Wang wrote: > > >> > Orphan skb in ndo_start_xmit() breaks socket accoun

Re: [PATCH RFC] virtio_net: enable tx interrupt

2014-10-15 Thread Jason Wang
On 10/15/2014 05:53 AM, Michael S. Tsirkin wrote: > On newer hosts that support delayed tx interrupts, > we probably don't have much to gain from orphaning > packets early. > > Based on patch by Jason Wang. > > Note: this will likely degrade performance for hosts without event idx > support. Vario

Re: [RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Jason Wang
On 10/15/2014 06:25 PM, Michael S. Tsirkin wrote: > On Wed, Oct 15, 2014 at 03:25:24PM +0800, Jason Wang wrote: >> According to David, proper accounting and queueing (at all levels, not >> just TCP sockets) is more important than trying to skim a bunch of >> cycles by avoiding TX interrupts. > He a

Re: [RFC PATCH net-next 1/6] virtio: make sure used event never go backwards

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:44:41PM +0800, Jason Wang wrote: > On 10/15/2014 06:32 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 06:13:19PM +0800, Jason Wang wrote: > >> On 10/15/2014 05:34 PM, Michael S. Tsirkin wrote: > >>> On Wed, Oct 15, 2014 at 03:25:25PM +0800, Jason Wang wrote: >

Re: [RFC PATCH net-next 2/6] virtio: introduce virtio_enable_cb_avail()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 06:58:15PM +0800, Jason Wang wrote: > On 10/15/2014 06:41 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 06:19:15PM +0800, Jason Wang wrote: > >> On 10/15/2014 05:28 PM, Michael S. Tsirkin wrote: > >>> On Wed, Oct 15, 2014 at 03:25:26PM +0800, Jason Wang wrote: >

Re: [PATCH RFC] virtio_net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 07:04:20PM +0800, Jason Wang wrote: > On 10/15/2014 05:53 AM, Michael S. Tsirkin wrote: > > On newer hosts that support delayed tx interrupts, > > we probably don't have much to gain from orphaning > > packets early. > > > > Based on patch by Jason Wang. > > > > Note: this w

Re: [RFC PATCH net-next 0/6] Always use tx interrupt for virtio-net

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 07:14:14PM +0800, Jason Wang wrote: > On 10/15/2014 06:25 PM, Michael S. Tsirkin wrote: > > On Wed, Oct 15, 2014 at 03:25:24PM +0800, Jason Wang wrote: > >> According to David, proper accounting and queueing (at all levels, not > >> just TCP sockets) is more important than t

Re: [RFC PATCH net-next 3/6] virtio-net: small optimization on free_old_xmit_skbs()

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 10:51:32AM +, David Laight wrote: > From: Michael S. Tsirkin > > On Wed, Oct 15, 2014 at 09:49:01AM +, David Laight wrote: > > > From: Of Michael S. Tsirkin > > > > On Wed, Oct 15, 2014 at 03:25:27PM +0800, Jason Wang wrote: > > > > > Accumulate the sent packets and

[PATCH] virtio_net: fix use after free

2014-10-15 Thread Michael S. Tsirkin
commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 net: Remove ndo_xmit_flush netdev operation, use signalling instead. added code that looks at skb->xmit_more after the skb has been put in TX VQ. Since some paths process the ring and free the skb immediately, this can cause use after free. Fix

RE: [PATCH] virtio_net: fix use after free

2014-10-15 Thread David Laight
From: Michael S. Tsirkin > commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 > net: Remove ndo_xmit_flush netdev operation, use signalling instead. > > added code that looks at skb->xmit_more after the skb has > been put in TX VQ. Since some paths process the ring and free the skb > immediately,

Re: [PATCH] virtio_net: fix use after free

2014-10-15 Thread Michael S. Tsirkin
On Wed, Oct 15, 2014 at 01:24:57PM +, David Laight wrote: > From: Michael S. Tsirkin > > commit 0b725a2ca61bedc33a2a63d0451d528b268cf975 > > net: Remove ndo_xmit_flush netdev operation, use signalling instead. > > > > added code that looks at skb->xmit_more after the skb has > > been put i

[PATCH RFC v2 1/3] virtio_net: enable tx interrupt

2014-10-15 Thread Michael S. Tsirkin
On newer hosts that support delayed tx interrupts, we probably don't have much to gain from orphaning packets early. Based on patch by Jason Wang. Note: this might degrade performance for hosts without event idx support. Should be addressed by the next patch. Signed-off-by: Michael S. Tsirkin -

[PATCH RFC v2 3/3] virtio-net: optimize free_old_xmit_skbs stats

2014-10-15 Thread Michael S. Tsirkin
From: Jason Wang We already have counters for sent packets and sent bytes. Use them to reduce the number of u64_stats_update_begin/end(). Take care not to bother with stats update when called speculatively. Based on a patch by Jason Wang. Cc: Rusty Russell Signed-off-by: Jason Wang Signed-of

[PATCH RFC v2 2/3] virtio_net: bql

2014-10-15 Thread Michael S. Tsirkin
Improve tx batching using byte queue limits. Should be especially effective for MQ. Signed-off-by: Michael S. Tsirkin --- drivers/net/virtio_net.c | 20 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a

[PATCH 1/2] virtio_balloon: return the amount of freed memory from leak_balloon()

2014-10-15 Thread Denis V. Lunev
From: Raushaniya Maksudova This value would be useful in the next patch to provide the amount of the freed memory for OOM killer. Signed-off-by: Raushaniya Maksudova Signed-off-by: Denis V. Lunev CC: Rusty Russell CC: Michael S. Tsirkin --- drivers/virtio/virtio_balloon.c | 5 - 1 file

[PATCH v3 0/2] shrink virtio baloon on OOM in guest

2014-10-15 Thread Denis V. Lunev
Excessive virtio_balloon inflation can cause invocation of OOM-killer, when Linux is under severe memory pressure. Various mechanisms are responsible for correct virtio_balloon memory management. Nevertheless it is often the case that these control tools does not have enough time to react on fast c

[PATCH 2/2] virtio_balloon: free some memory from balloon on OOM

2014-10-15 Thread Denis V. Lunev
From: Raushaniya Maksudova Excessive virtio_balloon inflation can cause invocation of OOM-killer, when Linux is under severe memory pressure. Various mechanisms are responsible for correct virtio_balloon memory management. Nevertheless it is often the case that these control tools does not have e

Re: [PATCH] virtio_net: fix use after free

2014-10-15 Thread David Miller
From: "Michael S. Tsirkin" Date: Wed, 15 Oct 2014 16:23:28 +0300 > You used __netif_subqueue_stopped but that seems to use > a slightly more expensive test_bit internally. More expensive in what sense? It should be roughly the same as "x & y" sans the volatile. Anyways I'm ambivalent and I wan