Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Michael S. Tsirkin]

On Tue, Oct 27, 2015 at 10:58:25AM +0800, Jason Wang wrote:
> 
> 
> On 10/26/2015 04:30 PM, Michael S. Tsirkin wrote:
> > On Mon, Oct 26, 2015 at 02:53:38PM +0800, Jason Wang wrote:
> >>
> >> On 10/26/2015 02:09 PM, Michael S. Tsirkin wrote:
> >>> On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
>  On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
> > On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
> >> We don't have fraglist support in TAP_FEATURES. This will lead
> >> software segmentation of gro skb with frag list. Fixes by having
> >> frag list support in TAP_FEATURES.
> >>
> >> With this patch single session of netperf receiving were restored from
> >> about 5Gb/s to about 12Gb/s on mlx4.
> >>
> >> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
> >> Cc: Vlad Yasevich 
> >> Cc: Michael S. Tsirkin 
> >> Signed-off-by: Jason Wang 
> > Thanks!
> > Does this mean we should look at re-adding NETIF_F_FRAGLIST
> > to virtio-net as well?
>  Not sure I get the point, but probably not. This is for receiving and
>  skb_copy_datagram_iter() can deal with frag list.
> >>> Point is:
> >>> - bridge within guest
> >>> - assigned device creating gro skbs with frag list bridged to virtio
> >> I see, but this problem looks not specific to virtio. Most cards does
> >> not support frag list.
> > These will be slower when used with a bridge then, won't they?
> 
> For forwarding, not sure. GRO has latency and cpu overhead anyway.

Right but that's up to the user. You aren't disabling GRO
on source, you are just splitting it up.

> Anyway I can try to add the support for this.

Which reminds me: on modern devices there are commands to control
offloads, so for these, we should support turning offloads on/off using
ethtool.

-- 
MST
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Jason Wang]

On 10/27/2015 05:05 PM, Michael S. Tsirkin wrote:
> On Tue, Oct 27, 2015 at 10:58:25AM +0800, Jason Wang wrote:
>>
>> On 10/26/2015 04:30 PM, Michael S. Tsirkin wrote:
>>> On Mon, Oct 26, 2015 at 02:53:38PM +0800, Jason Wang wrote:
 On 10/26/2015 02:09 PM, Michael S. Tsirkin wrote:
> On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
>> On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
>>> On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
 We don't have fraglist support in TAP_FEATURES. This will lead
 software segmentation of gro skb with frag list. Fixes by having
 frag list support in TAP_FEATURES.

 With this patch single session of netperf receiving were restored from
 about 5Gb/s to about 12Gb/s on mlx4.

 Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
 Cc: Vlad Yasevich 
 Cc: Michael S. Tsirkin 
 Signed-off-by: Jason Wang 
>>> Thanks!
>>> Does this mean we should look at re-adding NETIF_F_FRAGLIST
>>> to virtio-net as well?
>> Not sure I get the point, but probably not. This is for receiving and
>> skb_copy_datagram_iter() can deal with frag list.
> Point is:
> - bridge within guest
> - assigned device creating gro skbs with frag list bridged to virtio
 I see, but this problem looks not specific to virtio. Most cards does
 not support frag list.
>>> These will be slower when used with a bridge then, won't they?
>> For forwarding, not sure. GRO has latency and cpu overhead anyway.
> Right but that's up to the user. You aren't disabling GRO
> on source, you are just splitting it up.
>
>> Anyway I can try to add the support for this.
> Which reminds me: on modern devices there are commands to control
> offloads, so for these, we should support turning offloads on/off using
> ethtool.
>

Trying to implement frag list but see a problem. Looks like driver need
to scan the possible number of io vectors? (Since vhost support max to
UIO_MAXIOV number of io vectors). Looks like there's no clarification on
this in the spec. (Which only limit the length of descriptor chain to
Queue size).
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Michael S. Tsirkin]

On Mon, Oct 26, 2015 at 02:53:38PM +0800, Jason Wang wrote:
> 
> 
> On 10/26/2015 02:09 PM, Michael S. Tsirkin wrote:
> > On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
> >>
> >> On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
> >>> On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
>  We don't have fraglist support in TAP_FEATURES. This will lead
>  software segmentation of gro skb with frag list. Fixes by having
>  frag list support in TAP_FEATURES.
> 
>  With this patch single session of netperf receiving were restored from
>  about 5Gb/s to about 12Gb/s on mlx4.
> 
>  Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
>  Cc: Vlad Yasevich 
>  Cc: Michael S. Tsirkin 
>  Signed-off-by: Jason Wang 
> >>> Thanks!
> >>> Does this mean we should look at re-adding NETIF_F_FRAGLIST
> >>> to virtio-net as well?
> >> Not sure I get the point, but probably not. This is for receiving and
> >> skb_copy_datagram_iter() can deal with frag list.
> >
> > Point is:
> > - bridge within guest
> > - assigned device creating gro skbs with frag list bridged to virtio
> 
> I see, but this problem looks not specific to virtio. Most cards does
> not support frag list.

These will be slower when used with a bridge then, won't they?

-- 
MST
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Michael S. Tsirkin]

On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
> 
> 
> On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
> > On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
> >> We don't have fraglist support in TAP_FEATURES. This will lead
> >> software segmentation of gro skb with frag list. Fixes by having
> >> frag list support in TAP_FEATURES.
> >>
> >> With this patch single session of netperf receiving were restored from
> >> about 5Gb/s to about 12Gb/s on mlx4.
> >>
> >> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
> >> Cc: Vlad Yasevich 
> >> Cc: Michael S. Tsirkin 
> >> Signed-off-by: Jason Wang 
> > Thanks!
> > Does this mean we should look at re-adding NETIF_F_FRAGLIST
> > to virtio-net as well?
> 
> Not sure I get the point, but probably not. This is for receiving and
> skb_copy_datagram_iter() can deal with frag list.


Point is:
- bridge within guest
- assigned device creating gro skbs with frag list bridged to virtio

> >
> >> ---
> >>  drivers/net/macvtap.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/net/macvtap.c b/drivers/net/macvtap.c
> >> index 248478c..197c939 100644
> >> --- a/drivers/net/macvtap.c
> >> +++ b/drivers/net/macvtap.c
> >> @@ -137,7 +137,7 @@ static const struct proto_ops macvtap_socket_ops;
> >>  #define TUN_OFFLOADS (NETIF_F_HW_CSUM | NETIF_F_TSO_ECN | NETIF_F_TSO | \
> >>  NETIF_F_TSO6 | NETIF_F_UFO)
> >>  #define RX_OFFLOADS (NETIF_F_GRO | NETIF_F_LRO)
> >> -#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG)
> >> +#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG | NETIF_F_FRAGLIST)
> >>  
> >>  static struct macvlan_dev *macvtap_get_vlan_rcu(const struct net_device 
> >> *dev)
> >>  {
> >> -- 
> >> 1.8.3.1
> > --
> > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > the body of a message to majord...@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Jason Wang]

On 10/26/2015 02:09 PM, Michael S. Tsirkin wrote:
> On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
>>
>> On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
>>> On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
 We don't have fraglist support in TAP_FEATURES. This will lead
 software segmentation of gro skb with frag list. Fixes by having
 frag list support in TAP_FEATURES.

 With this patch single session of netperf receiving were restored from
 about 5Gb/s to about 12Gb/s on mlx4.

 Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
 Cc: Vlad Yasevich 
 Cc: Michael S. Tsirkin 
 Signed-off-by: Jason Wang 
>>> Thanks!
>>> Does this mean we should look at re-adding NETIF_F_FRAGLIST
>>> to virtio-net as well?
>> Not sure I get the point, but probably not. This is for receiving and
>> skb_copy_datagram_iter() can deal with frag list.
>
> Point is:
> - bridge within guest
> - assigned device creating gro skbs with frag list bridged to virtio

I see, but this problem looks not specific to virtio. Most cards does
not support frag list.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Jason Wang]

On 10/26/2015 04:30 PM, Michael S. Tsirkin wrote:
> On Mon, Oct 26, 2015 at 02:53:38PM +0800, Jason Wang wrote:
>>
>> On 10/26/2015 02:09 PM, Michael S. Tsirkin wrote:
>>> On Mon, Oct 26, 2015 at 11:15:57AM +0800, Jason Wang wrote:
 On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
> On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
>> We don't have fraglist support in TAP_FEATURES. This will lead
>> software segmentation of gro skb with frag list. Fixes by having
>> frag list support in TAP_FEATURES.
>>
>> With this patch single session of netperf receiving were restored from
>> about 5Gb/s to about 12Gb/s on mlx4.
>>
>> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
>> Cc: Vlad Yasevich 
>> Cc: Michael S. Tsirkin 
>> Signed-off-by: Jason Wang 
> Thanks!
> Does this mean we should look at re-adding NETIF_F_FRAGLIST
> to virtio-net as well?
 Not sure I get the point, but probably not. This is for receiving and
 skb_copy_datagram_iter() can deal with frag list.
>>> Point is:
>>> - bridge within guest
>>> - assigned device creating gro skbs with frag list bridged to virtio
>> I see, but this problem looks not specific to virtio. Most cards does
>> not support frag list.
> These will be slower when used with a bridge then, won't they?

For forwarding, not sure. GRO has latency and cpu overhead anyway.

Anyway I can try to add the support for this.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Jason Wang]

On 10/23/2015 09:37 PM, Michael S. Tsirkin wrote:
> On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
>> We don't have fraglist support in TAP_FEATURES. This will lead
>> software segmentation of gro skb with frag list. Fixes by having
>> frag list support in TAP_FEATURES.
>>
>> With this patch single session of netperf receiving were restored from
>> about 5Gb/s to about 12Gb/s on mlx4.
>>
>> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
>> Cc: Vlad Yasevich 
>> Cc: Michael S. Tsirkin 
>> Signed-off-by: Jason Wang 
> Thanks!
> Does this mean we should look at re-adding NETIF_F_FRAGLIST
> to virtio-net as well?

Not sure I get the point, but probably not. This is for receiving and
skb_copy_datagram_iter() can deal with frag list.

>
>> ---
>>  drivers/net/macvtap.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/macvtap.c b/drivers/net/macvtap.c
>> index 248478c..197c939 100644
>> --- a/drivers/net/macvtap.c
>> +++ b/drivers/net/macvtap.c
>> @@ -137,7 +137,7 @@ static const struct proto_ops macvtap_socket_ops;
>>  #define TUN_OFFLOADS (NETIF_F_HW_CSUM | NETIF_F_TSO_ECN | NETIF_F_TSO | \
>>NETIF_F_TSO6 | NETIF_F_UFO)
>>  #define RX_OFFLOADS (NETIF_F_GRO | NETIF_F_LRO)
>> -#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG)
>> +#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG | NETIF_F_FRAGLIST)
>>  
>>  static struct macvlan_dev *macvtap_get_vlan_rcu(const struct net_device 
>> *dev)
>>  {
>> -- 
>> 1.8.3.1
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[David Miller]

From: Jason Wang 
Date: Fri, 23 Oct 2015 00:57:05 -0400

> We don't have fraglist support in TAP_FEATURES. This will lead
> software segmentation of gro skb with frag list. Fixes by having
> frag list support in TAP_FEATURES.
> 
> With this patch single session of netperf receiving were restored from
> about 5Gb/s to about 12Gb/s on mlx4.
> 
> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
> Cc: Vlad Yasevich 
> Cc: Michael S. Tsirkin 
> Signed-off-by: Jason Wang 

Applied and queued up for -stable, thanks Jason.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH net] macvtap: unbreak receiving of gro skb with frag list

[Michael S. Tsirkin]

On Fri, Oct 23, 2015 at 12:57:05AM -0400, Jason Wang wrote:
> We don't have fraglist support in TAP_FEATURES. This will lead
> software segmentation of gro skb with frag list. Fixes by having
> frag list support in TAP_FEATURES.
> 
> With this patch single session of netperf receiving were restored from
> about 5Gb/s to about 12Gb/s on mlx4.
> 
> Fixes a567dd6252 ("macvtap: simplify usage of tap_features")
> Cc: Vlad Yasevich 
> Cc: Michael S. Tsirkin 
> Signed-off-by: Jason Wang 

Thanks!
Does this mean we should look at re-adding NETIF_F_FRAGLIST
to virtio-net as well?

> ---
>  drivers/net/macvtap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/macvtap.c b/drivers/net/macvtap.c
> index 248478c..197c939 100644
> --- a/drivers/net/macvtap.c
> +++ b/drivers/net/macvtap.c
> @@ -137,7 +137,7 @@ static const struct proto_ops macvtap_socket_ops;
>  #define TUN_OFFLOADS (NETIF_F_HW_CSUM | NETIF_F_TSO_ECN | NETIF_F_TSO | \
> NETIF_F_TSO6 | NETIF_F_UFO)
>  #define RX_OFFLOADS (NETIF_F_GRO | NETIF_F_LRO)
> -#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG)
> +#define TAP_FEATURES (NETIF_F_GSO | NETIF_F_SG | NETIF_F_FRAGLIST)
>  
>  static struct macvlan_dev *macvtap_get_vlan_rcu(const struct net_device *dev)
>  {
> -- 
> 1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html