On 2016年11月08日 19:17, Jason Wang wrote:
>On 2016年11月08日 19:13, Jason Wang wrote:
>> Cc Michael
>>
>> On 2016年11月08日 16:34, Zhangming (James, Euler) wrote:
>>>
>>> In container scenario, OVS is installed in the Virtual machine, and
>>> all the containers connected to the OVS will communicated through
>>> VXLAN encapsulation.
>>>
>>> By now, virtio_net does not support TSO offload for VXLAN
>>> encapsulated TSO package. In this condition, the performance is not
>>> good, sender is bottleneck
>>>
>>> I googled this scenario, but I didn’t find any information. Will
>>> virtio_net support VXLAN encapsulation package TSO offload later?
>>>
>>
> >Yes and for both sender and receiver.
>>
>>> My idea is virtio_net open encapsulated TSO offload, and transport
>>> encapsulation info to TUN, TUN will parse the info and build skb with
>>> encapsulation info.
>>>
>>> OVS or kernel on the host should be modified to support this. Using
>>> this method, the TCP performance aremore than 2x as before.
>>>
>>> Any advice and suggestions for this idea or new idea will be greatly
>>> appreciated!
>>>
>>> Best regards,
>>>
>>> James zhang
>>>
>>
>> Sounds very good. And we may also need features bits
>> (VIRTIO_NET_F_GUEST|HOST_GSO_X) for this.
>>
>> This is in fact one of items in networking todo list. (See
>> http://www.linux-kvm.org/page/NetworkingTodo). While at it, we'd
>> better support not only VXLAN but also other tunnels.
>
>Cc Vlad who is working on extending virtio-net headers.
>
>>
>> We can start with the spec work, or if you've already had some bits
>> you can post them as RFC for early review.
>>
>> Thanks
Below is my demo code
Virtio_net.c
static int virtnet_probe(struct virtio_device *vdev), add belows codes:
if (virtio_has_feature(vdev, VIRTIO_NET_F_MRG_RXBUF) ||
// avoid gso segment, it should be negotiation later, because in the
demo I reuse num_buffers.
virtio_has_feature(vdev, VIRTIO_F_VERSION_1)) {
dev->hw_enc_features |= NETIF_F_TSO;
dev->hw_enc_features |= NETIF_F_ALL_CSUM;
dev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL;
dev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
dev->hw_enc_features |= NETIF_F_GSO_TUNNEL_REMCSUM;
dev->features |= NETIF_F_GSO_UDP_TUNNEL;
dev->features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
dev->features |= NETIF_F_GSO_TUNNEL_REMCSUM;
}
static int xmit_skb(struct send_queue *sq, struct sk_buff *skb), add below to
pieces of codes
if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL)
hdr->hdr.gso_type |= VIRTIO_NET_HDR_GSO_TUNNEL;
if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL_CSUM)
hdr->hdr.gso_type |= VIRTIO_NET_HDR_GSO_TUNNEL_CSUM;
if (skb_shinfo(skb)->gso_type & SKB_GSO_TUNNEL_REMCSUM)
hdr->hdr.gso_type |= VIRTIO_NET_HDR_GSO_TUNNEL_REMCSUM;
if (skb->encapsulation && skb_is_gso(skb)) {
inner_mac_len = skb_inner_network_header(skb) -
skb_inner_mac_header(skb);
tnl_len = skb_inner_mac_header(skb) - skb_mac_header(skb);
if ( !(inner_mac_len >> DATA_LEN_SHIFT) && !(tnl_len >>
DATA_LEN_SHIFT) ) {
hdr->hdr.flags |= VIRTIO_NET_HDR_F_ENCAPSULATION;
hdr->num_buffers = (__virtio16)((inner_mac_len <<
DATA_LEN_SHIFT) | tnl_len); //we reuse num_buffers for simple , we
should add extend member for later.
} else
hdr->num_buffers = 0;
}
Tun.c
if (memcpy_fromiovecend((void *)&hdr, iv, offset,
tun->vnet_hdr_sz)) //read header with negotiation length
return -EFAULT;
if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL)
//set tunnel gso info
skb_shinfo(skb)->gso_type |= SKB_GSO_UDP_TUNNEL;
if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL_CSUM)
skb_shinfo(skb)->gso_type |= SKB_GSO_UDP_TUNNEL_CSUM;
if (hdr.gso_type & VIRTIO_NET_HDR_GSO_TUNNEL_REMCSUM)
skb_shinfo(skb)->gso_type |= SKB_GSO_TUNNEL_REMCSUM;
if (hdr.flags & VIRTIO_NET_HDR_F_ENCAPSULATION) {
//read tunnel info from header and set to built skb.
tnl_len = tun16_to_cpu(tun, hdr.num_buffers) & TUN_TNL_LEN_MASK;
payload_mac_len = tun16_to_cpu(tun, hdr.num_buffers) >>
TUN_DATA_LEN_SHIFT;
mac_len = skb_network_header(skb) - skb_mac_header(skb);
skb_set_inner_mac_header(skb, tnl_len - mac_len);
skb_set_inner_network_header(skb, tnl_len + payload_mac_len -
mac_len);
skb->encapsulation = 1;
}
