On 2018/11/15 14:46, jiangyiwen wrote: > On 2018/11/15 12:19, Jason Wang wrote: >> >> On 2018/11/15 上午11:56, jiangyiwen wrote: >>> Hi Stefan, Michael, Jason and everyone, >>> >>> Several days ago, I discussed with jason about "Vsock over Virtio-net". >>> This idea has two advantages: >>> First, it can use many great features of virtio-net, like batching, >>> mergeable rx buffer and multiqueue, etc. >>> Second, it can reduce many duplicate codes and make it easy to be >>> maintained. >>> >>> Before the implement, I want to discuss with everyone again, and >>> want to know everyone's suggestions. >>> >>> After the discussion, based on this point I will try to implement >>> this idea, but I am not familiar with the virtio-net, that is a >>> pity. :( >> >> >> I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. >> And host should fail the negotiation if guest doesn't support this to avoid >> confusion. When this feature is negotiated, we will use it only for VOSCK >> transport. This can simplify things somehow. >> >> >>> -------------------------Simple idea------------------------------ >>> >>> 1. The packet layout will become as follows: >>> >>> +---------------------------------+ >>> | Virtio-net header | >>> |(struct virtio_net_hdr_mrg_rxbuf)| >>> +---------------------------------+ >>> | Vsock header | >>> | (struct virtio_vsock_hdr) | >>> +---------------------------------+ >>> | payload | >>> | (until end of packet) | >>> +---------------------------------+ >>> >>> 2. The Guest->Host basic code flow as follow: >>> +------------+ >>> | Client | >>> +------------+ >>> | >>> | >>> +------------------------------------------------------------------+ >>> |VSOCK Core Module | >>> |ops->sendmsg; (vsock_stream_sendmsg) | >>> | -> alloc_skb; /* it will packet a skb buffer, and include vsock | >>> | * hdr and payload */ | >>> | -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */| >>> |vsock hdr and payload, and then call | >>> +------------------------------------------------------------------+ >> >> >> Note, if we've negotiated the feature, virtio-net driver must not use >> register_netdev to register it to network core. This can avoid lots of >> confusion. > > Hi Jason, > > You mean we should not register netdev if use vsock, and in order to > avoid confusion, then I think whether we should keep vsock and export > some virtio-net's functions that can be shared. In this way, first, vsock > may keep existing architecture and will not affect virtio-net. In addition, > vsock doesn't need to use virtio_net header too, then it don't need to pack > skb structure. > > Thanks, > Yiwen. > >> >> >>> | >>> | >>> +------------------------------------------------------------------+ >>> |Virtio-net Module | >>> |start_xmit | >>> | -> add virtio_net_hdr and pack sg in ring desc, notify Host | >>> +------------------------------------------------------------------+ >>> | >>> | >>> +------------------------------------------------------------------+ >>> |Vhost-net Module | >>> |handle_tx | >>> | -> get tx buffer, skip virtio_net_hdr and call Vsock function. | >>> | /* This point has some differences, vhost-net use ->sendmsg to | >>> | * forward information, however vsock only need to notify server | >>> | * that data ready. */ | >>> +------------------------------------------------------------------+ >> >> >> When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, >> we can then forward it to vsock core. >> >> >>> | >>> | >>> +------------------------------------------------------------------+ >>> |VSOCK Core Module | >>> |alloc_pkt, copy skb data to pkt. | >>> |add pkt to rx_queue and notify server to get data. | >>> +------------------------------------------------------------------+ >>> >>> 3. To Host->Guest >>> I have a problem and difficult, mainly I know about virtio-net a little), >>> because I have been doing work related with storage and file system. >>> >>> The problem as follows: >>> we should monitor all of socket of vsock in handle_rx, when there are >>> data coming, and copy data to vq desc. Vhost-net use ->recvmsg to >>> get data, it is different with socket. To vsock, I think host will >>> not call ->recvmsg when it need to send message to guest. To net, >>> vhost-net only as forwarding layer. >> >> Know not much here, but is it possible to have a vsock(tap) to be passed to >> vhost_net and let vhost call its recvmgs()? Bascially it was a socket on >> host as well I believe? > > For vsock, Host->Guest, it's code flow as follows: > Server call send() > -> sendmsg(); (vsock_stream_sendmsg) > -> virtio_trasnport_send_pkt_info > -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker > > Vhost_worker > -> vhost_transport_send_pkt_work > -> get pkt from send_pkt_list > -> get vq input desc and then fill data to desc addr > -> update used ring and then signal guest > > In the whole process, host don't call recvmsg() because it is a net device, > and > it also receives any messages.
Sorry, it is *not* a net device, it *does not* receive any messages. Thanks. > > For vhost-net, I understand it is a tap device, so it can receive messages > from other net device. > > This is my understanding, it may have some errors. > > Thanks. > >> >> If this doesn't work, we can have vsock specific receiving routine in >> vhost_net if VIRTIO_NET_F_VOSCK is negotiated. >> >> Generally, I think we should try out best to keep the exist >> sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet >> some real blocker. >> >> Thanks >> >> >>> >> >> . >> > _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
