On 2018/11/15 17:21, Jason Wang wrote: > > On 2018/11/15 下午5:02, jiangyiwen wrote: >> On 2018/11/15 16:19, Jason Wang wrote: >>> On 2018/11/15 下午2:46, jiangyiwen wrote: >>>> On 2018/11/15 12:19, Jason Wang wrote: >>>>> On 2018/11/15 上午11:56, jiangyiwen wrote: >>>>>> Hi Stefan, Michael, Jason and everyone, >>>>>> >>>>>> Several days ago, I discussed with jason about "Vsock over Virtio-net". >>>>>> This idea has two advantages: >>>>>> First, it can use many great features of virtio-net, like batching, >>>>>> mergeable rx buffer and multiqueue, etc. >>>>>> Second, it can reduce many duplicate codes and make it easy to be >>>>>> maintained. >>>>>> >>>>>> Before the implement, I want to discuss with everyone again, and >>>>>> want to know everyone's suggestions. >>>>>> >>>>>> After the discussion, based on this point I will try to implement >>>>>> this idea, but I am not familiar with the virtio-net, that is a >>>>>> pity. :( >>>>> I think we should have a new feature flag for this. E.g >>>>> VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't >>>>> support this to avoid confusion. When this feature is negotiated, we will >>>>> use it only for VOSCK transport. This can simplify things somehow. >>>>> >>>>> >>>>>> -------------------------Simple idea------------------------------ >>>>>> >>>>>> 1. The packet layout will become as follows: >>>>>> >>>>>> +---------------------------------+ >>>>>> | Virtio-net header | >>>>>> |(struct virtio_net_hdr_mrg_rxbuf)| >>>>>> +---------------------------------+ >>>>>> | Vsock header | >>>>>> | (struct virtio_vsock_hdr) | >>>>>> +---------------------------------+ >>>>>> | payload | >>>>>> | (until end of packet) | >>>>>> +---------------------------------+ >>>>>> >>>>>> 2. The Guest->Host basic code flow as follow: >>>>>> +------------+ >>>>>> | Client | >>>>>> +------------+ >>>>>> | >>>>>> | >>>>>> +------------------------------------------------------------------+ >>>>>> |VSOCK Core Module | >>>>>> |ops->sendmsg; (vsock_stream_sendmsg) | >>>>>> | -> alloc_skb; /* it will packet a skb buffer, and include vsock | >>>>>> | * hdr and payload */ | >>>>>> | -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */| >>>>>> |vsock hdr and payload, and then call | >>>>>> +------------------------------------------------------------------+ >>>>> Note, if we've negotiated the feature, virtio-net driver must not use >>>>> register_netdev to register it to network core. This can avoid lots of >>>>> confusion. >>>> Hi Jason, >>>> >>>> You mean we should not register netdev if use vsock, and in order to >>>> avoid confusion, then I think whether we should keep vsock and export >>>> some virtio-net's functions that can be shared. In this way, first, vsock >>>> may keep existing architecture and will not affect virtio-net. >>> >>> At least it needs new feature bits which will be a functional duplication >>> of virtio-net (e.g mrg_rxbuf). >> Hi Jason, >> >> Actually I mean only use some shared function to make vsock support these >> features, in that way, guest see the device is still vsock device instead of >> virtio-net device, in addition, it can have less codes and easier to be >> compatible with old vsock version. > > > Yes, I think we're talking about same thing. Both of us want to share codes. > What you want is to export and share some common helpers between virtio-net > and vsock. What I meant is to e.g probe vsock device and merge vsock specific > codes into virtio-net driver. I agree it's not a small project. We can start > from e.g patches that try to share the codes. This could also give us > inspiration for how to unify them. > >
Then we have two ways to implement it. 1. For Guest, it is a virtio-net device(VIRTIO_ID_NET), use a feature bit(e.g VIRTIO_NET_F_VSOCK) to distinguish vsock special device and other virtio-net device. For old vsock device, it still use old driver to drive it. 2. For Guest, it is a virtio-vsock device(VIRTIO_ID_VSOCK), use virtio device id to distinguish them, it will integrate old driver, but it may be more complicated, because it needs to consider the compatibility with old vsock device. >> >> Thanks, >> Yiwen. >> >>> >>>> In addition, >>>> vsock doesn't need to use virtio_net header too, then it don't need to pack >>>> skb structure. >>> >>> For mergeable rx buffer it need I think? >> As said above, I will define the related structure in the virtio-vsock >> module. >> >>> >>>> Thanks, >>>> Yiwen. >>>> >>>>>> | >>>>>> | >>>>>> +------------------------------------------------------------------+ >>>>>> |Virtio-net Module | >>>>>> |start_xmit | >>>>>> | -> add virtio_net_hdr and pack sg in ring desc, notify Host | >>>>>> +------------------------------------------------------------------+ >>>>>> | >>>>>> | >>>>>> +------------------------------------------------------------------+ >>>>>> |Vhost-net Module | >>>>>> |handle_tx | >>>>>> | -> get tx buffer, skip virtio_net_hdr and call Vsock function. | >>>>>> | /* This point has some differences, vhost-net use ->sendmsg to | >>>>>> | * forward information, however vsock only need to notify server | >>>>>> | * that data ready. */ | >>>>>> +------------------------------------------------------------------+ >>>>> When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock >>>>> transport, we can then forward it to vsock core. >>>>> >>>>> >>>>>> | >>>>>> | >>>>>> +------------------------------------------------------------------+ >>>>>> |VSOCK Core Module | >>>>>> |alloc_pkt, copy skb data to pkt. | >>>>>> |add pkt to rx_queue and notify server to get data. | >>>>>> +------------------------------------------------------------------+ >>>>>> >>>>>> 3. To Host->Guest >>>>>> I have a problem and difficult, mainly I know about virtio-net a little), >>>>>> because I have been doing work related with storage and file system. >>>>>> >>>>>> The problem as follows: >>>>>> we should monitor all of socket of vsock in handle_rx, when there are >>>>>> data coming, and copy data to vq desc. Vhost-net use ->recvmsg to >>>>>> get data, it is different with socket. To vsock, I think host will >>>>>> not call ->recvmsg when it need to send message to guest. To net, >>>>>> vhost-net only as forwarding layer. >>>>> Know not much here, but is it possible to have a vsock(tap) to be passed >>>>> to vhost_net and let vhost call its recvmgs()? Bascially it was a socket >>>>> on host as well I believe? >>>> For vsock, Host->Guest, it's code flow as follows: >>>> Server call send() >>>> -> sendmsg(); (vsock_stream_sendmsg) >>>> -> virtio_trasnport_send_pkt_info >>>> -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker >>>> >>>> Vhost_worker >>>> -> vhost_transport_send_pkt_work >>>> -> get pkt from send_pkt_list >>>> -> get vq input desc and then fill data to desc addr >>>> -> update used ring and then signal guest >>>> >>>> In the whole process, host don't call recvmsg() because it is a net >>>> device, and >>>> it also receives any messages. >>> >>> If I understand this correctly, there's no a socket object on host that >>> represent vosck in guest? If yes, maybe we can invent one. >>> >>> Thanks >>> >> Sorry, I am not understanding you very much, vsock only a socket >> channel, it does not have a network device entity, so it only >> transmit the data between server and client, the data is only >> saved in server and client. Instead of vhost-net, I feel it >> has a network device that can saved the data, so when host >> send message to guest, it can use recvmsg() from the >> network device(tap). For Vsock, recvmsg() interface will >> read message from tx vq. > > > So I understand the model is not a real socket pair on host which AF_UNIX > did. Maybe it's better to have one. What I meant is, have a socket that > represent for each guest vsock device on host (guest socket). Then when you > transfer packets from host to guest, you can queue the packets into the > receive queue of the guest socket and wake up vhost-net and it will call > recvmsg() for the guest socket. And when you want to transfer packets form > guest to host, vhost_net will call sendmsg() to the guest socket on host then > it can search the correct destination and queue packet on the receive of host > socket. This can make vsock much more easier to be integrated with vhost_net. > > Thanks > >> >>>> For vhost-net, I understand it is a tap device, so it can receive messages >>>> from other net device. >>>> >>>> This is my understanding, it may have some errors. >>>> >>>> Thanks. >>>> >>>>> If this doesn't work, we can have vsock specific receiving routine in >>>>> vhost_net if VIRTIO_NET_F_VOSCK is negotiated. >>>>> >>>>> Generally, I think we should try out best to keep the exist >>>>> sendmsg()/recvmsg() interfaces and only consider the alternatives if we >>>>> meet some real blocker. >>>>> >>>>> Thanks >>>>> >>>>> >>>>> . >>>>> >>> . >>> >> > > . > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
