Re: [Qemu-devel] "socket" host network backend: suggested improvements and fixes
On 2018年06月01日 20:30, Stefan Hajnoczi wrote: On Wed, May 30, 2018 at 04:34:03PM +0600, Artem Pisarenko wrote: Hi to all. Hi Artem, I have CCed Jason Wang, the QEMU networking maintainer, so your email gets attention. QEMU is a high-traffic mailing list and you may not get responses unless you CC relevant people (use scripts/get_maintainer.pl -f path/to/source.c to look up maintainers). I'm working on integrating QEMU networking to simulation environment and found socket backend very convenient: it's simple, easy to use (i.e no intermediate things required, such as tap/tun adapter, vde switch, etc.) and transparent to host environment (i.e. it doesn't pollutes system with bunch of virtual devices, etc.). Although, it have some problems, closely related to each other. I've investigated source code and played with it a little, but I'm not ready submit a complete patch. So, here are my thoughts... You're very welcome to send patches. If you think it's not ready to be merged, you can send RFC for early discussion. 1. Internal protocol (only qemu instances can join its network devices between). You mean the simple protocol on top of TCP? We probably need at least a version for socket backend. (We change the protocol recently for vnet header support). For UDP, I believe external program could join? I suggest to make it available to plug with external software, i.e freeze communication protocol at current state and document it in docs/interop/ directory. Any pointer to freeze communication protocol? 2. Transport options wrongly documented. Section "2.3.6 Network options" lists "-netdev socket,..." entries. It gives very different basic understanding of how it works from actual one. 2.1. It has two entries: listen/connect (TCP connecton) and mcast (multicast UDP), but 'qemu --help' outputs additional one - udp (UDP tunnel), which is undocumented, but looks like working. Yes. 2.2. Each entry has fd=h parameter, which looks like it's an optional parameter, but code analysis (net/socket.c) shows that, in fact, it's a separate transport option exclusive to listed ones. It used as follows: user creates/opens whatever (almost) custom socket/file it wants, connects it with other endpoint and passes file descriptor (handle) to qemu, which just recv/send over it and nothing more. Right. 2.3. As a consequence, if you try to invoke any transport/variant option with "fd=", you'll get an error: "exactly one of listen=, connect=, mcast= or udp= is required". And again, error message is incomplete - it misses "fd=" option. This needs to be fixed. 3. "fd=" transport doesn't work with SOCK_DGRAM type sockets. It's due to an implementation error in net/socket.c. Function net_socket_receive_dgram() uses sendo() call with s->dgram_dst value which is undefined for this case (and, of course, cannot be defined). I think maybe we need a new qmp method for setting the destination for socket backend. Although net_socket_fd_init() execution branch is smart enough to detect type of socket passed with "fd=", but its "connected" state forgotten afterwards. Suggested fix: replace sendto() with send(), which correctly assumes already connected socket, and add corresponding connect() calls for "mcast=" and "udp=" init sequences. Appreciated if you could send the aboves fixes. Some unit tests are even better. (For those, who interested, currently I've got working network communication with unmodified qemu 2.12.0 in Linux using UNIX domain sockets created by socketpair(AF_LOCAL, SOCK_STREAM, ...), one of which passed to spawned child qemu process via -netdev socket,fd=... and other one, used in parent application process to send/receive packets. Protocol, used by qemu, is simple and implements only data plane: it just transfers raw ethernet frames in binary form, for datagram-type sockets it's straightforward, and for stream-type sockets each frame prepended with uint32 length in network byte order, without any delimiters and escaping.) -- С уважением, Артем Писаренко Thanks
Re: [Qemu-devel] "socket" host network backend: suggested improvements and fixes
On Wed, May 30, 2018 at 04:34:03PM +0600, Artem Pisarenko wrote: > Hi to all. Hi Artem, I have CCed Jason Wang, the QEMU networking maintainer, so your email gets attention. QEMU is a high-traffic mailing list and you may not get responses unless you CC relevant people (use scripts/get_maintainer.pl -f path/to/source.c to look up maintainers). > I'm working on integrating QEMU networking to simulation environment and > found socket backend very convenient: it's simple, easy to use (i.e no > intermediate things required, such as tap/tun adapter, vde switch, etc.) > and transparent to host environment (i.e. it doesn't pollutes system with > bunch of virtual devices, etc.). > > Although, it have some problems, closely related to each other. I've > investigated source code and played with it a little, but I'm not ready > submit a complete patch. So, here are my thoughts... > > 1. Internal protocol (only qemu instances can join its network devices > between). I suggest to make it available to plug with external software, > i.e freeze communication protocol at current state and document it in > docs/interop/ directory. > > 2. Transport options wrongly documented. Section "2.3.6 Network options" > lists "-netdev socket,..." entries. It gives very different basic > understanding of how it works from actual one. > 2.1. It has two entries: listen/connect (TCP connecton) and mcast > (multicast UDP), but 'qemu --help' outputs additional one - udp (UDP > tunnel), which is undocumented, but looks like working. > 2.2. Each entry has fd=h parameter, which looks like it's an optional > parameter, but code analysis (net/socket.c) shows that, in fact, it's a > separate transport option exclusive to listed ones. It used as follows: > user creates/opens whatever (almost) custom socket/file it wants, connects > it with other endpoint and passes file descriptor (handle) to qemu, which > just recv/send over it and nothing more. > 2.3. As a consequence, if you try to invoke any transport/variant option > with "fd=", you'll get an error: "exactly one of listen=, connect=, mcast= > or udp= is required". And again, error message is incomplete - it misses > "fd=" option. > > 3. "fd=" transport doesn't work with SOCK_DGRAM type sockets. It's due to > an implementation error in net/socket.c. Function > net_socket_receive_dgram() uses sendo() call with s->dgram_dst value which > is undefined for this case (and, of course, cannot be defined). > Although net_socket_fd_init() execution branch is smart enough to detect > type of socket passed with "fd=", but its "connected" state forgotten > afterwards. Suggested fix: replace sendto() with send(), which correctly > assumes already connected socket, and add corresponding connect() calls for > "mcast=" and "udp=" init sequences. > > (For those, who interested, currently I've got working network > communication with unmodified qemu 2.12.0 in Linux using UNIX domain > sockets created by socketpair(AF_LOCAL, SOCK_STREAM, ...), one of which > passed to spawned child qemu process via -netdev socket,fd=... and other > one, used in parent application process to send/receive packets. Protocol, > used by qemu, is simple and implements only data plane: it just transfers > raw ethernet frames in binary form, for datagram-type sockets it's > straightforward, and for stream-type sockets each frame prepended with > uint32 length in network byte order, without any delimiters and escaping.) > -- > > С уважением, > Артем Писаренко signature.asc Description: PGP signature
[Qemu-devel] "socket" host network backend: suggested improvements and fixes
Hi to all. I'm working on integrating QEMU networking to simulation environment and found socket backend very convenient: it's simple, easy to use (i.e no intermediate things required, such as tap/tun adapter, vde switch, etc.) and transparent to host environment (i.e. it doesn't pollutes system with bunch of virtual devices, etc.). Although, it have some problems, closely related to each other. I've investigated source code and played with it a little, but I'm not ready submit a complete patch. So, here are my thoughts... 1. Internal protocol (only qemu instances can join its network devices between). I suggest to make it available to plug with external software, i.e freeze communication protocol at current state and document it in docs/interop/ directory. 2. Transport options wrongly documented. Section "2.3.6 Network options" lists "-netdev socket,..." entries. It gives very different basic understanding of how it works from actual one. 2.1. It has two entries: listen/connect (TCP connecton) and mcast (multicast UDP), but 'qemu --help' outputs additional one - udp (UDP tunnel), which is undocumented, but looks like working. 2.2. Each entry has fd=h parameter, which looks like it's an optional parameter, but code analysis (net/socket.c) shows that, in fact, it's a separate transport option exclusive to listed ones. It used as follows: user creates/opens whatever (almost) custom socket/file it wants, connects it with other endpoint and passes file descriptor (handle) to qemu, which just recv/send over it and nothing more. 2.3. As a consequence, if you try to invoke any transport/variant option with "fd=", you'll get an error: "exactly one of listen=, connect=, mcast= or udp= is required". And again, error message is incomplete - it misses "fd=" option. 3. "fd=" transport doesn't work with SOCK_DGRAM type sockets. It's due to an implementation error in net/socket.c. Function net_socket_receive_dgram() uses sendo() call with s->dgram_dst value which is undefined for this case (and, of course, cannot be defined). Although net_socket_fd_init() execution branch is smart enough to detect type of socket passed with "fd=", but its "connected" state forgotten afterwards. Suggested fix: replace sendto() with send(), which correctly assumes already connected socket, and add corresponding connect() calls for "mcast=" and "udp=" init sequences. (For those, who interested, currently I've got working network communication with unmodified qemu 2.12.0 in Linux using UNIX domain sockets created by socketpair(AF_LOCAL, SOCK_STREAM, ...), one of which passed to spawned child qemu process via -netdev socket,fd=... and other one, used in parent application process to send/receive packets. Protocol, used by qemu, is simple and implements only data plane: it just transfers raw ethernet frames in binary form, for datagram-type sockets it's straightforward, and for stream-type sockets each frame prepended with uint32 length in network byte order, without any delimiters and escaping.) -- С уважением, Артем Писаренко
