-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Last year, I posted a question on implementation strategy of HTTP-over-UDP(multicast) module to modules-dev list. I was able to implement fully functional "UDP echo" service, but for HTTP/U module, it suffered severe limitation due to how core_filter handles output.
- - http://www.mail-archive.com/[email protected]/msg00129.html Here were my findings at the time (with 2.2.4-dev): - - For initial UDP socket setup, current httpd implementation (listen.c) is OK, as I was able to bypass it by inserting custom ap_listen_rec (with custom accept_func function) into ap_listeners at open_log stage. - - Another issue was ap_lingering_close called after request processing, but that too, can be worked around by doing ap_set_module_config(c->conn_config, &core_module, NULL); at process_connection stage. - - Probe error from dummy_connection was an issue, but was not a blocker as I can suppress that by listening on same TCP port (Type2 implementation in above URL). - - There was a timeout issue which caused server to stop responding when handling UDP socket. I worked around that by setting custom timeout value at process_connection stage, but this was only a kludge and requires deeper look for complete solution. - - core_output_filter (core_filters.c) was a major blocker. This (to be precise, underlying apr_socket_sendv called) expects output socket to be able to return data with simple writev(2). That can't be done for un-connected socket. To workaround last issue, I did apr_socket_connect to UDP socket, but that destroys listening UDP socket. In order to handle this, I tried several methods below: - - Do a connect(2) on current socket, pass it to bucket handler, and dup2(2) newly created listening socket into socket registered to ap_listeners. - -- This almost worked, but apache never responded to further request even though it continued to listen on UDP port. - - Disconnect "connected" UDP socket by connecting with AF_UNSPEC - -- Same as above...forgot the detail, but it didn't work. - - Pass newly created "connected" socket to bucket handler - -- This was a stupid idea as there's no way to pass received data to this new socket (only if there's accept(2) for UDP...). All failed, and as a result this HTTP/U module handled only first HTTP/UDP request properly, but no more. Although I have no specific idea on implementation, I think the patch posted can be made simpler by focusing on core_filter issue. But maybe this is an APR issue rather than of httpd? > I'm working on a protocol module for Apache 2.2 and ran into the lack of > UDP support in httpd. I'd like to try and remedy the situation in a > manner best suited for merging to trunk + backporting where possible. I > know that people have asked about it in the past, and if we really want > to be "d", we need support beyond TCP/SOCK_STREAM sockets. > > I tried to go through the socket-related routines in httpd (those of the > server; not of modules, clients, etc (mod_ssl, mod_proxy, mod_cgid, > etc)) and came up with a list of areas which needed attention: > > listen.c - alloc_listener is the focal point of the socket creation in > the server. There are a few issues that I can see with patching this > function: > 1) Currently it calls apr_socket_create with SOCK_STREAM, 0 (for type, > protocol). For adding UDP support only, we could make do with adding a > flag to the function to toggle between the existing values and > SOCK_DGRAM, APR_PROTO_UDP (which I'm personally against). To be more > portable, we'd pass both type and protocol as parameters (which I > personally prefer). Neither of these options makes much of a mess, as > AFAIK only ap_set_listener calls alloc_listener. > 2) In the code to check if we can re-use an existing listener, compare > the socket-descriptor's type and protocol against the ones passed. > 3) Additionally, I'd suggest making alloc_listener a public API function > (we've already added arguments, so no extra harm in renaming it to > ap_alloc_listener and adding it to ap_listen.h is there?) so that module > authors can create directives for protocols/types beyond SOCK_STREAM/TCP > and SOCK_DGRAM/UDP (see below). [This would be a minor bump, right? It > adds functionality without any breakage or backwards compatibility > issues, since as mentioned above only ap_set_listener currently calls > alloc_listener today] > > listen.c - make_sock sets a bunch of socket options. I don't profess to > be expert enough in socket programming to know what's kosher or not for > passing to non-"SOCK_STREAM TCP" sockets. More importantly, make_sock > calls apr_socket_listen, which isn't correct for datagram sockets. To > get around this, perhaps wrap the call in "if (s->type == SOCK_STREAM) " > (or whatever should be there to know if this type of socket needs to be > listen()ed to). Finally, similar logic (to detect if the socket needs > to be accept()ed) should be applied to the accept_func for MPMs which > utilize it. IMHO, rewriting standard MPM's that use an accept_func to > check if it's NULL before running it seems the smartest way to do it > (would that be a bigger bump than the ap_alloc_listeners described > above?). Barring that, we could define an "empty" default listener > function which does nothing, and have non-listening sockets have that > assigned as their alloc_func instead of MPM_ACCEPT_FUNC > > listen.c - ap_apply_accept_filter sets the accept filter for the > protocol (["http", "https", etc], not [TCP, UDP, etc]) As above, I'm > not sure how "kosher" this is, but TCP_DEFER_ACCEPT on UDP sounds wrong > :-) That could be solved by enclosing the filter in an "if (s->protocol > == APR_PROTO_TCP)" block > > listen.c - ap_set_listener is the user's interface to adding a listener. > I can't think of a nice graceful way to add the stream-type and > protocol to the existing listen directive while remaining backwards > compatible and keeping some resemblance of simplicity to the directive, > so I propose adding a ListenUDP directive which does the same logic > except that it calls alloc_listener with SOCK_DGRAM, APR_PROTO_UDP > (since UDP has been asked about several times). For other combinations, > module authors can call ap_alloc_listener as described above > > core.c - core_pre_connection disables the nagle algorithm by setting > APR_TCP_NODELAY. Can this be done safely to a non-TCP socket? If not, > just check "if (csd->protocl == APRC_PROTO_TCP)" as above > > mpm_common.c - ap_sock_disable_nagle disables the nagle algorithm as > above. > > mpm_common.c - dummy_connection currently makes assumptions about > SOCK_STREAM and TCP. However, it also only connects to the port in the > first listener's (in global ap_listener) port. I suppose changing it to > use ap_listeners->sd->type, ap_listeners->sd->protocol shouldn't hurt... > should it? > > Connection I/O should also be fairly easy. The core input filter just > uses a socket bucket to read from; if apr_bucket_type_socket supports > UDP sockets, we should be good to go on that end. The core output > filter is also fairly simple - just check that we're a SOCK_STREAM > socket before attempting to sendfile. > > I'm also assuming (possibly incorrectly) that polling and socket timeout > related code should all work as-is. > > I *think* that's all that should be needed... I have a patch against > trunk demonstrating most of this (attached). I'd really appreciate it > if people who have a firmer understanding of the relevant bits of code > that I'm proposing changes to could chime in and let me know if and what > I'm doing wrong, or not taking into consideration. - -- Taisuke Yamada <[EMAIL PROTECTED]>, http://rakugaki.org/ 2268 E9A2 D4F9 014E F11D 1DF7 DCA3 83BC 78E5 CD3A Message to my public address may not be handled in a timely manner. For a direct contact, please use my private address on my namecard. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF3q6b3KODvHjlzToRAq6JAJ9HB3XWDBPb1t+d8HMOwNC8P3rlFwCghLUW +UqYm2BkxGa9rK5aHoIHe7w= =7TEe -----END PGP SIGNATURE-----
