I'm working on a protocol module for Apache 2.2 and ran into the lack of
UDP support in httpd. I'd like to try and remedy the situation in a
manner best suited for merging to trunk + backporting where possible. I
know that people have asked about it in the past, and if we really want
to be "d", we need support beyond TCP/SOCK_STREAM sockets.
I tried to go through the socket-related routines in httpd (those of the
server; not of modules, clients, etc (mod_ssl, mod_proxy, mod_cgid,
etc)) and came up with a list of areas which needed attention:
listen.c - alloc_listener is the focal point of the socket creation in
the server. There are a few issues that I can see with patching this
function:
1) Currently it calls apr_socket_create with SOCK_STREAM, 0 (for type,
protocol). For adding UDP support only, we could make do with adding a
flag to the function to toggle between the existing values and
SOCK_DGRAM, APR_PROTO_UDP (which I'm personally against). To be more
portable, we'd pass both type and protocol as parameters (which I
personally prefer). Neither of these options makes much of a mess, as
AFAIK only ap_set_listener calls alloc_listener.
2) In the code to check if we can re-use an existing listener, compare
the socket-descriptor's type and protocol against the ones passed.
3) Additionally, I'd suggest making alloc_listener a public API function
(we've already added arguments, so no extra harm in renaming it to
ap_alloc_listener and adding it to ap_listen.h is there?) so that module
authors can create directives for protocols/types beyond SOCK_STREAM/TCP
and SOCK_DGRAM/UDP (see below). [This would be a minor bump, right? It
adds functionality without any breakage or backwards compatibility
issues, since as mentioned above only ap_set_listener currently calls
alloc_listener today]
listen.c - make_sock sets a bunch of socket options. I don't profess to
be expert enough in socket programming to know what's kosher or not for
passing to non-"SOCK_STREAM TCP" sockets. More importantly, make_sock
calls apr_socket_listen, which isn't correct for datagram sockets. To
get around this, perhaps wrap the call in "if (s->type == SOCK_STREAM) "
(or whatever should be there to know if this type of socket needs to be
listen()ed to). Finally, similar logic (to detect if the socket needs
to be accept()ed) should be applied to the accept_func for MPMs which
utilize it. IMHO, rewriting standard MPM's that use an accept_func to
check if it's NULL before running it seems the smartest way to do it
(would that be a bigger bump than the ap_alloc_listeners described
above?). Barring that, we could define an "empty" default listener
function which does nothing, and have non-listening sockets have that
assigned as their alloc_func instead of MPM_ACCEPT_FUNC
listen.c - ap_apply_accept_filter sets the accept filter for the
protocol (["http", "https", etc], not [TCP, UDP, etc]) As above, I'm
not sure how "kosher" this is, but TCP_DEFER_ACCEPT on UDP sounds wrong
:-) That could be solved by enclosing the filter in an "if (s->protocol
== APR_PROTO_TCP)" block
listen.c - ap_set_listener is the user's interface to adding a listener.
I can't think of a nice graceful way to add the stream-type and
protocol to the existing listen directive while remaining backwards
compatible and keeping some resemblance of simplicity to the directive,
so I propose adding a ListenUDP directive which does the same logic
except that it calls alloc_listener with SOCK_DGRAM, APR_PROTO_UDP
(since UDP has been asked about several times). For other combinations,
module authors can call ap_alloc_listener as described above
core.c - core_pre_connection disables the nagle algorithm by setting
APR_TCP_NODELAY. Can this be done safely to a non-TCP socket? If not,
just check "if (csd->protocl == APRC_PROTO_TCP)" as above
mpm_common.c - ap_sock_disable_nagle disables the nagle algorithm as
above.
mpm_common.c - dummy_connection currently makes assumptions about
SOCK_STREAM and TCP. However, it also only connects to the port in the
first listener's (in global ap_listener) port. I suppose changing it to
use ap_listeners->sd->type, ap_listeners->sd->protocol shouldn't hurt...
should it?
Connection I/O should also be fairly easy. The core input filter just
uses a socket bucket to read from; if apr_bucket_type_socket supports
UDP sockets, we should be good to go on that end. The core output
filter is also fairly simple - just check that we're a SOCK_STREAM
socket before attempting to sendfile.
I'm also assuming (possibly incorrectly) that polling and socket timeout
related code should all work as-is.
I *think* that's all that should be needed... I have a patch against
trunk demonstrating most of this (attached). I'd really appreciate it
if people who have a firmer understanding of the relevant bits of code
that I'm proposing changes to could chime in and let me know if and what
I'm doing wrong, or not taking into consideration.
Issac
Index: include/ap_listen.h
===================================================================
--- include/ap_listen.h (revision 509448)
+++ include/ap_listen.h (working copy)
@@ -77,6 +77,26 @@
AP_DECLARE(void) ap_listen_pre_config(void);
/**
+ * Allocate a new listener to be created during ap_setup_listeners.
+ * @param process The process_rec of the parent process
+ * @param addr The IP address to bind the socket to (uses same format as
+ * the Listen directive in httpd.conf)
+ * @param port The port to bind the socket to
+ * @param type The socket type (SOCK_STREAM, etc)
+ * @param protocol The socket protocol (APR_PROTO_TCP, etc)
+ * @param proto The optional protocol argument is not required for most
+ * configurations. If not specified, https is the default for port 443
+ * and http the default for all other ports. The protocol is used to
+ * determine which module should handle a request, and to apply
+ * protocol specific optimizations with the AcceptFilter directive.
+ * @return NULL on success, or an error message on failure
+ */
+
+AP_DECLARE(const char *) ap_alloc_listener(process_rec *process, char *addr,
+ apr_port_t port, int type, int protocol,
+ const char* proto);
+
+/**
* Loop through the global ap_listen_rec list and create all of the required
* sockets. This executes the listen and bind on the sockets.
* @param s The global server_rec
@@ -99,6 +119,9 @@
AP_DECLARE_NONSTD(const char *) ap_set_listenbacklog(cmd_parms *cmd, void
*dummy, const char *arg);
AP_DECLARE_NONSTD(const char *) ap_set_listener(cmd_parms *cmd, void *dummy,
int argc, char *const argv[]);
+AP_DECLARE_NONSTD(const char *) ap_set_udp_listener(cmd_parms *cmd,
+ void *dummy, int argc,
+ char *const argv[]);
AP_DECLARE_NONSTD(const char *) ap_set_send_buffer_size(cmd_parms *cmd, void
*dummy,
const char *arg);
AP_DECLARE_NONSTD(const char *) ap_set_receive_buffer_size(cmd_parms *cmd,
@@ -110,6 +133,10 @@
"Maximum length of the queue of pending connections, as used by listen(2)"),
\
AP_INIT_TAKE_ARGV("Listen", ap_set_listener, NULL, RSRC_CONF, \
"A port number or a numeric IP address and a port number, and an optional
protocol"), \
+AP_INIT_TAKE_ARGV("ListenTCP", ap_set_listener, NULL, RSRC_CONF, \
+ "A port number or a numeric IP address and a port number, and an optional
protocol"), \
+AP_INIT_TAKE_ARGV("ListenUDP", ap_set_udp_listener, NULL, RSRC_CONF, \
+ "A port number or a numeric IP address and a port number, and an optional
protocol"), \
AP_INIT_TAKE1("SendBufferSize", ap_set_send_buffer_size, NULL, RSRC_CONF, \
"Send buffer size in bytes"), \
AP_INIT_TAKE1("ReceiveBufferSize", ap_set_receive_buffer_size, NULL, \
Index: server/core_filters.c
===================================================================
--- server/core_filters.c (revision 509448)
+++ server/core_filters.c (working copy)
@@ -535,6 +535,7 @@
{
apr_bucket *bucket, *next;
apr_status_t rv;
+ int type;
struct iovec vec[MAX_IOVEC_TO_WRITE];
apr_size_t nvec = 0;
@@ -546,7 +547,12 @@
int did_sendfile = 0;
next = APR_BUCKET_NEXT(bucket);
#if APR_HAS_SENDFILE
- if (APR_BUCKET_IS_FILE(bucket)) {
+ /** Don't even bother with sendfile if we're not dealing with a
+ * SOCK_STREAM socket
+ */
+ if (APR_BUCKET_IS_FILE(bucket) &&
+ ((apr_socket_type_get(s, &type) == APR_SUCCESS) &&
+ (type == SOCK_STREAM))) {
apr_bucket_file *file_bucket = (apr_bucket_file *)(bucket->data);
apr_file_t *fd = file_bucket->fd;
/* Use sendfile to send this file unless:
Index: server/listen.c
===================================================================
--- server/listen.c (revision 509448)
+++ server/listen.c (working copy)
@@ -49,6 +49,7 @@
int v6only_setting = 1;
#endif
#endif
+ int type;
apr_status_t stat;
#ifndef WIN32
@@ -136,8 +137,17 @@
apr_socket_close(s);
return stat;
}
-
- if ((stat = apr_socket_listen(s, ap_listenbacklog)) != APR_SUCCESS) {
+
+ if ((stat = apr_socket_type_get(s, &type)) != APR_SUCCESS) {
+ ap_log_perror(APLOG_MARK, APLOG_STARTUP|APLOG_CRIT, stat, p,
+ "make_sock: could not determine socket type for "
+ "address %pI", server->bind_addr);
+ apr_socket_close(s);
+ return stat;
+ }
+
+ if (type == SOCK_STREAM &&
+ ((stat = apr_socket_listen(s, ap_listenbacklog)) != APR_SUCCESS)) {
ap_log_perror(APLOG_MARK, APLOG_STARTUP|APLOG_ERR, stat, p,
"make_sock: unable to listen for connections "
"on address %pI",
@@ -204,7 +214,8 @@
const char *accf;
apr_status_t rv;
const char *proto;
-
+ int protocol;
+
proto = lis->protocol;
if (!proto) {
@@ -225,10 +236,13 @@
}
#else
#ifdef APR_TCP_DEFER_ACCEPT
- rv = apr_socket_opt_set(s, APR_TCP_DEFER_ACCEPT, 30);
- if (rv != APR_SUCCESS && !APR_STATUS_IS_ENOTIMPL(rv)) {
- ap_log_perror(APLOG_MARK, APLOG_WARNING, rv, p,
- "Failed to enable APR_TCP_DEFER_ACCEPT");
+ rv = apr_socket_protocol_get(s, &protocol);
+ if ((rv == APR_SUCCESS) && (protocol == APR_PROTO_TCP)) {
+ rv = apr_socket_opt_set(s, APR_TCP_DEFER_ACCEPT, 30);
+ if (rv != APR_SUCCESS && !APR_STATUS_IS_ENOTIMPL(rv)) {
+ ap_log_perror(APLOG_MARK, APLOG_WARNING, rv, p,
+ "Failed to enable APR_TCP_DEFER_ACCEPT");
+ }
}
#endif
#endif
@@ -241,12 +255,14 @@
return APR_SUCCESS;
}
-static const char *alloc_listener(process_rec *process, char *addr,
- apr_port_t port, const char* proto)
+AP_DECLARE(const char *) ap_alloc_listener(process_rec *process, char *addr,
+ apr_port_t port, int type, int protocol,
+ const char* proto)
{
ap_listen_rec **walk, *last;
apr_status_t status;
apr_sockaddr_t *sa;
+ apr_socket_t *sd;
int found_listener = 0;
/* see if we've got an old listener for this address:port */
@@ -255,13 +271,20 @@
/* Some listeners are not real so they will not have a bind_addr. */
if (sa) {
ap_listen_rec *new;
+ sd = (*walk)->sd;
apr_port_t oldport;
+ int oldprotocol = -1;
+ int oldtype = -1;
oldport = sa->port;
- /* If both ports are equivalent, then if their names are
equivalent,
+ apr_socket_type_get(sd, &oldtype);
+ apr_socket_protocol_get(sd, &oldprotocol);
+ /* If both ports are equivalent and both socket type/protocols are
+ * equivalent, then if their names are equivalent,
* then we will re-use the existing record.
*/
- if (port == oldport &&
+ if ((port == oldport &&
+ (type == oldtype && protocol == oldprotocol)) &&
((!addr && !sa->hostname) ||
((addr && sa->hostname) && !strcmp(sa->hostname, addr)))) {
new = *walk;
@@ -284,7 +307,7 @@
process->pool))
!= APR_SUCCESS) {
ap_log_perror(APLOG_MARK, APLOG_CRIT, status, process->pool,
- "alloc_listener: failed to set up sockaddr for %s",
+ "ap_alloc_listener: failed to set up sockaddr for %s",
addr);
return "Listen setup failed";
}
@@ -309,7 +332,7 @@
sa = sa->next;
status = apr_socket_create(&new->sd, new->bind_addr->family,
- SOCK_STREAM, 0, process->pool);
+ type, protocol, process->pool);
#if APR_HAVE_IPV6
/* What could happen is that we got an IPv6 address, but this system
@@ -322,7 +345,7 @@
#endif
if (status != APR_SUCCESS) {
ap_log_perror(APLOG_MARK, APLOG_CRIT, status, process->pool,
- "alloc_listener: failed to get a socket for %s",
+ "ap_alloc_listener: failed to get a socket for %s",
addr);
return "Listen setup failed";
}
@@ -583,7 +606,6 @@
ap_listenbacklog = DEFAULT_LISTENBACKLOG;
}
-
AP_DECLARE_NONSTD(const char *) ap_set_listener(cmd_parms *cmd, void *dummy,
int argc, char *const argv[])
{
@@ -626,9 +648,57 @@
ap_str_tolower(proto);
}
- return alloc_listener(cmd->server->process, host, port, proto);
+ return ap_alloc_listener(cmd->server->process, host, port,
+ SOCK_STREAM, 0, proto);
}
+AP_DECLARE_NONSTD(const char *) ap_set_udp_listener(cmd_parms *cmd,
+ void *dummy, int argc,
+ char *const argv[])
+{
+ char *host, *scope_id, *proto;
+ apr_port_t port;
+ apr_status_t rv;
+ const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY);
+
+ if (err != NULL) {
+ return err;
+ }
+
+ if (argc < 1 || argc > 2) {
+ return "Listen requires 1 or 2 arguments.";
+ }
+
+ rv = apr_parse_addr_port(&host, &scope_id, &port, argv[0], cmd->pool);
+ if (rv != APR_SUCCESS) {
+ return "Invalid address or port";
+ }
+
+ if (host && !strcmp(host, "*")) {
+ host = NULL;
+ }
+
+ if (scope_id) {
+ /* XXX scope id support is useful with link-local IPv6 addresses */
+ return "Scope id is not supported";
+ }
+
+ if (!port) {
+ return "Port must be specified";
+ }
+
+ if (argc != 2) {
+ proto = "http";
+ }
+ else {
+ proto = apr_pstrdup(cmd->pool, argv[1]);
+ ap_str_tolower(proto);
+ }
+
+ return ap_alloc_listener(cmd->server->process, host, port,
+ SOCK_DGRAM, APR_PROTO_UDP, proto);
+}
+
AP_DECLARE_NONSTD(const char *) ap_set_listenbacklog(cmd_parms *cmd,
void *dummy,
const char *arg)
Index: server/mpm/experimental/event/event.c
===================================================================
--- server/mpm/experimental/event/event.c (revision 509448)
+++ server/mpm/experimental/event/event.c (working copy)
@@ -1010,7 +1010,9 @@
apr_pool_tag(ptrans, "transaction");
- rc = lr->accept_func(&csd, lr, ptrans);
+ rc = APR_SUCCESS;
+ if (lr->accept_func)
+ rc = lr->accept_func(&csd, lr, ptrans);
/* later we trash rv and rely on csd to indicate
* success/failure
Index: server/mpm/experimental/leader/leader.c
===================================================================
--- server/mpm/experimental/leader/leader.c (revision 509448)
+++ server/mpm/experimental/leader/leader.c (working copy)
@@ -786,7 +786,9 @@
}
got_fd:
if (!workers_may_exit) {
- rv = lr->accept_func(&csd, lr, ptrans);
+ rv = APR_SUCCESS;
+ if (lr->accept_func)
+ rv = lr->accept_func(&csd, lr, ptrans);
/* later we trash rv and rely on csd to indicate success/failure */
AP_DEBUG_ASSERT(rv == APR_SUCCESS || !csd);
Index: server/mpm/experimental/perchild/perchild.c
===================================================================
--- server/mpm/experimental/perchild/perchild.c (revision 509448)
+++ server/mpm/experimental/perchild/perchild.c (working copy)
@@ -736,7 +736,9 @@
}
got_fd:
if (!workers_may_exit) {
- rv = lr->accept_func(&csd, lr, ptrans);
+ rv = APR_SUCCESS;
+ if (lr->accept_func)
+ rv = lr->accept_func(&csd, lr, ptrans);
if (rv == APR_EGENERAL) {
/* E[NM]FILE, ENOMEM, etc */
workers_may_exit = 1;
Index: server/mpm/experimental/threadpool/threadpool.c
===================================================================
--- server/mpm/experimental/threadpool/threadpool.c (revision 509448)
+++ server/mpm/experimental/threadpool/threadpool.c (working copy)
@@ -855,7 +855,9 @@
}
}
if (!listener_may_exit) {
- rv = lr->accept_func(&csd, lr, ptrans);
+ rv = APR_SUCCESS;
+ if (lr->accept_func)
+ rv = lr->accept_func(&csd, lr, ptrans);
/* later we trash rv and rely on csd to indicate success/failure */
AP_DEBUG_ASSERT(rv == APR_SUCCESS || !csd);
Index: server/mpm/prefork/prefork.c
===================================================================
--- server/mpm/prefork/prefork.c (revision 509448)
+++ server/mpm/prefork/prefork.c (working copy)
@@ -618,7 +618,9 @@
/* if we accept() something we don't want to die, so we have to
* defer the exit
*/
- status = lr->accept_func(&csd, lr, ptrans);
+ status = APR_SUCCESS;
+ if (lr->accept_func)
+ status = lr->accept_func(&csd, lr, ptrans);
SAFE_ACCEPT(accept_mutex_off()); /* unlock after "accept" */
Index: server/mpm/worker/worker.c
===================================================================
--- server/mpm/worker/worker.c (revision 509448)
+++ server/mpm/worker/worker.c (working copy)
@@ -737,7 +737,9 @@
apr_allocator_owner_set(allocator, ptrans);
}
apr_pool_tag(ptrans, "transaction");
- rv = lr->accept_func(&csd, lr, ptrans);
+ rv = APR_SUCCESS;
+ if (lr->accept_func)
+ rv = lr->accept_func(&csd, lr, ptrans);
/* later we trash rv and rely on csd to indicate success/failure */
AP_DEBUG_ASSERT(rv == APR_SUCCESS || !csd);
Index: server/mpm_common.c
===================================================================
--- server/mpm_common.c (revision 509448)
+++ server/mpm_common.c (working copy)
@@ -421,6 +421,13 @@
*
* In spite of these problems, failure here is not a shooting offense.
*/
+
+ int protocol;
+ if (!((apr_socket_protocol_get(s, &protocol) == APR_SUCCESS) &&
+ protocol == APR_PROTO_TCP))
+ /** Don't do anything unless we're a TCP socket */
+ return;
+
apr_status_t status = apr_socket_opt_set(s, APR_TCP_NODELAY, 1);
if (status != APR_SUCCESS) {
@@ -584,6 +591,13 @@
apr_socket_t *sock;
apr_pool_t *p;
apr_size_t len;
+ int protocol;
+ int type;
+
+ if (apr_socket_type_get(ap_listeners->sd, &type) != APR_SUCCESS)
+ type = SOCK_STREAM;
+ if (apr_socket_protocol_get(ap_listeners->sd, &protocol) != APR_SUCCESS)
+ type = 0;
/* create a temporary pool for the socket. pconf stays around too long */
rv = apr_pool_create(&p, pod->p);
@@ -592,7 +606,7 @@
}
rv = apr_socket_create(&sock, ap_listeners->bind_addr->family,
- SOCK_STREAM, 0, p);
+ type, protocol, p);
if (rv != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_WARNING, rv, ap_server_conf,
"get socket to connect to listener");
