[Resending because apparently it didn't get through yesterday.]
Hi,
my HTTP server thingy came to a screeching halt when serving many
large files, so I added a form of Linux sendfile support that requires
just a few changes and supports some interesting uses (e.g. sending
the HTTP headers normally but the HTTP body via sendfile).
The server now runs very fast, with very little CPU and RAM. This is
preliminary, probably buggy, and I don't understand the evbuffer
functionality very well (e.g. draining).
In addition to the normal data evbuffers, this patch adds two types of
evbuffers: (1) sendfile buffers, and (2) data buffers to which a
sendfile buffer has been added. This distinction is recorded in the
sf_fd field. There are two additonal fields, sf_off and sf_buf.
*** libevent-1.4.3-stable/event.h 2008-02-23 02:36:12.0 +0100
--- libevent-1.4.3-stable-sendfile/event.h 2008-05-07
15:40:32.0 +0200
***
*** 724,729
--- 724,746
size_t totallen;
size_t off;
+ /*
+ sf_fd == -1. A standard data buffer. sf_off and sf_buf are
+ unused.
+
+ sf_fd == -2. A buffer to which a sendfile buffer has been
+ added at the end (sf_buf). off holds the combined size of
+ the RAM data of this buffer and the added sendfile buffer.
+ sf_off is unused.
+
+ sf_fd = 0. A pure sendfile buffer. sf_fd, sf_off, and off
+ are the arguments in_fd, offset, and count to sendfile(),
+ respectively. sf_buf is unused.
+ */
+ int sf_fd;
+ off_t sf_off;
+ struct evbuffer *sf_buf;
+
void (*cb)(struct evbuffer *, size_t, size_t, void *);
void *cbarg;
};
The adding and writing of buffers (evbuffer_add_buffer,
evbuffer_write) is changed:
To keep it simple this system only allows adding a sendfile buffer to
a data buffer. So, the sendfile buffer must be the last step in the
chain, which works fine for e.g. many HTTP uses cases. You cannot add
a sendfile buffer to another sendfile buffer, or add a data buffer to
a sendfile buffer. I think these things would theoretically be
possible, but require a lot more housekeeping than the current
solution, which plugs directly into the existing draining code.
When a sendfile buffer is added after a data buffer, we trick and
increment the data buffer's length by the length of the sendfile
buffer, without actually copying the data to the buffer (which would
completely defeat the purpose of sendfile, of course). Then, later
when it comes to writing this combined buffer, we just need to check
whether all normal data has been written, and then switch over to
writing the sendfile data. This *seems* to work nicely but I am not
100% sure that every situation is handled OK.
*** libevent-1.4.3-stable/buffer.c 2007-11-12 03:37:32.0 +0100
--- libevent-1.4.3-stable-sendfile/buffer.c 2008-05-07
16:36:44.0 +0200
***
*** 61,66
--- 61,68
#include unistd.h
#endif
+ #include sys/sendfile.h
+
#include event.h
#include config.h
***
*** 71,76
--- 73,81
buffer = calloc(1, sizeof(struct evbuffer));
+ /* Make all buffers non-sendfile buffers initially. */
+ buffer-sf_fd = -1;
+
return (buffer);
}
***
*** 100,105
--- 105,130
{
int res;
+ if (outbuf-sf_fd == -1) {
+ /* The output buffer is a data buffer. */
+ if (inbuf-sf_fd = 0) {
+ /* The input buffer is a sendfile buffer.
+ Mark this buffer as containing a sendfile
+ buffer and virtually extend the buffer by
+ the size of the sendfile buffer. */
+ outbuf-sf_fd = -2;
+ outbuf-sf_buf = inbuf;
+ outbuf-off += inbuf-off;
+ return 0;
+ }
+ } else {
+ /* No support for nested or concatenated sendfile
+ buffers, i.e. if the output buffer is or contains a
+ sendfile buffer, we cannot add another sendfile
+ buffer to it. */
+ return -1;
+ }
+
/* Short cut for better performance */
if (outbuf-off == 0) {
struct evbuffer tmp;
***
*** 415,421
int n;
#ifndef WIN32
! n = write(fd, buffer-buffer, buffer-off);
#else
n = send(fd, buffer-buffer, buffer-off, 0);
#endif
--- 440,467
int n;
#ifndef WIN32
! switch(buffer-sf_fd) {
! case -1:
! /* A data buffer. */
! n = write(fd, buffer-buffer, buffer-off);
! break;
! case -2:
! /* A data buffer with a sendfile buffer at the end.
! First write the actual data in the buffer, then
! switch to
