On Sat, Jun 06, 2015 at 07:05:46PM +0000, Florian Obser wrote:
> On Fri, Jun 05, 2015 at 05:15:09PM -0500, Matthew Martin wrote:
> > >Synopsis: Serving large files with httpd eats fscktons of memory.
> > >Category: system
> > >Environment:
> > System : OpenBSD 5.7
> > Details : OpenBSD 5.7 (GENERIC) #0: Thu Apr 30 22:01:01 CEST 2015
> >
> > [email protected]:/binpatchng/work-binpatch57-amd64/src/sys/arch/amd64/compile/GENERIC
> >
> > Architecture: OpenBSD.amd64
> > Machine : amd64
> > >Description:
> > A couple of people concurrently downloading iso sized files can drag
> > the server into using all of swap (there's only a gig) in a
> > frighteningly short time.
> >
> > Revision 1.30 of
> > http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/httpd/server_file.c
> > had message "Adjust the read/write watermarks according to the TCP
> > send buffer. This fixes sending of large files. Previously, httpd was
> > reading the input file too quickly and could run out of memory when
> > filling the input buffer."
> >
> > That pretty well describes the situation I'm seeing now with 5.7
> > (-RELEASE + m:tier binpatches). It also happens with -CURRENT (as of
> > Jun 5) httpd compiled and slapped in its place.
> >
> > >How-To-Repeat:
> > Something along the lines of
> > hostA# dd if=/dev/zero of=/var/www/htdocs/test_512.dat bs=1M count=512
> > hostB$ ftp http://hostA/test_512.dat
> >
> > >Fix:
> > I've attempted various settings of 'tcp socket buffer' and shotgunning
> > around the lines modified in 1.30 to no avail. Trying to limit its
> > consumption via login.conf just makes it die when it hits the limit.
>
> the problem is that we have to bufferevents. clt->clt_bev which is
> handleing the accept(2)ed socket and clt->clt_srvbev which is
> handleing the fd of the served file.
>
> it goes like this:
> clt_bev: Here I am, brain the size of a planet, and they ask me to serve
> a file.
> clt_srvbev: Whee, I can read clt_sndbufsiz bytes from the file. Hello
> clt_bev, here, have some data.
> clt_bev: Ok, I can't send it right now, but I have a buffer, I'll put
> the data at the end.
> [last message repeated n times]
> clt_bev: Phew, I can send some data to the client, hopfully clt_bev
> stops to send me data soon.
>
> and so on... In the end the whole iso ends up in clt_bev.
>
> I'm not sure yet about the 32 and 256 magic numbers, they seem to it
> some kind of sweet spot on *my* system. Make them to small and
> everything slows down, make them to large and you waste memory.
>
> Please try this:
>
this gets rid of a compiler warning:
diff --git server.c server.c
index ca67a47..8725e2c 100644
--- server.c
+++ server.c
@@ -704,7 +704,7 @@ server_input(struct client *clt)
/* Adjust write watermark to the socket buffer output size */
bufferevent_setwatermark(clt->clt_bev, EV_WRITE,
- clt->clt_sndbufsiz, 0);
+ 32 * clt->clt_sndbufsiz, 0);
/* Read at most amount of data that fits in one fcgi record. */
bufferevent_setwatermark(clt->clt_bev, EV_READ, 0, FCGI_CONTENT_SIZE);
@@ -729,6 +729,10 @@ server_write(struct bufferevent *bev, void *arg)
goto done;
bufferevent_enable(bev, EV_READ);
+
+ if (clt->clt_srvbev && !(clt->clt_srvbev->enabled & EV_READ))
+ bufferevent_enable(clt->clt_srvbev, EV_READ);
+
return;
done:
(*bev->errorcb)(bev, EVBUFFER_WRITE|EVBUFFER_EOF, bev->cbarg);
@@ -769,6 +773,11 @@ server_read(struct bufferevent *bev, void *arg)
goto fail;
if (clt->clt_done)
goto done;
+
+ if (EVBUFFER_LENGTH(EVBUFFER_OUTPUT(clt->clt_bev)) > (size_t) 256 *
+ clt->clt_sndbufsiz)
+ bufferevent_disable(bev, EV_READ);
+
return;
done:
(*bev->errorcb)(bev, EVBUFFER_READ|EVBUFFER_EOF, bev->cbarg);
--
I'm not entirely sure you are real.
