Bug#919316: bugs.debian.org: DoS with libravatar.cgi and version.cgi
On Tue, 15 Jan 2019, Julien Cristau wrote: > On Mon, Jan 14, 2019 at 21:27:31 -0800, Don Armstrong wrote: > > Is there any reason why we're using RLimitNPROC instead of setting > > MaxClients? > > > MaxRequestWorkers (new name for MaxClients) is set to 150 currently. We > could try lowering that or bumping the RLimitNPROC value a bit? That sounds reasonable to me; I think at the most, there should be twice as many processes as maxclients, so probably setting it to something like 450 should be more than sufficient, and still stop things from completely blowing up if I do something really stupid. -- Don Armstrong https://www.donarmstrong.com As nightfall does not come at once, neither does oppression. In both instances, there is a twilight when everything remains seemingly unchanged. And it is in such twilight that we all must be most aware of change in the air -- however slight -- lest we become unwitting victims of the darkness. -- William O. Douglas
Bug#919316: bugs.debian.org: DoS with libravatar.cgi and version.cgi
On Mon, Jan 14, 2019 at 21:27:31 -0800, Don Armstrong wrote: > On Mon, 14 Jan 2019, Julien Cristau wrote: > > the last few days our two bugs web hosts have been struggling. We've > > got apache set up with RLimitNPROC 256, which means once www-data has > > that many processes fork() dies with EAGAIN. > > > > In the case of version.cgi that means perl forking to run dot, for > > libravatar.cgi it looks like it's either cp or convert. Either way, the > > failure mode of fork returning EAGAIN is that perl does sleep(5) and > > tries again. Over and over. Which means by the time something happens > > the client has long gone, but we're still holding up one of the 256 > > nproc slots not doing anything useful for quite a bunch of time. > > Is there any reason why we're using RLimitNPROC instead of setting > MaxClients? > MaxRequestWorkers (new name for MaxClients) is set to 150 currently. We could try lowering that or bumping the RLimitNPROC value a bit? Cheers, Julien
Bug#919316: bugs.debian.org: DoS with libravatar.cgi and version.cgi
On Mon, 14 Jan 2019, Julien Cristau wrote: > the last few days our two bugs web hosts have been struggling. We've > got apache set up with RLimitNPROC 256, which means once www-data has > that many processes fork() dies with EAGAIN. > > In the case of version.cgi that means perl forking to run dot, for > libravatar.cgi it looks like it's either cp or convert. Either way, the > failure mode of fork returning EAGAIN is that perl does sleep(5) and > tries again. Over and over. Which means by the time something happens > the client has long gone, but we're still holding up one of the 256 > nproc slots not doing anything useful for quite a bunch of time. Is there any reason why we're using RLimitNPROC instead of setting MaxClients? There's not much on bugs.debian.org which isn't served by a cgi script, so there's limited difference between the two for it, and as you can see, returning EAGAIN isn't particularly useful. [Those two CGI scripts actually just die() when system() returns non-zero, so it's probably perl internal system() bits which are sleep() and retrying.] -- Don Armstrong https://www.donarmstrong.com Those who begin coercive elimination of dissent soon find themselves exterminating dissenters. Compulsory unification of opinion achieves only the unanimity of the graveyard. -- Justice Roberts in 319 U.S. 624 (1943)
Bug#919316: bugs.debian.org: DoS with libravatar.cgi and version.cgi
Package: bugs.debian.org Severity: important X-Debbugs-Cc: debian-ad...@lists.debian.org Hi, the last few days our two bugs web hosts have been struggling. We've got apache set up with RLimitNPROC 256, which means once www-data has that many processes fork() dies with EAGAIN. In the case of version.cgi that means perl forking to run dot, for libravatar.cgi it looks like it's either cp or convert. Either way, the failure mode of fork returning EAGAIN is that perl does sleep(5) and tries again. Over and over. Which means by the time something happens the client has long gone, but we're still holding up one of the 256 nproc slots not doing anything useful for quite a bunch of time. So for now, I've chmodded -x libravatar.cgi on beach and buxtehude, but we should figure out a better solution. Cheers, Julien
