Hello,
I've run into a problem with the Twiggy web server where it became
unresponsive with 100% CPU consumption if many (1020 plus/minus a few)
clients were trying to connect to a streaming server. The problem is
actually in AnyEvent::Socket's tcp_server, as the following example
demonstrates:
###################
#!/usr/bin/perl
use strict;
use warnings;
use feature qw/say/;
use AnyEvent;
use AnyEvent::Socket;
use AnyEvent::Handle;
my $timer = AE::timer rand(1), 0, \&send_to_clients;
my $handles = {};
sub send_to_clients {
$_->push_write("message\n") for values %$handles;
$timer = AE::timer rand(1), 0, \&send_to_clients;
}
tcp_server undef, 5000, sub {
my ($fh, $host, $port) = @_;
my $hdl; $hdl = AnyEvent::Handle->new(
fh => $fh,
on_error => sub {
my ($hdl, $fatal, $msg) = @_;
delete $handles->{$hdl};
say STDERR "Client closed connection: $msg";
$hdl->destroy;
}
);
$handles->{$hdl} = $hdl;
say STDERR "Client connected on port $port, number of clients ".scalar
keys %$handles;
}, sub {
my ($fh, $thishost, $thisport) = @_;
say STDERR "Bound to $thishost, port $thisport.";
};
AnyEvent->condvar->recv;
##################
You can drive it with the following bash command (after starting the
above server of course):
for i in {1..1021}; do curl -Ns localhost:5000 > /dev/null & done
The server won't print the line with "number of clients 1021", and with
(h)top you can see that it eats 100% CPU until a few clients are killed.
I realize that what actually happens is that the process runs out of
file descriptors (of which the default limit is indeed 1024), so it's no
surprise that the server can't serve any more clients. However, the 100%
CPU usage seems like a bug to me, as this behavior can be abused to deny
resources from other services running on the same machine.
The problem seems to come from the following part of tcp_server in
AE::Socket:
################
$rstate->{aw} = AE::io $rstate->{fh}, 0, sub {
# this closure keeps $state alive
while ($rstate->{fh} && (my $peer = accept my $fh,
$rstate->{fh})) {
AnyEvent::fh_unblock $fh; # POSIX requires inheritance, the
outside world does not
my ($service, $host) = unpack_sockaddr $peer;
$accept->($fh, format_address $host, $service);
}
};
################
where accept fails, sets $! to Errno::EMFILE, but for some reason the
callback seems to be called repeatedly.
I used AnyEvent version 7.12, the problem persists with both the EV and
pure perl backends.
Best regards,
Peter Juhasz
_______________________________________________
anyevent mailing list
[email protected]
http://lists.schmorp.de/mailman/listinfo/anyevent
