Glyph, thanks for the great feedback. I'm looking into all of this now. I don't think it's a ulimit issue as it seems to be "unlimited"
r...@proxy1:~# ulimit unlimited I think my first step will be to just make a log observer that will simply restart the proxy script when it sees (EMFILE) so that I don't have to keep logging in at night to restart it myself :). Thanks for filing that ticket. It would be great if there were a way to be notified of the particular error inside the code. I like your suggestion of keeping a list of my open connections. I could also keep a time stamp with them and close down the oldest, or any that are over a certain time period. I'm new to twisted. Do you know in what object of the Proxy the new connections are made and how to access them to store in a list? Thanks again! Dave Fowler On Thu, Aug 6, 2009 at 9:31 PM, Glyph Lefkowitz <[email protected]>wrote: > On Thu, Aug 6, 2009 at 6:22 PM, dave fowler <[email protected]> wrote: > >> I guys. I'm running a twisted web proxy and after a day or so of usage it >> stops working and start spitting out these errors. >> >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> 2009/08/05 14:04 +0000 [__main__.ProxyFactory] Could not accept new >> connection (EMFILE) >> >> Can anyone expand on what this error is or suggestions on fixing it or >> debugging further? >> >> What is the EMFILE? >> > > It means you're out of file descriptors for your process. > > See here for the root cause: <http://linux.die.net/man/2/accept>, > specifically the part about EMFILE in the Errors section. I'm actually glad > to see this error report - Twisted used to fail in a *much* worse way in > this error condition (<http://twistedmatrix.com/trac/ticket/662>), this is > the first proof I've seen that our fix works in the wild :). > > To increase your per-process file-descriptor limit, use "ulimit -n". Be > aware that in order to increase this above 1024, you'll need to be root; I > think there's a way to set it up to be bigger than that for non-root users > but I don't know how. > > In order to free up file descriptors, keep a list of your open connections > and shut down some of them. Sadly, your Factory object won't be notified of > this problem, so you might have to hack something gross together (like a log > observer that watches for the string "(EMFILE)"). I just filed this ticket > to provide a nice, clean notification: < > http://twistedmatrix.com/trac/ticket/3958>. This may be a duplicate of > something, but if so it should be closed with a link to what it's a > duplicate of. > > In some cases, connections won't shut down properly because there is still > some buffered data and the other end is not available to acknowledge the > connection's closure. If this is the case, you can forcibly close the file > descriptor (although this may cause some ugliness). To get a good, > supported way to destroy file descriptors in Twisted, see this ticket: < > http://twistedmatrix.com/trac/ticket/78>. > > Good luck! Sorry Twisted has not handled this error in the best way for > you, I hope you help us fix some of these bugs for the next guy :). > > > _______________________________________________ > Twisted-web mailing list > [email protected] > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web > >
_______________________________________________ Twisted-web mailing list [email protected] http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
