On Fri, 13 Apr 2012, Andy Pellett wrote:
I'm working with curl via pycurl via tornado, so I might be barking up the wrong tree; let me know if I am.
You are. Well, to be honest it isn't that easy to tell but since you're using tornado you are not using libcurl directly and I really have no idea what tornado does or doesn't do in between.
The best idea is then to first work it out with the binding developers, the tornado team in this case. If they think they do everything by the book libcurl-wise, it will help us a lot of you then try to repeat the issue in plain C instead. I understand that may not be that easy, but then it isn't easy for us to straighten out how to debug a libcurl issue intertwined in tornado either...
But you also didn't say which libcurl version on which operating system you're using. Is this a "stock" libcurl or does it have any particular changes applied?
I've noticed that when my app starts performing poorly, the number of connections reported by: $ lsof -i is larger than 'normal'.
How much larger are we talking about here?
Sometimes I decide that I don't want to process an HTTP response's body by looking at the headers. So, in the curl header callback (set with setopt HEADERFUNCTION), I look for certain signs, and set a flag to ignore the body of the request. If the flag is set, then in the curl body callback (set with setopt WRITEFUNCTION) I return a number different than the number of bytes passed in. From reading the libcurl documentation, it seems to me that this should abort that transfer and (I'm hoping) free up the easy handle for another connection.
Yes, that is the result of such an action! -- / daniel.haxx.se ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
