Hi I'm working with curl via pycurl via tornado, so I might be barking up the wrong tree; let me know if I am.
I'm having an issue where my tornado app seems to work fine for a while, but over time degrades until it basically isn't working at all (e.g. lots of timeouts on processing requests). This makes me think the issue has something to do with resource usage. I've watched memory, there doesn't seem to be a leak, so now I'm looking at the number of network connections. I've noticed that when my app starts performing poorly, the number of connections reported by: $ lsof -i is larger than 'normal'. I'm worried that I'm doing something wrong when it comes to handling curl connections. Here's where I think I might be doing something wrong: I'm using tornado's curl_httpclient, and basically just plugging in customized header and body callbacks. Sometimes I decide that I don't want to process an HTTP response's body by looking at the headers. So, in the curl header callback (set with setopt HEADERFUNCTION), I look for certain signs, and set a flag to ignore the body of the request. If the flag is set, then in the curl body callback (set with setopt WRITEFUNCTION) I return a number different than the number of bytes passed in. From reading the libcurl documentation, it seems to me that this should abort that transfer and (I'm hoping) free up the easy handle for another connection. So, is this a reasonable thing to do to abort a curl connection? Or am I shooting myself in the foot? Any input is appreciated. Thanks for your time.
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
