[twitter-dev] Search API cURL strangeness
I'm building an application that uses the search API to check for data related to particular domains, and suddenly (within the last week or so), I have started to experience a strange problem. Some of my requests are coming back with a cURL error Empty reply from server, but only when I am searching for a specific set of domains (all of the other domains work fine). I wrote a small test script to try and track down the problem, and it seems that the UserAgent I am setting with cURL seems to be causing the problem (or part of the problem). If I change the UserAgent to anything else, I get a normal response. I remember reading in the documentation that Twitter expects a unique UserAgent for the application, so that's what I did, but that seems to be causing problems. This seems like it's likely some sort of blacklist problem, but I can't figure out why it would work in this manner (only blocking a small subset of my queries, and not IP-based). Here are some sample queries I am trying to cURL: http://search.twitter.com/search.json?q=batteryoperatedcandles.netrpp=100since_id=9431322892177408since=until= http://search.twitter.com/search.json?q=carnationbreakfastessentials.comrpp=100since_id=since=2010-12-14until= http://search.twitter.com/search.json?q=apple.comrpp=100since_id=since=2010-12-14until= The first two don't work unless I change my UserAgent to something else, but the last one works no matter what. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Search API cURL strangeness
And your UserAgent is? Tom On 12/14/10 5:02 PM, Brian Medendorp wrote: I'm building an application that uses the search API to check for data related to particular domains, and suddenly (within the last week or so), I have started to experience a strange problem. Some of my requests are coming back with a cURL error Empty reply from server, but only when I am searching for a specific set of domains (all of the other domains work fine). I wrote a small test script to try and track down the problem, and it seems that the UserAgent I am setting with cURL seems to be causing the problem (or part of the problem). If I change the UserAgent to anything else, I get a normal response. I remember reading in the documentation that Twitter expects a unique UserAgent for the application, so that's what I did, but that seems to be causing problems. This seems like it's likely some sort of blacklist problem, but I can't figure out why it would work in this manner (only blocking a small subset of my queries, and not IP-based). Here are some sample queries I am trying to cURL: http://search.twitter.com/search.json?q=batteryoperatedcandles.netrpp=100since_id=9431322892177408since=until= http://search.twitter.com/search.json?q=carnationbreakfastessentials.comrpp=100since_id=since=2010-12-14until= http://search.twitter.com/search.json?q=apple.comrpp=100since_id=since=2010-12-14until= The first two don't work unless I change my UserAgent to something else, but the last one works no matter what. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
