I'm building an application that uses the search API to check for data related to particular domains, and suddenly (within the last week or so), I have started to experience a strange problem. Some of my requests are coming back with a cURL error "Empty reply from server", but only when I am searching for a specific set of domains (all of the other domains work fine).
I wrote a small test script to try and track down the problem, and it seems that the UserAgent I am setting with cURL seems to be causing the problem (or part of the problem). If I change the UserAgent to anything else, I get a normal response. I remember reading in the documentation that Twitter expects a unique UserAgent for the application, so that's what I did, but that seems to be causing problems. This seems like it's likely some sort of blacklist problem, but I can't figure out why it would work in this manner (only blocking a small subset of my queries, and not IP-based). Here are some sample queries I am trying to cURL: http://search.twitter.com/search.json?q=batteryoperatedcandles.net&rpp=100&since_id=9431322892177408&since=&until= http://search.twitter.com/search.json?q=carnationbreakfastessentials.com&rpp=100&since_id=&since=2010-12-14&until= http://search.twitter.com/search.json?q=apple.com&rpp=100&since_id=&since=2010-12-14&until= The first two don't work unless I change my UserAgent to something else, but the last one works no matter what. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk