Matt & Doug, Here's some more information to help fingerprint search requests:
The MGTwitterEngine library sends the following X headers by default: X-Twitter-Client: MGTwitterEngine X-Twitter-Client-Url: http://mattgemmell.com/source X-Twitter-Client-Version: 1.0 These can be overridden by the developer. For Twitterrific, we're using: X-Twitter-Client: Twitterrific X-Twitter-Client-Url: http://iconfactory.com/twitterrific X-Twitter-Client-Version: iPhone 2.0 In addition, connections initiated from an iPhone will likely be going through CFNetwork. This API adds a user agent that contains the application name, version as well as the version of the framework and operating system. For example: User-Agent: Twitterrific/2.1a3 CFNetwork/445.6 Darwin/10.0.0d3 Hope this helps! -ch On Jun 16, 2:05 pm, Matt Sanford <m...@twitter.com> wrote: > Hi there, > > While all of this flame is keeping my feet warm it's not really > productive. This isn't Slashdot comments, let's try and remain on > topic rather the getting into RFC debates. To be even more explicit > than my previous email: Use the user-agent. Referrer will be taken > care of by browsers and I see as a fallback for client-side JSON users > rather than a replacement for a user-agent. > > The subsequent reply from Michael Ivey about how this helps is > dead on. With no context at all I'm forced to block all of ECS/ > AppEngine/Yahoo Pipes is one person misbehaves. Nobody likes that. > Since search is not authenticated OAuth does not really help here. We > may be forced to make search authenticated if we can't find a > reasonable way to sort the good from the bad. This is a first attempt > at helping us cut out poorly build spam scripts and shorten the time I > spend researching each abuser. It saves time and lets me fix more > bugs, assuming I don't spend the newly saved time in RFC debates, that > is :) > > Thanks; > – Matt Sanford / @mzsanford > Twitter Dev > > On Jun 16, 2009, at 12:39 PM, Stuart wrote: > > > > > 2009/6/16 Naveen Kohli <naveenko...@gmail.com> > > Redefining HTTP spec, eh :-) > > Whatever makes twitter boat float. Lets hope for the best. Just > > concerned that some firewalls or proxies tend to remove "referrer". > > > What a completely ridiculous thing to say. It's not "redefining" > > anything. If Twitter want to require something in order to access > > their service they absolutely have that right. It's not like they're > > saying every HTTP server should start requiring these headers. > > > It's true that some firewalls and proxies remove the referrer > > header, and some also remove the user agent header. > > > I'm somewhat unclear on exactly how this stuff is supposed to help. > > If an application sets out to abuse the system they'll simply set > > the headers so they look like a normal browser. I don't see what > > purpose requiring these headers to be something useful will actually > > serve. IMHO you might as well "require" the source parameter for all > > API requests that use basic auth which is simple for all apps to > > implement; OAuth clearly carries identification with it already. > > > -Stuart > > > -- > >http://stut.net/projects/twitter > > > On Tue, Jun 16, 2009 at 1:05 PM, Stuart <stut...@gmail.com> wrote: > > > It's optional in the HTTP spec, but mandatory for the Twitter Search > > API. I don't see a problem with that. > > > Doug: Presumably the body of the 403 response will contain a suitable > > descriptive error message in the usual format? > > > -Stuart > > > -- > >http://stut.net/projects/twitter > > > 2009/6/16 Naveen Kohli <naveenko...@gmail.com>: > > > Why would you make decision based on "Referrer" which is an > > OPTIONAL header > > > field in HTTP protocol? Making decision based on something that is > > > "REQUIRED" may be more appropriate. > > > > On Tue, Jun 16, 2009 at 12:33 PM, Doug Williams <d...@twitter.com> > > wrote: > > > >> Hi all, > > >> The Search API will begin to require a valid HTTP Referrer, or at > > the very > > >> least, a meaningful and unique user agent with each request. Any > > request not > > >> including this information will be returned a 403 Forbidden > > response code by > > >> our web server. > > > >> This change will be effective within the next few days, so please > > check > > >> your applications using the Search API and make any necessary > > code changes. > > > >> Thanks, > > >> Doug > > > > -- > > > Naveen K Kohli > > >http://www.netomatix.com > > > -- > > Naveen K Kohli > >http://www.netomatix.com
