[twitter-dev] Re: API Limit of 150 is Obsolete
I've discovered that the API rate limit is 450 per hour for pages/ cursors within a followers_ids or friends_ids call, if that helps. But I really think that increasing the API rate limit for basic HTML auth is a bad idea - let's make oAuth work! On Jan 20, 3:04 pm, Josh Roesslein jroessl...@gmail.com wrote: Yeah an increase in API requests would be nice to have with the addition of new API features. I would almost like a solution where twitter sets a guaranteed hits/hour soft limit. By soft limit I mean if you go above this limit you may be rate limited if the twitter cluster is currently under heavy load or you are being too rough with the API. If the cluster has unused capacity, why start limiting users? For non-whitelisted applications a guarantee of 250 would be nice. Whiltelisted apps would get a higher guaranteed limit still to meet their demands. I'm sure twitter has floated this idea around. Not sure how big of a technical hurdle it would be to implement. Just my two cents on the subject of API rate limits. Josh On Wed, Jan 20, 2010 at 4:48 PM, Eric Woodward e...@nambu.com wrote: I will come straight to the point: we need to an increase to the API limit to properly implement Twitter within a desktop client application given the addition of: 1) three retweets timelines; 2) checking the account's saved searches; and 3) up to 10-20 Twitter Lists timelines. Twitter Lists alone are causing real problems if a user follows more than 5 or so. We cant poll Twitter List subscriptions with one API call that combines them altogether, which we could then split apart client-side with some attached meta-data. That alone would have been a big help, and without it we are left polling each List as if it was a separate timeline, since that is what they are. Implementing proper Lists management is a non-starter within this limit, so is regular confirmation of a relationship between two users when asked for by the user (on Lists or search results). There is simply a lot of stuff I cannot do properly that is standard on twitter.com, all because I am subject to the API limit while twitter.com is not. Users simply do not understand this distinction in possibilities. I would like to formally ask on behalf of all client developers that the API limit increase to 250, from 150, for all applications whether they use OAuth or HTTP Basic Authentication. We are simply not able to implement Twitter properly within a limit of 150, but dont need a lot more, only another 100-200 API calls or so. If Twitter can even technically contemplate a 10x API limit increase to 1,500 for OAuth applications, surely an increase to 250 based on the addition of core features like official retweets and Lists is a reasonable request. A limit of 150 is simply obsolete, and has been for a long time. I do not want to wait for the UX repairs around OAuth for desktop applications, and I dont like being forced into OAuth sooner than we are ready just because we need the extra API hits just to do basic features properly. And besides, that was announced as two weeks away three weeks ago. I dont want to wait any longer. I want to properly implement the basics, like Lists polling, now. This is a considered email because I care about the quality of our Twitter implementation and I care about the Twitter ecosystem. I would appreciate a considered reply. --ejw Eric Woodward Email: e...@nambu.com
[twitter-dev] Re: API Limit of 150 is Obsolete
On top of all that, AFAIK the 1500 limit for OAuth is still vaporware at this point, so everybody is capped at 150. To inform the discussion, I wonder if Twitter could share any figures like what's the actual API use distribution? Like what combination of users/apps hit the cap regularly and cause massive load? If it was an equal distribution (i.e most users/apps are around the same level) that gives them heavy load, then I would see why they need to be careful about raising limits (any increase would bring more fail whales). But I suspect that it's highly asymmetrical... i.e there are very few users/apps who actually cause any meaningful load. Another hunch: desktop apps are negligible and the real load comes from web apps who spider asynchronously 24/7. Should the load be differentiated across client and web apps? Client apps are typically only one user per device at a time, whereas the web app may be spidering on behalf of who knows how many people. On Jan 20, 5:48 pm, Eric Woodward e...@nambu.com wrote: I will come straight to the point: we need to an increase to the API limit to properly implement Twitter within a desktop client application given the addition of: 1) three retweets timelines; 2) checking the account's saved searches; and 3) up to 10-20 Twitter Lists timelines. Twitter Lists alone are causing real problems if a user follows more than 5 or so. We cant poll Twitter List subscriptions with one API call that combines them altogether, which we could then split apart client-side with some attached meta-data. That alone would have been a big help, and without it we are left polling each List as if it was a separate timeline, since that is what they are. Implementing proper Lists management is a non-starter within this limit, so is regular confirmation of a relationship between two users when asked for by the user (on Lists or search results). There is simply a lot of stuff I cannot do properly that is standard on twitter.com, all because I am subject to the API limit while twitter.com is not. Users simply do not understand this distinction in possibilities. I would like to formally ask on behalf of all client developers that the API limit increase to 250, from 150, for all applications whether they use OAuth or HTTP Basic Authentication. We are simply not able to implement Twitter properly within a limit of 150, but dont need a lot more, only another 100-200 API calls or so. If Twitter can even technically contemplate a 10x API limit increase to 1,500 for OAuth applications, surely an increase to 250 based on the addition of core features like official retweets and Lists is a reasonable request. A limit of 150 is simply obsolete, and has been for a long time. I do not want to wait for the UX repairs around OAuth for desktop applications, and I dont like being forced into OAuth sooner than we are ready just because we need the extra API hits just to do basic features properly. And besides, that was announced as two weeks away three weeks ago. I dont want to wait any longer. I want to properly implement the basics, like Lists polling, now. This is a considered email because I care about the quality of our Twitter implementation and I care about the Twitter ecosystem. I would appreciate a considered reply. --ejw Eric Woodward Email: e...@nambu.com
Re: [twitter-dev] Re: API Limit of 150 is Obsolete
Another hunch: desktop apps are negligible and the real load comes from web apps who spider asynchronously 24/7. Should the load be differentiated across client and web apps? Client apps are typically only one user per device at a time, whereas the web app may be spidering on behalf of who knows how many people. The problem here is distinguishing the two. OAuth doesn't (and I was told this by one of the people on the OAuth committee) specifically allow you to unambiguously and securely identify an application just because it has a certain app key, and Twitter's Basic Auth implementation uses source keys pretty much purely cosmetically. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- BOND THEME NOW PLAYING: Die Another Day --
[twitter-dev] Re: API Limit of 150 is Obsolete
On Jan 20, 4:50 pm, Cameron Kaiser spec...@floodgap.com wrote: The problem here is distinguishing the two. OAuth doesn't (and I was told this by one of the people on the OAuth committee) specifically allow you to unambiguously and securely identify an application just because it has a certain app key Huh? Can you translate this into either English or pseudo-code? I fill out a form. The app gets a name, which must be unique. And I choose between a desktop exclusive-or server app (PIN workflow exclusive-or callback workflow) with a radio button. I get a consumer key and consumer secret, also, I'm assuming, unique. So now I run that app. I send packets back and forth between the app / my IP address and Twitter's servers / Twitter's IP addresses. Are you saying Twitter can't distinguish my oAuth app running on my IP address from another oAuth app running on a different IP address? You don't know where I am and what I'm running? You don't know which of 30 users of my app from different machines is acting abusively?
Re: [twitter-dev] Re: API Limit of 150 is Obsolete
Not really that hard to distinguish between 5 IPs making 20k API hits and 20k IPs making 5 API hits each... Abraham On Wed, Jan 20, 2010 at 16:50, Cameron Kaiser spec...@floodgap.com wrote: Another hunch: desktop apps are negligible and the real load comes from web apps who spider asynchronously 24/7. Should the load be differentiated across client and web apps? Client apps are typically only one user per device at a time, whereas the web app may be spidering on behalf of who knows how many people. The problem here is distinguishing the two. OAuth doesn't (and I was told this by one of the people on the OAuth committee) specifically allow you to unambiguously and securely identify an application just because it has a certain app key, and Twitter's Basic Auth implementation uses source keys pretty much purely cosmetically. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- BOND THEME NOW PLAYING: Die Another Day -- -- Abraham Williams | Moved to Seattle | May cause email delays Project | Intersect | http://intersect.labs.poseurtech.com Hacker | http://abrah.am | http://twitter.com/abraham This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States
Re: [twitter-dev] Re: API Limit of 150 is Obsolete
Another hunch: desktop apps are negligible and the real load comes from web apps who spider asynchronously 24/7. Should the load be differentiated across client and web apps? Client apps are typically only one user per device at a time, whereas the web app may be spidering on behalf of who knows how many people. The problem here is distinguishing the two. OAuth doesn't (and I was told this by one of the people on the OAuth committee) specifically allow you to unambiguously and securely identify an application just because it has a certain app key, and Twitter's Basic Auth implementation uses source keys pretty much purely cosmetically. Not really that hard to distinguish between 5 IPs making 20k API hits and 20k IPs making 5 API hits each... But it's not a guaranteed one either. Also, it doesn't allow you to distinguish between equally popular services, one which might be kosher and the other not. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- BOND THEME NOW PLAYING: The James Bond Theme from Dr. No -
Re: [twitter-dev] Re: API Limit of 150 is Obsolete
The problem here is distinguishing the two. OAuth doesn't (and I was told this by one of the people on the OAuth committee) specifically allow you to unambiguously and securely identify an application just because it has a certain app key Huh? Can you translate this into either English or pseudo-code? I fill out a form. The app gets a name, which must be unique. And I choose between a desktop exclusive-or server app (PIN workflow exclusive-or callback workflow) with a radio button. I get a consumer key and consumer secret, also, I'm assuming, unique. Didn't we just get done with a thread where people complained, correctly, that a desktop app containing a consumer key/secret can't keep those things secret? -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Of course, what I really want is total world domination. -- Linus Torvalds -
[twitter-dev] Re: API Limit of 150 is Obsolete
I would also like to request the API limit to be raised as soon as possible. I am not able to roll out an update for my desktop Twitter client, simply because the API limit is too low for the features I have implemented. The new version of my client utilizes Lists related APIs, which consumes a lot of API calls. Moreover, multiple API calls are currently required to get a chain of replies (conversation), which consumes significant number of API calls. The previously discussed features were implemented, because there was an official announcement to change the API limit to 1500. I know it is expected to happen in few weeks, but it would be good to know the estimated date for this, so the developers can make appropriate decisions on the application deployment timeline. -- Aki On Jan 21, 9:48 am, Eric Woodward e...@nambu.com wrote: I will come straight to the point: we need to an increase to the API limit to properly implement Twitter within a desktop client application given the addition of: 1) three retweets timelines; 2) checking the account's saved searches; and 3) up to 10-20 Twitter Lists timelines. Twitter Lists alone are causing real problems if a user follows more than 5 or so. We cant poll Twitter List subscriptions with one API call that combines them altogether, which we could then split apart client-side with some attached meta-data. That alone would have been a big help, and without it we are left polling each List as if it was a separate timeline, since that is what they are. Implementing proper Lists management is a non-starter within this limit, so is regular confirmation of a relationship between two users when asked for by the user (on Lists or search results). There is simply a lot of stuff I cannot do properly that is standard on twitter.com, all because I am subject to the API limit while twitter.com is not. Users simply do not understand this distinction in possibilities. I would like to formally ask on behalf of all client developers that the API limit increase to 250, from 150, for all applications whether they use OAuth or HTTP Basic Authentication. We are simply not able to implement Twitter properly within a limit of 150, but dont need a lot more, only another 100-200 API calls or so. If Twitter can even technically contemplate a 10x API limit increase to 1,500 for OAuth applications, surely an increase to 250 based on the addition of core features like official retweets and Lists is a reasonable request. A limit of 150 is simply obsolete, and has been for a long time. I do not want to wait for the UX repairs around OAuth for desktop applications, and I dont like being forced into OAuth sooner than we are ready just because we need the extra API hits just to do basic features properly. And besides, that was announced as two weeks away three weeks ago. I dont want to wait any longer. I want to properly implement the basics, like Lists polling, now. This is a considered email because I care about the quality of our Twitter implementation and I care about the Twitter ecosystem. I would appreciate a considered reply. --ejw Eric Woodward Email: e...@nambu.com
