Of course they do :-) Each server gets an IP and gets 20000 requests per hour. With 20 servers that means 400000 requests per hour. Besides, I'd assume that these services spread their requests properly and don't update every hour. Just because you have 20000 requests per hour doesn't mean you can use them as much as you want. (API Terms of Service, I think.)
Tom On 9/20/10 5:44 PM, Vijay wrote: > OK Tom, will queue it and spread it, that should give me more room. > > But that still doesn't answer the question - how do the big companies > manage? > Do they have multiple IPs? > > On Mon, Sep 20, 2010 at 11:41 AM, Tom van der Woerdt <[email protected] > <mailto:[email protected]>> wrote: > > Just don't and you'll be fine :-) Just queue the requests and spread > them over a few hours. It may also help to setup a Twitter account, as > it will allow you to make 350 requests per hour instead. > > Tom > > > On 9/20/10 5:38 PM, Vijay wrote: > > > > "You really shouldn't be planning for the case where every single > Twitter > > user uses your application" > > > > Haha, I agree. Its just that I took a few thousand users > randomly, and > > was playing > > with their publicly available stats, tweets etc and quickly hit > the rate > > limit. Hence > > this question. > > > > On Mon, Sep 20, 2010 at 11:21 AM, Tom van der Woerdt <[email protected] > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > You really shouldn't be planning for the case where every > single Twitter > > user uses your application ;-) When you get to 100 users, you > request > > whitelisting. When you get to 20000 users, your server will have a > > problem handling all the information and you'll need to get a > second > > server anyway. > > > > Tom > > > > > > On 9/20/10 5:17 PM, Vijay wrote: > > > I am not sure about 20k, but 150 is miniscule. > > > If I am collecting stats on a bunch of users every hour for > example, I > > > can only > > > collect on 150 users, which is tiny, compared to 140 million > users > > > Twitter has. > > > > > > How do the big companies manage? For example, twittercounter > > claims to > > > have stats on 10 million users...that is a LOT of data. > > > > > > On Mon, Sep 20, 2010 at 11:14 AM, Tom van der Woerdt > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > > > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > > > 20k per IP is a lot. It means that a server should make more > > than 5 > > > requests per second to hit the limit, which is a lot. > > > > > > About your options: no idea. Just make sure to use the > proper > > functions > > > and try not to hit the limits? :-) 150 is a lot as well, > most > > Desktop > > > clients don't hit it (didn't, until Lists came ^^). > > > > > > Tom > > > > > > > > > On 9/20/10 4:56 PM, Vijay wrote: > > > > Hi, > > > > > > > > Newbie question, so please bear with me. > > > > > > > > I am experimenting with twitter API, but quickly found > > myself hitting > > > > the rate limit > > > > (150 without authentication, correct?) > > > > > > > > How do the big sites get over the rate limit (tweetstats, > > > twittercounter > > > > etc)? > > > > Twitter's documentation says they can white list IPs, but > > would still > > > > allow only > > > > 20k requests per hour. Also, they would only white list > > apps that are > > > > already in > > > > production. So if I just want to experiment / learn, this > > > wouldn't work. > > > > > > > > What are my options? There is no way I can have > multiple IPs. > > > > > > > > Vijay. > > > > > > > > -- > > > > Twitter developer documentation and resources: > > > http://dev.twitter.com/doc > > > > API updates via Twitter: http://twitter.com/twitterapi > > > > Issues/Enhancements Tracker: > > > > http://code.google.com/p/twitter-api/issues/list > > > > Change your membership to this group: > > > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > -- > > > Twitter developer documentation and resources: > > > http://dev.twitter.com/doc > > > API updates via Twitter: http://twitter.com/twitterapi > > > Issues/Enhancements Tracker: > > > http://code.google.com/p/twitter-api/issues/list > > > Change your membership to this group: > > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > > > > -- > > > Twitter developer documentation and resources: > > http://dev.twitter.com/doc > > > API updates via Twitter: http://twitter.com/twitterapi > > > Issues/Enhancements Tracker: > > > http://code.google.com/p/twitter-api/issues/list > > > Change your membership to this group: > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > -- > > Twitter developer documentation and resources: > > http://dev.twitter.com/doc > > API updates via Twitter: http://twitter.com/twitterapi > > Issues/Enhancements Tracker: > > http://code.google.com/p/twitter-api/issues/list > > Change your membership to this group: > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > -- > > Twitter developer documentation and resources: > http://dev.twitter.com/doc > > API updates via Twitter: http://twitter.com/twitterapi > > Issues/Enhancements Tracker: > > http://code.google.com/p/twitter-api/issues/list > > Change your membership to this group: > > http://groups.google.com/group/twitter-development-talk?hl=en > > -- > Twitter developer documentation and resources: > http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk?hl=en > > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk?hl=en -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
