Right, that makes sense. On Mon, Sep 20, 2010 at 11:52 AM, Tom van der Woerdt <[email protected]> wrote:
> Of course they do :-) Each server gets an IP and gets 20000 requests per > hour. With 20 servers that means 400000 requests per hour. Besides, I'd > assume that these services spread their requests properly and don't > update every hour. Just because you have 20000 requests per hour doesn't > mean you can use them as much as you want. (API Terms of Service, I think.) > > Tom > > > On 9/20/10 5:44 PM, Vijay wrote: > > OK Tom, will queue it and spread it, that should give me more room. > > > > But that still doesn't answer the question - how do the big companies > > manage? > > Do they have multiple IPs? > > > > On Mon, Sep 20, 2010 at 11:41 AM, Tom van der Woerdt <[email protected] > > <mailto:[email protected]>> wrote: > > > > Just don't and you'll be fine :-) Just queue the requests and spread > > them over a few hours. It may also help to setup a Twitter account, > as > > it will allow you to make 350 requests per hour instead. > > > > Tom > > > > > > On 9/20/10 5:38 PM, Vijay wrote: > > > > > > "You really shouldn't be planning for the case where every single > > Twitter > > > user uses your application" > > > > > > Haha, I agree. Its just that I took a few thousand users > > randomly, and > > > was playing > > > with their publicly available stats, tweets etc and quickly hit > > the rate > > > limit. Hence > > > this question. > > > > > > On Mon, Sep 20, 2010 at 11:21 AM, Tom van der Woerdt <[email protected] > > <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > > > You really shouldn't be planning for the case where every > > single Twitter > > > user uses your application ;-) When you get to 100 users, you > > request > > > whitelisting. When you get to 20000 users, your server will > have a > > > problem handling all the information and you'll need to get a > > second > > > server anyway. > > > > > > Tom > > > > > > > > > On 9/20/10 5:17 PM, Vijay wrote: > > > > I am not sure about 20k, but 150 is miniscule. > > > > If I am collecting stats on a bunch of users every hour for > > example, I > > > > can only > > > > collect on 150 users, which is tiny, compared to 140 million > > users > > > > Twitter has. > > > > > > > > How do the big companies manage? For example, twittercounter > > > claims to > > > > have stats on 10 million users...that is a LOT of data. > > > > > > > > On Mon, Sep 20, 2010 at 11:14 AM, Tom van der Woerdt > > <[email protected] <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>> > > > > <mailto:[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>>> wrote: > > > > > > > > 20k per IP is a lot. It means that a server should make > more > > > than 5 > > > > requests per second to hit the limit, which is a lot. > > > > > > > > About your options: no idea. Just make sure to use the > > proper > > > functions > > > > and try not to hit the limits? :-) 150 is a lot as well, > > most > > > Desktop > > > > clients don't hit it (didn't, until Lists came ^^). > > > > > > > > Tom > > > > > > > > > > > > On 9/20/10 4:56 PM, Vijay wrote: > > > > > Hi, > > > > > > > > > > Newbie question, so please bear with me. > > > > > > > > > > I am experimenting with twitter API, but quickly found > > > myself hitting > > > > > the rate limit > > > > > (150 without authentication, correct?) > > > > > > > > > > How do the big sites get over the rate limit > (tweetstats, > > > > twittercounter > > > > > etc)? > > > > > Twitter's documentation says they can white list IPs, > but > > > would still > > > > > allow only > > > > > 20k requests per hour. Also, they would only white > list > > > apps that are > > > > > already in > > > > > production. So if I just want to experiment / learn, > this > > > > wouldn't work. > > > > > > > > > > What are my options? There is no way I can have > > multiple IPs. > > > > > > > > > > Vijay. > > > > > > > > > > -- > > > > > Twitter developer documentation and resources: > > > > http://dev.twitter.com/doc > > > > > API updates via Twitter: http://twitter.com/twitterapi > > > > > Issues/Enhancements Tracker: > > > > > http://code.google.com/p/twitter-api/issues/list > > > > > Change your membership to this group: > > > > > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > > > -- > > > > Twitter developer documentation and resources: > > > > http://dev.twitter.com/doc > > > > API updates via Twitter: http://twitter.com/twitterapi > > > > Issues/Enhancements Tracker: > > > > http://code.google.com/p/twitter-api/issues/list > > > > Change your membership to this group: > > > > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > > > > > > > -- > > > > Twitter developer documentation and resources: > > > http://dev.twitter.com/doc > > > > API updates via Twitter: http://twitter.com/twitterapi > > > > Issues/Enhancements Tracker: > > > > http://code.google.com/p/twitter-api/issues/list > > > > Change your membership to this group: > > > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > -- > > > Twitter developer documentation and resources: > > > http://dev.twitter.com/doc > > > API updates via Twitter: http://twitter.com/twitterapi > > > Issues/Enhancements Tracker: > > > http://code.google.com/p/twitter-api/issues/list > > > Change your membership to this group: > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > > > > -- > > > Twitter developer documentation and resources: > > http://dev.twitter.com/doc > > > API updates via Twitter: http://twitter.com/twitterapi > > > Issues/Enhancements Tracker: > > > http://code.google.com/p/twitter-api/issues/list > > > Change your membership to this group: > > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > -- > > Twitter developer documentation and resources: > > http://dev.twitter.com/doc > > API updates via Twitter: http://twitter.com/twitterapi > > Issues/Enhancements Tracker: > > http://code.google.com/p/twitter-api/issues/list > > Change your membership to this group: > > http://groups.google.com/group/twitter-development-talk?hl=en > > > > > > -- > > Twitter developer documentation and resources: > http://dev.twitter.com/doc > > API updates via Twitter: http://twitter.com/twitterapi > > Issues/Enhancements Tracker: > > http://code.google.com/p/twitter-api/issues/list > > Change your membership to this group: > > http://groups.google.com/group/twitter-development-talk?hl=en > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk?hl=en > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
