Thanks all - I managed to get this fixed. The main thing to remember is to build in some kind of delay between successive API calls. In my case I made it 200ms with linear backoff on retries. I know this is documented for the streaming api's - wasn't aware that it also holds for the REST API.
Thanks Tom for your help - greatly appreciated! I'm getting the full 3200 tweets now. The thing to remember here is, include_rts doesn't default to true, so if you don't set it, retweets are omitted, but still counted. This is documented, I just didn't rtfm properly ... Adriaan Pelzer putting you in touch with your crowds http://www.wewillraakyou.com <http://www.wewillraakyou.com>twitter: http://www.twitter.com/adriaan_pelzer linkedIn: http://uk.linkedin.com/pub/adriaan-pelzer/4/874/860/ skype: adriaan_pelzer <http://uk.linkedin.com/pub/adriaan-pelzer/4/874/860/> +4478 7978 1743 On Mon, Jul 4, 2011 at 11:44 AM, Adriaan Pelzer <adri...@wewillraakyou.com>wrote: > Ah - thanks. That makes sense. > > Still, having to retry on empty results doesn't make sense, or is this > normal behaviour when not using pagination? > > Adriaan Pelzer > > putting you in touch with your crowds > http://www.wewillraakyou.com > <http://www.wewillraakyou.com>twitter: > http://www.twitter.com/adriaan_pelzer > linkedIn: http://uk.linkedin.com/pub/adriaan-pelzer/4/874/860/ > skype: adriaan_pelzer > <http://uk.linkedin.com/pub/adriaan-pelzer/4/874/860/> > +4478 7978 1743 > > > > On Mon, Jul 4, 2011 at 11:37 AM, Tom van der Woerdt <i...@tvdw.eu> wrote: > >> Check for whether you include retweets. >> >> Tom >> >> >> >> On 7/4/11 12:34 PM, Adriaan Pelzer wrote: >> >> I'm not paging, but rather using a combination of max_id and count=200. >> Also - not getting nearly 200 every time, mostly below 100 >> >> Adriaan Pelzer >> >> putting you in touch with your crowds >> http://www.wewillraakyou.com >> twitter: http://www.twitter.com/adriaan_pelzer >> linkedIn: http://uk.linkedin.com/pub/adriaan-pelzer/4/874/860/ >> skype: adriaan_pelzer >> +4478 7978 1743 >> >> >> >> On Mon, Jul 4, 2011 at 11:18 AM, Correa Denzil <mcen...@gmail.com> wrote: >> >>> I am unable to reproduce the following error. I am able to paginate 200 >>> tweets for 16 times. >>> >>> I am using OAuth for authentication. >>> >>> --Regards, >>> Denzil >>> >>> >>> >>> >>> On Mon, Jul 4, 2011 at 2:43 PM, Adriaan Pelzer < >>> adri...@wewillraakyou.com> wrote: >>> >>>> We realise that we're not supposed to get 3200 (but 3200 minus spammed >>>> tweets), but we're only getting at most 2000 tweets. We don't want to retry >>>> too aggressively, it eats into the rate limits, and we're concerned the >>>> account might be flagged as abusive. >>>> >>> >>> -- >>> Twitter developer documentation and resources: >>> https://dev.twitter.com/doc >>> API updates via Twitter: https://twitter.com/twitterapi >>> Issues/Enhancements Tracker: >>> https://code.google.com/p/twitter-api/issues/list >>> Change your membership to this group: >>> https://groups.google.com/forum/#!forum/twitter-development-talk >>> >> >> -- >> Twitter developer documentation and resources: >> https://dev.twitter.com/doc >> API updates via Twitter: https://twitter.com/twitterapi >> Issues/Enhancements Tracker: >> https://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> https://groups.google.com/forum/#!forum/twitter-development-talk >> >> >> -- >> Twitter developer documentation and resources: >> https://dev.twitter.com/doc >> API updates via Twitter: https://twitter.com/twitterapi >> Issues/Enhancements Tracker: >> https://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> https://groups.google.com/forum/#!forum/twitter-development-talk >> > > -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
