> Is this a known problem or are we getting throttled? You are likely getting rate limited. http://dev.twitter.com/pages/streaming_api_concepts#filter-limiting "Limiter periodicity is aligned with statuses/sample sampling periodicity" Basically, this means that if you're predicates would return MORE than the sample hose (at whatever level your account has), then you will be rate limited. The default is now 1% I believe...and 10% if you've been granted "garden hose" role. So if your predicates would capture more than 1% of all tweets (10% for gardenhose role), you will be rate limited. The wording isn't quite like that, but that is what we were experiencing.
> If we are throttled, what should we do to get back to normal? A few routes for you. 1 Contact twitter to see if you can move to a higher tier (ie gardenhose). See http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting 2 Try and send the least noisy subset possible that will still capture the data for your users (if possible). Depending on how your predicates are provided, you can possibly reduce them down to the least noisy subset that would still capture most of what you want. 3 Pay for higher access roles or the full hose. You'll need to work with twitter on this, and I'm honestly not sure where they stand on continuing this model given the gnip relationship. 4 Contact http://gnip.com/twitter (i don't work for them) and see if the pricing works out for you > And should we be able to detect throttling ourselves? Yes. You will get "limit" messages in the stream. http://dev.twitter.com/pages/streaming_api_concepts#parsing-responses "Track streams may also contain limitation notices, where the integer track is an enumeration of statuses that, since the start of the connection, matched the track predicate but were rate limited." Notice that it is "since the start of the connection"...which is a bit strange and threw me at first. James On Tue, May 31, 2011 at 12:09 AM, Jacob Friis Saxberg <[email protected]> wrote: > We are using statuses/filter.json and have lately experienced that > some tweets are not delivered to our consumer. > > Is this a known problem or are we getting throttled? > > If we are throttled, what should we do to get back to normal? > And should we be able to detect throttling ourselves? > > Jacob > > -- > Twitter developer documentation and resources: https://dev.twitter.com/doc > API updates via Twitter: https://twitter.com/twitterapi > Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > https://groups.google.com/forum/#!forum/twitter-development-talk > -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
