Thanks for the quick feedback, John.  I'm checking it out and will let
you know what happens.

On Aug 6, 10:15 am, John Kalucki <> wrote:
> Josh,
> It seems that you can accomplish most of your goals by using the /
> track feature in the Streaming API. You can then make far fewer calls
> to the Search API to cover dynamic cases, or fill in whatever else is
> left. I suspect you'll have a better user experience with far fewer
> coding and rate limiting hassles.
> Let me know if you have any questions or issues with the Streaming
> API, or just post to this list.
> -John Kalucki
> Services, Twitter Inc.
> On Aug 5, 12:11 pm, Josh Shabtai <> wrote:
> > Hi there.  I was just about to start a thread on this topic myself, as
> > I've developed a Web application that seems to be running into some
> > issues related to the search API.
> > A disclaimer: I'm pretty inexperienced as a developer, so apologies
> > for any redundancy and/or misuse of terminology.
> > I recently launched, a Web application/parody
> > designed to apply relatively sophisticated search and analytical tools
> > to the basest of subjects (the URL is a dead giveaway).  We've been
> > whitelisted, but recently, we experienced a surge in traffic and usage
> > that illuminated potential issues with our ability to access the
> > search and REST APIs.
> > Some background on the app, before going into my questions... It
> > revolves around a few key modules:
> >     * A search engine that lets users compare the number of people
> > talking about #2 in the last 24 hours (according to a handful of
> > predetermined phrases) against any other keywords
> >     * A real-time feed that pulls in live tweets using the same set of
> > predetermined keywords
> >     * A leaderboard mechanism that identifies the most active keyword
> > 'abusers' on Twitter and scores their profiles according to frequency
> > of keyword usage
> > Now, even though our taste in subject is questionable, we've made it a
> > priority to ensure that our search engine and leaderboard are as
> > accurate and useful as possible (ideally, we'd like to extend these
> > tools to other applications).  To do this, we're making up to 19,000
> > search API calls/hour.
> > On the search engine front, we've built a database and cron job that
> > stores user-inputted keywords and publicly trending words that max out
> > at 1,500 results (around 1,000 different words).  Then, we make a
> > search API call against each word every 5 minutes to ensure reasonably
> > accurate results.
> > Our real-time feed makes a live search API call every 10 seconds (360
> > API calls/hour) and we also make search API calls related to
> > approximately 20 distinct #2-focused keywords every 5 minutes (240
> > search API calls/hour).
> > After our initial surge in traffic, we've noticed some strange issues,
> > all of which seem to relate to us being unable to access data.  So,
> > after that long-winded explanation, here are my questions:
> >     * First of all, are we within an acceptable rate limit for Search
> > API? What's the ballpark?
> >     * We are using both REST and Search API calls to access Twitter
> > data. Does using both simultaneously (as we do) cause any problems you
> > are aware of? Do you have any known restrictions we may have missed?
> >     * We make quite a few search API requests consecutively.  (For
> > example, we will make simultaneous calls against various keywords.)
> > Is there a timing restriction that we should be aware of?
> >     * Our site continually updates users who are talking about our
> > topic. Thus the page is almost always dynamic. However, almost always
> > when we refresh the page or reload the page we have problems fetching
> > data which of course distorts the page and often does not load
> > correctly. Could this be the result of using both methods REST and
> > Search to acquire data and doing it from the same IP address? If yes,
> > how do we solve this? If not, any ideas why this is happening?
> > Thanks for the time and apologies for subjecting you to toilet humor.

Reply via email to