I am building an application that monitors tweets about movies(for now
with... other interesting things planned). I have my id whitelisted
but I want to avoid overusing it.
The challenge that I face is that ideally I want to make full use of
the opportunity to retrieve 100 tweets per call and for that I need
information on the frequency with which users are tweeting about a
movie and then set my call frequency (to call twitter search api)
accordingly so that I maximize the number of tweets returned per call
Since I presume there is no way to know what frequency is someone
tweeting about a movie - I need help is what is the best way to
optimize for such a situation.
The challenge is complicated by the fact that users tweet about
different movies at different rates and the rates generally decrease
I have tried combining searches - but the challenge is that lets say I
(Movie A OR Movie B)
(Movie C OR Movie D)
it could be the case that people tweet about Movie A & B a lot and
litle to none about C or D or there is a combination in which they
continue to tweet about A but not about B - So I still can end up in a
situation where I am not optimizing my calls. Also situations such as
Oscars can dramatically change what people talk about even about
movies out months ago.
I have thought of writing something such as a variable frequency
caller that can check the frequency of tweets for the last 3 calls in
order to appreciate the frequency of tweets for a given search and
then continuously vary the time between calls so that I can get as
close to 100 tweets as possible in a call.
Any ideas suggestions that can suggest ways to alleviate the above
will be highly appreciated.