I would do anything (including paying good amounts of money) to be
able to purchase access to older datasets that I could transfer to my
database through non-rest-api methods. I'm envisioning being able to
download a CSV or SQL file that I could merge with my database easily,
but only have to make a single request to the server to get a month of
data. I'd sign agreements and pay money for such.
On Jul 28, 12:03 pm, owkaye <owk...@gmail.com> wrote:
> I agree with you Dave. I have several thought about new
> services based on searching Twitter's historical data.
> Unfortunately my ideas appear to be getting less and less
> Twitter claims to have all its data stored in disk-based
> databases from what I understand ... yet without access to
> this data it is worthless.
> It seems to me they could allow searches of this historical
> data via a new "History API" then let us cache the results
> on our own servers. Most of the services I've conceived
> would do this infrequently -- never in real time -- and
> would not impact their existing "cached" server data because
> this historical data would exist on separate data storage
> servers ... theoretically anyways.
> > I am a bit concerned. I remember at one point it being
> > between 30-45 days. Now it seems to be getting smaller by
> > about 1-day per month. Last month it was closer to 10
> > days.
> > Is it basically going to keep getting smaller and smaller
> > until we get V2 of the API, or will we be forced to all
> > use only streaming services and then locally cache
> > everything that we'd want to search for any time period?
> > I know there are a LOT of problems inherent in the
> > massive scaling out of Twitter, and this is just a
> > symptom of them- but at the same time I can only imagine
> > how unusable Google would be if you only had a 7-day
> > window to Search in, and couldn't get any content made
> > prior to that. Very worried about this soon being a 2-3
> > day window.
> > dave