----- "Raffi Krikorian" <ra...@twitter.com> wrote:
> hi dewald.
> we obviously feel that users want the most relevant tweets first (the
> use of "popular" is unfortunate here). and the web interface of
> search.twitter.com has begun an evolution in that direction.
> it's still unclear what Twitter is going to do with the API (hence the
> silence), however, to go with your argument: "time indexed" search is,
> potentially, something a third party service could do. we do provide
> the streaming API to get much-better-than-search-real-time results.
Yes, and the real-time work I'm doing I do with Streaming. Building your own
"time indexed search" on top of Streaming, however, has an *extensive*
investment requirement on the part of said third-party services. You've got
Firehose-scale bandwidth requirements, Cassandra-scale persistence
requirements, and Hadoop-scale algorithmic requirements just for openers.
It's an *extremely* competitive marketplace. Hell, there are profitable
businesses out there *giving away* Twitter-based services. You've got to be
compelling, cheap, correct, pretty and fast out of the box to compete with
them. You can't "make it work, then make it pretty, go sell it and then make it
scale" any more. Using Streaming in its current state means duplicating large
chunks of Twitter's infrastructure. That's inefficient, and off the top of my
head, I can't think of a *single* example of an inefficient business that
survived in the long run.
M. Edward (Ed) Borasky
"A mathematician is a device for turning coffee into theorems." ~ Paul Erdős