On Sat, Jul 18, 2009 at 12:49 AM, David Fisher <tib...@gmail.com> wrote:

> Show me these killer companies doing great NLP with social networks. I
> find the ones that are doing stuff right now themselves are far behind
> the curve and not really pushing stuff to the edge. They are often
> marketing companies that have hired one NLP guy (and underpaid them)
> and are just pushing the marketing side. I have yet to see anything
> truly revolutionary come from most of these monitoring companies yet
> and they are all too narrow focused. Plus, none of them have the VC
> funding to really expand and grow (and not many people are getting new
> funding these days)

Who says that a company has to create something "truly revolutionary" to be
successful?  There are plenty of big successes that got where they are by
packaging and distributing better than anyone else, not with great
breakthroughs.  Heard of Microsoft?

Sentiment analysis, like everything else that depends on computers figuring
out language, isn't great.  Nor is anyone really close to writing software
that comes understands language with context, nuance, etc.  Language isn't
even well understood enough for anyone to write code to emulate it; it is at
the core of human intelligence.  Language in 140 character chunks is
*really* hard.

If you think there are no well-funded, successful companies in this domain,
take a look at Nielsen/Buzzmetrics.  They've been at this for more than 10
years.  They acquired my patents, from a startup where we demonstrated basic
sentiment analysis in 2000 and 2001, showing that our software could rate
the sentiment of Usenet movie reviews with 80 percent accuracy and forecast
box office.

I would love to see more people tackling this kind of problem, but nobody is
likely to succeed if they don't realize what has worked and what hasn't over
the last decade and more.  Intelligence agencies and law enforcement have
used relevant techniques for 20-30 years.  For example, traffic analysis is
fundamental and doesn't require any NLP, just as the NSA is able to identify
command and control centers by their behavior without having to decode a
single encrypted transmission.  The danger of focusing on NLP and other
really hard problems is that you fail to apply known techniques in new ways.

Having said all that, I'll add that a lot of what I saw over the last few
years in social media analytics was pretty eye candy without much behind
it.  If that's all you look at, then yes, it seems quite shallow.  But I
would hope that serious developers know that that's not all there is.  The
systems I've built over the last decade have been based first on traffic
analysis, then social network analysis, and last, text/lingustic analysis...
and to do the latter well, humans were involved in the final summarization
of topics, trends and so forth.


Reply via email to