On Sat, Jul 18, 2009 at 7:15 AM, Nick Arnett<nick.arn...@gmail.com> wrote: > If you think there are no well-funded, successful companies in this domain, > take a look at Nielsen/Buzzmetrics. They've been at this for more than 10 > years. They acquired my patents, from a startup where we demonstrated basic > sentiment analysis in 2000 and 2001, showing that our software could rate > the sentiment of Usenet movie reviews with 80 percent accuracy and forecast > box office.
Netflix, even without the contributions of the contest teams, is doing pretty well too. ;-) > I would love to see more people tackling this kind of problem, but nobody is > likely to succeed if they don't realize what has worked and what hasn't over > the last decade and more. Intelligence agencies and law enforcement have > used relevant techniques for 20-30 years. For example, traffic analysis is > fundamental and doesn't require any NLP, just as the NSA is able to identify > command and control centers by their behavior without having to decode a > single encrypted transmission. The danger of focusing on NLP and other > really hard problems is that you fail to apply known techniques in new ways. > > Having said all that, I'll add that a lot of what I saw over the last few > years in social media analytics was pretty eye candy without much behind > it. If that's all you look at, then yes, it seems quite shallow. But I > would hope that serious developers know that that's not all there is. The > systems I've built over the last decade have been based first on traffic > analysis, then social network analysis, and last, text/lingustic analysis... > and to do the latter well, humans were involved in the final summarization > of topics, trends and so forth. Man, it is so good to hear this from someone who's actually done it! The other point, though, is that the "real thing", even traffic / social network analysis, is compute-resource intensive and requires a kind of programming knowledge that few have. So if something simple, like emoticon counting, provides *some* clues about sentiment, it may be worth doing. I'm not convinced, though, that it is worth doing. -- M. Edward (Ed) Borasky http://borasky-research.net "I've always regarded nature as the clothing of God." ~Alan Hovhaness