There is an article in the NY Times on the Netflix competition to upgrade the software they use for handling film ratings and other things. Instead of hiring a group of programmers to do the job for them, Netflix decided to make it into a world-wide competition in which the winner(s) (i.e., producing a program that performs according to certain criteria) would win $US 1 million.
The characterization of this contest process is interesting. Consider the following quote from the article: |The emerging prize economy, according to some labor market |analysts, does carry the danger of being a further shift in the balance |of power toward the buyers — corporations — and away from most |workers. | |Thousands of teams from more than 100 nations competed in the |Netflix prize contest. And it was a good deal for Netflix. “You look |at the cumulative hours and you’re getting Ph.D.’s for a dollar an hour,” |Mr. Hastings said in an interview. | |Netflix, Mr. Hastings said, did not do a crisp cost-benefit analysis of |its investment in the contest. But several crucial techniques garnered |from the contest have been folded into the company’s in-house movie |recommendation software, Cinematch, and customer retention rates |have improved slightly. Better recommendations, Netflix says, enhance |customer satisfaction. | |“We strongly believe this has been a big winner for Netflix,” Mr. Hastings |said.The prize winner was a team of statisticians, machine-learning experts |and computer engineers from the United States, Austria, Canada and Israel, |calling itself BellKor’s Pragmatic Chaos. The group was actually a merger |of teams that came together late in the contest. http://www.nytimes.com/2009/09/22/technology/internet/22netflix.html?_r=1&th=&emc=th&pagewanted=all I think that we may be able understand how and why skilled scientists might be motivated to tackle an interesting problem with the goal of trying to understand how to solve the problem but what might be less obvious is that this "contest" is essentially a high-stakes, winner takes all lottery. I don't know if anyone has done an real "economic analysis" (i.e., doing a cost-benefit analysis to determine whether the "prize" covers the basic costs of engaging in solving the problem) but if the "$1 per hour" comment by Mr. Hastings is anywhere close to being correct, then there should have been 1 million person-hours of work invested in the work (some of which was redundant because the work wasn't coordinated until individual teams joined into alliances with others teams). If 1 million person-hours were spent in coming up with the solution, how much should the prize have been to represent a fair return on the amount of time spend (NOTE: computer equipment, software, processing times, and other costs are not considered here and would also have to be covered by the prize money)? What would these academics and professional be paid on an hourly basis if they had been paid at that rate instead of the "$1 per hour" the project wound up costing. NOTE: since this is a "winner take all" competition, losers don't even see that "$1 per hour" return for their investment. My main point isn't that Netflix made out like a bandit with this competition and will continue to earn barrels of cash as result (indeed all sponsors of this type of competition or "lottery" win big) but why would statisticians, other highly skilled academics, computer scientists, programmers, etc. would allow themselves to be suckered into a game where essentially everyone loses except for the people who run the competition? Wouldn't they have done better if they took the nominal "$1 per hour" times the number of hours contributed to the project and used this sum to buy lottery tickets? Wouldn't buying lottery tickets, by comparison, produce more of a return than the net loss experienced by all of the losing teams? I await the rationalizations that what they learned in the process of trying to solve the problem might make up for some of their losses but clearly their solutions were not good enough. Remember Crick and Watson's double helix model of DNA? They were in competition with other teams for determing what the actual structure was but the other teams focused on the wrong structures -- does anyone remember who those researchers were? At least the search for the structure of DNA was part of their job, what they were being paid for. I don't think that most of the people working on the Netflix prize were doing it as a part of their "job" unless they could decide what their "job" was supposed to be. So, should we be more cautious in critizing people when they go off and buy lottery tickets for a prize they have a low probability of winning or should we reflect on why scientists doing the same thing when the situation has been re-framed as in the Netflix prize? -Mike Palij New York University [email protected] --- To make changes to your subscription contact: Bill Southerly ([email protected])
