SEATTLE POST-INTELLIGENCER http://seattlepi.nwsource.com/business/217538_msftstudy25.html
Microsoft funding of security report decried Finding that system is superior to Linux is biased, critics say Friday, March 25, 2005 By TODD BISHOP SEATTLE POST-INTELLIGENCER REPORTER Two researchers surprised the audience at a computer-security convention last month with their finding that a version of Microsoft Windows was more secure than a competing Linux operating system. SEE THE STUDY Download a copy of the Windows vs. Linux study in PDF format (265K). This week, the researchers released their finished report, and it included another surprise: Microsoft was funding the project all along. The researchers, from the Florida Institute of Technology and Boston-based Security Innovation Inc., defend their process and conclusions as valid. They say they had "complete editorial control over all research and analysis" involved in the project. Their report details their methods, and they invite other experts to examine and duplicate their work. But their disclosure of the project's funding source this week is stirring new debate over what had otherwise been viewed as encouraging news for Microsoft in an area in which it has struggled. The researchers had made the presentation at last month's RSA Conference, which attracts some of the biggest names in the computer-security business. "It was evidence that Microsoft was doing better, and now the evidence is tainted," said Counterpane Internet Security founder Bruce Schneier, a longtime RSA Conference speaker. "The results might be accurate, but now nobody's going to care, because all they'll see is a bias that was undisclosed." But one of the researchers, Herbert Thompson of Security Innovation, said he and his colleague considered the final report, not the earlier presentation, the proper place to make the disclosure. In addition, he said, the report's detailed presentation of the project's research methods should resolve any concerns about potential bias. "We knew that some of the criticism that would be levied on the report would come from Microsoft's funding of it," Thompson said. As a result, he said, "Our own requirement for the methodology was that it had to be very open and transparent. We wanted to give people the recipe so they could go out and recalculate the numbers for themselves." The 37-page final report, released Tuesday, is explicit about the Redmond company's role: "This study and our analysis were funded under a research contract from Microsoft," it explains on the fourth page. However, during their Feb. 16 presentation at the RSA Conference, Thompson and fellow researcher Richard Ford of the Florida Institute of Technology did not mention that one of the subjects of their research was the one funding the project. Thompson said yesterday that they had decided it would be better to wait until releasing the final report to make that disclosure. In part, he said, the idea was to avoid some of the divisiveness that often characterizes the Windows vs. Linux debate. The presentation at the RSA Conference "just didn't seem like the appropriate venue to get into that religious warfare," Thompson said. But in some ways, the decision not to disclose the source of funding at the conference could intensify the debate between the two camps. One Linux advocate compared the situation to Microsoft's past labeling of studies as "independent," even though the text of the study reports identified the company as the source of funding. "Redmond strikes again," said Russell Pavlicek, a Linux expert and columnist. "Here again is a Microsoft study that isn't put forward as a Microsoft study to begin with ... and is generally out of step with the rest of the assessments that I've seen." Microsoft has funded a series of studies comparing its software to Linux and other open-source programs as part of a campaign it calls "Get the Facts on Windows and Linux." The company points out that it's a common industry practice, and the studies themselves include disclaimers disclosing the source of funding. However, sponsored research can provide a distorted picture if companies release only those studies that they consider favorable. Thompson said he didn't know whether anything in the research contract with Microsoft would have prevented release of the study if the company considered the results unfavorable. The study by Thompson and Ford compared Microsoft Windows Server 2003 to Red Hat Enterprise Linux 3.0 on such factors as the number of reported security vulnerabilities in 2004 and "days of risk" -- the amount of time between the public disclosure of a vulnerability and the availability of a fix. Windows Server benefited in part from Microsoft's reduction of security vulnerabilities in the latest version of the software -- with 52 reported vulnerabilities for the year, compared with 132 vulnerabilities for the Linux version, according to the report. The researchers also calculated an average of about 31 days of risk for the Windows software in 2004, compared with an average of about 70 days of risk for the Linux version. Thompson said he and Ford developed the methodology on their own and submitted a proposal to Microsoft last year. He declined to say how much Microsoft paid to fund the research, but he said the company didn't have a say in the methodology. He said they told the company, "Here's our methodology, here's what we're going to do; fund it if you like, but the results will be the results. It's pretty clear and straightforward how those things are calculated." Microsoft declined to make an executive available to discuss the research contract or the RSA presentation. In a statement, the company said it was pleased that the researchers "delivered a repeatable methodology that customers can easily understand and duplicate themselves to validate the findings in the report." It was important for the researchers to disclose the funding source in the final report, said University of Virginia business professor R. Edward Freeman, director of the Olsson Center for Applied Ethics at the university's Darden Graduate School of Business Administration. However, he said, the issue of disclosure during a conference isn't as clear. "It certainly would be a good thing to disclose, a smart thing to do, just for the sake of credibility," Freeman said. But it also depends on the context and the nature of the conference. "It might just be that they saw this as informal," he said. In fact, the researchers cast their RSA presentation almost as a comedy routine. It was billed as a "security showdown," with Thompson taking the position of a staunch Windows advocate and Ford presenting himself as a die-hard Linux fan -- cracking repeated jokes at Microsoft's expense and betting Thompson $20 over the outcome of the security comparison. But even that part of the presentation wasn't entirely as it seemed. Although Thompson does tend to favor Windows, and Ford does lean toward Linux, neither man is quite as extreme as it may have seemed during the event, Thompson said. "I'm not the kind of guy who comes home at night and wears the underwear with the Windows logo on it," he said. "But taking sides like that, especially in a talk, is great, because you get to flesh out the big issues and keep the audience engaged." And for the record, Thompson said, he kept the $20 in winnings. You are a subscribed member of the infowarrior list. Visit www.infowarrior.org for list information or to unsubscribe. This message may be redistributed freely in its entirety. Any and all copyrights appearing in list messages are maintained by their respective owners.
