Business Week Online JANUARY 31, 2005
COVER STORY Linux Inc. Linus Torvalds once led a ragtag band of software geeks. Not anymore. Here's an inside look at how the unusual Linux business model increasingly threatens Microsoft Five years ago, Linus Torvalds faced a mutiny. The reclusive Finn had taken the lead in creating the Linux computer operating system, with help from thousands of volunteer programmers, and the open-source software had become wildly popular for running Web sites during the dot-com boom. But just as Linux was taking off, some programmers rebelled. Torvalds' insistence on manually reviewing everything that went into the software was creating a logjam, they warned. Unless he changed his ways, they might concoct a rival software package -- a threat that could have crippled Linux. "Everybody knew things were falling apart," recalls Larry McVoy, a programmer who played peacemaker. "Something had to be done." The crisis came to a head during a tense meeting at McVoy's house, on San Francisco's Twin Peaks. A handful of Linux' top contributors took turns urging Torvalds to change. After an awkward dinner of quiche and croissants, they sat on the living room floor and hashed things out. Four hours later, Torvalds relented. He agreed to delegate more and use a software program for automating the handling of code. When the program was ready in 2002, Torvalds was able to process contributions five times as fast as he had in the past. The Twin Peaks truce is just one of the dramatic changes during the past few years in the way Linux is made and distributed. The phenomenon that Torvalds kicked off as a student at the University of Helsinki in 1991 had long been a loosey-goosey effort, with little structure or organization. Young students and caffeine-jazzed iconoclasts wrote much of the code in their spare time, while the overtaxed Torvalds stitched in improvements almost singlehandedly. TURNING PRO Today, that approach is quaint history. Little understood by the outside world, the community of Linux programmers has evolved in recent years into something much more mature, organized, and efficient. Put bluntly, Linux has turned pro. Torvalds now has a team of lieutenants, nearly all of them employed by tech companies, that oversees development of top-priority projects. Tech giants such as IBM (IBM ), Hewlett-Packard (HPQ ), and Intel (INTC ) are clustered around the Finn, contributing technology, marketing muscle, and thousands of professional programmers. IBM alone has 600 programmers dedicated to Linux, up from two in 1999. There's even a board of directors that helps set the priorities for Linux development. The result is a much more powerful Linux. The software is making its way into everything from Motorola (MOT ) cell phones and Mitsubishi robots to eBay (EBAY ) servers and the NASA supercomputers that run space-shuttle simulations. Its growing might is shaking up the technology industry, challenging Microsoft Corp.'s (MSFT ) dominance and offering up a new model for creating software. Indeed, Torvalds' onetime hobby has become Linux Inc. "People thought this wouldn't work. There are just too many people and companies to hang together. But now it's clear it does work," says Mark Blowers, an analyst at market researcher Butler Group. Not that this Inc. operates like a traditional corporation. Hardly. There's no headquarters, no CEO, and no annual report. And it's not a single company. Rather, it's a cooperative venture in which employees at about two dozen companies, along with thousands of individuals, work together to improve Linux software. The tech companies contribute sweat equity to the project, largely by paying programmers' salaries, and then make money by selling products and services around the Linux operating system. They don't charge for Linux itself, since under the cooperative's rules the software is available to all comers for free. How do companies benefit from free software? In several different ways. Distributors, including Red Hat Inc. (RHAT ) and Novell Inc., (NOVL ) package Linux with helpful user manuals, regular updates, and customer service, and then charge customers annual subscription fees for all the extras. Those fees range from $35 a year for a basic desktop version of Linux to $1,500 for a high-end server version. The dollars can add up. Red Hat, which employs 200 programmers, is expected to see profits triple, to $53 million, in its current fiscal year, as revenues surge 56%, to $195 million. Those numbers are dwarfed by the winnings for computer makers that sell PCs and servers preloaded with Linux. IBM, HP, and others capitalize on the ability to sell machines without any up-front charge for an operating-system license, which can range up to several thousand dollars for some versions of Windows and Unix. At HP, sales of servers that run the Linux operating system hit nearly $3 billion during the past fiscal year, almost double the tally three years ago. In the Linux community, this kind of red-meat capitalism is combined with the sharing philosophy of the open-source movement. Dick Porter, a T-shirted coder who often works under an apple tree in his garden in Wales, is on the same team with Jim Stallings, a hard-charging ex-Marine who travels the world making deals for IBM. What they have in common is a keen interest in making Linux ever more capable. The result is a culture that's cooperative, meritocratic -- and Darwinian at the same time. Any company or person is free to participate in Linux Inc., and those with the most to offer win recognition and prominent roles. "Linux is the first natural business ecosystem," says James F. Moore, a senior fellow at the Berkman Center for Internet & Society at Harvard Law School. STRANGE GROUND To understand the inner workings of Linux Inc., BusinessWeek took a journey through the fast-evolving ecosystem. The unusual trip included everything from sitting in on gritty developer meetings to interviewing dozens of tech execs and engineers from Germany to China. One stop was Torvalds' home, just south of Portland, Ore. The 34-year-old moved from Silicon Valley last summer, in part because he was hired by the Beaverton (Ore.) Linux advocacy group Open Source Development Labs Inc. (OSDL). He spent several hours talking about Linux as his three towheaded daughters played nearby. Something of a rock star in techie circles, he was preparing for a flight to Los Angeles for the premiere of Shark Tale -- which was animated on Linux computers -- and was taking along his oldest daughter, Patricia, then 7 years old. What's clear from these interviews is that the organization supporting Linux has matured more dramatically than most outsiders realize. While Torvalds remains at its center, he has ceded some control and accepted lots of help, thanks to some prodding from individual programmers like McVoy and some coaxing from tech giants whose fortunes have become inextricably linked to Linux. One important step was the move by IBM, Intel, and others to set up OSDL as the focal point for accelerating Linux adoption. Perhaps most surprising, the legal attacks on Linux over the past year have unified the community. There continue to be some internal tensions -- for instance, Linux backers fret that different versions of the software will become incompatible with one another. Yet a suit by SCO Group Inc., a software company that claims IBM handed some of SCO's intellectual property to Linux, gave Linux aficionados the motivation to coordinate their efforts as never before. Tech companies have opened their checkbooks to pay for administrative support, including a legal staff that scans every stitch of code to make sure it can bear patent scrutiny. Even Linux' original idealists, who have grumbled at times about the corporatization of the community, put their complaints on hold and rallied to defend their baby. The SCO suit against IBM is slated for trial late this year. Put it all together, and Linux has become the strongest rival that Microsoft has ever faced. In servers, researcher IDC predicts Linux' market share based on unit sales will rise from 24% today to 33% in 2007, compared with 59% for Windows -- essentially keeping Microsoft at its current market share for the next three years and squeezing its profit margins. That's because, for the first time, Linux is taking a bite out of Windows, not just the other alternatives, and is forcing Microsoft to offer discounts to avoid losing sales. In a survey of business users by Forrester Research Inc. (FORR ), 52% said they are now replacing Windows servers with Linux. On the desktop side, IDC sees Linux' share more than doubling, from 3% today to 6% in 2007, while Windows loses a bit of ground. IDC expects the total market for Linux devices and software to jump from $11 billion last year to $35.7 billion by 2008. In response, Microsoft has launched a counterattack against what it calls its No. 1 threat. The software giant's "Get the Facts" publicity campaign claims that Windows is more secure and less expensive to own than Linux. Microsoft has notched some victories. The city government of Paris, for instance, decided in October against a complete switchover to Linux, citing the costs of such a change. Now that Linux distributors are charging more for subscriptions, Microsoft figures that it can use the same cost-benefit arguments that helped bury old rivals, such as Netscape Communications Corp. "It's getting to be much more like the old world instead of the new world for us, and we know how to compete with that kind of phenomenon," says Microsoft Chief Executive Steve Ballmer. But Ballmer may have a tough time persuading customers that Windows is cheaper than Linux. It often isn't. With Windows, end users pay an up-front fee that ranges from several hundred dollars for a PC to several thousand for a server, while there's no such charge for Linux. The total cost over three years for a small server used by 30 people, including licensing fees, support, and upgrade rights, would be about $3,500 for Windows, compared with $2,400 for a Red Hat subscription, say analysts. The situation where Microsoft can have an edge is when a company already is using Windows. Then, in some cases, it can be cheaper to upgrade to a newer version of Microsoft's software, rather than replacing it with Linux -- once you take into account the retraining expenses. Analyst George Weiss of market researcher Gartner Inc. says that Microsoft may trumpet those individual cases, but "there's no study that says Windows will be a better total cost of ownership in general." Microsoft isn't shying away from brass-knuckle tactics in an effort to win this battle. Several sources say that its executives have been warning corporations that they're taking a legal risk by using Linux. A spokesperson for one company whose CEO met with Ballmer says the implication of their conversation was that Microsoft is considering suing outfits that use the software and claiming that it infringes Microsoft patents. Although legal experts doubt Microsoft would actually sue its own customers, Linux supporters say such warnings are an effort to spread doubt and uncertainty. "Our friends in Redmond [Wash.] are rattling their swords. They're trying to scare people into not switching from Windows to Linux," says Jack Messman, CEO of Linux distributor Novell. Microsoft acknowledges discussing legal risks with customers but denies trying to intimidate them. It won't say whether it believes Linux infringes on its patents. COMMUNAL IMPULSES That Linux is more than holding its own against Microsoft's onslaught suggests it could become a model for others in the tech industry. Otherwise fierce competitors -- think IBM and Hewlett-Packard (HPQ ) -- are demonstrating that they can benefit from embracing the open-source philosophy of sharing work. By collaborating on the operating system, they all get a stable foundation on which to build tech projects and save millions in programming costs. "Much software will be developed this way. It's especially good for infrastructure -- stuff that affects everybody," says Torvalds. "In the long run, you can't sanely compete with the open-source mentality." Linux Inc. has become so mature that it's clear it could continue to thrive even without Torvalds. Already his chief lieutenant, Andrew Morton, shares leadership duties and makes all the public appearances. >From 1997 to 2003, when Torvalds worked for chipmaker Transmeta Corp., putting out Linux wasn't even his full-time job -- yet its market share in servers rose from 6.8% to 24%. Plus, this isn't the army: Programmers don't wait around for orders. Linux' legions know how the development process works, and they just do it. "I manage people, but not in the traditional sense," says Torvalds. "I can't say, 'You do this because here's your next paycheck.' It's more like we know what we want to do, but we don't know how to do it. We try directions. Sometimes somebody disagrees and has a vision. They go and sulk in their corner for a year. Then they come back and say, 'I'll show you it's much faster if you do it this way.' And sometimes they're right." This mix of commercial and communal impulses has its roots in the early days of personal computing. Academics and corporate researchers originally shared many of their software innovations. But that started to change in the 1980s as the industry took shape. In response, programmer Richard Stallman launched the Free Software movement. His answer: the GNU operating system, modeled on Unix, to be shared by a community of programmers. It was Torvalds who came along with a piece of software called the kernel, which is the control center of the operating system and coordinates the work of other pieces, such as the software that tells the printer to produce a page. Programmers called the kernel "Linux," a contraction of Linus and Unix, and Linux caught on as the name for the whole thing. Torvalds decided the group's mascot should be a friendly penguin, named Tux, partly because a pint-size Fairy penguin once nibbled his finger at an Australian zoo. Stallman is still an evangelist for free software, but with his wild long hair and odd behavior, he doesn't fit in with the suit-and-tie crowd. He doesn't even speak to Torvalds anymore -- since Torvalds decided to use a piece of software that wasn't open-source to help develop Linux. "The place he wants to lead people is a mistake. It isn't to freedom," says Stallman of Torvalds. During speaking engagements, Stallman often adopts the persona of "St. IGNUcius," donning a robe and a halo made of a computer disk. Chris Wright, a young programmer for OSDL, recalls a group dinner at a restaurant where the trade group hosted Stallman. Wright was impressed with Stallman's beliefs but put off by his style. "He wanted to taste everybody's food, so it was a little awkward," says Wright. Torvalds proved to be just the guy to lead the Linux charge. He was only a casual programmer in 1991 when he started writing software to run on a PC. But after he posted the first Linux code on the Internet for others to contribute to, he got the knack for spotting quality and handling the flow of fixes. Gradually, he developed a support organization of volunteers. Begun as a meritocracy, Linux continues to operate that way. In a world where everybody can look at every bit of code that is submitted, only the A+ stuff gets in and only the best programmers rise to become Torvalds' top aides. "The lieutenants get picked -- but not by me," explains Torvalds. "Somebody who gets things done, and shows good taste -- people just start sending them suggestions and patches. I didn't design it this way. It happens because this is the way people work naturally." One reason that Linux Inc. bears little resemblance to a traditional company is that Torvalds has almost nothing in common with classic, hard-driving, and autocratic tech-industry leaders. He rarely appears in public and largely lets other people set priorities for development. Once others come up with improvements, he shepherds them along. "Linus has power, but he doesn't have it by fiat," says Havoc Pennington, a Linux contributor who works for Red Hat. "He has power because people trust him. As long as he keeps making good decisions, people won't take it away from him." Yet for all of his seeming passivity, Torvalds is a strong leader. He stays scrupulously neutral, never taking one company's side over another. He focuses on the open-source development process. There, he demands high-quality work. Things must be just so, with the least amount of coding. As a result, Linux has few errors that can be exploited by virus writers. That gives it an edge on Windows, which has become a favorite target of hackers -- largely because it's so widely used, but also because it has vulnerabilities that Linux doesn't. "He has set a compelling vision and inspired people to follow it," says Larry Augustin, a venture capitalist at Azure Capital Partners and an OSDL board member: "It's leadership by example, rather than leadership by hype." Even today, Torvalds operates in a virtual world of e-mails and Web sites. He works almost entirely from a roomy house that sits on a wooded Oregon mountaintop and is decorated with taxidermic specimens, including a piranha and a crocodile. He gets up early, making strong cups of coffee for himself and his wife, Tove, a former karate champion in Finland. Then he settles in for hours of reviewing code and snapping off e-mail messages in his basement office. It's lined with science fiction and fantasy books, including classics such as Dune and the Wheel of Time series. In the afternoon, he coasts down the hill on his bicycle to a quaint village, stops at a Peet's coffee shop for a latte or Chai tea, and pumps back up the hill. Then he returns to his computers. Although Torvalds is physically near his comrades at OSDL, he almost never sees them face to face. He visited the organization's office only once in his first three months in the Portland area, and he rarely meets with Morton, an Aussie who lives in Silicon Valley. "It's a long-distance mind-meld," says Morton. In a rare encounter last summer, they shook hands and made small talk at a picnic. The Linux community, Torvalds says, is like a huge spider web, or better yet, multiple spider webs representing dozens of related open-source projects. His office is "near where those webs intersect." The Linux development process begins and ends with the programmers. While there are still some individual volunteers and government agencies that chip in, more than 90% of the patches now come from employees at tech companies. Many of those workers are formerly independent aces who have been scooped up over the past few years. Some of these people simply submit code, and others, called maintainers, are in charge of improving specific functions. >From there on, it's a continuous cycle. Individuals submit patches; maintainers improve them. Then they're passed off to Torvalds and Morton, who review the patches, ask for improvements, and update the kernel. Every four to six weeks, Torvalds releases a new test version so that thousands of people around the world can probe it for flaws. He puts out a major upgrade every three years or so. Unlike at traditional software companies, there are no deadlines. The Linux kernel is done when Torvalds decides it's ready. Linux Inc. is a series of concentric circles radiating out from Torvalds. In the first circle, you have Open Source Development Labs. The top tech companies with a stake in Linux -- including HP, IBM, and Intel -- have technical people on the board of directors. The board sets priorities, such as getting Linux running better for huge data centers and desktop PCs. In addition, the board is responsible for raising $10 million to protect customers from potential intellectual-property claims. TAKING THE SUBWAY The second circle is a dozen or so Linux distributors. Spearheaded by Red Hat and Novell, this group also includes such regional players as Red Flag Software in China and MandrakeLinux in Europe. They pick up the latest version of the kernel about once a year and package it with 1,000 or so related open-source programs, including the GNOME graphical-user interface, the Firefox browser, and the OpenOffice desktop application suite. The distributors race one another to be first out with Linux updates, but their engineers spend most of their time on projects they share with everybody else. For example, Novell employs open-source pioneer Miguel de Icaza, who is both a Novell vice-presi- dent and the leader of the Mono project -- software for building applications to run on Linux. The 34-year-old Mexican coordinates 25 Novell employees plus more than 300 other programmers, many of whom work for other tech companies. So far, de Icaza says, there have been no conflicts. His explanation: "Cooperating gets you further along than screwing your neighbor." These Linux companies have little in common with their brethren from the dot-com boom. They're typically frugal. Matthew J. Szulik, CEO of Red Hat, takes the subway rather than a cab when he visits customers in New York and Boston. And rather than being motivated by big money, Linux programmers say their goal is making Linux an ever-bigger force in computing. Red Hat's Pennington doesn't covet expensive wheels, proudly pointing to his 2001 Toyota Corolla in the parking lot, which he jokes is "fully loaded." For his part, Torvalds has been amply rewarded for his role, but he's no Bill Gates billionaire. OSDL pays him a salary of nearly $200,000. In addition, he sold initial public offering shares that he got as gifts from a couple of Linux companies, including VA Linux Systems. That helped him afford his house and put money away for his daughters' educations. ALL-PURPOSE SYSTEM In Linux society, there's no bowing and scraping before the rich and powerful. Executives and product managers at HP, IBM, Intel, and Oracle (ORCL ) don't even try to pressure Torvalds and Morton to further their interests. Instead, their input goes through their engineers, who, as members of the open-source community, submit patches for the kernel or other pieces of Linux software. The tech powerhouses have learned to play by new rules. You can't meet in private, come up with new features, and then drop massive changes on Torvalds. A handful of companies, including Intel and Nokia Corp. (NOK ), learned this lesson the hard way when they went about making Linux capable of running telecom gear. About two dozen of their engineers worked on the "carrier-grade" Linux project, and then, in late 2002, they posted hundreds of thousands of lines of code on a Web site. The response: outrage. "We were offended by the whole process," says Alan Cox, a top kernel programmer. The posting was quickly removed. Still, the cultures of open-source and commercial software are melding together. Red Hat used to scatter employees around the world, the typical open-source approach. Now the company brings its workers together so young programmers can cross-pollinate with gray-haired veterans. It works. Not only did 46-year-old Larry Woodman bond with 26-year-old Rik van Riel by teaching him how to drive a car, but the two are working in tandem on improvements to memory management in Linux. "We complement each other," says Woodman. These collaborations are turning Linux into an all-purpose operating system. It's secure enough that Lawrence Livermore National Laboratory loads it not only on desktop and server computers but also on supercomputers it uses to simulate the aging of nuclear materials. "Linux is definitely more secure than Windows," says Mark Seager, the lab's assistant department head for advanced technology. "There aren't as many ways to break the system." With the latest improvements, Linux now works on servers with more than 128 processors and can run the largest databases. The newest versions also have features, such as power management, that make them more suitable for laptop PCs. Linux is so solid that staid corporate purchasers are adopting it aggressively for run-the-company applications. Holcim Ltd. (HCMLY ), the Swiss cement giant, just switched from Unix to Linux for some of its accounting, manufacturing, and human-resource applications. The attraction: 50% savings on hardware and 20% on software. "It was a no-brainer to go with Linux," says Carl Wilson, chief operations manager for the company's North American data center. Cost isn't the only reason that companies are switching to Linux. The data processor Axciom Corp. recently shifted some servers to the operating system, after using Unix in the past. Alex Dietz, the company's chief information officer, says he's thinking about replacing the Windows operating system with Linux on the company's desktop computers. One important reason: Axciom doesn't want to be too dependent on Microsoft. "[Linux] has an innate guarantee that you won't be held hostage," says Dietz. Torvalds takes tremendous satisfaction in seeing his baby grow up. "It's like a river. It starts off a bouncy small stream and turns into a slower-moving big thing," he says. Indeed, Linux Inc. has emerged as a model for collaborating in a new way on software development, which could have reverberations throughout the business world. Its essence is captured in one of the mottoes of the open-source world: Give a little, take a lot. In a business environment where efficiency rules, that's a potent formula -- maybe even strong enough to knock mighty Microsoft down a peg. By Steve Hamm -- KPLUG-Newbie mailing list [email protected] http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-newbie
