July 3, 2006 A Search Engine That's Becoming an Inventor By SAUL HANSELL and JOHN MARKOFF NY Times
http://www.nytimes.com/2006/07/03/technology/03google.html?pagewanted=print When Google was a graduate-school project being run from a Silicon Valley garage, its founders, Larry Page and Sergey Brin, built their own computers out of cheap parts meant for personal computers. They wanted to save money, and they felt that they could design a network of computers that would search the Web more efficiently than those available from traditional manufacturers. Google no longer needs to pinch pennies. It is a solid member of the Fortune 500 with $9 billion in cash. Still, it is stubbornly sticking to its do-it-yourself approach to technology. Even as it spends more than $1.5 billion this year on operations centers and technology, most of the hundreds of thousands of servers it will deploy are being custom-made based on Google's own eccentric designs. To be closer to its users and speed response time, it is building a worldwide string of data centers, including a huge site in The Dalles, Ore., with technologies it designed to reduce its ravenous need for electricity. These computers in turn use software developed with advanced tools that Google also designed itself. There are signs that Google is even preparing to create its own custom microchips. "Google is as much about infrastructure as it is about the search engine," said Martin Reynolds, an analyst with the Gartner Group. "They are building an enormous computing resource on a scale that is almost unimaginable." He said he believed that Google was the world's fourth-largest maker of computer servers, after Dell, Hewlett-Packard and I.B.M. Google's biggest rivals, Microsoft and Yahoo, certainly write much of their own software, and they work to configure their computers and data centers to their own needs. But they largely buy machines from existing manufactures like Dell, Sun Microsystems and Rackable Systems. "At some point you have to ask yourself what is your core business," said Kevin Timmons, Yahoo's vice president for operations. "Are you going to design your own router, or are you going to build the world's most popular Web site? It is very difficult to do both." Google, in fact, has decided it will do both. In many ways, it still has the head of an graduate-school project grafted onto the body of an multinational corporation. The central tenet of its strategy is that its growing cadre of world-class computer scientists can design a network of machines that can store and process more information more efficiently than anyone else. Mr. Reynolds estimated that Google's computing costs are half those of other large Internet companies and a tenth those of traditional corporate technology users. Google will not comment on its costs, but it does claim an advantage. "We don't think our competitors can deploy systems cheaper, faster or at scale," Alan Eustace, Google's vice president for research and systems engineering, told analysts in March. "That will give us a two-, three-, five-year lead." Despite those boasts, some argue that Google's home-brew approach is unnecessary and inefficient, a headstrong indulgence masked for now by the growth and profitability of its advertising business. And Google's rivals say their networks are plenty efficient and powerful. "Google doesn't have anything magic here," Bill Gates, the Microsoft chairman, said in an interview. "We spend a little bit more per machine. But to do the same tasks, we have less machines." Google is notoriously secretive about its technology. Yet it also has published papers on some of its developments and been granted patents on others. These, along with the public statements of Google executives and interviews with current and former employees, vendors and other technology executives, paint a picture of a company devoted to pushing the boundaries of modern computer science, and applying those concepts on a vast scale. "Google took the best ideas from the supercomputer research community and wove them into a working system," said Stephen E. Arnold, a technology consultant to investors and the author of "The Google Legacy" (Infonortics, 2005), a book on Google's technology. Some of its innovations are designed to wring pennies from its growing spending on technology. Last year, it was granted a patent (06906920) on a "drive-cooling baffle," meant to funnel air into a rack of computers held together with Velcro, a Google design signature. But some innovations are bolder, like a series of software tools that simplify the way it can divide a problem to be handled by thousands of processors simultaneously, an approach called parallel processing. One such program, called MapReduce, is based on ideas discussed in computer science literature for decades, according to Urs Hölzle, Google's senior vice president for operations. "What surprised us was how useful it turned out to be in our environment," he said. MapReduce, he said, "allows Joe Schmo software engineer to process large amounts of data and take advantage of our infrastructure." Mr. Arnold, the consultant, said these tools created a significant cost advantage. "If you talk to guys who work in massively parallel computing operations, as much as 30 percent of their coding time is spent trying to figure out how to get the thing to run," he said. Google "has figured out how they can reduce a lot of the hassle and work of creating parallel applications." Mr. Gates acknowledged that MapReduce was a significant technology, but he asserted that Microsoft was building its own parallel processing software, opening another front in the technological war between the two companies. "They did MapReduce; we have this thing called Driad that's better," Mr. Gates said. "But they'll do one that's better." Moreover, Google's focus on building general purpose tools and systems is different from that of most companies, which develop systems tailored to specific applications. And it is building these systems rapidly, with the billions of dollars in cash it generates and the thousands of engineers it hires each year. It hopes that it can build a lead that will allow it to create products that do more, for less money, than its rivals. "If they can get a 30 percent cost advantage, in operating a service on the Internet that is a huge difference," said John M. Lervik, the chief executive of Fast Search & Transfer, a Norwegian search company. Google's academic approach can be traced not only to its founders' graduate work in computer science, but even to their early home life, Mr. Arnold said, noting that Mr. Page and Mr. Brin had come from families with expertise in computer science and mathematics. "The stuff they did in 1996 to 1998 was not as immature as it should have been," he said of the Google founders. He said that told him the two men learned a lot "when their parents were talking at the kitchen table." By the time Mr. Page and Mr. Brin were designing Google, parallel processing was more than an academic dream; it was enabled on a large scale by the low prices of processors, memory and disk drives used to make personal computers. These components were hardly of the highest quality and could be counted on to fail often. Mr. Page designed the initial Google servers, with the assumption that parts would fail on a regular basis. At first he tried to simplify assembly and reduce the presumed repair time by not fastening components to the servers at all but simply laying them on a bed of cork. This proved to be unstable, and so parts were connected with Velcro. "Nobody builds servers as unreliably as we do," Mr. Hölzle said in a speech last year at CERN, the Swiss particle physics institute. Google is reducing cost while maintaining performance by shifting the burden of reliability from hardware to software individual hardware components can fail, but software automatically shifts the local task and the data to other machines. For example, Google designed a software system it calls the Google File System that keeps copies of data in several places so Google does not have to worry when one of its cheap servers fails. This approach also means that it does not have to make regular backup copies of its data as other companies do. Another system, called the Google Work Queue, allows a big pool of servers to be assigned to various tasks as needed and reassigned to other projects later. This concept, called "virtualization," has become a trend among large data center operators, which also want to reduce the expense of having separate servers dedicated to each system. But most companies buy commercial software to track which computers are doing what, a complex process. While Google's servers are built on inexpensive parts, the designs it uses have been modified every year or so, to improve their efficiency and increasingly to customize them to Google's applications. The current generation uses the powerful Opteron chip from Advanced Micro Devices, which uses less power than the Intel chips Google had used. Google is among Advanced Micro's five largest clients, and the largest that does not make computers to resell, according to a semiconductor industry executive with knowledge of Advanced Micro's business. Google is increasingly doing business with Sun Microsystems as well. Sun, known for systems that are both reliable and expensive, would not seem a natural match for a company that emphasizes economy and self-sufficiency. But Eric E. Schmidt, Google's chief executive, is a former Sun executive, and Sun has developed a new microchip that is especially efficient in electricity use. Moreover, Google increasingly needs systems that are less likely to fail than those it uses for its search engine in order to handle important information, like e-mail and payments in its new Google Checkout service. Beyond servers, there are signs that Google is now designing its own microchips. The company has hired many of the engineers responsible for the Digital Equipment Corporation's well-regarded Alpha chip. "Google's next step is to build high-performance silicon," said Mark Stahlman, an independent technology analyst. Mr. Hölzle said Google had considered custom semiconductor design, but he declined to say if the company had built any. He said that, in general, Google did not want to build anything from scratch if it could buy something that was just as good. But he added that Google continued to believe that its approach to designing its own cheap and fast computer networks gave it an edge. "Having lots of relatively unreliable machines and turning them into a reliable service is a hard problem," Mr. Hölzle said. "That is what we have been doing for a while." ================================================= George Antunes Voice (713) 743-3923 Associate Professor Fax (713) 743-3927 Political Science Internet: antunes at uh dot edu University of Houston Houston, TX 77204-3011 Reply with a "Thank you" if you liked this post. _____________________________ MEDIANEWS mailing list medianews@twiar.org To unsubscribe send an email to: [EMAIL PROTECTED]
