The Planetary Society SETI@home Update, 5/20/2002:
A Question of Bandwidth by Amir Alexander Fascinated as we are by the idea of contacting alien civilizations, we sometimes forget that SETI@home is not just a search for intelligent beings among the stars. It is also a gigantic computer science experiment of unprecedented magnitude taking place here on Earth. With over 3.5 million users worldwide, SETI@home is by far the largest and most powerful computing network ever assembled. Keeping this gigantic network operating, it turns out, is no easy matter. Every day around 400,000 people from around the world connect to the SETI@home server in Berkeley, California. First-time users sign up, established users send in processed work units and download new ones. Each work unit is around 350 kilobytes in size, and the constant movement back and forth of thousands of units has created something of an information traffic jam on SETI@home's internet connection. The communications problem can be summed up in one word: "Bandwidth." Because of the enormous amount of web traffic it generates, SETI@home needs a very wide communications band to operate smoothly. And as time goes on, and more and more users join the SETI@home network, the bandwidth requirements only get larger and larger. On Cars and Bytes To visualize the problem, consider any road or a highway near your home. If you use it when traffic is light, say, on a weekend morning, all you see is open road. You can drive more or less as fast as you want, or as fast as your local highway patrol will allow. But try the same route during Monday morning rush hour and you will find things very different. Even the widest roadway will often be packed with vehicles as far as the eye can see, and traffic will slow to a crawl. The reason is that (given speed limitations) any roadway can only allow the movement of a certain number of vehicles at any given time. If more vehicles try to pass through, the road will become hopelessly clogged, with dire (but not uncommon) results for the morning commute. Eventually, one would hope, your local government will address the situation by widening the roadway and adding a lane or two. This will alleviate the problem, because the wider roadway will have a greater capacity and will be able to move a larger number of vehicles on their way. But if your area is at all like mine (I live in Los Angeles.) it will not be long before the maximum capacity of the wider road will be reached, and the problem will repeat itself. What is true of automobile traffic is also true of electronic web traffic. The fiber-optic cables that carry information to SETI@home headquarters in Berkeley can only move so many work units along at a given time. And as SETI@home users have recently noticed, that limit is fast approaching. A Clogged Highway The signs were unmistakable early in January. SETI@home users trying to send or receive work units found that the speed of uploading and downloading information had slowed to a crawl. A process that usually takes less than a minute would now drag on for hours, causing many to give up the effort altogether. And although conditions were considerably better during off-peak hours (such as night-time in California), even then the information was moving at a far slower pace than it should. The problem was particularly severe for users who run several computers with the SETI@home program simultaneously and are therefore constantly sending and receiving data. When every online transaction became an adventure of indeterminate duration, these dedicated users had to shut down their operations, or at least trim them down drastically. The source of these troubles, as you've probably surmised, was bandwidth. Since SETI@home is located at the Space Science Laboratory of the University of California at Berkeley, all web traffic to and from the SETI@home servers passes through the university's internet system. Now U.C. Berkeley's total bandwidth is 70 Megabits per second (or "70 Mbps"), meaning that it allows for the transfer of 70 million bits of digital information every second. This capacity must suffice for all of the university's e-mail and web traffic in all of its academic and administrative departments. When SETI@home went online in 1999, it initially didn't make much of a dent in this overall capacity. But over time, as millions around the world joined the program, it grew into the single largest user of U.C. Berkeley's bandwidth. By early 2002, SETI@home traffic ranged between 20 and 30 Megabytes per second, or about 40% of the university's total bandwidth. This proved to be too much for U.C. Berkeley's web system. The SETI@home traffic, when combined with the natural growth of the university's other internet requirements, was using up the university's maximum full bandwidth capacity. Since SETI@home data is a relatively low priority for the university compared to carrying on its day-to-day operations, users trying to upload and download information bore the brunt of this traffic jam. The flow of SETI@home data, like traffic on an overused roadway, ground nearly to a halt. What to Do? The SETI@home team had been aware of the growing problem for quite a while, and was considering possible solutions. Nevertheless, when the crisis came in early January, it surprised everyone with its suddenness and impact. According to Project Director David Anderson, they are still not sure what initiated the crisis at this particular time. For some reason, outgoing data from U.C. Berkeley had increased in January, using up the full capacity of the university's bandwidth. The first response of the SETI@home team was literally to buy time. For a monthly payment of $6000.- the university agreed to guarantee at least 20 Mbps in bandwidth for SETI@home's use at any given time. With this arrangement in place, SETI@home users could again send and receive their work units, though not quite at the rate they were used to. This, however, was only a temporary stopgap measure, and the SETI@home crew immediately started looking for a permanent solution. What was needed was an independent web connection, completely separate from the general U.C. Berkeley line and with sufficient capacity. They found exactly that: Cogent Communications, a company based in Washington D.C., was offering 100 Mbps worth of bandwidth at an affordable price on their network of fiber optic cables. While Cogent's proposal is highly attractive, there are still some obstacles on the way. Cogent's network, it turns out, cannot be accessed in Berkeley, but only in Palo Alto, on the other side of the San Francisco Bay. The SETI@home team must find a way to transmit and receive data across the 40 miles separating Berkeley from Palo Alto with sufficient bandwidth so as not to create a bottleneck. The U.C. Brekeley campus network administrators have proposed using their existing connection with Palo Alto, via a company named (some would say appropriately) UFO Communications. According to SETI@home chief scientist Dan Werthimer, however, it is not yet certain whether this particular arrangement is going to work. A preliminary test a few days ago failed, and more tests are scheduled for the coming days. Once all the problems are resolved, however, the new bandwidth will be an enormous boon for SETI@home. For one thing, 100 Mbps would provide the project with four times the capacity it needs today, and would enable users to contact SETI@home quickly and easily for years to come. For another, according to David Anderson, such bandwidth would also enhance the project's scientific value. With that kind of capacity at its disposal SETI@home will be able to receive many more data packages from Arecibo, and quickly distribute them to its millions of users. As a result, the project would be able to analyze radio signals recorded at a wider range of frequencies then the relatively modest 2.5 MHz it tracks today. A victim of its own phenomenal success, SETI@home has outgrown the academic environment in which it was conceived, and is now struggling with the constraints it places on the project's growth. When this transition is completed, SETI@home will reemerge as a larger, more efficient, and more scientifically ambitious project than ever before. Reference URL: http://www.planetary.org/html/UPDATES/seti/SETI@home/Update_1.htm == Unsubscribe instructions: http://www.talkspace.net/mlists/setiathome.html This list sponsored by talkspace.net: building space communities online. Mailing list services provided by klx.communications -- www.klx.com
