Title: Storage Innovators
a weekly newsletter fromTechTarget
 April 02, 2003 >> Receive this email as text   >> About this email 

SPONSORED BY: HP Storage
Leverage your IT storage investment for success. Free storage consolidation white paper from HP.

Unique archival system helps lab weather data storm
NCAR archives the environment -- one petabyte at a time.

To rein in the enormous amount of natural data created, The National Center for Atmospheric Research Foundation turns to a high-capacity archival system to help make sense of nature.

by Drew Bird

Trying to make sense of these seemingly random events created by Mother Nature is the function of the Boulder, CO-based National Center for Atmospheric Research (NCAR). Funded through the National Science Foundation, NCAR conducts research and provides facilities -- including data sets and computer models -- to improve the understanding of the skies, oceans, ice packs and landmasses of the world. As you can imagine, it's a function that requires a massive amount of computing power and an enormous, capacious storage system. The data and projections obtained and created by NCAR are used by scientists, academic institutions and numerous government agencies including NASA, NOAA, DOE and the EPA.

The demands of coping with a subject as vast as the environment has caused NCAR a few storage-related issues over the years. The burden has led them to develop an innovative high-capacity archival system -- one designed to accommodate huge amounts of critical, irreproducible and ever-increasing data.

SPONSORED BY:  HP Storage
Leverage your IT storage investment for success. Free storage consolidation white paper from HP.

Converting your direct attached storage into a networked storage configuration will allow your company to get a much more efficient return on your IT investment. And that's just the beginning. Storage consolidation will also enable your company to streamline business operations, accelerate time-to-market, fully leverage the power of your information, exploit new business opportunities and increase revenue. Sound good?

>> Request your free Architectures for Storage Consolidation white paper today!

In NCAR's case, the amount of data generated is driven by one simple factor -- the speed at which its two IBM supercomputers, the 10th and 33rd largest in the world, can create it. With the current processing power, the data growth rate sits at around 1.5 terabytes (TB) a day. Although it's taken 16 years to reach the 1 petabyte (PB) level, NCAR expects to break through the 2 PB barrier as early as next year. The increase in yield is directly related to the processing increases of the supercomputers, which process the raw numerical data for the models used by NCAR in its analysis.

In light of the sheer volume of data stored by the NCAR mass storage system (MSS), the only currently viable media is tape. At present, the backbone of the NCAR MSS is a tape-based archive with only a small amount of disk space for files. The tape technology consists of five PowderHorn 9310 tape libraries, 20 StorageTek 9840A silo-(library) attached tape drives, 28 StorageTek 9940A silo-attached tape drives, four StorageTek 9840A manual mount tape drives, etc.

One of the principal developers of the NCAR MSS is Gene Harano, manager of the high-performance systems section for the scientific computing division. For the past 16 years, Harano has been instrumental in the development and management of the MSS. Although Harano sees a point where there will be more disk capacity for the MSS, he still feels that tape will have its place. "Although the costs associated with disks are coming down, there is a long way to go before they are comparable to tape cost capacities."

According to Harano, the MSS software was initially deployed on an IBM mainframe under MVS, though this software is in the process of being migrated to open systems platforms. Some of the MSS functionality has already been moved. Direct tape access from 25 MSS heterogeneous Unix-based host platforms is accomplished with a combination of a high-performance parallel interface (HiPPI) fabric and StorageTek channel protocol bridges that attach directly to the control units of the storage devices.

Like others in his situation, Harano realizes that you can only keep throwing capacity at a storage problem for so long. Ultimately, he says, you need to find ways to manage and control the data you have. Harano's team felt that the current software offerings -- which allow data management and classification -- fall short of a suitable solution. As a result they are currently developing an in-house system that will allow them to manage their data more effectively. "We have a single user that owns over a million files," says Harano, "There can be no doubt that he does not know what they all are, and even more likely that he doesn't need them all."

Having seen the NCAR MSS grow over the last 16 years, Harano is able to offer one overriding piece of advice for those who are in the process of building and developing their own storage networks. "The amount of data being created in all environments is growing at an alarming rate. Never underestimate the size of the problem. There is always more data than you think there will be."

What do you think of the Storage Innovators e-mail? Take our quick survey and let us know.

For more information on National Center for Atmospheric Research (NCAR) visit its Web site.

Additional information on StorageTek can be found here.

MORE ON THIS TOPIC:
>>  Don't confuse archiving and backup

>>  Steel company builds sturdy archive infrastructure

>>  Best Web Links: Archiving


RESOURCES DIRECT FROM: HP Storage

>>  Learn about ENSAextended, HP's storage strategy for today and tomorrow. ENSAextended is the evolution of the storage utility vision, where the right information is accessible to anyone, anytime, anywhere.
>>  View the latest storage promotions and programs.
>>  See all of HP's storage blueprints, including blueprints for SANs, starter SANs, high availability, storage efficiency, disaster recovery and backup.
>>  HP offers the most comprehensive lineup of network storage solutions in the industry.
>>  Check out the HP U.S. Corporate Business Center for technology advice and resources.


A B O U T   T H I S   N E W S L E T T E R
Created by TechTarget
TechTarget - The Most Targeted IT Media
Copyright 2003, All Rights Reserved.

To unsubscribe from Storage Innovators: Leading-Edge Users of Enterprise Storage, a weekly TechTarget report:

>> Simply Reply to this Email with REMOVE within the Body or Subject
-OR-
>> Go to: http://searchStorage.techtarget.com/register
   - Log in to edit your profile.
   - Click on the link to Edit email subscriptions.
   - Uncheck the box next to the newsletter you wish to unsubscribe from.
   - When finished, click "Save Changes to My Profile".

Reply via email to