On Oct 05, 2007 11:02 -0600, Daniel Leaberry wrote: > Aaron Knister wrote: > > I'm planning to hang 58 terabytes off of a PowerEdge 1950 with 4 CPUS > > and 8 gigs of memory. My MDS is a dual core Opteron with a 250gig > > raid1 metadata volume and 2GB of ram. Do you think this hardware > > configuration is sane? > > We have about 85TB of disk (in 24 luns) hanging off 4 PE2950's with > those same specs. They are set up in failover pairs (each handles 6 > luns) but I can't run too long on a single machine before it starts > thrashing when it takes over the other nodes 6 luns.
If you have 12 OSTs on a single node, that means up to 12 * 400MB = 4.8GB of RAM pinned just by the ext3 journal. Either you need a lot more RAM than this (8TB for example), or you need to shrink the journal size like 128MB (tune2fs to remove then re-add it). Using 128MB should be fine unless you have many hundreds of clients doing concurrent IO. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
