Norman P B Joseph <[EMAIL PROTECTED]> writes:
> Our company is currently considering the purchase of an EMC Symmetrix
> box.
They rock.
All of our user data in AFS is now on EMC Symmetrix. They cost a small
fortune, and then they just run. We've had, I believe, two significant
hardware problems that caused us difficulties since we started putting
things on EMC (over a year ago now, I believe; I'd have to check for the
exact timing), only one of which caused user downtime. Disks just get
quietly replaced, performance is quite solid, and we're very pleased.
> In trying to determine if it would be beneficial to hook up our AFS
> servers to this box and how such a configuration might look, I began to
> wonder what exactly a "reasonable" AFS server configuration would look
> like otherwise, i.e., without using the Symmetrix box. What is an
> optimal layout of RO & RW volumes across AFS file servers? How does one
> maximize availability and reliability while minimizing the impact of
> unplanned outages?
We separate our servers into RW servers (nearly all user data), replica
servers (core file structure volumes and software for the most part, some
critical user data), and RW with replica servers that hold the read/write
volumes for replicated volumes plus one of the replicas. We do things
this way mostly for administrative convenience rather than for performance
and reliability reasons; it's easier to automate bulk volume moves and
balances when you know that everything on a given server has a particular
characteristic. It also helped in managing backups under our old backup
system.
Our replica servers for the most part use less reliable disk, on the
grounds that having one go down isn't as big of a deal. We also have some
RW servers with less reliable disk that we use to store large,
infrequently needed data (like server logs).
> And also to find out if any other sites have employed a Symmetrix box
> for their AFS server storage.
Yup. We're currently using three of them, purchased to replace quite a
few DEC StorageWorks and Sun SPARCStorage Arrays. Our replica servers and
our RW w/replica servers are on Sun A1000s now, except for one old replica
server that's still running on a Big Box 'O Disks.
> My scheme was to put all non-replicated (RW) volumes on the Sun RAID
> partitions (home directories & project spaces. etc.), put the RW
> versions of replicated volumes on the other Sun vice partitions, and
> dedicate the two SGI boxes to each hold identical copies of the RO
> versions of replicated volumes. I figured this would allow us to take
> either of the 2 SGI boxes out of commission (for maintenance, etc.)
> since the other SGI box would still be able to serve the same set of RO
> volumes.
That seems reasonable. The basic criteria are pretty obvious: Put
non-replicated RW content on your most reliable disk, since you have the
most problems when it goes down. Put your replicas on your least reliable
disk, since you can trivially replace that content if you lose that
storage. RWs for replicated volumes fall somewhere inbetween depending on
how you use replication; here, for the most part, the only unreleased
changes are trivial to recreate, so the RWs can live on less reliable disk
and we can recreate them from a replica plus reproducing the changes if
needed.
The exception is, of course, for non-replicated RW content that isn't
particularly important, like log storage or huge data sets that are just
dumps of tape or CD.
> My first cut at a Symmetrix configuration was to simply mirror our
> current configuration: 2 servers serving RO volumes and 2 servers
> serving RW volumes, each of the latter with enough extra capacity to
> hold all the RW volumes of the other server.
Putting RO volumes on a Symmetrix is probably a waste, since you really
don't need any disk reliability there.
> The downside then becomes ``do I really need 4 RO copies of the "man
> page" volume?'' So, maybe not -all- of the servers have to serve RO
> volumes, but what would be prudent?
Our philosophy on replication sites, in a nutshell, is that core structure
and critical data is replicated on four servers (root.afs, root.cell,
etc.), pretty much everything else that we replicate is replicated on
three servers, and things like frozen software trees for no-longer
supported platforms that are waiting to be deleted are replicated on two
servers.
Remember that you get the first replication site for "free" since it's an
inode clone on the same partition.
--
Russ Allbery ([EMAIL PROTECTED]) <http://www.eyrie.org/~eagle/>
