You wrote:
> To: [EMAIL PROTECTED]
> Subject: Volume dump performance.
> From: Harald Barth <[EMAIL PROTECTED]>
> Date: Wed, 02 Aug 2000 01:39:29 +0200
>
>
> When doing a ``vos dump 250-meg-volume'' I get performance between 3
> and 3.5Mbytes/sec. The funny thing is, hardware does not seem to
> matter. I tried this on different architectures for host and clients,
> with the network being Fast Ether, FDDI and loopback (!). Raw disk
> reads are in the 15-20Mbytes/sec range and network speed is around
> 10Mbytes/sec.
>
> Hardware this was tested on is for example Digital 344au, Sun450, IBM
> 7043-140. No big CPU loads were observerd during the tests.
> The AFS software levels on therse boxes are 3.4a and 3.6.
>
> If you get better performance (letting you make use of all the
> bandwidth you have put into your system), please let me know.
> (and how you did it).
>
> With today's performance by backups will take "forever".
>
> Thanks,
> Harald.
>
3.5 Mb/s doesn't sound all that slow, considering the other figures you
mentioned. Volserver is probably not capable of actually doing
overlapping I/O on the disk & network at the same time, and even if it
were, it almost certainly can't do this in the course of a single
volume dump. That means that in order to dump a volume, with your
claimed hardware peak rate of 20 Mb/s & 10 Mb/s, the theoretical best a
dump could run at would be 6.6 Mb/s. This is assuming an unladen
fileserver, deserted network, and otherwise pristine conditions.
There are a number of other factors that get in the way here; for
instance, raw disk reads would typically measure time to read large
contiguous blocks. Going through the filesystem will cost a fair bit -
disk reads will necessarily be much smaller and not contiguous, at
which point seeks became much more important to disk performance than
transfer rate. Just to complicate matters more, if the fileserver were
well endowed (say, had 256M of main memory), and the volume had *just*
been created and populated, much of it might still be in the system
disk cache, at which point disk performance could become virtually
irrelevant.
Network I/O also has its issues. Most of RX is designed around the
notion of 1.5K datagrams & hardwired windows and such (this is true
even when jumbograms are being used). That means RX is well tuned for
ethernet like media, but may not make effective use of higher speed
stuff. For instance, to get the most out of FDDI, it's necessary to
send larger packets--only possible if the jumbogram logic in rx is
enabled and actually effective. If the path between volserver & vos
traverses more than one network hop, then routers, and intermediate
media also become important. The client machine is also important: if
the client runing vos is a slow IBM RT, then the speed of the
fileserver is much less important than the speed of the RT, even if
it's just dumping stuff to /dev/null .
In the real world, there are other issues that are just as important.
For instance, the "vos dump" format is actually fairly CPU intensive.
It should be easiest to see this by creating a smallish volume,
populating it with lots of very small files, then "vos dump"ing that
several times to try to get it all in the disk cache on the file
server. Having a slow CPU on the file server (like an IBM RT or system
370 compatible) should make it even more obvious. If, on the other
hand, the volume wasn't recently made and isn't in the disk cache,
reading all those little files will probably require lots of random
seeks all over the disk, so may be a better measure of seek time than
anything else. In addition to small files, it's typical to have many
more volumes that one dumps at once. With many small volumes (perhaps
such as user home directories?), there is additional overhead in the
dump process to go to VLserver to locate the volume, and in the case of
Tranarc's backup programs, there's additional delay to talk to the
backup server as well. Finally, it's typical to run backups during the
day, dumping all the snapshots made the previous night. This means
instead of spending all of the fileserver's resources on backups, one
wants to have enough fileserver bandwidth (including network, disk &
CPU) to be able to get real things done with the fileserver.
One final thought: AFS is a distributed architecture. That means you
don't need to put all your eggs in one basket. Two file servers
running in parallel and dumping data at 3.5 Mb/s is just as fast as one
file server running alone at 7 Mb/s. As a general rule, you should
probably not put more file storage on one file server than you would
feel comfortable restoring from tape all at once, to recover from a
total file server failure (ie, the ceiling leaks right above *that*
computer, and the CPU and all its attached disks are a total loss and
have to be replaced with your hot spare.)
-Marcus Watts
UM ITD PD&D Umich Systems Group
