I concur with Rainer's comments regarding UDP socket overflows.
Also look at rxdebug -rxstats "no packets" and "no buffers" statistics.
Ideally, those would be very small, and you would never actually
_notice_ them increasing.
Rainer Toebbicke wrote:
>
> system with 128 MB cache. However, the maximum (not the average!) real
> service time for an individual I/O was almost 2.5 seconds (!!). That's an
> eternity for such a machine! I wonder what was going on in the system?
It seems like the SCSI driver may be trying so hard to optimize
throughput that one or more I/O operations remain queued for a very long
time. I don't have any proof, or source for their driver, so it's just
a hypothesis. Ask on comp.arch.storage.
> 'afsmonitor' shows two fields 'FetchData max' and 'StoreData max': on one of
> our server these read 415 and 306 seconds !! If they mean what their name
> leads me to think they mean I conclude that in one case one guy patiently
> turned his thumbs for 5 minutes whereas in the other he probably went for a
> coffee cursing... unless both were multi-megabyte-transfers. The highest
Fetchdata is limited to a single chunk (or a single directory) --
whatever size that would be in your configuration. Storedata tries to
return multiple chunks in a single RPC, so that could easily amount to
16 MB or more at one go.
> 'Storedata max' was 1121 seconds (!!). Plenty of time to pick up the phone
> and start yelling - to a point where I would be happy if somebody could
> speak up and say "look, it's not that bad because you misunderstood that and
> that".
Maximum and minimum are not very useful statistics. They are by
definition outliers. 1121 seconds looks more like a bug -- or somebody
resetting your clock. It's hard to imagine any store rpc taking 20
minutes unless it's happening via a WAN or dialup link.
> However, if somebody speaks about setting up a single 500 GB fileserver:
> think about how long it takes you to reload that much stuff from backup tapes
> should it ever get damaged. I believe that you can realistically pipe 2-2.5
> Gigabytes per *hour* onto a local tape drive. 200 hours working full time
> makes....
If I ever implemented a single 500 GB fileserver (which wasn't the
original question), I would make sure that a whole lot of the data was
replicated elsewhere. At the present time, I don't know of anyone who
stores more than 60 GB of unreplicated read-write data on a single
fileserver. (Here's where everyone chimes in and tells me otherwise :-)
I would also be sure that it is stored in a hot-swappable RAID storage
system. You have to worry, not only about the time to restore from tape,
but also the time to (maybe) fsck, (maybe) salvage, and (definitely)
attach all that data when a server restarts.
