On Monday, June 27, 2005 04:03:10 PM -0500 Troy Benjegerdes <[EMAIL PROTECTED]> wrote:
Has anyone looked into making volserver/vos dump go any faster? Is there any way to coalesce multiple small files into one RPC call, or pre-buffer files from disk?
This is not about multiple RPC's. A volume dump is transferred as part of one large RPC.
I'm seeing around an average of 10MB/sec, and performance goes in the toilet when the volume has lots of small files.
Volume dumps are created on the fly. Especially with lots of small files, much of the data is metadata emitted by the volserver based on the contents of the vnode index, rather than actual file data. There will also be delays related to opening files and reading their contents, which will be more prominent when lots of small files are involved.
This has basically prevented vos backup/etc from ever being able to go fast enough to get my DLT8000 drive to stream, and I think probably prematurely wore out one of my drives.
While volume dump performance is slow and certainly something we'd like to improve, it is unreasonable to expect the volserver to keep up a transfer rate sufficient for tape streaming given variability in network performance, fileserver load, and other factors. You should consider a backup system which spools volume dumps to disk, and writes spooled dumps to tape as part of a separate process which is not tightly coupled to the volume dump process.
-- Jeffrey T. Hutzelman (N3NHS) <[EMAIL PROTECTED]> Sr. Research Systems Programmer School of Computer Science - Research Computing Facility Carnegie Mellon University - Pittsburgh, PA _______________________________________________ OpenAFS-devel mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-devel
