Hey All, I ran the problem vault earlier last night so that I could watch the memory while it was running. I dumped the memory on both the client (the problem machine) and the dirvish backup server. The results can be found below. On the client we can see there's still 38MB free. That, I assume is not the problem. The server shows 5MB of free memory. That may be the problem, but I'm not sure. Can somebody please weigh in?
Problematic Client During Dirvish Run: MemTotal: 1026120 kB MemFree: 38516 kB Buffers: 216588 kB Cached: 340256 kB SwapCached: 8 kB Active: 414076 kB Inactive: 355944 kB HighTotal: 121376 kB HighFree: 204 kB LowTotal: 904744 kB LowFree: 38312 kB SwapTotal: 3903672 kB SwapFree: 3903256 kB Dirty: 2780 kB Writeback: 92 kB Mapped: 271948 kB Slab: 200884 kB CommitLimit: 4416732 kB Committed_AS: 296916 kB PageTables: 3304 kB VmallocTotal: 118776 kB VmallocUsed: 6300 kB VmallocChunk: 111972 kB Problematic Client Without Dirvish Running: MemTotal: 1026120 kB MemFree: 64520 kB Buffers: 200852 kB Cached: 498700 kB SwapCached: 32 kB Active: 557828 kB Inactive: 278900 kB HighTotal: 121376 kB HighFree: 144 kB LowTotal: 904744 kB LowFree: 64376 kB SwapTotal: 3903672 kB SwapFree: 3903228 kB Dirty: 28 kB Writeback: 0 kB Mapped: 208904 kB Slab: 108288 kB CommitLimit: 4416732 kB Committed_AS: 224536 kB PageTables: 3284 kB VmallocTotal: 118776 kB VmallocUsed: 6300 kB VmallocChunk: 111972 kB Here are the reports from the server: ------------------------------ While the backup is running: ------------------------------ MemTotal: 515612 kB MemFree: 5492 kB Buffers: 21264 kB Cached: 327824 kB SwapCached: 0 kB Active: 165856 kB Inactive: 319344 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 515612 kB LowFree: 5492 kB SwapTotal: 3903672 kB SwapFree: 3903500 kB Dirty: 68 kB Writeback: 0 kB AnonPages: 136156 kB Mapped: 9656 kB Slab: 16968 kB SReclaimable: 10216 kB SUnreclaim: 6752 kB PageTables: 1392 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 4161476 kB Committed_AS: 259660 kB VmallocTotal: 511992 kB VmallocUsed: 4716 kB VmallocChunk: 506992 kB ------------------------------- While the backup is NOT running ------------------------------- MemTotal: 515612 kB MemFree: 11316 kB Buffers: 16804 kB Cached: 426520 kB SwapCached: 0 kB Active: 50764 kB Inactive: 428252 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 515612 kB LowFree: 11316 kB SwapTotal: 3903672 kB SwapFree: 3903488 kB Dirty: 4 kB Writeback: 0 kB AnonPages: 35700 kB Mapped: 8660 kB Slab: 17668 kB SReclaimable: 10588 kB SUnreclaim: 7080 kB PageTables: 860 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 4161476 kB Committed_AS: 61604 kB VmallocTotal: 511992 kB VmallocUsed: 4716 kB VmallocChunk: 506992 kB [EMAIL PROTECTED] wrote: > If it worked well for a while and is now seemingly taking > exponentially long to finish, I'd make sure you're not swapping > excessively. A directory of mail probably has a much larger ratio of > files to bytes than other places in your filesystem, so just saying > "there's not much data there" may misdiagnose the problem. > > Since you're (presumably) asking rsync to transfer a large number of > files, rsync must hold information about all of them in its address > space while it runs, and it's possible that you've crossed some magic > threshold (as the number of users has grown) that is causing rsync not > to fit in physical memory after it's built that data structure. If > that's true, you're going to run very, very slowly as all those pages > get thrashed onto and off of the disk again. > > [And if you have dirvish passing -H to rsync, this is even more likely > (especially with older rsync versions)---and isn't fully fixed even in > the most-recent rsync release (since -H makes it hard for rsync to > know when it can forget about already-transferred directories if > there's a possibility of a hardlink in there somewhere).] > _______________________________________________ > Dirvish mailing list > Dirvish@dirvish.org > http://www.dirvish.org/mailman/listinfo/dirvish > > _______________________________________________ Dirvish mailing list Dirvish@dirvish.org http://www.dirvish.org/mailman/listinfo/dirvish
