On Sat, 6 Aug 2011, Claus-Justus Heine wrote:
So roughly 2.5 millon files, metadata is about 700M.
Still, this seems to be insane. Reading this month's archives (and given that
the last release of rdiff-backup was about '09) it seems that I would have to
fix that by myself, it seems. Or live with it. Or buy a backup server with
more memory ...
Of course, in principle the core-size of rdiff-backup is not a problem, on a
decent OS the part of the core currently not in use would just be swapped
out. There is only a problem if the program constantly traverses the
allocated buffers. This is what I suspect. It takes ages (2 or 3 days) to
finally finish the regression.
I don't know the internals of rdiff-backup, but I do know that there is a
runtime option to disable compression of .snapshot and .diff files. When
file operations are done properly, this might be helpful (using memory
mapped file access instead of decompressing/compressing, so effectively
using pointers to disk instead of ram).
But I don't know whether it would work that way. The only thing I do know
is that it will at best help you in the future, as it is not going to
change anything wrt historic files.
--
Maarten
_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
