-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christoph Litauer wrote: > Craig Barratt schrieb: >> Christoph writes: >> >>> If I take a look on the structure of incrementals, I can see lots of >>> empty directories. It seems as if the whole directory structure of the >>> backup-source is kind of "duplicated" to the backup disk - although most >>> of the directories (and the files in them) are unchanged. >>> This leads to _lots_ of files/directories on the backuppc-disk (about 20 >>> million now). Is it necessary? >> Yes - the directory structure needs to be complete, even for >> an incremental. The storage used should be small. > > Craig, > > can you explain why, please? > > You're right: The storage amount is very small. But one can get _very_ > large directory structures on the backup filesystem. My BackupPC volume > now uses 147,650,611 inodes in an XFS filesystem. (I think) this leads > to a very slow directory creation: > time for i in `seq 1 10000`; do mkdir $i; done > runs about 2.5 minutes! This is 66 directories per second, whereas the > same command on the same server but another (empty) xfs filesystem took > only 34 seconds (about 5 times faster). > > Let's take a look at one (out of 10) of the backup clients: > A level 0 dump consists of 285,207 directories and 1,140,363 files. > A level 5 dump consists of 285,041 directories and 94,449 files. > This level 5 dump contains 192069 emtpy directories. > > I am afraid of expanding my backup scheme ... more clients and longer > incremental keep periods will lead to lots of millions of empty > directories ...
I don't want to get into a war about filesystem formats, but perhaps this is a valid data point for XFS. I don't know about other filesystem types either... You might like to check out/talk to some XFS experts, and see what they say about your very slow performance.... There may be some options to tune/improve the performance, or they may simply suggest another FS format which is better suited to the workload in backuppc. Either way, let us all know the results, I'm sure you aren't the only person on the list with this issue. BTW, I use reiserfs on my backuppc server, and will let you know the number of directories/files under the backuppc directory, along with the time to create new directories .... will probably take some hours for the results :) Regards, Adam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIYR3bGyoxogrTyiURAlKEAKCGJlw8aaMaNF/jfJtG99zhpT/XYgCfaP/5 aR6ZCUVcONdl9MKSMbLoOYw= =x8LR -----END PGP SIGNATURE----- ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
