On 28/6/19 9:46 pm, Jean-Louis Biasini via BackupPC-users wrote:
Hi all,
I have a working installation on a centos7 server. I'm backing up 30+
linux server hosts with rsync. Since I was a bit surprised by the
growing space taken on the backup server, I started to investigate on
the pooling mechanism. So I started to create simple identical text
files on 2 servers to see if the file was showed as already in pool
while backing up the second server after having back up the first.
First I just created 2 times the same text file (ie same content, same
md5sum, same permission, same selinux but different creating time)
second I created it on the first server then rsync it to the second to
have it absolutely identical (rsync -aAXv). In both case the file is
showed as created by the second server's backup. Then I tried a bigger
file created with fallocate -l 2M test.img and rsynced it the same
way. In all case my file is created again on the second backup. I also
checked the hard link limit that I increased from 32000 to 64000 (ext4
file-system here) with no improvement. Am I missing something?
You didn't mention which version of backuppc you are using, so I can't
be sure, but from memory, with BPC 3.x that is the correct/expected
behaviour. However, while the file is transferred from the second host
to the BPC server, it *will* get pooled correctly, so there will only be
a single on-disk version of the file. You can check this by confirming
the hard link entry on the BPC server for the 2 hosts in question.
eg:
ls -i /var/lib/backuppc/pc/<host1>/123/f%2f/ftest.txt
/var/lib/backuppc/pc/<host2>/124/f%2f/ftest.txt
If the inode number is the same for both files, then they are correctly
de-duplicated/pooled.
This might be the same for BPC 4.x, although I recall (and have
confirmed ages ago) that BPC 4.x doesn't transfer the file if the
duplicate file is located on the same host. So, it may still transfer
the file in your example (different hosts), but again should not be
duplicated in the pool (I don't know how to verify this on BPC 4.x).
Maybe something like this:
/usr/lib/backuppc/bin/BackupPC_attribPrint
/var/lib/backuppc/pc/<host1>/123/f%2f/attrib_*
Check the inode entry, and do the same for your second host, make sure
the inode entry is the same, and I assume this would mean that the
pooling is working correctly.
PS, just because the xferlog shows a file is transferred, doesn't mean
it isn't pooled after BPC receives the full file, and confirms the
content is a match for the other file it already has.
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
--
The information in this e-mail is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this e-mail by anyone else
is unauthorised. If you are not the intended recipient, any disclosure,
copying, distribution or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful. If you have received this message
in error, please notify us immediately. Please also destroy and delete the
message from your computer. Viruses - Any loss/damage incurred by receiving
this email is not the sender's responsibility.
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
