Hi, Les Mikesell wrote on 2014-06-26 10:47:19 -0500 [Re: [BackupPC-users] Uneffective rsync transfers?]: > On Thu, Jun 26, 2014 at 7:40 AM, Franti??ek Starý <franti...@stary.net> wrote: > > > > I've started experimenting with rsync xfer method and I've found that the > > rsync transfers always the changes against the last backup. > > You don't need to experiment to find that - see $Conf{IncrLevels} in > the documentation.
and it's not "the last backup", it's "the reference backup". Of course, if you're only doing full backups, that's the same thing. > > Second full backup (when 0 files changed): > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > full backup started for directory /testdir/ (baseline backup #0) > > [...] > > Got fatal error during xfer (No files dumped for share /testdir/) > > Backup aborted (No files dumped for share /testdir/) > > Not saving this as a partial backup since it has fewer files than the prior > > one (got 0 and 0 files versus 0) > > > > Shouldn't be this considered as "non error" status? It means no files were > > changed. > > That's optional, See $Conf{BackupZeroFilesIsFatal} No, I believe the objection is correct. BackupZeroFilesIsFatal is for a share with 0 files, not 0 *changed* files. If there's something there, but it hasn't changed since the last backup, that is not an error. Apparently, rsync (or rather the rsync XferMethod) is complaining about not having any files in the backup - possibly due to failing hard linking. > > Second full backup (when 1 file added): > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > [...] > > Done: 1 files, 1048576 bytes. > > > > It seems to be okay. Only the new file was transferred. But when I look in > > the web gui at this full backup, I see only this file. The other files > > (testfile1,2,3) are not here. Shouldn't be the full backup really full > > (merged somehow with the previous full backup)? > > Full rsync > backups should link unchanged files from the previous full into > the current tree. Other XferMethods will "coincidentally" create a link to the same pool file because the content matches (hard link count permitting). They are, however, unaware that the file is unchanged. Only rsync creates "same" log entries - other XferMethods only have "pool". You should check your XferLogs for this. It would seem they might not contain "same" entries, although they really should. > Is your xfer log full of 'can't link' errors? Since the quote was from the xfer log, apparently it isn't. And we're talking about a different piece of code than usual - it would seem to be the XferMethod that is failing to create the hard links, not BackupPC_link (though it's hard to imagine *how* it would fail in such a way). > > Fourth full backup (when 0 files changed after the previous backup): > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > [...] > > And again. Only the files that are not present in the previous backup are > > present. > > > > Is this a normal behavior? > > No, you have some problem with the filesystem. Does it support hard links? In particular, does Perl link() report success although the file system is not creating hard links? Honestly, if there were a bug in BackupPC that would re-transfer unchanged data every second backup, someone would have noticed ;-). There are people here doing large backups over slow links who depend on the bandwidth savings from rsync transfer. The analysis that hard links may be failing fits what you are seeing very well, though from the code I can't see how BackupPC a) would not notice, b) still complain about a zero file backup. Linking would need to *appear* to be working for BackupPC to even start. Of course, it's hard to analyse the code when you don't know which code to look at. Which version of BackupPC are you using on which distribution (or rather: where did you get it from)? What type of file system is your pool on, and how do you access it? A different possibility would be that the code for building a view of the last backup is not working correctly for you, though I can't even begin to imagine why it wouldn't. > > Is there a way to configure backuppc to transfer > > only changed files and merge them with the previous backup? I want to backup > > 200GB of email files. > > Yes, that is the way it is supposed to work. If the files are mbox > format (many messages in one file) it can be rather slow to merge the > changes but it does work. Maildir format works more efficiently. The important thing to note here is that with large mbox files, you'll have an individual copy after every change and not much savings from BackupPC pooling. Maildir on the other hand fits in with BackupPC pooling perfectly. Regards, Holger ------------------------------------------------------------------------------ Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/