Re: [BackupPC-users] Understanding pros/cons of full vs. incremental and filled vs. unfilled for v4
Perhaps a better solution would be to do the "fill" part in the background after the new/incremental part of the background is recorded. Carl Soderstrom wrote at about 15:53:43 + on Monday, April 22, 2019: > If it is faster to duplicate the last filled backup and then update it, > would it be reasonable for a future version of BackupPC to pre-duplicate the > last backup during one of the maintenance operations? Then it would be > possible to simply update that when the time comes. > > Or am I misunderstanding this? > > On 04/21 05:15 , Craig Barratt via BackupPC-users wrote: > > I haven't done testing to see if having 100% fulls would be faster. On my > > ext4 system running on sw raid 10, it is actually quite slow duplicating a > > filled backup (which is the required step prior to starting a backup when > > you want the prior one to remain filled), since the whole directory tree > > has to be traversed. So that part is definitely slower. However, you are > > right that, after that, the backup is somewhat simpler since it is only > > modifying the current backup tree in place, since there's no need to update > > the prior unfilled backup with the reverse deltas. Another minor advantage > > of only having filled backups is deleting any one of them is easier, as you > > note. Currently BackupPC needs to merge deltas into the immediate prior > > backup (if unfilled) when you deleted a backup. > > > > Craig > > > -- > Carl Soderstrom > Systems Administrator > Real-Time Enterprises > www.real-time.com > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Understanding pros/cons of full vs. incremental and filled vs. unfilled for v4
If it is faster to duplicate the last filled backup and then update it, would it be reasonable for a future version of BackupPC to pre-duplicate the last backup during one of the maintenance operations? Then it would be possible to simply update that when the time comes. Or am I misunderstanding this? On 04/21 05:15 , Craig Barratt via BackupPC-users wrote: > I haven't done testing to see if having 100% fulls would be faster. On my > ext4 system running on sw raid 10, it is actually quite slow duplicating a > filled backup (which is the required step prior to starting a backup when > you want the prior one to remain filled), since the whole directory tree > has to be traversed. So that part is definitely slower. However, you are > right that, after that, the backup is somewhat simpler since it is only > modifying the current backup tree in place, since there's no need to update > the prior unfilled backup with the reverse deltas. Another minor advantage > of only having filled backups is deleting any one of them is easier, as you > note. Currently BackupPC needs to merge deltas into the immediate prior > backup (if unfilled) when you deleted a backup. > > Craig -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Understanding pros/cons of full vs. incremental and filled vs. unfilled for v4
Jeff, You statements about full vs incremental and filled vs unfilled are correct. In v3, all fulls are filled and all incrementals are unfilled. In v4, that's the default, but you can configure them differently. In particular, you could require that all backups are filled (whether they are incremental or full). I haven't done testing to see if having 100% fulls would be faster. On my ext4 system running on sw raid 10, it is actually quite slow duplicating a filled backup (which is the required step prior to starting a backup when you want the prior one to remain filled), since the whole directory tree has to be traversed. So that part is definitely slower. However, you are right that, after that, the backup is somewhat simpler since it is only modifying the current backup tree in place, since there's no need to update the prior unfilled backup with the reverse deltas. Another minor advantage of only having filled backups is deleting any one of them is easier, as you note. Currently BackupPC needs to merge deltas into the immediate prior backup (if unfilled) when you deleted a backup. Craig On Thu, Apr 11, 2019 at 9:50 AM wrote: > Presumably in an ideal world with no tradeoffs, every backup would be > full and filled as that would make sure each backup is standalone > independent without any need to merge or fill anything later on. > > *Full vs incremental* > If I understand it correctly, the difference between incrementals and > fulls is that for incrementals, if the attribs match, the files are > assumed to match, avoiding the need for full file checksums. > > So it makes sense to mix in incrementals between > full backups in order to speed up incremental backups relative to what > a full would take. > > > *Filled vs. Unfilled* > If I understand it correctly, unfilled backups don't include unchanged > files > in the backup file tree. > > For v3, this made sense as it eliminated the need to add yet more hard > linked files that take time to create and consume inodes. > > It's not as clear to me in v4 what the advantage is given that all the > file information is stored in the attrib file itself which needs to be > read anyway. It wouldn't seem that inefficent to me to just write out the > full directory attrib for all backups regardless of whether changed or not. > > - The inode wastage is a lot less in that you are only storing one > file per directory and you would need an attrib file anyway if any > file were to change in the directory. > > - There is no "messy" creation of hard links > > In all, I would even think that filled would still in general be > faster than unfilled in that you don't need to merge up the > incremental tree to see if there are changes -- all presumably much > slower than just writing the attrib file to each directory. > > Said another way, in v4, why not just make all backups filled while > still mixing incrementals between filled to overall speed up average > backup time. > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Understanding pros/cons of full vs. incremental and filled vs. unfilled for v4
Presumably in an ideal world with no tradeoffs, every backup would be full and filled as that would make sure each backup is standalone independent without any need to merge or fill anything later on. *Full vs incremental* If I understand it correctly, the difference between incrementals and fulls is that for incrementals, if the attribs match, the files are assumed to match, avoiding the need for full file checksums. So it makes sense to mix in incrementals between full backups in order to speed up incremental backups relative to what a full would take. *Filled vs. Unfilled* If I understand it correctly, unfilled backups don't include unchanged files in the backup file tree. For v3, this made sense as it eliminated the need to add yet more hard linked files that take time to create and consume inodes. It's not as clear to me in v4 what the advantage is given that all the file information is stored in the attrib file itself which needs to be read anyway. It wouldn't seem that inefficent to me to just write out the full directory attrib for all backups regardless of whether changed or not. - The inode wastage is a lot less in that you are only storing one file per directory and you would need an attrib file anyway if any file were to change in the directory. - There is no "messy" creation of hard links In all, I would even think that filled would still in general be faster than unfilled in that you don't need to merge up the incremental tree to see if there are changes -- all presumably much slower than just writing the attrib file to each directory. Said another way, in v4, why not just make all backups filled while still mixing incrementals between filled to overall speed up average backup time. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
